Video information display control method and device and electronic equipment

Abstract

The invention discloses a video information display control method and device, electronic equipment and a readable storage medium, and belongs to the field of artificial intelligence. The method comprises the following steps: displaying a program interface; and under the condition that the program interface is a video playing interface of the video, displaying at least one item of video content understanding information matched with the video type of the video.

Description

Technical Field

[0001] The present application belongs to the field of artificial intelligence, and specifically relates to a video information display control method, device and electronic equipment. Background Art

[0002] With the development of AI technology, the capabilities of multimodal video understanding models have gradually matured. They are generally used to understand the video content of the video through the model, and to classify or summarize based on the understanding of the video content.

[0003] In the related art, when a user is watching a music video, if the user is interested in a certain song or singer, for example, the user wants to further view other songs of the singer, or listen to the song, the user can manually switch to other applications to search for information about the singer or song, and then play the song. This makes the process of users viewing video-related content too cumbersome and inefficient. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a video information display control method, device and electronic device, which can avoid the tedious operation of users manually organizing video content or switching to other applications to search for video-related content, thereby improving the efficiency of querying video-related content.

[0005] In a first aspect, an embodiment of the present application provides a video information display control method, which includes: displaying a program interface; when the program interface is a video playback interface of a video, displaying at least one item of video content understanding information matching the video type of the video.

[0006] In a second aspect, an embodiment of the present application provides a video information display control device, which includes: a display module for displaying a program interface; the display module is also used to display at least one item of video content understanding information matching the video type of the video when the program interface is a video playback interface of the video.

[0007] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.

[0008] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0009] In a fifth aspect, an embodiment of the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run a program or instruction to implement the method described in the first aspect.

[0010] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method described in the first aspect.

[0011] In the embodiment of the present application, a program interface is displayed; when the program interface is a video playback interface of a video, at least one video content understanding information matching the video type of the video is displayed. In this solution, when the video playback interface of the video is displayed, the electronic device analyzes and understands the video to obtain video content understanding information matching the video type, and displays the obtained video content understanding information to the user for viewing, so that the user can directly view the recommended content related to the video while watching the video, avoiding the user manually organizing the video content, or manually opening the search program, and then entering the video content related content in the search program to obtain the information required by the user through the search program, thereby improving the efficiency of querying video related content. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1A is a schematic diagram of an educational video provided by some embodiments of the present application;

[0013] Figure 1B is a schematic diagram of an entertainment video provided by some embodiments of the present application;

[0014] Figure 2 is a flowchart of a video information display control method provided by some embodiments of the present application;

[0015] Figure 2A is a schematic diagram of function cards corresponding to educational videos provided in some embodiments of the present application;

[0016] Figure 2B is a schematic diagram of function cards corresponding to entertainment videos provided in some embodiments of the present application;

[0017] Figure 3A It is a schematic diagram of video content understanding information corresponding to mind map tags in educational videos provided by some embodiments of the present application;

[0018] Figure 3B It is a schematic diagram of video content understanding information corresponding to mind map tags in entertainment videos provided by some embodiments of the present application;

[0019] Figure 3C It is a schematic diagram of video content understanding information corresponding to time axis labels in educational videos provided by some embodiments of the present application;

[0020] Figure 3D is a schematic diagram of video content understanding information corresponding to video subtitle tags in educational videos provided by some embodiments of the present application;

[0021] Figure 3E is a schematic diagram of video content understanding information corresponding to translation tags in educational videos provided by some embodiments of the present application;

[0022] Figure 3F is a schematic diagram of video content understanding information corresponding to a tag in an educational video with one-click summary provided by some embodiments of the present application;

[0023] Figure 3G is a schematic diagram of video content understanding information corresponding to character introduction tags in educational videos provided by some embodiments of the present application;

[0024] Figure 3H It is a schematic diagram of video content understanding information corresponding to barrage hot tags in educational videos provided by some embodiments of the present application;

[0025] Figure 4A It is a schematic diagram of video content understanding information corresponding to song recognition tags in entertainment videos provided by some embodiments of the present application;

[0026] Figure 4B It is a schematic diagram of video content understanding information corresponding to video subtitle tags in entertainment videos provided by some embodiments of the present application;

[0027] Figure 4C It is a schematic diagram of video content understanding information corresponding to timeline labels in entertainment videos provided by some embodiments of the present application;

[0028] Figure 4D It is a schematic diagram of video content understanding information corresponding to a tag in an entertainment video with one-click summary provided by some embodiments of the present application;

[0029] Figure 4E It is a schematic diagram of video content understanding information corresponding to character introduction tags in entertainment videos provided by some embodiments of the present application;

[0030] Figure 4F It is a schematic diagram of video content understanding information corresponding to barrage hot tags in entertainment videos provided by some embodiments of the present application;

[0031] Figure 5Ais a schematic diagram of video content understanding information corresponding to rule introduction tags in sports videos provided by some embodiments of the present application;

[0032] Figure 5B is a schematic diagram of video content understanding information corresponding to score labels in sports videos provided by some embodiments of the present application;

[0033] Figure 5C is a schematic diagram of video content understanding information corresponding to timeline labels in sports videos provided by some embodiments of the present application;

[0034] Figure 5D is a schematic diagram of video content understanding information corresponding to video subtitle tags in sports videos provided by some embodiments of the present application;

[0035] Figure 5E is a schematic diagram of video content understanding information corresponding to translation tags in sports videos provided by some embodiments of the present application;

[0036] Fig. 5F is a schematic diagram of video content understanding information corresponding to character introduction tags in sports videos provided by some embodiments of the present application;

[0037] Figure 5G It is a schematic diagram of video content understanding information corresponding to barrage hot tags in sports videos provided by some embodiments of the present application;

[0038] Fig. 6A is a schematic diagram of an input method provided by some embodiments of the present application;

[0039] Figure 6B is a schematic diagram of an input method provided by some embodiments of the present application;

[0040] Figure 6C is a schematic diagram of an input method provided by some embodiments of the present application;

[0041] Figure 7 is a structural schematic diagram of a video information display control device provided by some embodiments of the present application;

[0042] Figure 8 is a structural schematic diagram of a video information display control device provided by some embodiments of the present application;

[0043] Fig. 9 is a structural schematic diagram of a video information display control device provided by some embodiments of the present application;

[0044] Fig.10 is a schematic diagram of the hardware structure of an electronic device provided in some embodiments of the present application;

[0045] Fig.11 It is a schematic diagram of the hardware structure of an electronic device provided in some embodiments of the present application. DETAILED DESCRIPTION

[0046] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0047] The following is an explanation of the technical terms involved in the embodiments of this application:

[0048] Video content understanding: mainly converts video frames into natural language or encoding vectors through multimodal models to facilitate subsequent processing, such as video classification, character recognition, and action recognition.

[0049] Prompt: It is mainly used to input commands to the large model, requiring the model to output the expected answer in a specified format or requiring it to output the expected answer.

[0050] User historical operations: mainly the interactive click operations after the user triggers the smart assistant function in the video scene.

[0051] Smart assistant: An assistant program with AI functions in electronic devices, which allows users to complete certain operations by operating or manually entering shortcut commands.

[0052] Specifically, the assistant program with AI function is an application in the electronic device, which is associated with an AI model, and the AI ​​model can be deployed in the electronic device or in the cloud. If the user wants the electronic device to perform a certain command operation, the user can trigger the interface of the intelligent assistant and enter the operation instruction in the interface, so that the intelligent assistant can input the operation instruction entered by the user into the AI ​​model, perform the operation, and output the information content that the user wants.

[0053] Video metadata: Metadata is a set of descriptive information about a video file that helps in the management of video assets. Metadata can be related to technical parameters, such as video format, file size and resolution, or descriptive parameters, such as the creation date or creator of the video.

[0054] Optical Character Recognition (OCR): refers to the process of processing and identifying images or videos containing text content, and extracting the text and layout information contained therein. Currently, there are many mature self-developed or industry solutions.

[0055] Card: is an information carrier with a closed outline, which provides various information of various applications in a condensed form, intuitively and quickly, and is used for displaying and interacting with various program information. In the Android system, a card is a window interface that can be stacked and presented. The shape of a card is usually a rounded rectangle, similar to the shape of a credit card, but it can also be other shapes. Cards make it more convenient and intuitive for users of electronic devices to obtain information and perform operations, and therefore are widely used. In the embodiments of the present application, the size, position, shape, etc. of the card are not limited.

[0056] Function card: usually a card with a specific functional property, such as a function card in an electronic device that displays weather application information, or a function card that displays clock application information, etc. In the embodiment of the present application, the function of the function card is to display video content understanding information.

[0057] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0058] The terms "at least one (item)", "at least one of" and the like in the specification and claims of this application refer to any one, any two or a combination of more than two of the objects included therein. For example, at least one (item) of a, b, and c can be represented by: "a", "b", "c", "a and b", "a and c", "b and c" and "a, b and c", where a, b, and c can be single or multiple. Similarly, "at least two (items)" means two or more, and its meaning is similar to that of "at least one (item)".

[0059] It should be noted that the video information display control method provided in the embodiments of the present application can be executed by electronic devices such as mobile phones, tablet computers, laptop computers, PDAs, and vehicle-mounted electronic devices. In some embodiments of the present application, the video information display control method provided in the embodiments of the present application is described by taking the electronic device as the execution subject to execute the video information display control method.

[0060] The video information display control method, device, electronic device and readable storage medium provided in the embodiments of the present application are described in detail below with reference to the accompanying drawings through specific embodiments and their application scenarios.

[0061] The video information display control method provided in the embodiment of the present application can be applied to a scenario where a user of an electronic device wants to query relevant information in a video.

[0062] In a specific application scenario, the video information display control method provided in the embodiment of the present application can be applied to a scenario where a user wants to query professional terms in an educational video, such as scenario 1.

[0063] Scenario 1: When a user is watching an educational video, Figure 1A As shown, assuming that the video title 11 of the educational video is "Breaking news! Mysterious "dark oxygen" discovered in the deep sea may overturn traditional theories of the origin of life", the user wants to check the meaning of the academic term "dark oxygen" in the video without closing the video.

[0064] In another specific application scenario, the video information display control method provided in the embodiment of the present application can be applied to a user who wants to search for the same clothing in an entertainment video.

[0065] Scenario 2: When users watch entertainment videos, Figure 1B As shown, assuming that the entertainment video is a variety show video, the current video playback interface 12 is playing Xiaohong's singing video, and the user wants to further query the shopping information of the dress without closing the video, such as the price, store or sales volume.

[0066] The execution subject of the video information display control method provided in this embodiment may be a video information display control device, which may be an electronic device, or a control module or processing module in the electronic device, etc. The technical solution provided in the embodiment of the present application is described below using an electronic device as an example.

[0067] The present application provides a method for controlling video information display. Figure 2 FIG. 1 is a flowchart of a video information display control method provided by an embodiment of the present application, which can be applied to electronic devices. Figure 2 As shown, the video information display control method provided in the embodiment of the present application may include the following steps 201 and 202.

[0068] Step 201: The electronic device displays a program interface.

[0069] In some embodiments of the present application, the above-mentioned program interface may be an interface of an application program in an electronic device.

[0070] Exemplarily, the program interface may be a video playback interface of a video application program, or may be a game operation interface of a game application program.

[0071] Step 202: When the program interface is a video playback interface of a video, the electronic device displays at least one item of video content understanding information matching the video type of the video.

[0072] In some embodiments of the present application, the above-mentioned video playback interface is an interface for playing videos in any video application.

[0073] In some embodiments of the present application, the above video types include but are not limited to at least one of the following: education, entertainment, sports, and comprehensive.

[0074] For example, when the video type is educational, the video may be a popular science video, an online course video, an academic video, or the like.

[0075] Exemplarily, when the video type is entertainment type, the video may be a gossip video, a game video, a variety show video, or the like.

[0076] For example, when the video type is sports type, the video may be a game video, or a sports commentary video, or the like.

[0077] For example, when the video type is a comprehensive type, the video may be a food video, a pet video, or the like.

[0078] In some embodiments of the present application, one item of video understanding content information corresponds to one content tag.

[0079] In some embodiments of the present application, the above-mentioned content tags include at least one of the following: keyword tags, translation tags, character introduction tags, rule introduction tags, score tags, one-click summary tags, mind map tags, video subtitle tags, timeline tags, song recognition tags, barrage hot spots tags, and identification of the same style tags.

[0080] In some embodiments of the present application, the above-mentioned video understanding content information can be displayed on a tab page, on a window, or on a function card.

[0081] Optionally, in some embodiments of the present application, the above step 202 specifically includes step 202a:

[0082] Step 202a: The electronic device displays at least one function card.

[0083] In some embodiments of the present application, each of the above-mentioned function cards includes a content tag and video content understanding information corresponding to the content tag.

[0084] Exemplarily, the above-mentioned function cards can be displayed in an overlaid manner on the video playback interface, or can be displayed on other interfaces except the video playback interface.

[0085] In a possible embodiment, the electronic device may be an electronic device including at least two display screens, wherein the video playback interface is displayed on a first display screen of the electronic device.

[0086] Furthermore, the above step 202 specifically includes step 202b:

[0087] Step 202b: Display at least one item of video content understanding information matching the video type of the video on the second display screen of the electronic device.

[0088] Example 1, combined with the above scenario 1, such as Figure 2A As shown, the electronic device is a folding screen electronic device, and a video titled "Breaking News! Mysterious "dark oxygen" discovered in the deep sea may overturn the traditional theory of the origin of life" is played on the first display screen 21, and function cards 23 are displayed on the second display screen 22, for example, function cards 231 with keyword labels, function cards 232 with mind map labels, function cards 233 with timeline labels, function cards 234 with video subtitle labels, etc. Correspondingly, function cards with other content labels can also be displayed. Figure 2A Only some of the tags are shown.

[0089] Example 2, combined with the above scenario 2, such as Figure 2B As shown, the electronic device is a folding screen electronic device, and the first display screen 21 plays Xiao Hong's singing video, and the second display screen 22 displays function cards 24, such as a function card 241 for identifying song labels by listening to songs, a function card 242 for identifying same-sounding labels, a function card 243 for timeline labels, and a function card 244 for video subtitle labels. Correspondingly, function cards with other content labels can also be displayed. Figure 2B Only some of the tags are shown.

[0090] In the video information display control method provided in the embodiment of the present application, a program interface is displayed; when the program interface is a video playback interface of a video, at least one video content understanding information matching the video type of the video is displayed. In this solution, when the video playback interface of the video is displayed, the electronic device analyzes and understands the video, obtains video content understanding information matching the video type, and displays the obtained video content understanding information to the user for viewing, so that the user can directly view the recommended content related to the video while watching the video, thereby improving the efficiency of querying video-related content.

[0091] Optionally, in some embodiments of the present application, the above step 202 “the electronic device displays at least one video content understanding information” specifically includes at least one of the following steps:

[0092] Step A1: When the content tag is a keyword tag, display explanation information of at least one keyword in the video.

[0093] Exemplarily, in the case where the content tag is a keyword tag, the electronic device traverses the academic keywords, or uncommon words, or proper nouns, etc. in the video, and searches and displays the explanation information of these keywords from the dictionary saved by default in the electronic device or from other search applications through the search interface.

[0094] Exemplarily, the above explanation information may include a noun explanation of the keyword, an explanation of words related to the keyword, or synonyms and antonyms of the keyword.

[0095] For example, when the keyword is "photosynthesis", the keyword explanation information may be: photosynthesis generally refers to the process in which green plants absorb light energy, synthesize carbon dioxide (CO2) and water (H2O) into energy-rich organic matter, and release oxygen at the same time.

[0096] In this way, the electronic device can identify the keywords in the video content and automatically display the explanation information of these keywords, so that the user can quickly know the specific meaning of the keywords when watching the video. The user does not need to enter the keywords in the search program to search for the specific meaning of the keywords, which reduces the user's manual search operation for keyword information and actively helps the user understand the meaning of the keywords in the video, thereby effectively helping the user understand the video.

[0097] Step A2: When the content tag is a translation tag, display translation information of at least one keyword in the video.

[0098] Exemplarily, when the content tag is a translation tag, the electronic device traverses the academic keywords, or uncommon words, or proper nouns, etc. in the video, and searches and displays the translation information of these keywords from the dictionary saved by default in the electronic device or from other search applications through the translation interface.

[0099] Exemplarily, the above translation information includes a translation of the keyword into another language, such as Chinese to English, or English to Chinese, or Chinese to French, or French to Chinese, and this application does not limit this.

[0100] It is understandable that when the language set by the user is Chinese, if English or other languages ​​are detected in the text, the word or phrase is judged to be a professional word and uncommon according to the dictionary stored in the electronic device. When professional words or difficult words are detected, the translation interface can be called to directly display the translation of the relevant difficult words.

[0101] In this way, the electronic device recognizes keywords in different languages ​​in the video content, automatically translates these keywords, and displays the translation information on the interface, so that users can quickly know the specific meanings of these keywords in different languages ​​when watching the video. Users do not need to enter keywords in the search program to search for specific translations of keywords, which reduces the user's manual translation of keywords and actively helps users understand the explanations of keywords in different languages ​​in the video, thereby effectively helping users understand the video.

[0102] Step A3: When the content tag is a character introduction tag, at least one character introduction information of the video character in the video is displayed.

[0103] Exemplarily, the electronic device searches for characters appearing in the video through a search interface.

[0104] Exemplarily, the above-mentioned video character may be a character selected by the user, or may be all the characters appearing in the video.

[0105] In this way, the electronic device can identify various characters in the video content and automatically display introductions about these characters, so that users can quickly know the background and character information of each character in the video when watching the video. Users do not need to take screenshots of the characters and then perform image recognition operations in the search program to obtain the character introduction of the character. This reduces the user's manual search operations for character introductions and actively helps users identify the characters appearing in the video, thereby effectively helping users understand the video.

[0106] Step A4: When the content tag is a one-key summary tag, summary information of the video content of the video is displayed.

[0107] Exemplarily, the summary information is information about the summary of the main points of the video content of the video by the electronic device, or abstract information of the video.

[0108] Exemplarily, the electronic device inputs the video file and one-key summary prompt words of the video into the video comprehension model. The video comprehension model summarizes the video file with one key according to the video comprehension requirements contained in the one-key summary prompt words, and then outputs the summarized information to the video comprehension model and displays it.

[0109] Exemplarily, the one-key summary prompt is:

[0110] “You are now an expert in extracting key points of content. You can extract the key points of the video based on the title, tags and content of the current video.

[0111] Require:

[0112] 1. The key points need to be discussed separately;

[0113] 2. If the video is divided into chapters, the key points should be divided into chapters;

[0114] The current video related information is as follows:

[0115] Title: {title}

[0116]

Label

[0117]

Content

[0118]

Category

[0119] Output the key points of the video in JSON format. ”

[0120] Specifically, in combination with scenario 1, the one-key summary prompt word may be:

[0121] “You are now an expert in extracting key points of content. You can extract the key points of the video based on the title, tags and content of the current video.

[0122] Require:

[0123] 1. The key points need to be discussed separately;

[0124] 2. If the video is divided into chapters, the key points should be divided into chapters;

[0125] The current video related information is as follows:

[0126]

Title

[0127]

Tags

[0128]

Content

[0129]

Category

[0130] Output the key points of the video in json format".

[0131] In this way, the electronic device extracts and summarizes the key points of the video content through the video understanding model, and automatically displays the summary information of the video content, so that when the user watches educational videos such as learning or popular science, the user can directly view the key points of the video. The user does not need to use a note-taking program or manually record the summary information of the key points of the video content, thereby effectively helping the user understand the video.

[0132] Step A5: When the content tag is a mind map tag, display the video mind map.

[0133] Exemplarily, the above-mentioned video mind map is generated based on the video title and video content of the video.

[0134] Exemplarily, the electronic device inputs the video file and mind map prompts of the video into the video comprehension model. The video comprehension model generates a mind map for the video file according to the video comprehension requirements contained in the mind map prompts, and then outputs the generated mind map to the video comprehension model and displays it.

[0135] Exemplarily, the mind map prompts are:

[0136] “You are now an expert in extracting key points from the content. You can generate a corresponding mind map based on the title, tags, and content of the current video.

[0137] Require:

[0138] 1. The key points need to be discussed separately;

[0139] 2. If the video is divided into chapters, the key points should be divided into chapters;

[0140] The current video related information is as follows:

[0141] Title: {title}

[0142]

Label

[0143]

Content

[0144]

Category

[0145] Output the generated mind map in json format. "

[0146] Specifically, in combination with scenario 1, the mind map prompts can be:

[0147] “You are now an expert in extracting key points from the content. You can generate a corresponding mind map based on the title, tags, and content of the current video.

[0148] Require:

[0149] 1. The key points need to be discussed separately;

[0150] 2. If the video is divided into chapters, the key points should be divided into chapters;

[0151] The current video related information is as follows:

[0152]

Title

[0153]

Tags

[0154]

Content

[0155]

Category

[0156] Output the generated mind map in json format. "

[0157] In this way, the electronic device generates a mind map based on the video content through the video understanding model, and automatically displays the mind map information corresponding to the video content, so that the user can directly view the mind map of the video when watching the video. The user does not need to use a note-taking program or manually record the video content to make a mind map, thereby effectively helping the user understand the video.

[0158] Step A6: When the content tag is a video subtitle tag, display the video subtitles of at least one video frame of the video and the video timestamp corresponding to each video subtitle.

[0159] Step A7: When the content tag is a time axis tag, display the video timestamp of at least one key frame of the video and the frame content description information of each key frame.

[0160] Exemplarily, the electronic device inputs the video file and timeline prompt words of the video into the video understanding model. The video understanding model organizes the subtitle file and timeline of the video file according to the video understanding requirements contained in the timeline prompt words, and then outputs the organized timestamps and corresponding video subtitles to the video understanding model and displays them.

[0161] It is understandable that the electronic device directly generates video subtitles corresponding to the video frames according to the time, and when the user clicks the subtitles or video timestamp, it jumps to the corresponding moment.

[0162] For example, the timeline prompt words may be:

[0163] “You are now an expert in extracting key points of content. You can generate a corresponding timeline based on the title, tags, and content of the current video.

[0164] Require:

[0165] 1. Output the division of paragraphs according to the subtitle content;

[0166] 2. Keep the timestamp of the corresponding subtitles;

[0167] The current video related information is as follows:

[0168] Title: {title}

[0169]

Label

[0170]

Content

[0171]

Category

[0172] Output the generated timeline in json format.

[0173] Specifically, in combination with scenario 1, the timeline prompt words may be:

[0174] “You are now an expert in extracting key points of content. You can generate a corresponding timeline based on the title, tags, and content of the current video.

[0175] Require:

[0176] 1. Output the division of paragraphs according to the subtitle content;

[0177] 2. Keep the timestamp of the corresponding subtitles;

[0178] The current video related information is as follows:

[0179]

Title

[0180]

Tags

[0181]

Content

[0182]

Category

[0183] Output the generated timeline in json format.

[0184] In this way, the electronic device obtains the video subtitles in each video frame in the video and the timestamps corresponding to the subtitles through the video understanding model, and automatically displays the video subtitles in each video frame and the timestamps corresponding to the subtitles, so that when the user is watching the video, he can quickly reach the corresponding video frame position by clicking on the timestamp or video subtitles, reducing the steps for the user to manually pull the video progress. At the same time, it allows the user to more intuitively view the time position of the video subtitles, thereby effectively helping the user understand the video.

[0185] Step A8: When the content tag is a song recognition tag, the song recognition result information of the songs in the video is displayed.

[0186] Exemplarily, the electronic device performs binary classification through the transformer model, obtains the audio file in the video, searches for songs related to the audio file by calling the interface of the music application, and displays the song title, singer name and song file.

[0187] In this way, the electronic device can identify the background music in the video content and automatically display the song recognition result information about the background music, so that the user can quickly know the song information in the video when watching a video with music. The user does not need to search for the background music in the search program to obtain the relevant song information of the background music, which reduces the user's manual search operation for songs and actively helps the user identify the songs that appear in the video, thereby effectively helping the user understand the video.

[0188] Step A9: When the content tag is a bullet screen hot topic tag, display at least one bullet screen hot topic information.

[0189] Exemplarily, the above-mentioned barrage hot topic information may be the barrage content of a certain barrage.

[0190] For example, the electronic device can obtain the bullet comments in the video through the API, and then cluster the bullet comments according to vectors, where the vectors can be BERT vectors or transformer vectors. The frequency statistics show hot topics, and clicking on the corresponding bullet comments can jump to the related bullet comments.

[0191] In this way, the electronic device identifies the barrage information appearing in the video content through the interface, and automatically classifies and displays different barrage hot topic information, so that when the user is watching the video, the user can quickly know the barrage hot topics involved in the video. The user does not need to check the barrage one by one to obtain the hot topics and understand the content of the video, thereby effectively helping the user understand the video.

[0192] Step A10: When the content tag is a tag for identifying the same style, display the identification result information of the clothes that are the same style as the clothes of the video character in the video.

[0193] For example, if the user input is received, the electronic device obtains a video screenshot corresponding to the current video input, and then detects whether there is clothing in the video screenshot through an existing classification model, such as mobilenet v3 or a semantic segmentation model SAM. If there is clothing in the video screenshot, the interface of the shopping application is called to search for relevant shopping information of the clothing in the video screenshot, and the searched relevant shopping information, i.e., the above-mentioned clothing recognition result information, is displayed.

[0194] In this way, the electronic device can identify various clothes in the video content and automatically display the relevant shopping information of the clothes, so that the user can quickly know the recognition result information of each piece of clothing in the video when watching the video. The user does not need to take a screenshot of the person and then perform image recognition operations in the shopping program to obtain the shopping information of the clothes. This reduces the user's manual search operations and actively helps the user to identify the product information appearing in the video, thereby effectively helping the user understand the video.

[0195] Step A11: When the content tag is a rule introduction tag, the rule introduction information of the sports event in the video is displayed.

[0196] Exemplarily, the electronic device analyzes the sports in the video, searches for the game rules of the sports through the search interface, and displays the game rules, that is, the above-mentioned rule introduction information.

[0197] In this way, the electronic device can identify the competition rules of the sports events in the video content and automatically display the competition rules, so that the user can quickly know the competition events and competition rules in the video when watching the competition video. The user does not need to manually enter the competition events in the search program to obtain the competition rules, which reduces the user's manual search operations and actively helps the user identify the competition rules of the competition events in the video, thereby effectively helping the user understand the video.

[0198] Step A12: When the content tag is a score tag, the scores of the sports events in the video are displayed.

[0199] Exemplarily, the electronic device obtains the score in the video in real time, converts the score into text, and displays it as video content understanding information.

[0200] In this way, the electronic device can analyze and convert the scores involved in the sports events in the video content, and automatically display the score results, so that when the user is watching the game video, he can quickly know the progress and score of the game in the video, and actively help the user to identify the progress and score of the game in the video, thereby effectively helping the user to understand the video.

[0201] In this way, users can view different video content understanding information directly on different pages or interfaces when playing videos in the video playback interface according to different content tags, thereby reducing the steps and process of querying video-related content information, and helping users better understand the video content.

[0202] Optionally, in some embodiments of the present application, under different video types, the electronic device fully understands the video content according to the video type to obtain different content tags. Specifically, different video types have different content tags.

[0203] A first possible implementation:

[0204] In the case where the video type is educational, the above step 202 "the electronic device displays at least one video content comprehension information" specifically includes at least one of the following steps:

[0205] Step B1: The electronic device displays a keyword tag and explanation information of at least one keyword in the video.

[0206] For example, in conjunction with scenario 1, refer to Figure 2A As shown, the electronic device is a folding screen electronic device, and a video titled "Breaking News! Mysterious "dark oxygen" discovered in the deep sea may overturn traditional theories of the origin of life" is played on the first display screen 21, and video content comprehension information with keyword label 231 is displayed on the second display screen 22, that is, explanatory information with the keyword "dark oxygen": Scientists originally believed that only plant photosynthesis can produce oxygen in nature, but the latest research has found that the metal mineral "manganese nodules" in the deep sea can also produce oxygen through similar thunder and lightning in the thunder pool. Since the process does not require light, the research team named the oxygen produced in this way "dark oxygen."

[0207] Step B2: The electronic device displays the mind map label and the video mind map.

[0208] In some embodiments of the present application, the above-mentioned video mind map is generated based on the video title and video content of the video.

[0209] For example, combined with scenario 1, Figure 3A As shown, the electronic device is a folding screen electronic device, which plays a video titled "Breaking News! Mysterious "dark oxygen" discovered in the deep sea, which may overturn traditional theories of the origin of life" on the first display screen 21, and displays video content comprehension information 311 of the mind map label 232 on the second display screen 22.

[0210] For example, combined with scenario 2, Figure 3B As shown, the electronic device is a folding screen electronic device, which plays Xiaohong's singing video on the first display screen 21 and displays the video content comprehension information 312 of the mind map label 232 on the second display screen 22.

[0211] Step B3: The electronic device displays the time axis label and the video timestamp of at least one key frame of the video and the video subtitle information corresponding to each video timestamp.

[0212] For example, combined with scenario 1, Figure 3CAs shown, the electronic device is a folding screen electronic device, which plays a video titled "Breaking News! Mysterious "dark oxygen" discovered in the deep sea, which may overturn traditional theories of the origin of life" on the first display screen 21, and displays video content understanding information 313 of the timeline label 233 on the second display screen 22.

[0213] Step B4: The electronic device displays the video subtitle tag and the video subtitles of at least one video frame of the video and the video timestamp corresponding to each video subtitle.

[0214] For example, combined with scenario 1, Figure 3D As shown, the electronic device is a folding screen electronic device, which plays a video titled "Breaking News! Mysterious "dark oxygen" discovered in the deep sea may overturn traditional theories of the origin of life" on the first display screen 21, and displays video content understanding information 314 of the video subtitle label 234 on the second display screen 22.

[0215] Step B5: the electronic device displays the translation tag and translation information of at least one keyword in the video;

[0216] For example, combined with scenario 1, Figure 3E As shown, the electronic device is a folding screen electronic device, and a video titled "Breaking news! Mysterious "dark oxygen" discovered in the deep sea may overturn traditional theories of the origin of life" is played on the first display screen 21, and video content understanding information 315 of the translation tag 235 is displayed on the second display screen 22, namely, the translation information of "dark oxygen": dark oxygen, the translation information of "deep seafloor": the abyssal seafloor; and the translation information of the key vocabulary "seafloor; seabed": seafloor.

[0217] Step B6: The electronic device displays the one-key summary tag and summary information of the video content of the video.

[0218] For example, combined with scenario 1, Figure 3F As shown, the electronic device displays the video content understanding information 316 of the one-key summary label 236 on the second display screen 22: Mysterious "dark oxygen" was found in the deep sea, challenging the traditional theory of the origin of life. The latest research reveals that in the deep sea plain of the Pacific Ocean, in the dark environment at a depth of four thousand meters, the oxygen concentration is abnormally high, exceeding the expectations of the scientific community. This phenomenon is not caused by photosynthesis, but is related to the combination of multiple metals, which may act as a catalyst to decompose water molecules and produce oxygen. This discovery may not only subvert the theory of the origin of life, but also have a profound impact on the marine ecosystem and space exploration. Recently, scientists have made progress in superconductors, micro-aircraft, Mars exploration, etc., challenging the limits of science and technology.

[0219] Step B7: The electronic device displays a character introduction tag and at least one character introduction information of the video character in the video.

[0220] For example, Figure 3G As shown, the electronic device is a folding screen electronic device, and a popular science video with the video title "Intelligence Changes Life" is played on the first display screen 21, and the video content understanding information 317 of the character introduction label 237 is displayed on the second display screen 22, that is, the character introduction information of the character selected by the user: Zhang San is a character in Gu Long's martial arts novel "The Legend of Chu Liuxiang", a friend of Chu Liuxiang, and appeared in "The Legend of the Bat". Zhang San is called "Quick Net" Zhang San, and the fish he grills is the most delicious in the world. Zhang San is a less eye-catching character, a small character image, he is alert, eloquent, has mixed eyes, has many friends, and of course is very loyal to his friends.

[0221] Step B8: The electronic device displays a bullet screen hotspot tag and at least one bullet screen hotspot topic information.

[0222] For example, combined with scenario 1, Figure 3H The electronic device shown is a folding screen electronic device. A video titled "Breaking News! Mysterious "dark oxygen" discovered in the deep sea may overturn traditional theories of the origin of life" is played on the first display screen 21, and video content understanding information 318 of the barrage hotspot tag 238 is displayed on the second display screen 22.

[0223] It is understandable that users can directly click on the barrage to jump to the video-related barrage display location.

[0224] It should be noted that the above steps B1 to B2 are illustrations of the interface display diagrams of the above steps A1 to 12 in different types of videos. For the specific acquisition process and related description content, please refer to the above steps A1 to 12, which will not be repeated here.

[0225] In this way, when playing an educational video on the video playback interface, the corresponding video content understanding information can be viewed directly on different pages or interfaces, thereby reducing the steps and process of querying video-related content information, thereby helping users better understand the video content.

[0226] A second possible embodiment:

[0227] In the case where the video type is entertainment type, the above step 202 "the electronic device displays at least one video content understanding information" specifically includes at least one of the following steps:

[0228] Step C1, the electronic device displays the song recognition label and the song recognition result information of the song in the video.

[0229] For example, combined with scenario 2, Figure 4AAs shown, the electronic device is a folding screen electronic device. The first display screen 21 plays Xiao Hong's singing video, and the second display screen 22 displays the video content understanding information 411 of the song recognition label 241, that is, the relevant information of the recognized song, such as the song "Free as a Dream".

[0230] Step C2: The electronic device displays information on identifying the same label and the same clothing as the clothing of the video character in the video.

[0231] For example, in conjunction with scenario 2, refer to Figure 2B As shown, the electronic device is a folding screen electronic device, which plays Xiaohong's singing video on the first display screen 21 and displays video content understanding information identifying the same label 242 on the second display screen 22: photos of the clothes, links to purchase the clothes, prices of the clothes and brands of the clothes.

[0232] Step C3: The electronic device displays the time axis label and the video timestamp of at least one key frame of the video and the video subtitle information corresponding to each video timestamp.

[0233] For example, combined with scenario 2, Figure 4B As shown, the electronic device is a folding screen electronic device. Xiaohong's singing video is played on the first display screen 21, and the video content understanding information 412 of the video subtitle label 244 is displayed on the second display screen 22. Each timestamp corresponds to the video subtitle information, such as, "0:04 A competition called "Singer" shocked the whole country."

[0234] Step C4: The electronic device displays the video subtitle tag and the video subtitles of at least one video frame of the video and the video timestamp corresponding to each video subtitle.

[0235] For example, combined with scenario 2, Figure 4C As shown, the electronic device is a folding screen electronic device, which plays Xiaohong's singing video on the first display screen 21 and displays the video content understanding information 413 of the timeline label 243 on the second display screen 22.

[0236] Step C5: The electronic device displays the one-key summary tag and summary information of the video content of the video.

[0237] For example, combined with scenario 2, Figure 4D As shown, the electronic device is a folding screen electronic device, and the singing video of Xiao Hong is played on the first display screen 21, and the video content understanding information 414 of the one-key summary tag 245 is displayed on the second display screen 22: "The champion selected by "Singer 2024" may not be the most technically advanced, but must be the most controversial."

[0238] Step C6: The electronic device displays a character introduction tag and at least one character introduction information of the video character in the video.

[0239] For example, combined with scenario 2, Figure 4E As shown, the electronic device is a folding screen electronic device, and a singing video of Xiao Hong is played on the first display screen 21, and video content understanding information 415 of the character introduction label 246 is displayed on the second display screen 22: "Singer: Xiao Hong, participated in the "Singer" competition and won the first place. Her representative songs include "Song A", "Song B" and "Song C", etc."

[0240] Step C7: The electronic device displays a bullet screen hotspot tag and at least one bullet screen hotspot topic information.

[0241] For example, combined with scenario 2, Figure 4F As shown, the electronic device is a folding screen electronic device, which plays Xiaohong's singing video on the first display screen 21 and displays the video content understanding information 416 of the barrage hotspot tag 247 on the second display screen 22.

[0242] It is understandable that users can directly click on the barrage to jump to the video-related barrage display location.

[0243] It should be noted that the above steps B1 to B2 are illustrations of the interface display diagrams of the above steps A1 to 12 in different types of videos. For the specific acquisition process and related description content, please refer to the above steps A1 to 12, which will not be repeated here.

[0244] In this way, when playing entertainment videos in the video playback interface, the corresponding video content understanding information can be viewed directly on different pages or interfaces, thereby reducing the steps and process of querying video-related content information, thereby helping users better understand the video content.

[0245] A third possible embodiment:

[0246] In the case where the video type is sports type, the above step 202 "the electronic device displays at least one video content understanding information" specifically includes at least one of the following steps:

[0247] Step D1, the electronic device displays a rule introduction label and rule introduction information of the sports event in the video.

[0248] For example, Figure 5AAs shown, the electronic device is a folding screen electronic device, which plays a table tennis game on the first display screen 21 and displays the table tennis game rules 511 of the rule introduction label 251 on the second display screen 22: "Table tennis games include men's and women's singles, men's and women's doubles, and mixed doubles. Doubles games are played by two athletes on one side, and singles games are played by one athlete on one side. Game scoring: In a table tennis game, except for the round in which a re-serve is ordered, an athlete scores one point in the following situations. The opposing athlete fails to serve legally; the opposing athlete fails to return a legal blow;... the opposing athlete touches the playing surface with his or her non-racket hand; in doubles, the opposing athlete hits the ball in the wrong order; when executing the rotating serve method, the receiving athlete or his or her doubles partner, including the receiving blow, completes 13 legal returns."

[0249] Step D2: The electronic device displays a score label and the score of the sports event in the video.

[0250] For example, Figure 5B As shown, the electronic device is a folding screen electronic device, which plays a table tennis match on the first display screen 21 and displays the table tennis match score 512 of the score label 252 on the second display screen 22: "Currently, Red Team VS Blue Team: Overall score 1:1, this round 9:3".

[0251] Step D3: The electronic device displays the time axis label and the video timestamp of at least one key frame of the video and the video subtitle information corresponding to each video timestamp.

[0252] For example, Figure 5C As shown, the electronic device is a folding screen electronic device, which plays a table tennis match on the first display screen 21 and displays the video content understanding information 513 of the timeline label 253 on the second display screen 22: "05:46 The first round of the game ends".

[0253] Step D4: The electronic device displays the video subtitle tag and the video subtitles of at least one video frame of the video and the video timestamp corresponding to each video subtitle.

[0254] For example, Figure 5D As shown, the electronic device is a folding screen electronic device, a table tennis match is played on the first display screen 21, and video content understanding information 514 of the video subtitle label 254 is displayed on the second display screen 22: "00:01 Hello everyone, today we are playing the table tennis match of the first campus sports meeting."

[0255] Step D5: The electronic device displays the translation tag and translation information of at least one keyword in the video.

[0256] For example, Figure 5EAs shown, the electronic device is a folding screen electronic device, which plays a table tennis match on the first display screen 21 and displays the video content understanding information 515 of the translation tag 255 on the second display screen 22, namely, the translation information of "table tennis competition": Table tennis at the Summer Olympics, and the translation information of "score": score.

[0257] Step D6: The electronic device displays the one-key summary tag and summary information of the video content of the video.

[0258] For example, Fig. 5F As shown, the electronic device is a folding screen electronic device, which plays a table tennis match on the first display screen 21 and displays the video content understanding information 515 of the translation tag 255 on the second display screen 22, namely, the translation information of "table tennis competition": Table tennis at the Summer Olympics, and the translation information of "score": score.

[0259] Step D7: The electronic device displays a character introduction tag and at least one character introduction information of the video character in the video.

[0260] For example, Fig. 5F As shown, the electronic device is a folding screen electronic device. A table tennis match is played on the first display screen 21, and video content understanding information 516 of the character introduction label 256 is displayed on the second display screen 22: "Player: Xiao Ming, a member of the Guangming University table tennis team, has won the table tennis championship of the school sports meeting for three consecutive times."

[0261] Step D8: The electronic device displays a bullet screen hotspot tag and at least one bullet screen hotspot topic information.

[0262] For example, Figure 5G As shown, the electronic device is a folding screen electronic device, which plays a table tennis match on the first display screen 21 and displays the video content understanding information 517 of the barrage hotspot tag 257 on the second display screen 22.

[0263] It should be noted that the above steps B1 to B2 are illustrations of the interface display diagrams of the above steps A1 to 12 in different types of videos. For the specific acquisition process and related description content, please refer to the above steps A1 to 12, which will not be repeated here.

[0264] In this way, when playing a sports video in the video playback interface, the corresponding video content understanding information can be viewed directly on different pages or interfaces, thereby reducing the steps and process of querying video-related content information, thereby helping users better understand the video content.

[0265] Optionally, in some embodiments of the present application, before the above step 202 "the electronic device displays at least one video content understanding information matching the video type of the video", the video information display control method provided by the embodiment of the present application further includes step 301:

[0266] Step 301: The electronic device inputs a video file and a video comprehension prompt word of a video into a video comprehension model, determines at least one video content comprehension information in the video through the video comprehension model, and outputs at least one video content comprehension information.

[0267] In some embodiments of the present application, the video comprehension prompt words include video description information.

[0268] In some embodiments of the present application, the above-mentioned video description information includes at least one of the following: the video type of the video, the timeline of the video, the author of the video, the tag of the video, and the title of the video.

[0269] The following are examples of video description information for different video types, namely, video description information for educational, entertainment, and other types of videos.

[0270] Example 1:

[0271] Video type: Educational

[0272] Video link: https: / / www.dsafsef.com / video

[0273] Basic Information:

[0274] {Author: Science Lens

[0275] Date: 2024-07-25 11:51:30

[0276] Duration: 05:47

[0277] Title: Big news! Mysterious "dark oxygen" discovered in the deep sea may overturn traditional theories of the origin of life, [Science Express 161]

[0278] Video introduction: Related research: Sweetman, AK, Smith, AJ, de Jonge, DS Wetal. Evidence of dark oxygen production at the abyssal seafloor. Nat. Geosci. (2024).

[0279] Zhu,Y.,Peng,D.,Zhang,E.et al.Superconductivity in pressurizedtrilayer La4Ni3O10-δ single crystals.Nature 631,531–536(2024).

[0280] Shen,W.,Peng,J.,Ma,R.et al.Sunlight-powered sustained flight of anultralight micro aerial vehicle.Nature 631,537–543(2024).

[0281] Poggiali,V.,Brighi,G.,Hayes,AGet al.Surface properties of the seas of Titan as revealed by Cassini mission bistatic radar experiments.Nat Commun15;

[0282] Subtitles: None

[0283] Video tags: Institute of Everything, knowledge, science popularization, science, life, nature, universe, deep sea, astronomy, popular science, dark oxygen, photosynthesis, Institute of Everything·Summer Exploration Program}

[0284] Example 2:

[0285] Video Type: Entertainment

[0286] Video link: https: / / www.dsafsef.com / video

[0287] Basic Information:

[0288] {Author: LexBurner

[0289] Date: 2024-07-28 16:58:04

[0290] Duration: 09:26

[0291] Title: Comment on the popular variety show "Singer 2024"

[0292] Video Description: None

[0293] Subtitles: None

[0294] Video tags: entertainment, entertainment talk, singer 1, singer 2, singer 3, singer 2024, singer 4}

[0295] Example 3:

[0296] Video type: Other

[0297] Video link: https: / / www.dsafsef.com / video

[0298] Basic Information:

[0299] {Author: Jiang Xunqian

[0300] Date: 2024-07-25 21:16:44

[0301] Duration: 03:55

[0302] Title: A small green plum on the fingertips, skillfully turned into a jade chrysanthemum orchid

[0303] Video Description: None

[0304] Subtitles: None

[0305] Video tags: bilibili Food World Cup, food, food production, food circle, food production, plum, Bai nationality, carved plum}

[0306] In some embodiments of the present application, the above-mentioned video comprehension prompt words are used to instruct the video comprehension model to determine at least one video content comprehension information based on the video description information and the video file.

[0307] Exemplarily, the electronic device encapsulates the video description information into instruction words (prompt), namely the above-mentioned video understanding prompt words, and then inputs them into the video understanding model, so that the video understanding model can perform corresponding operations according to the video understanding prompt words.

[0308] Exemplarily, the video comprehension prompts may include but are not limited to: video classification prompts, one-key summary prompts, mind map prompts, timeline prompts, etc.

[0309] Exemplarily, the above-mentioned video classification prompt words are used to instruct the video understanding model to classify the video type based on the video file.

[0310] Exemplarily, the video classification prompt words are:

[0311] "You are now a video classification expert. According to the title, tags and content of the current video, the video is classified into the following categories: 1. Education 2. Entertainment 3. Others, etc.

[0312] The current video related information is as follows:

[0313] Title: {title}

[0314]

Label

[0315]

Content

[0316]

Category

[0317] Output the predicted video type in JSON format. ”

[0318] Specifically, in combination with Example 1, the video classification prompt words are as follows:

[0319] "You are now a video classification expert. According to the title, tags and content of the current video, the video is classified into the following categories: 1. Education 2. Entertainment 3. Others, etc.

[0320] The current video related information is as follows:

[0321]

Title

[0322]

Introduction

[0323] Zhu,Y.,Peng,D.,Zhang,E.et al.Superconductivity in pressurizedtrilayer La4Ni3O10-δ single crystals.Nature 631,531–536(2024).

[0324] Shen,W.,Peng,J.,Ma,R.et al.Sunlight-powered sustained flight of anultralight micro aerial vehicle.Nature 631,537–543(2024).

[0325] Poggiali,V.,Brighi,G.,Hayes,AGet al.Surface properties of the seas of Titan as revealed by Cassini mission bistatic radar experiments.Nat Commun15}

[0326]

Tags

[0327] Output the predicted video type in JSON format. ”

[0328] Specifically, in combination with Example 2, the video classification prompt words are as follows:

[0329] "You are now a video classification expert. According to the title, tags and content of the current video, the video is classified into the following categories: 1. Education 2. Entertainment 3. Others, etc.

[0330] The current video related information is as follows:

[0331]

Title

[0332]

Introduction

[0333]

Tags

[0334] Output the predicted video type in JSON format. ”

[0335] Specifically, in combination with Example 3, the video classification prompt words are as follows:

[0336] "You are now a video classification expert. According to the title, tags and content of the current video, the video is classified into the following categories: 1. Education 2. Entertainment 3. Others, etc.

[0337] The current video related information is as follows:

[0338]

Title

[0339]

Introduction

[0340] [Tags]: {Food World Cup, food, food production, food circle, food production, plum, Bai nationality, carved plum}

[0341] Output the predicted video type in JSON format. ”

[0342] In this way, by simultaneously inputting the video type of the video into the video understanding model, the video understanding model can quickly and accurately determine the key video information of the video according to the video type, thereby obtaining accurate video content information.

[0343] It should be noted that the one-key summary prompt words, mind map prompt words, and timeline prompt words can refer to the descriptions in the above steps A1 to A12, and will not be repeated here.

[0344] In some embodiments of the present application, the above-mentioned video understanding model is a multimodal model, that is, the video understanding model is composed of multiple models with different functions.

[0345] Exemplarily, the above-mentioned video understanding model may include but is not limited to: an image recognition model, a text recognition model, a video classification model, a semantic analysis model, etc.

[0346] In some embodiments of the present application, different video content information can be obtained by inputting videos of different video types into a video understanding model.

[0347] In this way, electronic devices can directly use large models to extract content information from videos, and then analyze and understand the video content to obtain different video content understanding information, so that users no longer need to use other applications or query related content, thereby improving the diversity of video-related content recommendations.

[0348] Optionally, in some embodiments of the present application, before the above step 202 "the electronic device displays at least one video content understanding information matching the video type of the video", the video information display control method provided by the embodiment of the present application further includes step 401:

[0349] Step 401: The electronic device receives a first input to a video playback interface.

[0350] Furthermore, the above step 202 specifically includes step 202c:

[0351] Step 202c: In response to the first input, the electronic device displays at least one item of video content comprehension information matching the video type of the video.

[0352] In some embodiments of the present application, the first input is used to trigger a video understanding model to analyze and understand the video.

[0353] Exemplarily, after receiving the first input from the user, the electronic device analyzes and understands the video based on the video understanding model in response to the first input to obtain video content understanding information matching the video type.

[0354] In some embodiments of the present application, the first input includes input to a video element in a video, or input to a video playback interface.

[0355] Exemplarily, the above video element may be the entire video or a partial area in the video.

[0356] Exemplarily, the partial area in the above video may be a person in the video, may be a subtitle in the video, or may be a text in the video.

[0357] In a possible embodiment, the first input may include an input of selecting a region on the video playback interface.

[0358] In some embodiments of the present application, the circle selection input is an input whose input track is a closed circle made by the user on the video playback interface or other display screen area.

[0359] Exemplarily, the closed circle may be a perfect circle, an ellipse, or an irregular circle, and the present application embodiment does not limit this.

[0360] In some embodiments of the present application, when the first input is an area selection input, the area selection input includes at least one of the following: a fingertip touch input and a knuckle touch input.

[0361] Exemplarily, in the case where the area selection input includes a knuckle touch input, the knuckle touch input includes an input of tapping the screen with a knuckle and selecting an area along a reference direction from the tapping position.

[0362] In some embodiments of the present application, when the first input is an area circle selection input, the first input is used to select a video element in the video, so that the electronic device can perform operations such as search or translation according to the video element selected by the user.

[0363] In one example, Fig. 6A As shown, the first input may be an input for circling a video element in a partial area 61 in the video playback interface.

[0364] In another possible embodiment, the first input may also include a single-finger long press input, a multi-finger long press input, a single-finger sliding input or a multi-finger sliding input on the video playback interface.

[0365] Exemplarily, the above-mentioned single-finger sliding input or multi-finger sliding input can be a sliding input in any direction, such as sliding up, sliding down, sliding to the left or sliding to the right, etc., which is not limited in the embodiments of the present application.

[0366] In some embodiments of the present application, when the above-mentioned first input is a single-finger long press input, a multi-finger long press input, a single-finger sliding input or a multi-finger sliding input on the video playback interface, the above-mentioned first input is used to trigger the electronic device to analyze and understand the video content.

[0367] In another example, Figure 6B and Figure 6CAs shown, the first input may be a single-finger or double-finger press input at any position on the video playback interface.

[0368] In this way, the electronic device can accurately determine the video content information that the user wants to query in the video by receiving the user's first input.

[0369] Optionally, in some embodiments of the present application, when there are at least two pieces of video content understanding information matching the video type of the video, the above step 202 specifically includes: step 202d and step 202e:

[0370] Step 202d: The electronic device determines the display order according to the historical click frequency of the content tag corresponding to each video content comprehension information.

[0371] Step 202e: The electronic device displays at least two items of video content understanding information in a display order.

[0372] In some embodiments of the present application, the electronic device obtains and counts the historical click frequency of each content tag clicked by the user.

[0373] Exemplarily, when a user clicks on video content understanding information corresponding to a certain content tag, the electronic device records the frequency of the content tag as 1.

[0374] For example, when a user clicks on a timeline content label, clicking on the corresponding content will jump to the corresponding video frame. At this time, the frequency of the corresponding timeline content label is +1.

[0375] In some embodiments of the present application, the electronic device calculates the weights of different content tags using Formula 1 based on the historical click frequency of each content tag.

[0376] In some embodiments of the present application, a larger weight value of a content tag indicates a higher historical click frequency of the content tag, which means that the user more often views the video content understanding information corresponding to the content tag.

[0377] For example, the above formula 1 is:

[0378]

[0379] Where S is the set of functions in this category, i.e., {education, entertainment, other}, i is the function under this category, such as {keywords, one-key summary, mind map, time axis, video axis} under education, etc., c i is the frequency of clicking the function. i This is the frequency of the function.

[0380] For example, in conjunction with scenario 1, refer to Figure 2A,In the educational video scenario, the display order of the above content tags can be keywords, one-key summary, translation, timeline, and video subtitles.

[0381] In this way, by combining the user's historical operations to determine the display order of content tags, the video content understanding information corresponding to the content tags first displayed to the user is usually frequently viewed by the user, thereby improving the convenience of the user in viewing video-related recommended content.

[0382] Each of the above-mentioned method embodiments, or various possible implementation methods in each method embodiment can be executed separately, or any two or more of them can be executed in combination with each other. The specific implementation method can be determined according to actual usage requirements, and the embodiments of the present application do not limit this.

[0383] The video information display control method provided in the embodiment of the present application can be executed by an electronic device or a video information display control device. In the embodiment of the present application, the video information display control device executing the video information display control method is taken as an example to illustrate the video information display control device provided in the embodiment of the present application.

[0384] Figure 7 A possible structural diagram of a video information display control device involved in an embodiment of the present application is shown. Figure 7 As shown, the video information display control device 700 may include: a display module 701.

[0385] The display module 701 is used to display the program interface; the display module 701 is also used to display at least one item of video content understanding information matching the video type of the video when the program interface is a video playback interface of the video.

[0386] Optionally, in some embodiments of the present application, combined with Figure 7 ,like Figure 8 As shown, the above-mentioned device 700 also includes: a receiving module 702, which is used to receive a first input to the video playback interface before the above-mentioned display module 701 displays at least one video content understanding information that matches the video type of the video; the above-mentioned display module 701 is specifically used to display at least one video content understanding information that matches the video type of the video in response to the first input received by the above-mentioned receiving module 702.

[0387] Optionally, in some embodiments of the present application, the receiving module 702 is specifically configured to:

[0388] Receive area selection input on the video playback interface;

[0389] Alternatively, a single-finger long press input, a multi-finger long press input, a single-finger sliding input, or a multi-finger sliding input on the video playback interface is received.

[0390] Optionally, in some embodiments of the present application, when the first input is an area selection input, the area selection input includes at least one of the following: a fingertip touch input, a knuckle touch input;

[0391] Optionally, in some embodiments of the present application, when the area selection input includes a knuckle touch input, the knuckle touch input includes an input of tapping the screen with a knuckle and selecting an area along a reference direction from the tapping position.

[0392] Optionally, in some embodiments of the present application, the video playback interface is displayed on a first display screen of the electronic device; the display module 701 is specifically used to display at least one item of video content understanding information matching the video type of the video on a second display screen of the electronic device.

[0393] Optionally, in some embodiments of the present application, the above-mentioned video understanding content information corresponds to a content tag; the above-mentioned content tag includes at least one of the following: a keyword tag, a translation tag, a character introduction tag, a rule introduction tag, a score tag, a one-click summary tag, a mind map tag, a video subtitle tag, a timeline tag, a song recognition tag, a barrage hotspot tag, and an identification of the same style tag.

[0394] Optionally, in some embodiments of the present application, the display module 701 is specifically used for:

[0395] In the case where the content tag is a keyword tag, displaying explanation information of at least one keyword in the video;

[0396] In the case where the content tag is a translation tag, displaying translation information of at least one keyword in the video;

[0397] In the case where the content tag is a character introduction tag, displaying at least one character introduction information of a video character in the video;

[0398] When the content tag is a one-key summary tag, summary information of the video content of the video is displayed;

[0399] When the content tag is a mind map tag, a video mind map is displayed, and the video mind map is generated based on the video title and video content of the video;

[0400] In the case where the content tag is a video subtitle tag, displaying the video subtitles of at least one video frame of the video and the video timestamp corresponding to each video subtitle;

[0401] In the case where the content tag is a timeline tag, a video timestamp of at least one key frame of the video and frame content description information of each key frame are displayed;

[0402] In the case where the content tag is a song recognition tag, the song recognition result information of the song in the video is displayed;

[0403] When the content tag is a bullet screen hot topic tag, at least one bullet screen hot topic information is displayed;

[0404] In the case where the content tag is a tag for identifying the same style, the identification result information of the clothes that are the same style as the clothes of the video character in the video is displayed;

[0405] In the case where the content tag is a rule introduction tag, the rule introduction information of the sports event in the video is displayed;

[0406] When the content tag is a score tag, the scores of the sports events in the video are displayed.

[0407] Optionally, in some embodiments of the present application, the above-mentioned video type includes at least one of the following: education, entertainment, sports, and comprehensive.

[0408] Optionally, in some embodiments of the present application, the display module 701 is specifically used to display at least one function card;

[0409] Each function card includes a content tag and video content understanding information corresponding to the content tag, and one content tag corresponds to one item of video understanding content information.

[0410] Optionally, in some embodiments of the present application, when the video type of the video is entertainment, the display module 701 is specifically used to display at least one of the following:

[0411] Display the song recognition result information of the song in the song recognition tag and the video;

[0412] Display the identification result information of the same label and the same clothes as the clothes of the video characters in the video;

[0413] Display the time axis label and the video timestamp of at least one key frame of the video and the video subtitle information corresponding to each video timestamp;

[0414] Display the video subtitle tag and the video subtitle of at least one video frame of the video and the video timestamp corresponding to each video subtitle;

[0415] Display summary information of the video content of the one-click summary tag and video;

[0416] Displaying a character introduction tag and at least one character introduction information of a video character in the video;

[0417] Displays hot topic labels and at least one hot topic information.

[0418] Optionally, in some embodiments of the present application, when the video type of the video is education, the display module 701 is specifically used to display at least one of the following:

[0419] Display keyword tags and explanation information of at least one keyword in the video;

[0420] Display mind map tags and video mind maps. Video mind maps are generated based on the video title and video content.

[0421] Display the time axis label and the video timestamp of at least one key frame of the video and the video subtitle information corresponding to each video timestamp;

[0422] Display the video subtitle tag and the video subtitle of at least one video frame of the video and the video timestamp corresponding to each video subtitle;

[0423] Display translation information of at least one keyword in the translation tag and the video;

[0424] Display summary information of the video content of the one-click summary tag and video;

[0425] Displaying a character introduction tag and at least one character introduction information of a video character in the video;

[0426] Displays hot topic labels and at least one hot topic information.

[0427] Optionally, in some embodiments of the present application, when the video type of the video is sports, the display module 701 is specifically used to display at least one of the following:

[0428] Display rules information for the sports in the rules description tab and video;

[0429] Display score tags and scores of sports in the video;

[0430] Display the time axis label and the video timestamp of at least one key frame of the video and the video subtitle information corresponding to each video timestamp;

[0431] Display the video subtitle tag and the video subtitle of at least one video frame of the video and the video timestamp corresponding to each video subtitle;

[0432] Display translation information of at least one keyword in the translation tag and the video;

[0433] Display summary information of the video content of the one-click summary tag and video;

[0434] Displaying a character introduction tag and at least one character introduction information of a video character in the video;

[0435] Displays hot topic labels and at least one hot topic information.

[0436] Optionally, in some embodiments of the present application, combined with Figure 7 ,like Fig. 9 As shown, the above-mentioned device 700 also includes: a processing module 703, which is used to input the video file and video understanding prompt words of the video into the video understanding model before the above-mentioned display module 701 displays at least one video content understanding information matching the video type of the video, and determine at least one video content understanding information in the video through the video understanding model, and output at least one video content understanding information; wherein the video understanding prompt words include video description information, and the video description information includes at least one of the following: the video type of the video, the timeline of the video, the author of the video, the tag of the video, and the title of the video; the video understanding prompt words are used to instruct the video understanding model to determine at least one video content understanding information based on the video description information and the video file.

[0437] Optionally, in some embodiments of the present application, when there are at least two items of video content understanding information matching the video type of the video, the above-mentioned display module 701 is specifically used to: determine the display order according to the historical click frequency of the content tag corresponding to each item of video content understanding information; and display at least two items of video content understanding information according to the display order.

[0438] In the video information display control device provided in the embodiment of the present application, a program interface is displayed; when the program interface is a video playback interface of a video, at least one video content understanding information matching the video type of the video is displayed. In this solution, when the video playback interface of the video is displayed, the electronic device analyzes and understands the video to obtain video content understanding information matching the video type, and displays the obtained video content understanding information to the user for viewing, so that the user can directly view the recommended content related to the video while watching the video, thereby improving the efficiency of querying video-related content.

[0439] The video information display control device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or it can be other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a vehicle-mounted electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0440] The video information display control device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0441] The video information display control device provided in the embodiment of the present application can implement each process implemented in the embodiment of the video information display control method, and will not be described again here to avoid repetition.

[0442] Alternatively, if Fig.10 As shown, an embodiment of the present application also provides an electronic device 800, including a processor 801 and a memory 802, wherein the memory 802 stores programs or instructions that can be executed on the processor 801, and when the program or instructions are executed by the processor 801, the various steps of the above-mentioned video information display control method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, they are not described here.

[0443] It should be noted that the electronic devices in the embodiments of the present application include the mobile electronic devices and non-mobile electronic devices mentioned above.

[0444] Fig.11 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0445] The electronic device 100 includes but is not limited to components such as a radio frequency unit 101, a network module 102, an audio output unit 103, an input unit 104, a sensor 105, a display unit 106, a user input unit 107, an interface unit 108, a memory 109, and a processor 110.

[0446] Those skilled in the art will appreciate that the electronic device 100 may also include a power source (such as a battery) for supplying power to various components, and the power source may be logically connected to the processor 110 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Fig.11 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0447] The display unit 106 is used to display the program interface; the display unit 106 is also used to display at least one item of video content understanding information matching the video type of the video when the program interface is a video playback interface of the video.

[0448] Optionally, in some embodiments of the present application, the user input unit 107 is used to receive a first input to the video playback interface before the display unit 106 displays at least one item of video content understanding information that matches the video type of the video; the display unit 106 is specifically used to display at least one item of video content understanding information that matches the video type of the video in response to the first input received by the user input unit 107.

[0449] Optionally, in some embodiments of the present application, the user input unit 107 is specifically used for:

[0450] Receive area selection input on the video playback interface;

[0451] Alternatively, a single-finger long press input, a multi-finger long press input, a single-finger sliding input, or a multi-finger sliding input on the video playback interface is received.

[0452] Optionally, in some embodiments of the present application, when the first input is an area selection input, the area selection input includes at least one of the following: a fingertip touch input, a knuckle touch input;

[0453] Optionally, in some embodiments of the present application, when the area selection input includes a knuckle touch input, the knuckle touch input includes an input of tapping the screen with a knuckle and selecting an area along a reference direction from the tapping position.

[0454] Optionally, in some embodiments of the present application, the video playback interface is displayed on a first display screen of the electronic device; the display unit 106 is specifically used to display at least one item of video content understanding information matching the video type of the video on a second display screen of the electronic device.

[0455] Optionally, in some embodiments of the present application, the above-mentioned video understanding content information corresponds to a content tag; the above-mentioned content tag includes at least one of the following: a keyword tag, a translation tag, a character introduction tag, a rule introduction tag, a score tag, a one-click summary tag, a mind map tag, a video subtitle tag, a timeline tag, a song recognition tag, a barrage hotspot tag, and an identification of the same style tag.

[0456] Optionally, in some embodiments of the present application, the display unit 106 is specifically used for:

[0457] In the case where the content tag is a keyword tag, displaying explanation information of at least one keyword in the video;

[0458] In the case where the content tag is a translation tag, displaying translation information of at least one keyword in the video;

[0459] In the case where the content tag is a character introduction tag, displaying at least one character introduction information of a video character in the video;

[0460] When the content tag is a one-key summary tag, summary information of the video content of the video is displayed;

[0461] When the content tag is a mind map tag, a video mind map is displayed, and the video mind map is generated based on the video title and video content of the video;

[0462] In the case where the content tag is a video subtitle tag, displaying the video subtitles of at least one video frame of the video and the video timestamp corresponding to each video subtitle;

[0463] In the case where the content tag is a timeline tag, a video timestamp of at least one key frame of the video and frame content description information of each key frame are displayed;

[0464] In the case where the content tag is a song recognition tag, the song recognition result information of the song in the video is displayed;

[0465] When the content tag is a bullet screen hot topic tag, at least one bullet screen hot topic information is displayed;

[0466] In the case where the content tag is a tag for identifying the same style, the identification result information of the clothes that are the same style as the clothes of the video character in the video is displayed;

[0467] In the case where the content tag is a rule introduction tag, the rule introduction information of the sports event in the video is displayed;

[0468] When the content tag is a score tag, the scores of the sports events in the video are displayed.

[0469] Optionally, in some embodiments of the present application, the above-mentioned video type includes at least one of the following: education, entertainment, sports, and comprehensive.

[0470] Optionally, in some embodiments of the present application, the display unit 106 is specifically used to display at least one function card;

[0471] Each function card includes a content tag and video content understanding information corresponding to the content tag, and one content tag corresponds to one item of video understanding content information.

[0472] Optionally, in some embodiments of the present application, when the video type of the video is entertainment, the display unit 106 is specifically configured to display at least one of the following:

[0473] Display the song recognition result information of the song in the song recognition tag and the video;

[0474] Display the identification result information of the same label and the same clothes as the clothes of the video characters in the video;

[0475] Display the time axis label and the video timestamp of at least one key frame of the video and the video subtitle information corresponding to each video timestamp;

[0476] Display the video subtitle tag and the video subtitle of at least one video frame of the video and the video timestamp corresponding to each video subtitle;

[0477] Display summary information of the video content of the one-click summary tag and video;

[0478] Displaying a character introduction tag and at least one character introduction information of a video character in the video;

[0479] Displays hot topic labels and at least one hot topic information.

[0480] Optionally, in some embodiments of the present application, when the video type of the video is education, the display unit 106 is specifically configured to display at least one of the following:

[0481] Display keyword tags and explanation information of at least one keyword in the video;

[0482] Display mind map tags and video mind maps. Video mind maps are generated based on the video title and video content.

[0483] Display the time axis label and the video timestamp of at least one key frame of the video and the video subtitle information corresponding to each video timestamp;

[0484] Display the video subtitle tag and the video subtitle of at least one video frame of the video and the video timestamp corresponding to each video subtitle;

[0485] Display translation information of at least one keyword in the translation tag and the video;

[0486] Display summary information of the video content of the one-click summary tag and video;

[0487] Displaying a character introduction tag and at least one character introduction information of a video character in the video;

[0488] Displays hot topic labels and at least one hot topic information.

[0489] Optionally, in some embodiments of the present application, when the video type of the video is sports, the display unit 106 is specifically configured to display at least one of the following:

[0490] Display rules information for the sports in the rules description tab and video;

[0491] Display score tags and scores of sports in the video;

[0492] Display the time axis label and the video timestamp of at least one key frame of the video and the video subtitle information corresponding to each video timestamp;

[0493] Display the video subtitle tag and the video subtitle of at least one video frame of the video and the video timestamp corresponding to each video subtitle;

[0494] Display translation information of at least one keyword in the translation tag and the video;

[0495] Display summary information of the video content of the one-click summary tag and video;

[0496] Displaying a character introduction tag and at least one character introduction information of a video character in the video;

[0497] Displays hot topic labels and at least one hot topic information.

[0498] Optionally, in some embodiments of the present application, the processor 110 is used to input the video file and video comprehension prompt words of the video into a video comprehension model before the display unit 106 displays at least one item of video content comprehension information matching the video type of the video, determine at least one item of video content comprehension information in the video through the video comprehension model, and output at least one item of video content comprehension information; wherein the video comprehension prompt words include video description information, and the video description information includes at least one item of the following: the video type of the video, the timeline of the video, the author of the video, the tags of the video, and the title of the video; the video comprehension prompt words are used to instruct the video comprehension model to determine at least one item of video content comprehension information based on the video description information and the video file.

[0499] Optionally, in some embodiments of the present application, when there are at least two items of video content understanding information matching the video type of the video, the above-mentioned display unit 106 is specifically used to: determine the display order according to the historical click frequency of the content tag corresponding to each item of video content understanding information; and display at least two items of video content understanding information according to the display order.

[0500] In the electronic device provided in the embodiment of the present application, a program interface is displayed; when the program interface is a video playback interface of a video, at least one video content understanding information matching the video type of the video is displayed. In this solution, when the video playback interface of the video is displayed, the electronic device analyzes and understands the video to obtain video content understanding information matching the video type, and displays the obtained video content understanding information to the user for viewing, so that the user can directly view the recommended content related to the video while watching the video, thereby improving the efficiency of querying video-related content.

[0501] It should be understood that in the embodiment of the present application, the input unit 104 may include a graphics processor (Graphics Processing Unit, GPU) 1041 and a microphone 1042, and the graphics processor 1041 processes the image data of a static picture or video obtained by an image capture device (such as a camera) in a video capture mode or an image capture mode. The display unit 106 may include a display panel 1061, and the display panel 1061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 107 includes a touch panel 1071 and at least one of other input devices 1072. The touch panel 1071 is also called a touch screen. The touch panel 1071 may include two parts: a touch detection device and a touch controller. Other input devices 1072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0502] The memory 109 can be used to store software programs and various data. The memory 109 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 109 may include a volatile memory or a non-volatile memory, or the memory 109 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 109 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0503] The processor 110 may include one or more processing units; optionally, the processor 110 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 110.

[0504] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned video information display control method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0505] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0506] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned video information display control method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0507] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0508] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned video information display control method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0509] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0510] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0511] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A video information display control method, characterized in that: The method comprises: Display the program interface; In a case where the program interface is a video playback interface of a video, at least one item of video content understanding information matching the video type of the video is displayed.

2. The method according to claim 1, characterized in that Before displaying at least one item of video content understanding information matching the video type of the video, the method further includes: Receiving a first input to the video playback interface; The displaying of at least one item of video content understanding information matching the video type of the video includes: In response to the first input, at least one item of video content comprehension information matching a video type of the video is displayed.

3. The method according to claim 2, characterized in that The receiving a first input to the video playback interface includes: Receive area selection input on the video playback interface; Alternatively, a single-finger long press input, a multi-finger long press input, a single-finger sliding input, or a multi-finger sliding input on the video playback interface is received.

4. The method according to claim 3, characterized in that: In the case where the first input is an area selection input, the area selection input includes at least one of the following: a fingertip touch input and a knuckle touch input.

5. The method according to claim 4, characterized in that In the case where the area selection input includes the knuckle touch input, the knuckle touch input includes an input of tapping the screen with a knuckle and selecting an area along a reference direction from the tapping position.

6. The method according to claim 1, characterized in that The video playback interface is displayed on a first display screen of the electronic device; The displaying of at least one item of video content understanding information matching the video type of the video includes: At least one item of video content understanding information matching the video type of the video is displayed on a second display screen of the electronic device.

7. The method according to claim 1, characterized in that One piece of video understanding content information corresponds to one content label; The content tags include at least one of the following: keyword tags, translation tags, character introduction tags, rule introduction tags, score tags, one-click summary tags, mind map tags, video subtitle tags, timeline tags, song recognition tags, barrage hot spots tags, and identification of the same tags.

8. The method according to claim 7, characterized in that The displaying of the at least one video content understanding information comprises: In the case where the content tag is a keyword tag, displaying explanation information of at least one keyword in the video; In the case where the content tag is a translation tag, displaying translation information of at least one keyword in the video; In the case where the content tag is a character introduction tag, displaying at least one character introduction information of a video character in the video; In the case where the content tag is a one-key summary tag, displaying summary information of the video content of the video; In the case where the content tag is a mind map tag, displaying a video mind map, wherein the video mind map is generated based on the video title and video content of the video; In the case where the content tag is a video subtitle tag, displaying video subtitles of at least one video frame of the video and a video timestamp corresponding to each video subtitle; In the case where the content tag is a timeline tag, displaying a video timestamp of at least one key frame of the video and frame content description information of each key frame; In the case where the content tag is a song recognition tag, displaying song recognition result information of the songs in the video; In the case where the content tag is a bullet screen hot topic tag, displaying at least one bullet screen hot topic information; In the case where the content tag is a tag for identifying the same style, displaying the identification result information of the clothes of the same style as the clothes of the video character in the video; In the case where the content tag is a rule introduction tag, displaying rule introduction information of the sports event in the video; When the content tag is a score tag, the scores of the sports events in the video are displayed.

9. The method according to claim 1, characterized in that: The video type includes at least one of the following: education, entertainment, sports, and comprehensive.

10. The method according to claim 1, characterized in that Displaying the at least one item of video content understanding information includes: Display at least one function card; Each function card includes a content tag and video content understanding information corresponding to the content tag, and one content tag corresponds to one item of video understanding content information.

11. The method according to claim 1, characterized in that In the case where the video type of the video is entertainment, the displaying of the at least one item of video content comprehension information includes at least one of the following: Displaying song recognition tags and song recognition result information of the songs in the video; Displaying identification result information of the same label and the same clothes as the clothes of the video character in the video; Display a time axis label and a video timestamp of at least one key frame of the video and video subtitle information corresponding to each video timestamp; Displaying a video subtitle tag and the video subtitles of at least one video frame of the video and a video timestamp corresponding to each video subtitle; Display a one-click summary tag and summary information of the video content of the video; Displaying a character introduction tag and at least one character introduction information of a video character in the video; Displays hot topic labels and at least one hot topic information.

12. The method according to claim 1, characterized in that In the case where the video type of the video is education, the displaying of the at least one video content comprehension information includes at least one of the following: Displaying keyword tags and explanation information of at least one keyword in the video; Displaying a mind map label and a video mind map, wherein the video mind map is generated based on the video title and video content of the video; Display a time axis label and a video timestamp of at least one key frame of the video and video subtitle information corresponding to each video timestamp; Displaying a video subtitle tag and the video subtitles of at least one video frame of the video and a video timestamp corresponding to each video subtitle; Displaying a translation tag and translation information of at least one keyword in the video; Display a one-click summary tag and summary information of the video content of the video; Displaying a character introduction tag and at least one character introduction information of a video character in the video; Displays hot topic labels and at least one hot topic information.

13. The method according to claim 1, characterized in that In a case where the video type of the video is sports, the displaying of the at least one item of video content comprehension information includes at least one of the following: Displaying a rules introduction label and rules introduction information of the sports event in the video; Displaying a score tag and the score of the sport in the video; Display a time axis label and a video timestamp of at least one key frame of the video and video subtitle information corresponding to each video timestamp; Displaying a video subtitle tag and the video subtitles of at least one video frame of the video and a video timestamp corresponding to each video subtitle; Displaying a translation tag and translation information of at least one keyword in the video; Display a one-click summary tag and summary information of the video content of the video; Displaying a character introduction tag and at least one character introduction information of a video character in the video; Displays hot topic labels and at least one hot topic information.

14. The method according to claim 1, characterized in that Before displaying at least one item of video content understanding information matching the video type of the video, the method further includes: Inputting the video file and the video comprehension prompt words of the video into a video comprehension model, determining at least one piece of video content comprehension information in the video through the video comprehension model, and outputting the at least one piece of video content comprehension information; Among them, the video comprehension prompt words include video description information, and the video description information includes at least one of the following: the video type of the video, the timeline of the video, the author of the video, the tags of the video, and the title of the video; the video comprehension prompt words are used to instruct the video comprehension model to determine the at least one video content comprehension information based on the video description information and the video file.

15. The method according to claim 4, characterized in that In a case where there are at least two pieces of video content understanding information matching the video type of the video, displaying at least one piece of video content understanding information matching the video type of the video includes: Determine the display order according to the historical click frequency of the content tag corresponding to each video content understanding information; At least two items of video content understanding information are displayed in the display order.

16. A video information display control device, characterized in that: The device comprises: Display module, used for displaying the program interface; The display module is further configured to display at least one item of video content understanding information matching the video type of the video when the program interface is a video playback interface of the video.

17. The device according to claim 16, characterized in that The device also includes: a receiving module, configured to receive a first input to the video playback interface before the display module displays at least one item of video content understanding information matching the video type of the video; The display module is specifically configured to display at least one item of video content understanding information matching the video type of the video in response to the first input received by the receiving module.

18. The device according to claim 17, characterized in that The receiving module is specifically used for: Receive area selection input on the video playback interface; Alternatively, a single-finger long press input, a multi-finger long press input, a single-finger sliding input, or a multi-finger sliding input on the video playback interface is received.

19. The device according to claim 18, characterized in that In the case where the first input is an area selection input, the area selection input includes at least one of the following: a fingertip touch input and a knuckle touch input.

20. The device according to claim 19, characterized in that In the case where the area selection input includes the knuckle touch input, the knuckle touch input includes an input of tapping the screen with a knuckle and selecting an area along a reference direction from the tapping position.

21. The device according to claim 16, characterized in that The video playback interface is displayed on a first display screen of the electronic device; The display module is specifically used to display at least one item of video content understanding information matching the video type of the video on the second display screen of the electronic device.

22. The device according to claim 16, characterized in that One piece of video understanding content information corresponds to one content label; The content tags include at least one of the following: keyword tags, translation tags, character introduction tags, rule introduction tags, score tags, one-click summary tags, mind map tags, video subtitle tags, timeline tags, song recognition tags, barrage hot spots tags, and identification of the same tags.

23. The method according to claim 22, characterized in that The display module is specifically used for: In the case where the content tag is a keyword tag, displaying explanation information of at least one keyword in the video; In the case where the content tag is a translation tag, displaying translation information of at least one keyword in the video; In the case where the content tag is a character introduction tag, displaying at least one character introduction information of a video character in the video; In the case where the content tag is a one-key summary tag, displaying summary information of the video content of the video; In the case where the content tag is a mind map tag, displaying a video mind map, wherein the video mind map is generated based on the video title and video content of the video; In the case where the content tag is a video subtitle tag, displaying video subtitles of at least one video frame of the video and a video timestamp corresponding to each video subtitle; In the case where the content tag is a timeline tag, displaying a video timestamp of at least one key frame of the video and frame content description information of each key frame; In the case where the content tag is a song recognition tag, displaying song recognition result information of the songs in the video; In the case where the content tag is a bullet screen hot topic tag, displaying at least one bullet screen hot topic information; In the case where the content tag is a tag for identifying the same style, displaying the identification result information of the clothes of the same style as the clothes of the video character in the video; In the case where the content tag is a rule introduction tag, displaying rule introduction information of the sports event in the video; When the content tag is a score tag, the scores of the sports events in the video are displayed.

24. The device according to claim 16, characterized in that The video type includes at least one of the following: education, entertainment, sports, and comprehensive.

25. The device according to claim 16, characterized in that The display module is specifically used to display at least one function card; Each function card includes a content tag and video content understanding information corresponding to the content tag, and one content tag corresponds to one item of video understanding content information.

26. The device according to claim 16, characterized in that When the video type of the video is entertainment, the display module is specifically used to display at least one of the following: Displaying song recognition tags and song recognition result information of the songs in the video; Displaying identification result information of the same label and the same clothes as the clothes of the video character in the video; Display a time axis label and a video timestamp of at least one key frame of the video and video subtitle information corresponding to each video timestamp; Displaying a video subtitle tag and the video subtitles of at least one video frame of the video and a video timestamp corresponding to each video subtitle; Display a one-click summary tag and summary information of the video content of the video; Displaying a character introduction tag and at least one character introduction information of a video character in the video; Displays hot topic labels and at least one hot topic information.

27. The device according to claim 16, characterized in that When the video type of the video is education, the display module is specifically used to display at least one of the following: Displaying keyword tags and explanation information of at least one keyword in the video; Displaying a mind map label and a video mind map, wherein the video mind map is generated based on the video title and video content of the video; Display a time axis label and a video timestamp of at least one key frame of the video and video subtitle information corresponding to each video timestamp; Displaying a video subtitle tag and the video subtitles of at least one video frame of the video and a video timestamp corresponding to each video subtitle; Displaying a translation tag and translation information of at least one keyword in the video; Display a one-click summary tag and summary information of the video content of the video; Displaying a character introduction tag and at least one character introduction information of a video character in the video; Displays hot topic labels and at least one hot topic information.

28. The device according to claim 16, characterized in that When the video type of the video is sports, the display module is specifically used to display at least one of the following: Displaying a rules introduction label and rules introduction information of the sports event in the video; Displaying a score tag and the score of the sport in the video; Display a time axis label and a video timestamp of at least one key frame of the video and video subtitle information corresponding to each video timestamp; Displaying a video subtitle tag and the video subtitles of at least one video frame of the video and a video timestamp corresponding to each video subtitle; Displaying a translation tag and translation information of at least one keyword in the video; Display a one-click summary tag and summary information of the video content of the video; Displaying a character introduction tag and at least one character introduction information of a video character in the video; Displays hot topic labels and at least one hot topic information.

29. The device according to claim 16, characterized in that The device also includes: a processing module, configured to input the video file and the video comprehension prompt word of the video into a video comprehension model before the display module displays at least one video content comprehension information matching the video type of the video, determine at least one video content comprehension information in the video through the video comprehension model, and output the at least one video content comprehension information; Among them, the video comprehension prompt words include video description information, and the video description information includes at least one of the following: the video type of the video, the timeline of the video, the author of the video, the tags of the video, and the title of the video; the video comprehension prompt words are used to instruct the video comprehension model to determine the at least one video content comprehension information based on the video description information and the video file.

30. The device according to claim 19, characterized in that In a case where there are at least two items of video content understanding information matching the video type of the video, the display module is specifically configured to: Determine the display order according to the historical click frequency of the content tag corresponding to each video content understanding information; At least two items of video content understanding information are displayed in the display order.

31. An electronic device, characterized in that: It includes a processor, a memory, and a program or instruction stored in the memory and executable on the processor, wherein the program or instruction, when executed by the processor, implements the steps of the video information display control method as described in any one of claims 1 to 15.

Images