Video content retrieval method and apparatus, electronic device, and storage medium

Abstract

The application provides a video content retrieval method and device, electronic equipment and storage medium, and relates to the technical field of data processing. The video content retrieval method comprises the following steps: in response to a triggered retrieval behavior, acquiring multi-modal retrieval information of a user; according to the retrieval information, retrieving at least one target content in a target video; based on the target content, generating a positioning pointer corresponding to the current retrieval behavior; and according to the positioning pointer, marking the target content in the target video, thereby solving the problem of inefficient retrieval by searching through a drag-and-drop progress bar, and greatly improving the retrieval efficiency and retrieval interactive experience of the user.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a video content retrieval method, apparatus, electronic device, and storage medium. Background Technology

[0002] Existing technologies typically use a timeline to sequentially play video frames, only recording the most recently viewed video frames. Given the rich and diverse nature of video content, the recent viewing history only records the last frame watched. Users cannot directly retrieve plot segments containing characters, items, or text of interest from this history, and searching for content by dragging and changing the progress bar is extremely time-consuming and laborious. Summary of the Invention

[0003] This application aims to at least partially address one of the technical problems in the related art.

[0004] Therefore, the first objective of this application is to propose a video content retrieval method to achieve accurate searching of plot segments containing characters, objects, and text of interest.

[0005] The second objective of this application is to propose a video content retrieval device.

[0006] The third objective of this application is to propose an electronic device.

[0007] The fourth objective of this application is to provide a computer-readable storage medium.

[0008] The fifth objective of this application is to provide a computer program product.

[0009] To achieve the above objectives, a first aspect of this application proposes a video content retrieval method, comprising:

[0010] In response to a triggered search action, obtain the user's multimodal search information; Based on the search information, at least one target content is obtained by searching the target video. Based on the target content, a location pointer corresponding to the current search behavior is generated; Based on the positioning pointer, the target content is positioned and marked within the target video.

[0011] To achieve the above objectives, a second aspect of this application provides a video content retrieval device, comprising: The acquisition module is used to acquire the user's multimodal search information in response to a triggered search action; The retrieval module is used to retrieve at least one target content from the target video based on the retrieval information. The generation module is used to generate a positioning pointer corresponding to the current search behavior based on the target content; The marking module is used to mark the target content within the target video according to the positioning pointer.

[0012] To achieve the above objectives, a third aspect of this application provides an electronic device, including: a processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method described in the first aspect embodiment.

[0013] To achieve the above objectives, a fourth aspect of this application provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, are used to implement the method described in the first aspect embodiment.

[0014] To achieve the above objectives, a fifth aspect of this application provides a computer program product including a computer program that, when executed by a processor, implements the method described in the first aspect.

[0015] The video content retrieval method, apparatus, electronic device, and storage medium provided in this application, after a user triggers a retrieval behavior, acquire the user's multimodal retrieval information to retrieve target content, supporting multimodal retrieval and improving retrieval flexibility. Based on the retrieval information, the method searches and matches within the target video to obtain the corresponding target content, and generates a positioning pointer based on the target content. The target content is then positioned and marked within the target video according to the positioning pointer that matches the user's retrieval intent. This allows for intuitive determination of the target content's location through the positioning pointer associated with it, solving the problem of inefficient retrieval by dragging a progress bar, and greatly improving the user's retrieval efficiency and interactive experience.

[0016] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0017] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 A flowchart illustrating a video content retrieval method provided in an embodiment of this application; Figure 2 A flowchart illustrating a video content retrieval method provided in an embodiment of this application; Figure 3 A schematic diagram illustrating an image retrieval method provided in an embodiment of this application; Figure 4 A flowchart illustrating a video content retrieval method provided in an embodiment of this application; Figure 5 This is a schematic diagram illustrating the positioning of a positioning pointer according to an embodiment of this application. Figure 6 A schematic diagram of a hotspot marker provided in an embodiment of this application; Figure 7 A schematic diagram of a pointer set provided in an embodiment of this application; Figure 8 This is a schematic diagram of the structure of a video content retrieval device provided in an embodiment of this application. Detailed Implementation

[0018] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0019] The video content retrieval method, apparatus, electronic device, and storage medium of this application are described below with reference to the accompanying drawings.

[0020] Figure 1 This is a flowchart illustrating a video content retrieval method provided in an embodiment of this application. Figure 1 As shown, the method includes the following steps: S101, in response to triggering a search behavior, obtain the user's multimodal search information.

[0021] In some embodiments, a smart search switch can be configured on the video page to perform intelligent retrieval of video content when the user turns on the smart search switch.

[0022] Optionally, the search information can be multimodal, such as text or images. For text, users can enter text in the smart search input box as search information, such as the name of a person or item to be searched. For images, users can upload images to the smart search input box or drag and drop images into the smart search input box as search information.

[0023] S102, based on the search information, at least one target content is obtained by searching the target video.

[0024] Optionally, information statistics can be performed on the target video to obtain information on the people and items appearing in the target video, which can be used as the information set corresponding to the target video. For example, if the target video is a TV series, information on all the characters and items appearing in the TV series can be obtained. The character information may include the character name, the corresponding actor, and the character image, while the item information may include the item name, the item category, and the item image.

[0025] After determining the search information, the search results can be matched in the information set based on the name or image of the person or item in the search information to determine the location of the person or item with the same name or image, which will be used as the target content retrieved from the target video.

[0026] S103, Based on the target content, generate the positioning pointer corresponding to the current search behavior.

[0027] In some embodiments, the image of the target content of the current search action can be compressed, and the compressed image can be used as the shape of the positioning pointer corresponding to the current search action.

[0028] In some embodiments, an image including the name of the search target corresponding to the current search action can be generated, and a compressed image of the image can be used as the shape of the positioning pointer corresponding to the current search action.

[0029] S104, Based on the positioning pointer, mark the target content within the target video.

[0030] Optionally, the time position of the target content can be marked in the progress bar area of ​​the target video according to the positioning pointer, so that users can quickly find the segment corresponding to the target content of interest according to the positioning pointer, thereby improving search efficiency.

[0031] In this embodiment, after a user triggers a search, the system acquires the user's multimodal search information. This is not limited to text-based searches but also offers diverse search modes. The system searches and matches the target video based on the search information to obtain the corresponding target content. A positioning pointer is then generated based on the target content, which better aligns with the user's search intent. The target content is then located and marked within the target video based on the positioning pointer that matches the user's search intent. This solves the inefficiency of searching by dragging a progress bar. Users can quickly locate all target content that matches the search information in a single search, greatly improving search efficiency.

[0032] Based on the above embodiments, Figure 2 This is a flowchart illustrating a video content retrieval method provided in an embodiment of this application. Figure 2 As shown, the method includes the following steps: S201, in response to triggering a search action, obtain the user's multimodal search information.

[0033] In some embodiments, multimodal search information input by the user can be received; the search information input by the user can be of image type and text type.

[0034] Alternatively, the frequency of each target content in the user's historical search behavior can be counted. It can be understood that the more times the target content appears in the historical search behavior, the more times the user has searched for the target content, and the more likely the target content is the user's preferred information. In this embodiment, the user's preferred search information can be determined and displayed based on the frequency of occurrence. The preferred search information can be the search information of the historical search behavior to obtain the target content, that is, the target content can be obtained by searching based on the search information. The search information is displayed as the user's preferred search information.

[0035] In response to the user's selection of any preferred search information, the preferred search information is determined as the user's search information. In other words, the user can directly click on the preferred search information in the display area to conduct a search without having to manually enter the search information, thus improving the flexibility of the search method.

[0036] In some embodiments, the process of obtaining user preference retrieval information based on the frequency of occurrence of target content can be as follows: Based on the frequency of occurrence of each target content, determine the matching degree between the target content and the user. For example, the number of all historical search behaviors of the user can be obtained to get the maximum number of searches. A first difference between the maximum number of searches and the minimum number of searches can be obtained. A second difference between the frequency of occurrence of the target content and the minimum number of searches can be obtained. The ratio of the second difference to the first difference can be calculated to obtain the matching degree between the target content and the user. In this embodiment, the minimum number of searches is 0. Therefore, the matching degree is the ratio of the frequency of occurrence of the target content to the maximum number of searches. The more times the target content appears, the greater the matching degree between the target content and the user.

[0037] Optionally, target content with a matching degree higher than a preset threshold can be selected as interest content. For example, target content with a matching degree greater than 0.5 is selected as interest content. Based on the search information corresponding to the interest content, the user's preferred search information is determined. In this embodiment, the search information corresponding to the interest content that the user is interested in is used as the user's preferred search information.

[0038] Understandably, after determining preferred search information based on previous search history, this preferred search information can be displayed and recommended to users for searching, eliminating the need for users to manually enter search information.

[0039] In some embodiments, a pre-trained large artificial intelligence model can be used to analyze historical search behavior to obtain search information that highly matches user preferences. In other words, the model selects content that the user is interested in and determines the user's preferred search information based on the content of interest to ensure that the preferred search information matches the user's preferences and improves search efficiency.

[0040] In this application embodiment, the implementation method of step S201 can be implemented in any of the various embodiments of this disclosure, and no limitation is made here, nor will it be described in detail.

[0041] S202, Based on the search information, at least one target content is obtained by searching the target video.

[0042] In response to the fact that the retrieved information is text information, the target content corresponding to the text information is determined from the target video based on at least one of the name database and subtitle database related to the target video.

[0043] Optionally, the text type of the text information can be obtained. In this embodiment, the text type can be a subtitle or a name. When the text type is a subtitle, the target content is determined through a subtitle database. When the text type is a name, the target content is determined through a name database.

[0044] Specifically, in response to the text type being subtitle text, the subtitle text is retrieved from the subtitle database to obtain the target content in the target video; optionally, the subtitle text input by the user can be vectorized to obtain a query vector. The subtitle database includes feature vectors corresponding to all subtitle texts in the target video. The query vector is matched with the feature vectors to determine the feature vector closest to the query vector, and the position of the subtitle text corresponding to the feature vector in the target video is determined, thereby obtaining the target content.

[0045] Optionally, in this embodiment, the target content can be set as a 5-second video content, and the position of the subtitle text corresponding to the feature vector in the target video is the starting position of the target content, that is, the 5-second video content starting from this position is the target content.

[0046] In some embodiments, vectors can be constructed based on the subtitle information corresponding to the target video to obtain feature vectors of all subtitle texts, thereby obtaining a subtitle database. In other embodiments, if the subtitle information of the target video is not obtained in advance, the text content in the video frame containing subtitles in the target video can be identified by optical character recognition (OCR) to obtain subtitle information, and vectors can be constructed based on the subtitle information to obtain a subtitle database.

[0047] In response to at least one of the text types being a person's name, an item's name, or a scene's name, a search is performed in the name database for at least one of the person's name, item's name, or scene's name to obtain the target content in the target video. It is understood that the current search information can be vectorized and a vector search can be performed in the name database to obtain the closest feature vector, thereby locating the target content in the target video. For example, if the search information is the person "Zhang San", then the target content is all the content in the target video where the person "Zhang San" appears, and each target content is a 5-second video frame.

[0048] In some embodiments, the name database stores all character names, item names, and scene names in the target video. In this embodiment, triples are constructed based on relevant information of the target video, such as actor information, character information, item information, and scene information. The triples can be: (Male lead 1, Actor 1, Character image 1), (Male lead 2, Actor 2, Character image 2), (Male lead n, Actor n, Character image n), (Item 1, Scene 1, Item image 1), (Item 2, Scene 2, Item image 2), (Item n, Scene n, Item image n), (Scene 1, Category 1, Scene image 1), (Scene 2, Category 2, Scene image 2), (Scene n, Category n, Scene image n), etc. All triples are vectorized to obtain information vectors and stored in the name database.

[0049] Optionally, if the relevant information of the target video does not include characters, objects, and scenes, the content in the target video can be identified using OCR technology. The image is preprocessed to remove noise and interference factors such as tilt. Next, text features (such as strokes and character structure) are extracted. Finally, the extracted features are matched with a predefined character library through classification recognition to determine the text content in the image, identify the names and relationships of characters, and the names of objects and scenes. Similarly, triples are established to obtain a name database.

[0050] For example, in TV dramas, characters' names and identities are typically displayed vertically on screen, or only their names are displayed vertically. Taking two vertical displays as an example, three screens (A, B, and C) each display three characters: screen A shows Zhang San, manager of company AA; screen B shows Li Si, director of company BB; and screen C shows Wang Wu, employee of company CC. OCR content recognition identifies the characters' names and identities from these different screens. Through artificial intelligence analysis, the names and identities are categorized, and a triplet is created for character information (identity + name), actor information, and character image information. At this point, the actor information is empty. Similarly, OCR can be used to identify object and scene information and create triplets.

[0051] Specifically, in response to the retrieval information being image information, the target content corresponding to the image information is retrieved from the target video based on the image information.

[0052] Optionally, the Fast Forward Moving Picture Experts Group (FFmpeg) can be used to extract keyframes from the video at a rate of 1 frame per second. The keyframes can then be compared and analyzed with the images in the search information using an artificial intelligence image matching algorithm. This process identifies the keyframes in the target video that match the content of the images in the search information and obtains the corresponding time points. Each time point is considered the first frame of the target content, meaning that the 5-second video frame starting from each time point is the retrieved target content.

[0053] In some embodiments, when the retrieved information is image information, the user can directly drag the main image from the video frame into the search area for retrieval. Specifically, the user can long-press the person or object they want to search for in the target video. At this time, the keyframe of the current video is processed to complete the image subject recognition. After the recognition is completed, a light blue outline mask will be displayed on the image subject. The closed path is returned through the subject outline points, and then a new draggable view is created. In the view, a mask is created and clipped using the outline path through the image context (Core Graphics Context, CGContext) to restrict the drawing area to within the path. Then, the image content within the subject outline area is drawn through the CGContext.

[0054] For example, such as Figure 3 As shown, after the user enables smart search, a search box and search area are displayed in the lower half of the video. Users can enter the name of the main character, actor, item, or subtitle text in the search box to perform a search, or drag the centered image (person or item) into the search area to perform an image search. In the image, dragging the male lead 1 into the search area results in two keyframe time points, A and B, displayed together in the lower half of the screen. Time points A and B represent the start time of target content A and target content B, respectively.

[0055] S203, Determine the compression icon for the target content.

[0056] Optionally, based on the target content, the user's interest targets can be determined, including at least one person and item; the interest targets are then image-compressed to obtain compressed icons.

[0057] It is understandable that a user's interest target can be their search target. For example, if the target content searched by a user is "Male Lead 1", then the user's interest target is "Male Lead 1". Similarly, if the target content searched by a user is "Item 1", then the user's interest target is "Item 1". The image frame of the interest target in the target video is obtained and compressed to obtain a compressed icon.

[0058] S204, Generate a positioning pointer corresponding to the target content based on the compressed icon.

[0059] Optionally, the compressed icon can be scaled down and used as a positioning pointer for the target content.

[0060] S205, Based on the positioning pointer, mark the target content within the target video.

[0061] In this application embodiment, the implementation method of step S205 can be implemented in any of the various embodiments of this disclosure, and no limitation is made here, nor will it be described in detail.

[0062] In this embodiment, after a user triggers a search, the system acquires the user's multimodal search information, supporting both text and image multimodal search formats. It can also determine search information based on the user's historical search preferences, improving search flexibility and efficiency. Based on the type of search information, the system acquires target content from the target video, further obtaining compressed icons of the target content and determining positioning pointers based on these icons. Positioning marks are then placed within the target video, visually presenting the search location and content, optimizing the user's visual interaction experience, and resolving the inefficiency of searching by dragging a progress bar. Users can quickly locate all target content matching the search information in a single search, greatly improving search efficiency.

[0063] Based on the above embodiments, Figure 4 This is a flowchart illustrating a video content retrieval method provided in an embodiment of this application. Figure 4 As shown, the method includes the following steps: S401, in response to triggering a search action, obtain the user's multimodal search information.

[0064] In this application embodiment, the implementation method of step S401 can be implemented in any of the various embodiments of this disclosure, and no limitation is made here, nor will it be described in detail.

[0065] S402, based on the search information, retrieve at least one target content from the target video.

[0066] In this application embodiment, the implementation method of step S402 can be implemented in any of the various embodiments of this disclosure, and no limitation is made here, nor will it be described in detail.

[0067] S403, Determine the compression icon for the target content.

[0068] In this application embodiment, the implementation method of step S403 can be implemented in any of the various embodiments of this disclosure, and no limitation is made here, nor will it be described in detail.

[0069] S404, Generate a positioning pointer corresponding to the target content based on the compressed icon.

[0070] In this application embodiment, the implementation method of step S404 can be implemented in any of the various embodiments of this disclosure, and no limitation is made here, nor will it be described in detail.

[0071] S405, Determine the start time of the target content.

[0072] Understandably, the initial location of each target content in the target video, based on the search information, is considered the starting time point of the target content.

[0073] S406, in the progress display area of ​​the target video, mark the starting time point of the target content according to the positioning pointer.

[0074] For example, such as Figure 5 As shown, positioning pointers are marked for target content A, B, and C in the progress bar area of ​​the target video. The positioning pointers are the compression icons corresponding to the target content.

[0075] In some embodiments, the marking of target content may further include pointer arrows. By default, the pointer arrows position the video at the starting time point of the first target content. Users can change the viewing time point by switching the pointer arrows at the positions of each positioning pointer, thereby obtaining video content at different time points. Assuming the user is currently viewing... Figure 5 If the target content is B, then the pointer arrow corresponding to the location of pointer B is a solid line, and the pointer arrow stays at the location of target content B, while the pointer arrows at the locations of target content A and C are dashed lines.

[0076] In some embodiments, the number of online user views corresponding to each target content can also be counted; based on the number of online user views, hot target content can be determined from the target content, for example, the target content with the most online user views can be determined as hot target content; hotspot markings can be made at the positioning pointers corresponding to the hot target content, such as... Figure 6As shown, target content C is the content with the most views by online users. A spark hotspot marker is added to the positioning pointer of target content C.

[0077] Optionally, the location pointer corresponding to the current search behavior can also be saved to a pointer set. The pointer set is used to manage the location pointers corresponding to all user search behaviors. Each location pointer is used to locate the target content corresponding to the marker. In response to the user's selection operation of any location pointer in the pointer set, the location pointer corresponding to the selection operation is displayed in the target video to locate the target content corresponding to the location pointer.

[0078] For example, the pointer set includes location pointer 1, location pointer 2, and location pointer 3. Location pointer 1 corresponds to male lead 1, location pointer 2 corresponds to male lead 2, and location pointer 3 corresponds to male lead 3. Users can switch between location pointers in the pointer set to watch video content they are interested in. When a user selects a time point to play, the video playback time is updated to the latest time point; see reference. Figure 7 The pointer set includes positioning pointer 1, positioning pointer 2 and positioning pointer 3, and different positioning pointers correspond to different target video content. By switching positioning pointers, you can view the target video content at different positions in the target video.

[0079] In some embodiments, all location pointers and associated target content of this search behavior can be stored in the database so that they can be used as historical search behavior information for user preference analysis in subsequent searches.

[0080] In this embodiment, after a user triggers a search, the system acquires the user's multimodal search information, supporting both text and image multimodal search formats. It can also determine search information based on the user's historical search preferences, improving search flexibility and efficiency. Based on the type of search information, the system acquires target content from the target video, further obtaining a compressed icon of the target content and determining a positioning pointer based on the compressed icon. A positioning mark is then placed at the start time of the target content within the target video. Users can quickly jump to the corresponding segment based on the positioning mark, simplifying the search operation. Furthermore, users can switch between different target contents by changing the positioning pointer, flexibly switching between different video segments. The operation logic is simple, greatly optimizing the user's visual interaction experience during the search and solving the inefficiency of searching by dragging a progress bar. Users can quickly locate all target content matching the search information in a single search, significantly improving search efficiency.

[0081] To achieve the above embodiments, this application also proposes a video content retrieval device.

[0082] Figure 8This is a schematic diagram of the structure of a video content retrieval device provided in an embodiment of this application. Figure 8 As shown, the video content retrieval device 800 includes: The acquisition module 801 is used to acquire the user's multimodal search information in response to the triggered search behavior; The retrieval module 802 is used to retrieve at least one target content from the target video based on the retrieval information. The generation module 803 is used to generate a positioning pointer corresponding to the current search behavior based on the target content; The marking module 804 is used to mark the target content within the target video based on the positioning pointer.

[0083] Furthermore, in one possible implementation of this application embodiment, the retrieval module 802 is used for: In response to the fact that the retrieved information is text information, the target content corresponding to the text information is determined from the target video based on at least one of the name database and subtitle database related to the target video; In response to the fact that the retrieved information is image information, the system searches the target video based on the image information to obtain the target content corresponding to the image information.

[0084] Furthermore, in one possible implementation of this application embodiment, the retrieval module 802 is also used for: The text type from which you retrieve text information; In response to the text type being subtitle text, the subtitle text is retrieved from the subtitle database to obtain the target content in the target video; In response to at least one of the text types being a person name, an item name, or a scene name, at least one of the person name, item name, or scene name is retrieved from the name database to obtain the target content in the target video.

[0085] Furthermore, in one possible implementation of this application embodiment, the generation module 803 is used for: Determine the compression icon for the target content; Generate a location pointer corresponding to the target content based on the compressed icon.

[0086] Furthermore, in one possible implementation of this application embodiment, the generation module 803 is used for: Based on the target content, determine the user's interest goals, which include at least one person and one item; Compress the image of the target of interest to obtain a compressed icon.

[0087] Furthermore, in one possible implementation of this application embodiment, the marking module 804 is used for: Determine the start and end times of the target content; In the progress display area of ​​the target video, mark the starting time point of the target content according to the positioning pointer.

[0088] Furthermore, in one possible implementation of this application embodiment, the device 800 is also used for: Count the number of online user views for each target content; Based on the number of online user views, identify trending target content from the target content; Mark the hotspot at the location pointer corresponding to the hotspot target content.

[0089] Furthermore, in one possible implementation of this application embodiment, the device 800 is also used for: Save the location pointer corresponding to the current search behavior to the pointer set. The pointer set is used to manage the location pointers corresponding to all user search behaviors. Each location pointer is used to locate the target content corresponding to the marker. In response to the user's selection operation on any positioning pointer in the pointer set, the positioning pointer corresponding to the selection operation is displayed in the target video to locate the target content corresponding to the positioning pointer.

[0090] Furthermore, in one possible implementation of this application embodiment, the acquisition module 801 is used for: Receive multimodal search information input by the user; or, Count the number of times each target content appears in the user's historical search behavior; Based on the frequency of occurrence, determine user preferences, retrieve information, and display it. In response to the user's selection of any preferred search information, the preferred search information is determined as the user's search information.

[0091] Furthermore, in one possible implementation of this application embodiment, the acquisition module 801 is used for: Determine the relevance between the target content and the user based on the frequency of occurrence; Select target content with a matching degree higher than a preset threshold as content of interest; Based on the search information corresponding to the content of interest, determine the user's preferred search information.

[0092] It should be noted that the foregoing explanation of the video content retrieval method embodiment also applies to the video content retrieval device of this embodiment, and will not be repeated here.

[0093] In this embodiment, after a user triggers a search, the system acquires the user's multimodal search information, supporting both text and image multimodal search formats. It can also determine search information based on the user's historical search preferences, improving search flexibility and efficiency. Based on the type of search information, the system acquires target content from the target video, further obtaining a compressed icon of the target content and determining a positioning pointer based on the compressed icon. A positioning mark is then placed at the start time of the target content within the target video. Users can quickly jump to the corresponding segment based on the positioning mark, simplifying the search operation. Furthermore, users can switch between different target contents by changing the positioning pointer, flexibly switching between different video segments. The operation logic is simple, greatly optimizing the user's visual interaction experience and solving the inefficiency of searching by dragging a progress bar. Users can quickly locate all target content matching the search information in a single search, significantly improving search efficiency.

[0094] To implement the above embodiments, this application also proposes an electronic device, including: a processor and a memory communicatively connected to the processor; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory to implement the method provided in the foregoing embodiments.

[0095] To implement the above embodiments, this application also proposes a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the methods provided in the foregoing embodiments.

[0096] To implement the above embodiments, this application also proposes a computer program product, including a computer program that, when executed by a processor, implements the methods provided in the foregoing embodiments.

[0097] The examples in this application may involve user data, data acquisition, and / or use. All of these aspects comply with relevant laws, regulations, and provisions. In the examples of this application, all data collection, acquisition, processing, manipulation, forwarding, and use are conducted with the user's knowledge and confirmation. Accordingly, in implementing each example, the type, scope of use, and usage scenarios of any data or information that may be involved should be communicated to the user and their authorization obtained through appropriate means in accordance with relevant laws and regulations. The specific methods of notification and / or authorization can vary depending on the actual situation and application scenario, and the scope of the solution is not limited in this regard.

[0098] In this manual and the sample solutions, any processing of personal information will be conducted only on a legal basis (such as with the consent of the data subject or as necessary for the performance of a contract) and will only be carried out within the scope stipulated or agreed upon. A user's refusal to process personal information beyond what is necessary for basic functions will not affect the user's use of basic functions.

[0099] In the foregoing descriptions of the embodiments, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0100] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0101] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

[0102] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-included system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (electronic device), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Alternatively, the computer-readable medium may be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in a computer memory.

[0103] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0104] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0105] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0106] The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc. Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of this application.

Claims

1. A video content retrieval method, characterized in that, The method includes: In response to a triggered search action, obtain the user's multimodal search information; Based on the search information, at least one target content is obtained by searching the target video. Based on the target content, a location pointer corresponding to the current search behavior is generated; Based on the positioning pointer, the target content is positioned and marked within the target video.

2. The method according to claim 1, characterized in that, The step of retrieving at least one target content from the target video based on the retrieval information includes: In response to the fact that the retrieved information is text information, the target content corresponding to the text information is determined from the target video based on at least one of the name database and subtitle database related to the target video; In response to the fact that the search information is image information, the target content corresponding to the image information is obtained by searching the target video based on the image information.

3. The method according to claim 2, characterized in that, In response to the retrieval information being text information, determining the target content corresponding to the text information from the target video based on at least one of a database related to the target video and subtitle information includes: Obtain the text type of the text information; In response to the text type being subtitle text, the subtitle text is retrieved from the subtitle database to obtain the target content in the target video; In response to the text type being at least one of a person's name, an item's name, or a scene's name, at least one of the person's name, item's name, or scene's name is retrieved from the name database to obtain the target content in the target video.

4. The method according to any one of claims 1-3, characterized in that, The step of generating a location pointer corresponding to the current search behavior based on the target content includes: Determine the compression icon for the target content; A positioning pointer corresponding to the target content is generated based on the compressed icon.

5. The method according to claim 4, characterized in that, The step of determining the compressed icon of the target content includes: Based on the target content, determine the user's interest goals, which include at least one person and one item; The image of the target of interest is compressed to obtain a compressed icon.

6. The method according to claim 5, characterized in that, The step of performing time-based positioning marking on the target content within the target video based on the positioning pointer includes: Determine the start time point of the target content; In the progress display area of ​​the target video, the starting time point of the target content is marked according to the positioning pointer.

7. The method according to claim 6, characterized in that, The method further includes: Count the number of online user views for each of the target contents; Based on the number of online user views, determine the trending target content from the target content; Mark the hotspot at the location pointer corresponding to the hotspot target content.

8. The method according to claim 7, characterized in that, The method further includes: Save the location pointer corresponding to the current search behavior to the pointer set. The pointer set is used to manage the location pointers corresponding to all user search behaviors. Each location pointer is used to locate the target content corresponding to the marker. In response to a user's selection operation on any location pointer within the pointer set, the location pointer corresponding to the selection operation is displayed in the target video to locate the target content corresponding to the location pointer.

9. The method according to claim 1, characterized in that, The acquisition of the user's multimodal retrieval information includes: Receive the multimodal retrieval information input by the user; or, Count the number of times each target content appears in the user's historical search behavior; Based on the frequency of occurrence, the user's preference retrieval information is determined and displayed; In response to the user's selection of any of the preferred search information, the preferred search information is determined to be the user's search information.

10. The method according to claim 9, characterized in that, Based on the frequency of occurrence, the user's preference retrieval information is determined, including: The matching degree between the target content and the user is determined based on the frequency of occurrence. Target content with a matching degree higher than a preset threshold is selected as content of interest. Based on the search information corresponding to the content of interest, the user's preferred search information is determined.

11. A video content retrieval device, characterized in that, include: The acquisition module is used to acquire the user's multimodal search information in response to a triggered search action; The retrieval module is used to retrieve at least one target content from the target video based on the retrieval information. The generation module is used to generate a positioning pointer corresponding to the current search behavior based on the target content; The marking module is used to mark the target content within the target video according to the positioning pointer.

12. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1-10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1-10.

14. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1-10.

Images