Video-based information display method, device, electronic device, and storage medium

Abstract

A video-based information display method, apparatus, electronic device, and storage medium. The video-based information display method comprises: during playback of a target video, displaying first resource information corresponding to a target object in the target video on the target video's playback page, where the target video includes M image frames, and the first resource information is pre-obtained based on matching the target objects in N image frames; in response to a first event being triggered during playback of the target video, obtaining second resource information corresponding to the target object in the current image frame based on at least one current image frame played on the playback page during the triggering of the first event; and displaying the second resource information, where N is an integer greater than 0 and M is an integer greater than or equal to N. This video-based information display method can enrich image recognition and search functions for videos, simplify operational processes, and enhance user experience.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to a video-based information display method, device, electronic device, and storage medium. Background Art

[0002] To make it easier for users to find content in images, some apps provide image recognition and search functions. Users can upload images to the app, which will then identify the image and search for related content based on the recognition results, and provide the relevant content to the user. If a user wants to search for content in a video while watching it, they need to capture the video and upload the captured image to the image recognition program for recognition and search. Alternatively, when displaying a video or image, the app can retrieve and recommend similar content based on content such as objects that appear in the video or image. Summary of the Invention

[0003] Image search functionality for dynamic media resources such as videos is relatively simple and cumbersome to operate. To address these issues, at least one embodiment of the present disclosure provides a video-based information display method, device, electronic device, and storage medium that can enrich image search functionality for videos, simplify the operation process, and enhance the user experience.

[0004] At least one embodiment of the present disclosure provides a video-based information display method, comprising: during playback of a target video, displaying first resource information corresponding to a target object in the target video on a playback page of the target video, the target video comprising M image frames, and the first resource information being pre-obtained based on matching target objects in N of the image frames; in response to triggering a first event during playback of the target video, acquiring second resource information corresponding to the target object in the current image frame based on at least one current image frame played on the playback page during the triggering of the first event; and displaying the second resource information, where N is an integer greater than 0 and M is an integer greater than or equal to N.

[0005] At least one embodiment of the present disclosure also provides a video-based information display device, including: a first display unit and a second display unit, the first display unit being configured to display first resource information corresponding to a target object in the target video on a playback page of the target video during playback of the target video, the target video including M image frames, and the first resource information being pre-obtained based on matching target objects in N image frames; the second display unit being configured to respond to a first event being triggered during playback of the target video, and based on at least one current image frame played on the playback page during the triggering of the first event, obtain second resource information corresponding to the target object in the current image frame, and display the second resource information, where N is an integer greater than 0, and M is an integer greater than or equal to N.

[0006] At least one embodiment of the present disclosure also provides an electronic device, comprising: a processor; a memory, comprising one or more computer program modules; wherein the one or more computer program modules are stored in the memory and configured to be executed by the processor, and the one or more computer program modules include instructions for implementing the video-based information display method described in any embodiment of the present disclosure.

[0007] At least one embodiment of the present disclosure further provides a computer-readable storage medium for storing non-transitory computer-readable instructions. When the non-transitory computer-readable instructions are executed by a computer, the video-based information presentation method described in any embodiment of the present disclosure can be implemented.

[0008] At least one embodiment of the present disclosure further provides a computer program product, including a computer program carried on a non-transitory computer-readable medium, wherein the computer program includes program code for executing the video-based information presentation method described in any embodiment of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, like reference numerals represent like elements. It should be understood that the drawings are schematic and that the originals and elements are not necessarily drawn to scale.

[0010] Figure 1 A flowchart of a video-based information display method provided in some embodiments of the present disclosure;

[0011] Figure 2 A schematic diagram of a target video playback page provided in some embodiments of the present disclosure;

[0012] Figure 3 A schematic diagram of another target video playback page provided in some embodiments of the present disclosure;

[0013] Figure 4A A schematic diagram of another target video playback page provided in some embodiments of the present disclosure;

[0014] Figure 4B A schematic diagram of another target video playback page provided in some embodiments of the present disclosure;

[0015] Figure 5 A schematic diagram of another target video playback page provided in some embodiments of the present disclosure;

[0016] Figure 6 A schematic diagram of a page after a screenshot operation is performed on a playback page provided in some embodiments of the present disclosure;

[0017] Figure 7 A schematic diagram of a page after a first event is triggered provided in some embodiments of the present disclosure;

[0018] Figure 8 A schematic diagram of a resource page provided for some embodiments of the present disclosure;

[0019] Figure 9A A schematic diagram of a frame selection page provided in some embodiments of the present disclosure;

[0020] Figure 9B A schematic diagram of another resource page provided for some embodiments of the present disclosure;

[0021] Figure 10A A schematic diagram of a progress page provided in some embodiments of the present disclosure;

[0022] Figure 10B Schematic diagram of another progress page provided in some embodiments of the present disclosure

[0023] Figure 11 A system that can be used to implement the video-based information display method provided in the embodiments of the present disclosure;

[0024] Figure 12 A schematic block diagram of a video-based information display device provided in some embodiments of the present disclosure;

[0025] Figure 13 A schematic block diagram of an electronic device provided for some embodiments of the present disclosure;

[0026] Figure 14 A schematic block diagram of another electronic device provided for some embodiments of the present disclosure; and

[0027] Figure 15 A schematic diagram of a storage medium provided for some embodiments of the present disclosure. DETAILED DESCRIPTION

[0028] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0029] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0030] As used herein, the term "including" and its variations are open-ended, i.e., "including but not limited to." The term "based on" means "based, at least in part, on." The term "one embodiment" means "at least one embodiment," the term "another embodiment" means "at least one additional embodiment," and the term "some embodiments" means "at least some embodiments." Other terms are defined in the following description.

[0031] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0032] It should be noted that the modifications of "one" and "plurality" mentioned in this disclosure are illustrative rather than restrictive, and those skilled in the art will understand that unless the context clearly indicates otherwise, they should be understood as "one or more." "Plurality" should be understood as two or more.

[0033] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0034] For video content recognition and search, one way is to capture images in the video and upload the captured images to an image recognition platform (such as an application) for recognition and search. However, the operation is relatively cumbersome, and the image recognition and search function for videos is relatively simple, resulting in a poor user experience.

[0035] At least one embodiment of the present disclosure provides a video-based information display method, device, electronic device and storage medium, which can enrich the image recognition and search functions for videos, simplify the operation process, and improve the user experience.

[0036] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

[0037] At least one embodiment of the present disclosure provides a video-based information display method. The video-based information display method includes: during the playback of a target video, displaying first resource information corresponding to a target object in the target video on the playback page of the target video, the target video includes M image frames, and the first resource information is pre-obtained based on the matching of target objects in N image frames; in response to triggering a first event during the playback of the target video, based on at least one current image frame played by the playback page during the triggering of the first event, obtaining second resource information corresponding to the target object in the current image frame; and displaying the second resource information, where N is an integer greater than 0 and M is an integer greater than or equal to N.

[0038] Figure 1 A flowchart of a video-based information display method provided in some embodiments of the present disclosure. Figure 1 As shown, in at least one embodiment, the method includes the following operations of steps S110 to S130.

[0039] Step S110: During the playback of the target video, first resource information corresponding to the target object in the target video is displayed on the playback page of the target video.

[0040] Step S120: In response to triggering a first event during the target video playback process, based on at least one current image frame played by the playback page during the triggering of the first event, obtaining second resource information corresponding to the target object in the current image frame.

[0041] Step S130: Display the second resource information.

[0042] For example, the video-based information display method of the embodiments of the present disclosure can be executed by a terminal device, including but not limited to a mobile phone, a tablet computer, a laptop computer, etc. The terminal device may include a display device, a processor, a data transceiver device, etc. The terminal device may transmit data to a server and / or a database via a communication network.

[0043] For example, the target video can be a video media resource such as a short video, a long video, or a live video. The target video can be uploaded by a terminal device to a corresponding platform (such as an application), and the target video can be stored in the server and / or memory of the platform. The terminal device (such as a client, a user) that uploads the target video and the device (such as a client, a user) that plays the target video can be the same or different. For example, after the first user uploads the target video to the platform (such as a server) through the first terminal device, the platform can make a corresponding request to push the target video to the second terminal device for playback so that the second user of the second terminal device can watch it.

[0044] For example, the target object may include objects that appear in the video, such as objects, people, and animals, and the resource information (such as the first resource information and the second resource information) may be recommendation information or explanatory information about the target object. In some examples, when the target object is an object (such as a commodity, an exhibit, etc.), the resource information may be item recommendation information corresponding to the object or explanatory information about the object. In other examples, when the target object is a person, the resource information may be explanatory information about the object. In some of the following embodiments, the target object is an object and the resource information is item recommendation information. However, the embodiments of the present disclosure are not limited to this. During application, the types of target objects and resource information can be set according to actual needs.

[0045] For example, in step S110 , the target video includes M image frames, and the first resource information is pre-obtained based on matching target objects in N image frames, where N is an integer greater than 0, and M is an integer greater than or equal to N.

[0046] For example, after a target video is uploaded to the platform's server, the server can perform identification and search operations on at least some of the image frames (i.e., N image frames) in the target video. The identification and search operations can be performed with user authorization. For example, some of the image frames can be a video clip in the target video, or they can be several key image frames in the target video. A key image frame can be an image frame whose image difference from the previous image frame exceeds a certain threshold. The image difference between different image frames can be determined by the difference in pixel values ​​at multiple corresponding positions in the different image frames.

[0047] For example, the recognition operation of the image frame can be performed using a pre-trained object recognition model, which can be a neural network model, a classification tree model, or other types of models. In the process of training the object recognition model, the object recognition model can be trained to be able to recognize the category and features of the target object in the image frame. For example, there is a target object "skirt" in the image frame. The object recognition model can be used to identify the type of the target object as a skirt, and the color, length, material, texture and other features of the skirt can be identified. For example, one or more target objects can be identified for each image frame. If a target object needs to be determined for each image frame, and the image frame contains multiple target objects, a main target object can be determined based on conditions such as the occupied area or coordinate position of each target object.

[0048] For example, after obtaining the recognition results of the target objects for N image frames, a search operation can be performed in a predetermined network platform based on the recognition results of each target object to obtain the resource information that matches each target object as the first resource information. For example, if the recognition result of the target object for a certain image frame is a yellow long skirt, a search can be performed in a predetermined shopping platform based on keywords such as "yellow" and "long skirt" to obtain one or more product information that matches the target object. In some examples, if the product information obtained by the search exceeds a first predetermined number (the first predetermined number is, for example, one), a filtering operation can be performed to filter out the first predetermined number of product information from the search results. For example, the target video includes 10 key image frames. If a target object is identified for each key image frame, 10 target objects can be obtained. If a matching resource information is searched for each target object, 10 resource information can be obtained, and the 10 resource information can be used as the first resource information. For the sake of distinction, each resource information in the first resource information is referred to as the first sub-resource information below.

[0049] For example, based on step S110, the first resource information about the target video can be obtained offline from a server before the target video is played. Thereafter, when the terminal device plays the target video, the first resource information can be displayed on the target video's playback page, so that the user can obtain relevant resource information about the target video without performing additional search operations.

[0050] Figure 2 This is a schematic diagram of a target video playback page provided in some embodiments of the present disclosure. Figure 2 As shown, for example, the first resource information includes product information 301 corresponding to the illustrated image frame 201. The product information 301 can be displayed during the playback of the target video. The product information 301 can be implemented as an interface control. For example, if a predetermined operation (e.g., a click) is triggered on the product information 301, the user can be redirected to the product details page corresponding to the product information 301.

[0051] For example, in step S120, if a first event is triggered during video playback, an online recognition and search operation can be performed on a current image frame that is being played when the first event is triggered. For example, the terminal device can obtain the current image frame and send the current image frame to the server, and the server performs recognition and search operations on the current image frame. The recognition operation and search operation on the current image frame can refer to the above-mentioned recognition operation and search operation on N image frames. For the target object in the current image frame, a second predetermined amount of resource information can be obtained (the second predetermined amount is, for example, a value between 10 and 500), and the second predetermined amount of resource information is used as the second resource information. For the sake of distinction, each resource information in the second resource information is referred to as the second sub-resource information below. After the server obtains the second resource information that matches the target object in the current image frame, it can send the second resource information to the corresponding terminal device.

[0052] For example, in step S130, after receiving the second resource information, the terminal device may display the second resource information. For example, the second resource information may be displayed directly on the play page, or the play page may be redirected to the resource page and the second resource information may be displayed on the resource page.

[0053] For example, based on steps S120 and S130, during the target video playback process, one or some image frames can be identified and searched online in response to user operations. Based on this method, when the user sees the target object of interest, the corresponding resource information can be quickly and conveniently obtained.

[0054] The video-based information display method of the disclosed embodiments can combine offline and online recognition. Without triggering the recognition and search operation, the user can be presented with resource information obtained through offline recognition and search. Once the user triggers a search operation for an image frame of interest, resource information matching the image frame of interest can be obtained online. This can enrich the image recognition and search functionality for videos, simplify the operation process, and enhance the user experience.

[0055] For example, Figure 2 As shown, the play page includes a first display area (eg Figure 2 The first resource information may be displayed in the first display area. The first resource information includes N first sub-resource information corresponding to the target objects in the N image frames. For example, one sub-resource information may be matched to the target object in each key image frame.

[0056] For example, displaying the first resource information corresponding to the target object in the target video on the playback page of the target video in step S110 includes: when displaying N image frames respectively on the playback page, displaying N first sub-resource information respectively in the first display area.

[0057] Figure 3 This is a schematic diagram of another target video playback page provided in some embodiments of the present disclosure. Figure 2 and Figure 3 As shown, the first resource information may include Figure 2 The product information 301 corresponding to the image frame 201 and the Figure 3 The product information 302 corresponding to the image frame 202 shown is shown. When the playback page plays to the image frame 201, the product information 301 can be displayed in the first display area. When the playback page plays to the image frame 202, the product information 302 can be displayed in the first display area. In other words, when the playback page plays to each of the N image frames, the first sub-resource information matching the image frame is displayed on the playback page. As the playback progress changes, the displayed first sub-resource information will also change accordingly. Based on this method, the resource information can be matched with the image frames in the playback page, which can make the interface more concise and make it easier for users to view the first sub-resource information of interest.

[0058] For example, the N image frames include the i-th image frame and the j-th image frame, and the first resource information includes the i-th first sub-resource information corresponding to the i-th image frame and the j-th first sub-resource information corresponding to the j-th image frame, where i is an integer greater than 0, and j is an integer greater than i and less than M. For example, displaying the first resource information corresponding to the target object in the target video on the playback page of the target video in step S110 includes: displaying the i-th first sub-resource information in the first display area during the process of displaying the i-th image frame and the image frames between the i-th image frame and the j-th image frame on the playback page.

[0059] For example, the i-th image frame is Figure 2 The image frame 201 shown, the j-th image frame is Figure 3 As shown in the image frame 202, the first resource information includes Figure 2 The product information 301 corresponding to the image frame 201 and the Figure 3The product information 302 corresponding to the image frame 202 shown is shown. When the playback page plays to image frame 201, the product information 301 is displayed in the first display area. For a period of time before the playback reaches image frame 202, the product information 301 can be continuously displayed in the first display area until the playback page reaches image frame 202, at which point the product information in the first display area can be replaced by product information 302 instead of product information 301. In other words, during the playback of the image frames between two adjacent key image frames, the first sub-resource information corresponding to the previous key image frame between the two adjacent key image frames can be continuously displayed until the next key image frame is played. Because the image difference between the intermediate image frame and the previous key image frame is relatively small, there is a high probability that it corresponds to the same first sub-resource information as the previous key image frame. Therefore, the previous first sub-resource information can be continuously displayed during the playback of the intermediate image frame, essentially ensuring that the playback image matches the first sub-resource information throughout the playback of the target video.

[0060] For example, in other examples, during the entire target video playback period, all first sub-resource information included in the first resource information may be displayed on the playback page, and the user may select the first sub-resource information of interest to view.

[0061] For example, the M image frames also include a p-th image frame located between the i-th image frame and the j-th image frame, where p is an integer greater than i and less than or equal to j. The information display method may further include: during display of the i-th image frame and image frames between the i-th image frame and the p-th image frame on the playback page, displaying a first scanning pattern in the first display area that changes correspondingly with changes in the image frames displayed on the playback page.

[0062] Figure 4A A schematic diagram of another target video playback page provided in some embodiments of the present disclosure, Figure 4B A schematic diagram of another target video playback page provided in some embodiments of the present disclosure.

[0063] like Figure 2 、 3 , 4A and 4B, following the above example, for example, the i-th image frame is Figure 2 and Figure 4A The image frame 201 shown, the j-th image frame is Figure 3 The image frame 202 shown, the pth image frame is Figure 4BThe image frame 203 shown is located between the image frame 201 and the image frame 202. When the playback page plays to the image frame 201, the first scanning graphic 401 may appear in the first display area, and in the process of playing from the image frame 201 to the image frame 203, the first scanning graphic 401 is continuously displayed in the first display area, and during this period, as the playback screen continues to change, the first scanning graphic 401 also changes accordingly. It can also be understood that the first scanning graphic 401 changes as time goes by. For example, as the image frame displayed on the playback page changes, the first scanning graphic at least partially moves along a predetermined direction in the first display area. The predetermined direction may be the longitudinal direction of the playback page (for example, Figure 4A and Figure 4B to create a scanning effect.

[0064] For example, the first scanning graphic 401 may not be displayed during the process of playing from image frame 203 to image frame 202. Then, when the next key image frame (i.e., image frame 202) is played, new first sub-resource information will appear on the playback page. At this time, the first scanning image 401 may appear again, and the first scanning image 401 moves and scans for a period of time before disappearing, until the first scanning image 401 appears again when the next key image frame is played. This cycle continues until the target video is finished playing. In other words, the first scanning graphic may appear and move and scan for a period of time each time a new first sub-resource information appears on the playback page. Based on this method, the first scanning graphic may be used to indicate that new first sub-resource information is appearing, so as to attract the user's attention and prompt the user to view the newly appeared first sub-resource information.

[0065] For example, the first scanning pattern 401 can be in the form of a straight line, a curve, a frame, a dotted line, etc., and can be specifically set according to actual needs, and the embodiments of the present disclosure do not limit this. The first scanning pattern 401 can move in the vertical direction, the horizontal direction, or the diagonal direction. In addition to the first scanning pattern 401 being able to change in the form of movement, in other examples, the first scanning pattern 401 can also change in the form of rotation, flashing, deformation, etc.

[0066] Figure 5 This is a schematic diagram of another target video playback page provided in some embodiments of the present disclosure. Figure 5 As shown, for example, the video-based information display method of the embodiment of the present disclosure may further include: during the playback of the target video, displaying the first control 501 on the playback page of the target video. In some examples, triggering the first event during the playback of the target video in step S120 includes: triggering a first predetermined operation for the first control 501 during the playback of the target video.

[0067] For example, the first predetermined operation can be a click operation. During playback of the target video, a first control 501 is displayed on the playback page. If the user is interested in a target object in a certain image frame, the user can click the first control 501 to trigger an online recognition and search operation for that image frame. In other examples, the first predetermined operation can also be a double-click, a slide, or other operation. The embodiments of the present disclosure do not limit the specific form of the first predetermined operation.

[0068] For example, in some other examples, triggering a first event during the playback of the target video in step S120 may include: triggering a playback pause operation for the target video in response to the first event.

[0069] For example, the first event may be an event that triggers playback pause, such as clicking a pause button or clicking a certain area on the playback page. During playback of the target video, if the user is interested in a target object in a certain image frame, the target video may be paused, thereby triggering an online recognition and search operation for the image frame.

[0070] For example, in some other examples, triggering a first event during the playback of the target video in step S120 may include: triggering a screenshot operation of the playback page of the target video in response to the first event.

[0071] For example, the first event may be an event that can trigger a screenshot, such as pressing a specific button. During the target video playback, if the user is interested in a target object in a certain image frame, the user may take a screenshot, which may trigger an online recognition and search operation for the image frame.

[0072] For example, the above three methods of triggering online recognition and search operations (first control, pause and screenshot) are all simple to operate and easy to implement, and can make the methods of triggering online recognition and search operations more diversified to adapt to the different operating habits of different users and improve user experience.

[0073] Figure 6 This is a schematic diagram of a page after a screenshot operation is performed on the playback page provided in some embodiments of the present disclosure. Figure 6 As shown, for example, the video-based information display method of an embodiment of the present disclosure may also include: in response to a screenshot operation of the playback page of the target video, displaying a second control 601 and a third control 602, the second control 601 being configured to trigger an operation of displaying the second resource information, and the third control 602 being configured to trigger an operation of sharing the target video to a platform or user corresponding to the third control 602.

[0074] For example, when the playback page plays to the image frame 201, a screenshot operation is triggered, and then the page can be redirected to the sharing page 600. The sharing page 600 can display a second control 601 and a third control 602. The second control 601 can be a control about the second resource information matching the image frame 201. For example, the second resource information matching the image frame 201 includes several second sub-resource information (for example, including product information "skirt", "bag" and "shoes"). A second control 601 can be displayed for each second sub-resource information. If the user clicks on any second control 601, the user can jump to the details page of the corresponding second sub-resource information. The third control 602 can be a platform sharing control. If the user clicks on any platform sharing control, the user can jump to the corresponding platform to perform the sharing operation. The third control 602 can also be a user sharing control. If the user clicks on any user sharing control, the user can jump to the sharing interface with the corresponding user. The purpose of taking a screenshot may be to share the screenshot or to trigger recognition and search operations for the current image frame. If the user's intention is uncertain, displaying two controls for the user to choose from can avoid misoperation.

[0075] Figure 7 A schematic diagram of a page after a first event is triggered provided in some embodiments of the present disclosure. Figure 7 As shown, for example, the video-based information display method of the embodiment of the present disclosure may further include: during the process of obtaining the second resource information, displaying the current image frame on the playback page, and superimposing a dynamic second scanning pattern on the current image frame. The dynamic second scanning pattern may include a first sub-scanning pattern 701 that moves in a predetermined direction and / or a second sub-scanning pattern 702 that moves or flashes at the location of the target object in the current image frame.

[0076] For example, when playing to Figure 2 The first event is triggered when the image frame 201 is shown. The page can stay on the image frame 201, and, while waiting for the server to perform the recognition and search operations, the first sub-scanning graphic 701 can be displayed on the page. The first sub-scanning image can, for example, move along a predetermined direction (such as the up and down direction shown in the figure) to present a scanning effect. After the server determines the second sub-resource information that matches each target object in the image frame 201, it can send each second sub-resource information and the location information of each target object to the terminal device. The terminal device can display the second sub-scanning graphic 702 at the location of each target object, and the second sub-scanning graphic 702 can be presented in a dynamic manner such as moving or flashing to indicate that the target objects have been found at these locations and the relevant second sub-resource information has been searched and obtained. Subsequently, each second sub-resource information can be presented.

[0077] For example, displaying the second resource information in step S130 may include: displaying a resource page, and displaying the second resource information in the resource page. The current image frame includes E target objects, and the second resource information includes a plurality of second sub-resource information corresponding to the E target objects. The resource page may include a second display area and E fourth controls corresponding to the E target objects, each of which is configured to trigger display of the second sub-resource information corresponding to the fourth control in the second display area, where E is an integer greater than 0.

[0078] Figure 8 Schematic diagram of resource page provided by some embodiments of the present disclosure. Figure 8 As shown, for example, when playing to Figure 2 The first event is triggered when the image frame 201 is shown. After obtaining the second sub-resource information matching each target object in the image frame 201, a resource page 800 can be displayed. Resource page 800 can be displayed overlaid on the image frame 201 or redirected to a separate resource page 800 from the playback page. For example, if two target objects are identified in the image frame 201: a skirt and a bag, multiple second sub-resource information can be searched for each target object. Resource page 800 can include a second display area 802 and two fourth controls 801 corresponding to the two target objects. Clicking each control 801 displays the corresponding multiple second sub-resource information in the second display area 802. For example, if the fourth control 801 corresponding to the target object "skirt" is clicked, multiple second sub-resource information matching the target object "skirt" can be displayed in the second display area 802; if the fourth control 801 corresponding to the target object "bag" is clicked, multiple second sub-resource information matching the target object "bag" can be displayed in the second display area 802. Based on this approach, the second sub-resource information corresponding to different types of target objects can be displayed separately, making it easier for users to find information of interest.

[0079] For example, the video-based information display method of an embodiment of the present disclosure may also include: in response to a second predetermined operation on a resource page, displaying a frame selection page, the frame selection page displaying the current image frame; in response to receiving a frame selection operation on the current image frame in the frame selection page, based on the image area defined by the frame selection operation, obtaining third resource information corresponding to the target object in the image area; and displaying the third resource information.

[0080] Figure 9A A schematic diagram of a frame selection page provided in some embodiments of the present disclosure. Figure 9B A schematic diagram of another resource page provided for some embodiments of the present disclosure.

[0081] Continuing with the above example, if the user Figure 8If the item of interest is not found in the second resource information shown, the user can perform a second predetermined operation (such as a sliding operation) on the resource page 800 to jump to the Figure 9A The frame selection page 900 shown in the figure may display an image frame 201 corresponding to the first event. If the user performs a frame selection operation on a certain area in the image frame 201, the frame 901 may be displayed and the image area corresponding to the frame 901 may be sent to the server to notify the server to perform an identification and search operation on the image area. After the terminal device receives the target object and the third resource information corresponding to the image area, it may display the following information: Figure 9B In the resource page 900 shown, a new fourth control 903 can be added to the resource page. The newly added fourth control 903 corresponds to the target object in the selection box 901, and multiple third sub-resource information in the third resource information can be displayed in the second display area 902. Based on this approach, if the target object of the user's interest is not automatically identified, the user can manually select the area of ​​interest.

[0082] For example, the video-based information display method of the embodiment of the present disclosure may also include: in response to the failure of the operation to obtain the second resource information corresponding to the target object in the current image frame or the failure to obtain the second resource information corresponding to the target object in the current image frame within a predetermined time period from the triggering of the first event, displaying a frame selection page, the frame selection page displaying the current image frame; in response to receiving a frame selection operation on the current image frame in the frame selection page, obtaining third resource information corresponding to the target object in the image area based on the image area defined by the frame selection operation; and displaying the third resource information.

[0083] For example, if after triggering the first event for the current image frame, the second resource information feedback received from the server is empty (i.e., the target object in the current image frame is not identified or the second resource information matching the target object is not found) or no feedback information for the second resource information from the server is received for a long time, then the display Figure 9A The frame selection page 900 shown is for the user to manually frame the area of ​​interest. Based on this approach, by manually framing the image area, the recognition range can be narrowed to a more precise range, which can speed up the recognition speed and improve the recognition accuracy to a certain extent.

[0084] For example, the video-based information display method of an embodiment of the present disclosure may also include: in response to a third predetermined operation on a resource page, displaying a progress page, the progress page including a progress bar control and an image display area, the image display area displaying the current image frame; in response to a fourth predetermined operation on the progress bar control, switching the image display area from displaying the current image frame to displaying a target image frame corresponding to the fourth predetermined operation; obtaining fourth resource information corresponding to the target object in the target image frame; and displaying the fourth resource information.

[0085] Figure 10A A schematic diagram of a progress page provided in some embodiments of the present disclosure is provided. Figure 10B A schematic diagram of another progress page provided for some embodiments of the present disclosure.

[0086] For example, using the above example, if the user is browsing Figure 8 After viewing the second resource information corresponding to the image frame 201, if you want to view the resource information corresponding to other image frames, you can perform a third predetermined operation (such as a sliding operation) on the resource page 800 to jump to Figure 10A The progress page 1000 shown includes a progress bar control 1001 and an image display area 1002. For example, by sliding the progress bar, the progress value of the progress bar can be changed, and the image display area 1002 can be switched to an image frame corresponding to the progress value. For example, by adjusting the progress so that the image display area 1002 is switched to the image frame 202, the recognition and search operation for the image frame 202 can be triggered to obtain the target object and the fourth resource information corresponding to the image frame 202, and then the image frame 202 can be displayed. Figure 8 The resource page 800 is shown, and the fourth resource information is displayed in the resource page 800. Based on this approach, after browsing the resource information of an image frame, the user does not need to return to the playback page of the target video. Instead, they only need to drag the progress bar to trigger the recognition and search operation for another image frame, which simplifies the operation procedure.

[0087] It should be noted that, in the embodiments of the present disclosure, the execution order of the various steps of the video-based information display method is not limited. Although the execution process of each step is described above in a specific order, this does not constitute a limitation on the embodiments of the present disclosure. The various steps in the video-based information display method can be executed serially or in parallel, which can be determined according to actual needs. The video-based information display method can also include more or fewer steps, for example, in order to achieve a better display effect, some preprocessing steps are added, or some intermediate process data are stored and used for subsequent processing and calculation to omit some similar steps.

[0088] Figure 11A system that can be used to implement the video-based information display method provided by the embodiment of the present disclosure. Figure 11 As shown, the system 1100 may include one or more user terminals (i.e., terminal devices) 1111, a network 1112, a server 1113, and a database 1114. For example, the system 1110 may be used to implement the video-based information display method provided by any embodiment of the present disclosure.

[0089] The user terminal 1111 is, for example, a computer 1111-1. It is understandable that the user terminal 1111 can be any other type of electronic device capable of performing data processing, which may include but is not limited to a desktop computer, a laptop computer, a tablet computer, a workstation, etc. The user terminal 1111 can also be any equipment provided with an electronic device. The embodiments of the present disclosure do not limit the hardware configuration or software configuration of the user terminal (for example, the type or version of the operating system (for example, Windows, MacOS, Android, Hongmeng, etc.)).

[0090] The user can operate the application installed on the user terminal 1111 or the website logged in on the user terminal 1111. The application or website transmits data such as image frames and requests to the server 1113 through the network 1112. The user terminal 1111 can also receive data transmitted by the server 1113 through the network 1112.

[0091] For example, the user terminal 1111 is installed with software having a video playback function, and the user uses the video playback function of the software to play the target video on the user terminal 1111. The user terminal 1111 executes the video-based information display method provided by the embodiment of the present disclosure by running code.

[0092] The network 1112 may be a single network or a combination of at least two different networks, which may be wireless communication networks, wired communication networks, etc. For example, the network 1112 may include, but is not limited to, a local area network, a wide area network, a public network, a private network, etc., or a combination of several of them.

[0093] Server 1113 can be a single server, a server group, or a cloud server, where each server in the server group is connected via a wired or wireless network. A server group can be centralized, such as a data center, or distributed. Server 1113 can be local or remote.

[0094] Database 1114 can generally refer to a device with storage functions. Database 1114 is mainly used to store various data used, generated, and output by user terminal 1111 and server 1113 during operation. It can be various types of databases, such as relational databases or non-relational databases. Database 1114 can be local or remote. Database 1114 can include corresponding operating software and various memories, such as random access memory (RAM) and read-only memory (ROM). The storage devices mentioned above are just some examples, and the storage devices that can be used by system 1110 are not limited to them.

[0095] The database 1114 may be interconnected or communicated with the server 1113 or a portion thereof via the network 1112, or directly interconnected or communicated with the server 1113, or a combination of the two.

[0096] In some examples, database 1114 may be a standalone device. In other examples, database 1114 may be integrated into at least one of user terminal 1111 and server 1113. For example, database 1114 may be located on user terminal 1111 or on server 1113. In another example, database 1114 may be distributed, with a portion located on user terminal 1111 and another portion located on server 1113.

[0097] For example, the target video and the first resource information may be stored in a database 1114. When the terminal device needs to play the target video, the user terminal 1111 accesses the database 1114 through the network 1112 and obtains the target video and the first resource information stored in the database 1114 through the network 1112. The embodiments of the present disclosure do not limit the type of database, which may be a relational database or a non-relational database.

[0098] At least one embodiment of the present disclosure also provides a video-based information display device that combines offline and online recognition. Without triggering a recognition operation, the device displays offline recognition and search resource information to the user. Once the user triggers a search operation for an image frame of interest, resource information matching the image frame of interest can be retrieved online. This enriches the image recognition and search functionality for videos, simplifies the operational process, and enhances the user experience.

[0099] Figure 12 This is a schematic block diagram of a video-based information display device provided in some embodiments of the present disclosure. Figure 12As shown, the video-based information display device 1200 includes a first display unit 1210 and a second display unit 1220. For example, the video-based information display device 1200 can be applied to a user terminal, or to any device or system that needs to implement video playback and identification search functions, and the embodiments of the present disclosure are not limited to this.

[0100] The first display unit 1210 is configured to display first resource information corresponding to a target object in the target video on the target video playback page during the target video playback process. The target video includes M image frames, and the first resource information is pre-matched based on the target object in the N image frames. For example, the first display unit 1210 may perform the following operations: Figure 1 Step S110 of the video-based information display method is shown.

[0101] The second display unit 1220 is configured to, in response to a first event being triggered during the target video playback process, obtain second resource information corresponding to the target object in the current image frame based on at least one current image frame played by the playback page during the triggering of the first event, and display the second resource information, where N is an integer greater than 0 and M is an integer greater than or equal to N. For example, the second display unit 1220 may perform the following operations: Figure 1 Steps S120 and S130 of the video-based information presentation method are shown.

[0102] For example, the first display unit 1210 and the second display unit 1220 can be hardware, software, firmware, or any feasible combination thereof. For example, the first display unit 1210 and the second display unit 1220 can be dedicated or general-purpose circuits, chips, or devices, or can be a combination of a processor and memory. The embodiments of the present disclosure do not limit the specific implementation of the first display unit 1210 and the second display unit 1220.

[0103] It should be noted that in the embodiment of the present disclosure, the various units of the video-based information display device 1200 correspond to the various steps of the aforementioned video-based information display method. For the specific functions of the video-based information display device 1200, please refer to the relevant description of the video-based information display method above, which will not be repeated here. Figure 12 The components and structures of the video-based information display device 1200 shown are merely exemplary and non-limiting. The video-based information display device 1200 may further include other components and structures as needed.

[0104] For example, in some examples, the playback page includes a first display area, and the first resource information is displayed in the first display area; the first resource information includes N first sub-resource information corresponding to target objects in N image frames. The first display unit 1210 can be further configured to: when the playback page displays N image frames, respectively, display the N first sub-resource information in the first display area.

[0105] For example, in some examples, the N image frames include the i-th image frame and the j-th image frame, and the first resource information includes the i-th first sub-resource information corresponding to the i-th image frame and the j-th first sub-resource information corresponding to the j-th image frame. The first display unit 1210 can be further configured to: during the process of displaying the i-th image frame and the image frames between the i-th image frame and the j-th image frame on the playback page, display the i-th first sub-resource information in the first display area, where i is an integer greater than 0 and j is an integer greater than i.

[0106] For example, in some examples, the M image frames also include a p-th image frame located between the i-th image frame and the j-th image frame. The video-based information display device may further include a first image unit, wherein the first graphic unit is configured to: during the process of displaying the i-th image frame and image frames between the i-th image frame and the p-th image frame on the playback page, display a first scanning graphic in the first display area that changes accordingly as the image frames displayed on the playback page change, where p is an integer greater than i and less than or equal to j.

[0107] For example, in some examples, the first graphic unit is further configured to: as the image frame displayed on the playback page changes, the first scanning graphic at least partially moves along a predetermined direction in the first display area.

[0108] For example, in some examples, the video-based information display device may further include a first control unit configured to: display a first control on the playback page of the target video during playback of the target video. The second display unit 1220 is further configured to: trigger a first predetermined operation for the first control during playback of the target video.

[0109] For example, in some examples, the second display unit 1220 is further configured to: trigger a playback pause operation for the target video in response to the first event; or trigger a screenshot operation for the playback page of the target video in response to the first event.

[0110] For example, in some examples, the video-based information display device may also include a screenshot unit, which is configured to: display a second control and a third control in response to a screenshot operation on the playback page of the target video, the second control is configured to trigger the operation of displaying the second resource information, and the third control is configured to trigger the operation of sharing the target video to the platform or user corresponding to the third control.

[0111] For example, in some examples, the video-based information display device may further include a second graphic unit configured to: display the current image frame on the playback page and overlay a dynamic second scanning graphic on the current image frame during the process of obtaining the second resource information. The dynamic second scanning graphic includes a first sub-scanning graphic that moves in a predetermined direction and / or a second sub-scanning graphic that moves or flashes at the location of the target object in the current image frame.

[0112] For example, in some examples, the second display unit 1220 is further configured to: display a resource page and display second resource information in the resource page; the current image frame includes E target objects, the second resource information includes multiple second sub-resource information corresponding to the E target objects respectively, the resource page includes a second display area and E fourth controls corresponding to the E target objects respectively, each fourth control is configured to trigger an operation of displaying the second sub-resource information corresponding to the fourth control in the second display area, and E is an integer greater than 0.

[0113] For example, in some examples, the video-based information display device may also include a first framing unit, which is configured to: in response to a second predetermined operation on a resource page, display a framing page, and the framing page displays the current image frame; in response to receiving a framing operation on the current image frame in the framing page, obtain third resource information corresponding to the target object in the image area based on the image area defined by the framing operation; and display the third resource information.

[0114] For example, in some examples, the video-based information display device may also include a second framing unit, which is configured to: in response to the failure of an operation to obtain second resource information corresponding to the target object in the current image frame or the failure to obtain second resource information corresponding to the target object in the current image frame within a predetermined time period from the triggering of the first event, display a framing page, wherein the framing page displays the current image frame; in response to receiving a framing operation on the current image frame in the framing page, obtain third resource information corresponding to the target object in the image area based on the image area defined by the framing operation; and display the third resource information.

[0115] For example, in some examples, the video-based information display device may also include a progress unit, which is configured to: in response to a third predetermined operation on the resource page, display a progress page, the progress page including a progress bar control and an image display area, the image display area displaying the current image frame; in response to a fourth predetermined operation on the progress bar control, switch the image display area from displaying the current image frame to displaying a target image frame corresponding to the fourth predetermined operation; obtain fourth resource information corresponding to the target object in the target image frame; and display the fourth resource information.

[0116] Figure 13 This is a schematic block diagram of an electronic device provided in some embodiments of the present disclosure. Figure 13 As shown, electronic device 1300 includes a processor 1310 and a memory 1320. Memory 1320 is used to store non-transitory computer-readable instructions (e.g., one or more computer program modules). Processor 1310 is used to execute non-transitory computer-readable instructions. When the non-transitory computer-readable instructions are executed by processor 1310, one or more steps in the video-based information presentation method described above can be performed. Memory 1320 and processor 1310 can be interconnected via a bus system and / or other form of connection mechanism (not shown).

[0117] For example, the processor 1310 may be a central processing unit (CPU), a digital signal processor (DSP), or other processing units with data processing capabilities and / or program execution capabilities, such as a field programmable gate array (FPGA). For example, the central processing unit (CPU) may be an X86 or ARM architecture. The processor 1310 may be a general-purpose processor or a dedicated processor, and may control other components in the electronic device 1300 to perform desired functions.

[0118] For example, the memory 1320 may include any combination of one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), a hard disk, an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), a USB memory, a flash memory, etc. One or more computer program modules may be stored on the computer-readable storage medium, and the processor 1310 may execute one or more computer program modules to implement various functions of the electronic device 1300. The computer-readable storage medium may also store various applications and various data, as well as various data used and / or generated by the applications.

[0119] It should be noted that, in the embodiment of the present disclosure, the specific functions and technical effects of the electronic device 1300 can be referred to the above description of the video-based information display method, which will not be repeated here.

[0120] Figure 14 This is a schematic block diagram of another electronic device provided in some embodiments of the present disclosure. The electronic device 300 is suitable for implementing the video-based information display method provided in the embodiments of the present disclosure. The electronic device 1400 can be a user terminal, etc. It should be noted that, Figure 14 The electronic device 1400 shown is merely an example and does not limit the functions and scope of use of the embodiments of the present disclosure.

[0121] like Figure 14 As shown, the electronic device 1400 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 1410, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1420 or a program loaded from a storage device 1480 into a random access memory (RAM) 1430. Various programs and data required for the operation of the electronic device 1400 are also stored in the RAM 1430. The processing device 1410, the ROM 1420, and the RAM 1430 are connected to each other via a bus 1440. An input / output (I / O) interface 1450 is also connected to the bus 1440.

[0122] Typically, the following devices may be connected to the I / O interface 1450: an input device 1460 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 1470 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1480 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1490. The communication device 1490 may allow the electronic device 1400 to communicate with other electronic devices wirelessly or by wire to exchange data. Although Figure 14 The electronic device 1400 is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown, and the electronic device 1400 may alternatively implement or possess more or fewer devices.

[0123] For example, according to an embodiment of the present disclosure, a video-based information display method can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the above-mentioned video-based information display method. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 1490, or installed from the storage device 1480, or installed from the ROM 1420. When the computer program is executed by the processing device 1410, the functions defined in the video-based information display method provided by the embodiment of the present disclosure can be executed.

[0124] At least one embodiment of the present disclosure further provides a storage medium for storing non-transitory computer-readable instructions, which, when executed by a computer, can implement the video-based information presentation method described in any embodiment of the present disclosure.

[0125] Figure 15 A schematic diagram of a storage medium provided in some embodiments of the present disclosure. Figure 15 As shown, the storage medium 1500 is used to store non-transitory computer-readable instructions 410. For example, when the non-transitory computer-readable instructions 410 are executed by a computer, one or more steps in the video-based information presentation method described above can be performed.

[0126] For example, the storage medium 1500 can be applied to the electronic device 1300. Figure 13 The memory 1320 in the electronic device 1300 is shown. For example, the description of the storage medium 1500 can be referred to Figure 13 The corresponding description of the memory 1320 in the electronic device 1300 is not repeated here.

[0127] In the above, combined with Figures 1 to 15 The present disclosure describes a video-based information display method, video-based information display device, electronic device, storage medium, and program product provided by embodiments of the present disclosure. The video-based information display method provided by embodiments of the present disclosure can combine offline and online recognition. Without triggering a recognition operation, offline recognition and search resource information can be displayed to the user. After the user triggers a search operation for an image frame of interest, resource information matching the image frame of interest can be obtained online. This can enrich the image recognition and search functionality for videos, simplify the operational process, and enhance the user experience.

[0128] It should be noted that the storage medium (computer-readable medium) mentioned above in the present disclosure may be a computer-readable signal medium or a non-transitory computer-readable storage medium or any combination of the two. Non-transitory computer-readable storage media may be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or devices, or any combination of the above. More specific examples of non-transitory computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a non-transitory computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a non-transitory computer-readable storage medium that can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0129] In some embodiments, the client and server can communicate using any currently known or future developed network protocol, such as the Hyper Text Transfer Protocol (HTTP), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), internets (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed networks.

[0130] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0131] The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: during the playback of the target video, displays first resource information corresponding to the target object in the target video on the playback page of the target video, the target video includes M image frames, and the first resource information is pre-obtained based on the matching of the target objects in N image frames; in response to triggering a first event during the playback of the target video, based on at least one current image frame played on the playback page during the triggering of the first event, obtains second resource information corresponding to the target object in the current image frame; and displays the second resource information.

[0132] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, such as a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0133] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or can be implemented using a combination of dedicated hardware and computer instructions.

[0134] The units involved in the embodiments described in this disclosure may be implemented in software or hardware, wherein the name of a unit does not necessarily limit the unit itself.

[0135] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0136] In the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0137] The above description is only a partial embodiment of the present disclosure and an illustration of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, the above-mentioned features can be replaced with (but not limited to) technical features with similar functions disclosed in this disclosure.

[0138] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0139] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims

1. A video-based information display method, comprising: During playback of a target video, first resource information corresponding to a target object in the target video is displayed on a playback page of the target video, wherein the target video includes M image frames, and the first resource information is pre-obtained based on matching target objects in N of the image frames; In response to triggering a first event during playback of the target video, based on at least one current image frame played by the playback page during the triggering of the first event, obtaining second resource information corresponding to a target object in the current image frame; and Display the second resource information, The second resource information is obtained by performing online recognition and search operations on the current image frame in response to the first event; The method further comprises: In response to a third predetermined operation on the resource page displaying the second resource information, displaying a progress page, wherein the progress page includes a progress bar control and an image display area, and the image display area displays the current image frame; In response to a fourth predetermined operation on the progress bar control, switching the image display area from displaying the current image frame to displaying a target image frame corresponding to the fourth predetermined operation; Obtaining fourth resource information corresponding to the target object in the target image frame; and displaying the fourth resource information; Wherein, N is an integer greater than 0, and M is an integer greater than or equal to N.

2. The method according to claim 1, wherein The playback page includes a first display area, and the first resource information is displayed in the first display area; the first resource information includes N first sub-resource information corresponding to the target objects in the N image frames respectively; Displaying first resource information corresponding to the target object in the target video on the playback page of the target video includes: When the N image frames are displayed respectively on the playback page, the N first sub-resource information are displayed respectively in the first display area.

3. The method according to claim 2, wherein: The N image frames include an i-th image frame and a j-th image frame, and the first resource information includes an i-th first sub-resource information corresponding to the i-th image frame and a j-th first sub-resource information corresponding to the j-th image frame; Displaying first resource information corresponding to the target object in the target video on the playback page of the target video includes: In the process of displaying the i-th image frame and the image frames between the i-th image frame and the j-th image frame on the playback page, the i-th first sub-resource information is displayed in the first display area, Wherein, i is an integer greater than 0, and j is an integer greater than i.

4. The method according to claim 3, wherein: The M image frames also include a p-th image frame located between the i-th image frame and the j-th image frame, The information display method further includes: During the process of displaying the i-th image frame and the image frames between the i-th image frame and the p-th image frame on the playback page, a first scanning graphic that changes accordingly as the image frames displayed on the playback page change is displayed in the first display area, where p is an integer greater than i and less than or equal to j.

5. The method according to claim 4, wherein Displaying a first scanning pattern in the first display area that changes as the image frame displayed on the playback page changes includes: As the image frame displayed on the play page changes, at least a portion of the first scanning pattern moves along a predetermined direction in the first display area.

6. The method according to claim 1, further comprising: During playback of the target video, displaying a first control on the playback page of the target video; Triggering a first event during the target video playback process includes: triggering a first predetermined operation for the first control during the target video playback process.

7. The method according to claim 1, wherein Triggering a first event during playback of the target video includes: In response to the first event, triggering a playback pause operation for the target video; or In response to the first event, a screenshot operation of the playback page of the target video is triggered.

8. The method according to claim 7, further comprising: In response to a screenshot operation of the play page of the target video, a second control and a third control are displayed, Among them, the second control is configured to trigger the operation of displaying the second resource information, and the third control is configured to trigger the operation of sharing the target video to the platform or user corresponding to the third control.

9. The method according to claim 1, further comprising: In the process of obtaining the second resource information, the current image frame is displayed on the play page, and a dynamic second scanning graphic is superimposed on the current image frame. The dynamic second scanning pattern includes a first sub-scanning pattern moving along a predetermined direction and / or a second sub-scanning pattern moving or flashing at a position where the target object is located in the current image frame.

10. The method according to claim 1, wherein Display the second resource information, including: Displaying a resource page, and displaying the second resource information on the resource page; The current image frame includes E target objects, and the second resource information includes a plurality of second sub-resource information corresponding to the E target objects respectively. The resource page includes a second display area and E fourth controls corresponding to the E target objects respectively, each of the fourth controls is configured to trigger an operation of displaying the second sub-resource information corresponding to the fourth control in the second display area, Wherein, E is an integer greater than 0.

11. The method according to claim 10, further comprising: In response to a second predetermined operation on the resource page, displaying a frame selection page, wherein the frame selection page displays the current image frame; In response to receiving a frame selection operation on the current image frame in the frame selection page, obtaining third resource information corresponding to a target object in the image area based on the image area defined by the frame selection operation; The third resource information is displayed.

12. The method according to claim 1, further comprising: In response to failure in performing an operation to obtain second resource information corresponding to the target object in the current image frame or failure to obtain the second resource information corresponding to the target object in the current image frame within a predetermined time period starting from triggering the first event, displaying a selection page, wherein the selection page displays the current image frame; In response to receiving a frame selection operation on the current image frame in the frame selection page, obtaining third resource information corresponding to a target object in the image area based on the image area defined by the frame selection operation; The third resource information is displayed.

13. A video-based information display device comprising: a first display unit configured to display, on a playback page of a target video during playback of the target video, first resource information corresponding to a target object in the target video, wherein the target video includes M image frames, and the first resource information is pre-obtained based on matching target objects in N of the image frames; The second display unit is configured to, in response to a first event being triggered during playback of the target video, obtain second resource information corresponding to the target object in the current image frame based on at least one current image frame played by the playback page during the triggering of the first event, and display the second resource information. The second resource information is obtained by performing online recognition and search operations on the current image frame in response to the first event; The device further includes a progress unit configured to: In response to a third predetermined operation on the resource page displaying the second resource information, displaying a progress page, wherein the progress page includes a progress bar control and an image display area, and the image display area displays the current image frame; In response to a fourth predetermined operation on the progress bar control, switching the image display area from displaying the current image frame to displaying a target image frame corresponding to the fourth predetermined operation; Obtaining fourth resource information corresponding to the target object in the target image frame; and displaying the fourth resource information; Wherein, N is an integer greater than 0, and M is an integer greater than or equal to N.

14. An electronic device comprising: processor; a memory comprising one or more computer program modules; The one or more computer program modules are stored in the memory and configured to be executed by the processor, and the one or more computer program modules include instructions for implementing the video-based information presentation method according to any one of claims 1 to 12.

15. A computer-readable storage medium for storing non-transitory computer-readable instructions, which, when executed by a computer, can implement the video-based information presentation method according to any one of claims 1 to 12. 16 . A computer program product, comprising a computer program carried on a non-transitory computer-readable medium, the computer program comprising program code for executing the video-based information presentation method according to claim 1 .

Images