Object recognition method and device, computer equipment and computer readable storage medium

Abstract

The invention relates to an object recognition method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: displaying a playing interface, wherein the playing interface is used for playing film and television works; in response to an object recognition trigger event for the film and television work, displaying the playing interface in an object recognition mode; and when the playing interface in the object identification mode displays a film and television picture of the film and television work and an identifiable object serving as a plot role in the film and television work exists in the film and television picture, displaying plot role information of at least one identifiable object in the film and television work. By adopting the method, the popularization degree of film and television works can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an object recognition method, apparatus, computer device, storage medium and computer program product. Background Art

[0002] With the development of computer technology, image recognition technology has emerged. Image recognition is an important field of artificial intelligence. It refers to the technology of using computers to process, analyze and understand images to identify targets and objects of various different patterns.

[0003] For example, when watching a TV series, users can take screenshots of the characters they are interested in and save them. After exiting the video application, the recognition software can be used to identify the relevant information of the characters in the screenshots. However, TV series are often long, such as movies that last for hours, TV series that have dozens of episodes, and each episode that lasts for dozens of minutes. There are also many characters in them. When users are interested in many characters, they may frequently switch between the TV series playback interface and the recognition software interface, making it easy for users to lose interest in the TV series they are watching, resulting in low promotion of the TV series. Summary of the invention

[0004] Based on this, it is necessary to provide an object recognition method, device, computer equipment, computer-readable storage medium and computer program product that can improve the promotion of film and television works in response to the above technical problems.

[0005] In a first aspect, the present application provides an object recognition method. The method comprises:

[0006] Displaying a playback interface, wherein the playback interface is used to play a film or TV work;

[0007] In response to an object recognition trigger event for the film and television work, displaying the playback interface in the object recognition mode;

[0008] When the playback interface in the object recognition mode displays a video screen of the film or television work, and there is a identifiable object as a plot character in the film or television work in the video screen, plot role information of at least one of the identifiable objects in the film or television work is displayed.

[0009] In a second aspect, the present application also provides an object recognition device. The device comprises:

[0010] An interface display module, used to display a playback interface, where the playback interface is used to play a film or TV work;

[0011] An object recognition trigger module, configured to display the playback interface in an object recognition mode in response to an object recognition trigger event for the film and television work;

[0012] The plot role display module is used to display the plot role information of at least one identifiable object in the film and television work when the playback interface in the object recognition mode displays the film and television work's film and television pictures and there are identifiable objects as plot characters in the film and television work in the film and television pictures.

[0013] In one embodiment, the object recognition trigger module is further used to display an object recognition interactive control on the playback interface; in response to an object recognition trigger event for the object recognition interactive control, display the playback interface in object recognition mode.

[0014] In one embodiment, the plot character display module is also used to display the film and television pictures of the film and television work in the playback interface in the object recognition mode; when there is a identifiable object that is a plot character in the film and television work in the film and television picture, in response to an object selection event for the identifiable object, the plot role information of the identifiable object selected by the object selection event in the film and television work is displayed.

[0015] In one embodiment, the apparatus further comprises:

[0016] The object detection module is used to perform object detection on the video picture to obtain identifiable objects in the video picture.

[0017] The role recognition module is used to perform role recognition on the identifiable object in the film and television screen to obtain the plot role played by the identifiable object.

[0018] The search module is used to search the film and television works to which the plot character belongs, and the plot character information corresponding to the plot character in the film and television works from a preset knowledge graph.

[0019] In one embodiment, the role recognition module is further used to extract features of the identifiable object to obtain facial features of the identifiable object; perform role recognition on the identifiable object based on the facial features to obtain the plot role played by the identifiable object.

[0020] In one embodiment, the apparatus further comprises:

[0021] The acquisition module is used to acquire each film and television clip of the film and television work, and the clip plot information corresponding to each of the film and television clips.

[0022] The determination module is used to determine, for each of the video clips, video frames containing identifiable objects in the video clip.

[0023] The generation module is used to generate plot role information corresponding to the identifiable object in the targeted film and television segment according to the film and television pictures in which the identifiable object exists and the corresponding segment plot information.

[0024] A construction module is used to construct a knowledge graph based on each of the film and television clips, the plot information of each of the clips, each of the identifiable objects and the corresponding plot character information to obtain a preset knowledge graph.

[0025] In one embodiment, the plot role display module is further used to display role association information between the plot roles played by the multiple identifiable objects in the film and television work when there are multiple identifiable objects in the film and television work.

[0026] In one embodiment, the plot role display module is also used to display multiple object categories and the film and television pictures of the film and television work in the playback interface in the object recognition mode; in response to a category selection event for the object category, when there is a identifiable object as a plot character in the film and television work in the film and television picture, and the identifiable object belongs to the object category selected by the category selection event, the plot role information of the identifiable object in the film and television work is displayed.

[0027] In one embodiment, the plot role display module is also used to display the film and television pictures of the film and television work in the playback interface in the object recognition mode; when there are multiple identifiable objects as plot characters in the film and television work in the film and television picture, display multiple object categories; in response to a category selection event for the object category, display the plot role information in the film and television work of the identifiable objects belonging to the object category selected by the category selection event.

[0028] In one embodiment, the plot character display module is further configured to cancel the display of the plot character information and exit the object recognition mode when the display duration of the plot character information reaches a preset duration.

[0029] In one embodiment, the film and television work includes multiple film and television clips, and the plot role information includes the clip role information of the plot role in each of the film and television clips; the device also includes a jump module, which is used to jump from the film and television screen to the film and television clip to which the clip role information belongs in response to a clip jump operation triggered by the clip role information of the plot role.

[0030] In one embodiment, the jump module is further used to exit the object recognition mode in response to jumping to the film and television segment to which the segment role information belongs, and play the film and television segment to which the segment role information belongs on the playback interface after exiting the object recognition mode.

[0031] In one embodiment, the plot character display module is also used to display object information of at least one identifiable object in the real world when the playback interface in the object recognition mode displays the film and television work's screen, and there are identifiable objects in the film and television work that are plot characters in the film and television work.

[0032] In one embodiment, the plot character display module is further used to display a plot character display option and an object identity display option when the playback interface in the object recognition mode displays the film and television work's film and television picture, and there is a identifiable object as a plot character in the film and television work in the film and television picture; in response to an option selection event for the plot character display option, display the plot role information of at least one of the identifiable objects in the film and television work; in response to an option selection event for the object identity display option, display the object information of at least one of the identifiable objects in the real world.

[0033] In one embodiment, the object information includes identity information of the identifiable object in the real world and the works in which the identifiable object has participated in history; the plot role display module is further used to display the identity information of the identifiable object in the real world when the playback interface in the object recognition mode displays a video screen of the film and television work, and there is a identifiable object as a plot role in the film and television work in the film and television screen;

[0034] The works in which the identifiable object has historically participated are classified and displayed according to preset work categories.

[0035] In one embodiment, the device also includes a jump module; the jump module is used to jump from the film and television screen to the playback interface of the work selected by the selected event in response to a selection event of the work in which the identifiable object has historically participated; and play the selected work on the playback interface of the work selected by the selected event.

[0036] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0037] Displaying a playback interface, wherein the playback interface is used to play a film or TV work;

[0038] In response to an object recognition trigger event for the film and television work, displaying the playback interface in the object recognition mode;

[0039] When the playback interface in the object recognition mode displays a video screen of the film or television work, and there is a identifiable object as a plot character in the film or television work in the video screen, plot role information of at least one of the identifiable objects in the film or television work is displayed.

[0040] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0041] Displaying a playback interface, wherein the playback interface is used to play a film or TV work;

[0042] In response to an object recognition trigger event for the film and television work, displaying the playback interface in the object recognition mode;

[0043] When the playback interface in the object recognition mode displays a video screen of the film or television work, and there is a identifiable object as a plot character in the film or television work in the video screen, plot role information of at least one of the identifiable objects in the film or television work is displayed.

[0044] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0045] Displaying a playback interface, wherein the playback interface is used to play a film or TV work;

[0046] In response to an object recognition trigger event for the film and television work, displaying the playback interface in the object recognition mode;

[0047] When the playback interface in the object recognition mode displays a video screen of the film or television work, and there is a identifiable object as a plot character in the film or television work in the video screen, plot role information of at least one of the identifiable objects in the film or television work is displayed.

[0048] The above-mentioned object recognition method, device, computer equipment, storage medium and computer program product, by displaying a playback interface, the displayed playback interface is used to play a film and television work, responds to an object recognition trigger event for the film and television work to enter the object recognition mode, and displays the playback interface in the object recognition mode, so that in the object recognition mode, it can be determined whether there is an identifiable object as a plot character in the film and television screen of the film and television work. If so, the plot role information of at least one identifiable object in the film and television work is displayed, and corresponding plot information prompts can be provided to the user, so that the user can quickly understand the relevant plot of the plot character in the film and television work, improve the user's drama chasing experience, and thus effectively improve the promotion of the film and television work. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 A diagram of an application environment of an object recognition method in an embodiment;

[0050] Figure 2 is a schematic flow chart of an object recognition method in one embodiment;

[0051] Figure 3 A schematic diagram of an interface for displaying plot role information of an identifiable object in a film or television work in one embodiment;

[0052] Figure 4 A schematic diagram of an interface for triggering an object recognition interactive control to display a playback interface in an object recognition mode in an embodiment;

[0053] Figure 5 A schematic diagram of an interface for displaying plot role information of a selected identifiable object in a film or television work in one embodiment;

[0054] Figure 6 A schematic diagram of a process of searching for corresponding plot character information of a plot character in a film or television work from a preset knowledge graph in one embodiment;

[0055] Figure 7 A schematic diagram of an interface for displaying plot role information of identifiable objects belonging to the character class in a film or television work in one embodiment;

[0056] Figure 8 A schematic diagram of an interface that can identify plot role information of an object in a film or television work and object information in the real world in one embodiment;

[0057] Fig. 9 A schematic diagram of an interface showing plot character information and object information of an identifiable object selected by an option selection event in one embodiment;

[0058] Fig.10 A schematic diagram of a construction process of a preset knowledge graph in one embodiment;

[0059] Fig.11 A schematic diagram of an interface showing relevant information of various plot characters in a film or TV series in one embodiment;

[0060] Fig.12 A schematic diagram of a process of identifying an identifiable object in one embodiment;

[0061] Fig.13 An architectural diagram for partitioning all models in an embodiment;

[0062] Fig.14 is a structural block diagram of an object recognition device in one embodiment;

[0063] Fig.15 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0064] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0065] The object recognition method provided in the embodiment of the present application can be applied to Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be set separately, can be integrated on the server 104, or can be placed on the cloud or other servers. The terminal 102 obtains the film and television work from the server 104. The terminal 102 displays the playback interface, and the playback interface is used to play the film and television work. In response to the object recognition trigger event for the film and television work, the terminal 102 displays the playback interface in the object recognition mode. When the playback interface in the object recognition mode displays the film and television screen of the film and television work, and there is a recognizable object as a plot role in the film and television work in the film and television screen, the terminal 102 obtains the plot role information of at least one recognizable object in the film and television work from the server 104, and the terminal 102 displays the plot role information of at least one recognizable object in the film and television work. Among them, the terminal 102 can be, but is not limited to, various display devices, such as desktop computers, laptops, smart phones, tablet computers, Internet of Things devices and portable wearable devices, and the Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart car devices, etc. The portable wearable device may be a smart watch, a smart bracelet, a head-mounted device, etc. The terminal 102 may run an application, which may be a communication application, an email application, a film and television work playback application, etc. The server 104 may be an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0066] In one embodiment, Figure 2 As shown, an object recognition method is provided, which can be executed by a computer device, and the computer device can be a terminal or a server. In the embodiment of the present application, the method is applied to Figure 1 The terminal in is used as an example to illustrate, including the following steps:

[0067] Step S202, displaying a play interface, where the play interface is used to play a film or TV work.

[0068] Among them, the playback interface is an interface used to play film and television works, and the playback interface can be a pop-up window, a floating layer or an independent interface.

[0069] Film and television works are videos formed by a series of pictures with or without sound. Film and television works can be completed works recorded by terminals and played by players, for example, videos recorded by cameras or mobile phone cameras. Film and television works can be complete works or parts of complete works. Film and television works can be divided into long videos, medium videos and short videos. Long videos, medium videos and short videos can be divided according to the playing time of the video, such as videos with a playing time greater than or equal to 60 minutes are long videos, videos with a playing time greater than 30 minutes and less than 60 minutes are medium videos, and videos with a playing time less than or equal to 30 minutes are short videos, etc. The specific settings can be made according to needs. Long videos, medium videos and short videos can all be TV series, movies, documentaries, etc. Medium videos and short videos can also be an episode of a TV series or a clip in an episode.

[0070] Specifically, a playback application with a function of playing a film or TV work may be run on the terminal, and a playback interface of the playback application may be entered. The user may play the film or TV work on the playback interface.

[0071] A playback application refers to an application that has the function of playing film and television works. The playback application can be presented to the user in the form of an application, and the user can play the film and television works through the application. The application can refer to a client installed in the terminal. An application can also refer to an installation-free application, that is, an application that can be used without downloading and installing. This type of application can also be called a mini-program. It usually runs as a sub-program in the client. The client is called the parent application, and the sub-program running in the client is called the sub-application. The application can also refer to a web application opened through a browser, etc. The playback application can specifically be a long video application, a short video application, a music application, a live broadcast application, etc., but is not limited to this.

[0072] The playback application can also be a cloud application, which refers to an application running in the cloud. A cloud application is an application that interacts between the terminal and the cloud. The operation mode of a cloud application is to use the powerful computing power of the cloud simulator to encode the running process into an audio and video stream, and transmit it to the terminal through the network to realize interaction with the user.

[0073] The cloud is a cloud server, also known as a cloud server. A cloud server is based on a large-scale distributed computing system that integrates computer resources through virtualization technology to provide Internet infrastructure services. The network that provides resources is called a "cloud". The resources in the "cloud" are infinitely expandable in the eyes of users, and can be obtained at any time, used on demand, expanded at any time, and paid for by use. Cloud computing is a computing model that distributes computing tasks on a resource pool composed of a large number of computers, enabling various application systems to obtain computing power, storage space and information services as needed.

[0074] Step S204, in response to an object recognition triggering event for a film or TV work, displaying a playback interface in an object recognition mode.

[0075] Among them, the object recognition trigger event is used to trigger the object recognition mode. The object recognition trigger event can specifically be a touch operation, a cursor operation, a key operation or a voice operation. The touch operation can be a touch click operation, a touch press operation or a touch slide operation, and the touch operation can be a single-point touch operation or a multi-point touch operation; the cursor operation can be an operation of controlling the cursor to click or an operation of controlling the cursor to press; the key operation can be a virtual key operation or a physical key operation, etc. The physical key operation can be a key on the interactive controller, and the interactive controller can control the visual elements in the playback interface displayed by the terminal. For example, the interactive controller can be a remote control of a TV, a mouse, etc.

[0076] The object recognition mode is a mode for recognizing plot role information of an identifiable object in a film or television work. In other embodiments, the object recognition mode is also used to recognize object information of an identifiable object in the real world.

[0077] Specifically, the terminal plays the video screen of the film and television work in the playback interface, and the user can trigger the object recognition mode of the film and television work. In response to the object recognition trigger event for the film and television work triggered by the user, the terminal enters the object recognition mode and displays the playback interface in the object recognition mode.

[0078] In this embodiment, the user may also first trigger the object recognition mode of the film or television work. The terminal responds to the object recognition trigger event for the film or television work, displays the playback interface in the object recognition mode, and displays the film or television screen of the film or television work in the playback interface in the object recognition mode.

[0079] It is understandable that there is no limit to the order between triggering the object recognition mode and playing the film and television work. The object recognition mode can be triggered during the playing of the film and television work or when the playing is paused, or the film and television work can be played after the object recognition mode is triggered.

[0080] Step S206, when the playback interface in the object recognition mode displays a video screen of a film or TV work, and there is a recognizable object as a plot character in the film or TV work in the video screen, display the plot character information of at least one recognizable object in the film or TV work.

[0081] Among them, the identifiable object is a visual element displayed in the playback interface and available for user operation. The identifiable object can be identified in the object recognition mode and can be operated by the user. The identifiable object can specifically be a person object, an animal object, a plant object, a tool object, etc., but is not limited to this. Person objects include various characters in film and television works, animal objects include various animals in film and television works, plant objects include various plants in film and television works, and tool objects include tools used by characters in film and television works.

[0082] A plot role refers to a specific object played by an actor in a film or television work, which can be a person, animal, plant, etc.

[0083] Plot character information refers to plot information related to a plot character in a film or television work. The plot information includes at least one of the following: description information of the plot character in the film or television work, time information of the plot character appearing in the film or television work, location information of the plot character in the film or television screen, and role association information between multiple plot characters.

[0084] The description information of the plot character in the film and television work includes at least one of the identity information of the plot character in the film and television work or the image of the plot character. The identity information includes at least one of the name, age, or body type of the plot character in the film and television work. The plot character information may include, for example, at least one of the following: the plot information corresponding to the plot character in each film and television clip of the film and television work, the plot information corresponding to each film and television screen where the plot character appears in the film and television clip, and the role association information between multiple plot characters existing in each film and television screen.

[0085] Specifically, the user plays a film or TV work in the playback interface in the object recognition mode to display the film or TV screen of the film or TV work. The terminal performs object detection on the displayed film or TV screen to detect whether there is an identifiable object as a plot character in the film or TV screen. When there is an identifiable object as a plot character in the film or TV screen, the terminal can obtain the plot character information corresponding to at least one identifiable object in the film or TV work. That is, if there is a plot character in the film or TV screen, the plot character information of the plot character in the film or TV work is displayed.

[0086] like Figure 3 As shown, a film or TV work is played in a playback interface 300 to display the film or TV work's screen. When there is a recognizable object 302 as a plot character in the film or TV screen, plot character information 304 of the at least one recognizable object 302 in the film or TV work is displayed.

[0087] In this embodiment, when there is a recognizable object as a plot character in the film or television picture, the plot character information of the recognizable object in the film or television work is displayed.

[0088] In this embodiment, when there are multiple identifiable objects as plot characters in the film or television picture, the plot character information of at least one identifiable object in the film or television work can be displayed. For example, if there are three plot characters in the film or television picture, the plot character information of one, two, or three plot characters can be displayed.

[0089] In one embodiment, when the playback interface in object recognition mode displays a film or television work, and there are identifiable objects in the film or television work that are plot characters in the film or television work, the playback interface in object recognition mode displays plot role information of at least one identifiable object in the film or television work.

[0090] In other embodiments, the plot role information of the identifiable object in the film and television work can also be displayed through a pop-up window, a floating layer, etc. It is also possible to jump to the information display interface to display the plot role information.

[0091] In this embodiment, in response to the display of the plot role information, the playing of the film and television work is temporarily stopped. Further, in response to the cancellation of the display of the plot role information, the playing of the film and television work continues.

[0092] In the above-mentioned object recognition method, a playback interface is displayed, and the displayed playback interface is used to play the film and television work. In response to the object recognition trigger event for the film and television work, the object recognition mode is entered, and the playback interface in the object recognition mode is displayed, so that in the object recognition mode, it can be judged whether there is a identifiable object as a plot character in the film and television screen of the film and television work. If so, the plot role information of at least one identifiable object in the film and television work is displayed, and the corresponding plot information can be provided to the user to prompt the user of the relevant plot of the currently appearing plot character in the film and television work, so as to facilitate the user to quickly understand the relevant information of the character in the film and television work, making the identification of the character more flexible, which is conducive to improving the user's drama chasing experience, thereby effectively improving the promotion of the film and television work.

[0093] In one embodiment, in response to an object recognition trigger event for a film or television work, displaying a playback interface in an object recognition mode includes:

[0094] An object recognition interactive control is displayed on the playback interface; and in response to an object recognition trigger event for the object recognition interactive control, a playback interface in an object recognition mode is displayed.

[0095] Among them, the object recognition interactive control is a visual element. Visual elements refer to elements that can be displayed and visible to human eyes and used to convey information. The control can be a button, a fill-in-the-blank box, a radio button, or an option group.

[0096] The object recognition trigger event is a preset operation acting on the object recognition interactive control. The object recognition trigger event can specifically be a touch operation, cursor operation, key operation or voice operation.

[0097] Specifically, the terminal displays a playback interface, and displays an object recognition interactive control on the playback interface. The user can trigger the object recognition interactive control to trigger the entry into the object recognition mode. The terminal enters the object recognition mode in response to the object recognition trigger event of the object recognition interactive control by the user, and displays the playback interface in the object recognition mode.

[0098] like Figure 4 As shown, an object recognition interactive control 402 is displayed on the playback interface 400. The object recognition interactive control 402 is presented on the playback interface 400 with the text "Who is Ta?". The user clicks the object recognition interactive control 402 to display the playback interface 400 in the object recognition mode.

[0099] In this embodiment, the terminal displays a playback interface, and plays the video screen of the film and television work on the playback interface; when the playback progress of the film and television work reaches the control display progress, the object recognition interaction control is displayed; in response to the object recognition trigger event for the object recognition interaction control, the playback interface in the object recognition mode is displayed. Furthermore, when the playback progress of the film and television work reaches the control hiding progress, the display of the object recognition interaction control is cancelled.

[0100] In this embodiment, when the playback progress of the film and television work reaches the control hiding progress, the display of the object recognition interactive control is cancelled, and the object recognition mode is automatically exited.

[0101] In one embodiment, in response to an object recognition cancellation event for the object recognition interactive control, the object recognition mode is exited. The user can cancel the object recognition mode by triggering the object recognition interactive control.

[0102] In other embodiments, when the display time of the object recognition interactive control reaches a preset time, the display of the object recognition interactive control is canceled.

[0103] In other embodiments, when the display duration of the plot character information reaches a preset duration, the display of the plot character information is canceled, the object recognition mode may be automatically exited, and the display of the object recognition interactive control may be canceled.

[0104] In this embodiment, the object recognition interactive control is displayed on the playback interface, which can provide the user with a functional control for entering the object recognition mode, and the user can freely choose whether to enter the object recognition mode. In response to the object recognition trigger event of the object recognition interactive control by the user, the playback interface in the object recognition mode is displayed to identify the plot characters in the film and television works, thereby obtaining the relevant plot information prompts of the plot characters in the film and television works, so as to facilitate the user to understand the plot information related to the plot characters.

[0105] In one embodiment, when a playback interface in an object recognition mode displays a video screen of a film or TV work, and there is a recognizable object as a plot character in the film or TV work in the video screen, displaying plot character information of at least one recognizable object in the film or TV work includes:

[0106] In the playback interface in the object recognition mode, the film and television screen of the film and television work is displayed; when there is a identifiable object that is a plot character in the film and television work in the film and television screen, in response to an object selection event for the identifiable object, the plot role information of the identifiable object selected by the object selection event in the film and television work is displayed.

[0107] Specifically, the terminal displays a playback interface in the object recognition mode, and displays the video screen of the film and television work in the playback interface in the object recognition mode. The terminal performs object detection and role recognition on the film and television screen to determine whether there is a recognizable object as a plot role in the film and television work in the film and television screen. In the case that there is a recognizable object as a plot role in the film and television work in the film and television screen, the user can select the recognizable object in the film and television screen, and the terminal responds to the object selection event of the recognizable object, determines the recognizable object selected by the object selection event, obtains the plot role information of the recognizable object in the film and television work, and thus displays the plot role information of the recognizable object.

[0108] In this embodiment, when there is a recognizable object in the film and television screen that is a plot character in the film and television work, the recognizable object in the film and television screen can be marked. The user selects the marked recognizable object, and the terminal responds to the object selection event of the recognizable object, determines the recognizable object selected by the object selection event, obtains the plot character information of the selected recognizable object in the film and television work, and displays the plot character information corresponding to the selected recognizable object.

[0109] like Figure 5 As shown, in the playback interface 400 in the object recognition mode, a video screen of a video work is displayed, and there are identifiable objects 404 as plot characters in the video screen. When the user selects one of the identifiable objects 404, the plot character information 406 of the selected identifiable object 404 in the video work is displayed.

[0110] In this embodiment, when the user selects one of the identifiable objects 404 , the selected identifiable object 404 is marked and displayed, and the plot role information 406 of the marked identifiable object 404 is displayed.

[0111] In this embodiment, the user can select multiple identifiable objects, and the terminal displays the plot role information of each selected identifiable object in response to the object selection event of the multiple identifiable objects.

[0112] In this embodiment, when there are multiple identifiable objects in a film or TV screen, the user can select a identifiable object to display the plot role information of the identifiable object; when the user selects another identifiable object in the film or TV screen, the displayed plot role information is hidden, and the plot role information of the currently selected identifiable object is displayed. For example, if the user selects identifiable object A in the film or TV screen, the plot role information of identifiable object A in the film or TV work is displayed; when the user selects another identifiable object B in the film or TV screen, the plot role information of identifiable object A is hidden, and the plot role information of identifiable object B is displayed.

[0113] In this embodiment, when the playback interface in the object recognition mode displays a video screen of a film or TV work, and there is a recognizable object as a plot character in the film or TV work in the film or TV work, the plot character information of at least one recognizable object in the film or TV work is displayed, including:

[0114] In the playback interface in the object recognition mode, the film and television pictures of the film and television work are displayed; in response to a selection event of a identifiable object in the film and television picture that is a plot character in the film and television work, the plot role information of the identifiable object selected by the selection event in the film and television work is displayed.

[0115] In this embodiment, the video screen of the film and television work is displayed in the playback interface in the object recognition mode. When there is a recognizable object as a plot character in the film and television work in the film and television work in the film and television work displayed in the object recognition mode, the user can freely select the recognizable object to be recognized, and based on the user's free selection, the plot character information of the recognizable object selected by the user in the film and television work is displayed, making the display of the plot character information more flexible.

[0116] In one embodiment, Figure 6 As shown, in the case where there is a recognizable object as a plot character in a film and television work in a film and television picture, in response to an object selection event for the recognizable object, before displaying the plot character information of the recognizable object selected by the object selection event in the film and television work, the method further includes:

[0117] Step S602: perform object detection on the video image to obtain identifiable objects in the video image.

[0118] Specifically, the terminal may perform object detection on the video picture to detect identifiable objects in the video picture.

[0119] Step S604, performing role recognition on the identifiable object in the film and television screen to obtain the plot role played by the identifiable object.

[0120] Specifically, after detecting the identifiable objects existing in the film and television screen, the terminal can perform role recognition on each identifiable object respectively to obtain the plot role played by each identifiable object.

[0121] Furthermore, the terminal can extract features of each identifiable object to obtain object features corresponding to each identifiable object. For each identifiable object, the terminal performs role recognition based on the object of the identifiable object to obtain the plot role played by the identifiable object in the film and television work.

[0122] The object features may include facial features of the identifiable object, body features, clothing features, etc., but are not limited thereto.

[0123] In other embodiments, the identifiable object in the film or television screen is subjected to role recognition to obtain the plot role played by the identifiable object, including:

[0124] The identifiable object is matched with each candidate object in the preset knowledge graph, and the plot role bound to the successfully matched candidate object is used as the plot role played by the identifiable object.

[0125] Furthermore, the similarity between the identifiable object and each candidate object can be calculated to obtain each similarity; the plot role bound to the candidate object that reaches the similarity threshold is obtained as the plot role played by the identifiable object.

[0126] Step S606, searching the preset knowledge graph for the film and television works to which the plot character belongs, as well as the plot character information corresponding to the plot character in the film and television works.

[0127] The preset knowledge graph is a pre-built graph containing the plot role information of each plot role in the film and television work, as well as the binding relationship between each plot role and each candidate object. The candidate object is the real object in the real world with the plot role with the binding relationship. For example, candidate object A in the real world plays plot role C in film and television work B.

[0128] In other embodiments, the preset knowledge graph may also include role association information between various plot characters.

[0129] In other embodiments, the preset knowledge graph also includes object information of each plot character in the real world.

[0130] Specifically, after the terminal identifies the plot role played by the identifiable object in the film or television work, it obtains a preset knowledge graph, searches the preset knowledge graph for the film or television work to which the plot role belongs, and obtains the plot role information corresponding to the plot role in the film or television work.

[0131] In this embodiment, object detection is performed on the film and television screen to obtain identifiable objects existing in the film and television screen, and role recognition is performed on the identifiable objects in the film and television screen to obtain the plot role played by the identifiable object, so that the film and television work to which the plot role belongs can be accurately found from the preset knowledge graph, thereby accurately finding the corresponding plot role information in the film and television work.

[0132] In one embodiment, character recognition is performed on a recognizable object in a film or TV screen to obtain a plot role played by the recognizable object, including:

[0133] Feature extraction is performed on the identifiable object to obtain facial features of the identifiable object; role recognition is performed on the identifiable object based on the facial features to obtain the plot role played by the identifiable object.

[0134] Specifically, the terminal extracts features from the identifiable object to obtain facial features of the identifiable object. Further, the terminal extracts features from the identifiable object to obtain facial features of different levels corresponding to the identifiable object.

[0135] The terminal performs role recognition on the identifiable object based on facial features or facial features at different levels to obtain the plot role played by the identifiable object.

[0136] In this embodiment, after extracting the facial features of the identifiable object, the terminal matches the facial features of the identifiable object with the facial features of each candidate object in the preset knowledge graph to obtain a candidate object corresponding to the identifiable object. A plot role having a binding relationship with the candidate object is obtained as the plot role played by the identifiable object in the film and television work.

[0137] In other embodiments, the identifiable object in the film or television screen is subjected to role recognition to obtain the plot role played by the identifiable object, including:

[0138] The identifiable object is matched with each candidate object in the preset knowledge graph, and the plot role in the film and television work bound to the successfully matched candidate object is used as the plot role played by the identifiable object in the film and television work.

[0139] In this embodiment, the same object plays different plot roles in different film and television works, but the facial features are the same or highly similar, so feature extraction is performed on the identifiable object to obtain the facial features of the identifiable object, so as to accurately perform role recognition on the identifiable object based on the facial features, and the plot role played by the identifiable object can be accurately identified.

[0140] In one embodiment, the preset knowledge graph is obtained through a construction step, and the construction step includes:

[0141] Obtain each film and television clip of the film and television work, as well as the corresponding clip plot information of each film and television clip; for each film and television clip, determine the film and television screen in which a identifiable object exists in the targeted film and television clip; based on the film and television screen in which the identifiable object exists in the targeted film and television clip and the corresponding clip plot information, generate the plot role information corresponding to the identifiable object in the targeted film and television clip; construct a knowledge graph based on each film and television clip, each clip plot information, each identifiable object and the corresponding plot role information to obtain a preset knowledge graph.

[0142] Specifically, when a film or TV work is uploaded to a playback application, the terminal obtains each film or TV clip of the film or TV work and the corresponding plot information of each film or TV clip. For each film or TV clip, the terminal can determine the film or TV screens in which the identifiable object exists in the film or TV clip, thereby selecting the film or TV screens in which the identifiable object exists in the film or TV work.

[0143] The terminal generates the plot character information corresponding to the identifiable object in the targeted film and television clip according to the film and television screen with the identifiable object in the targeted film and television clip and the corresponding clip plot information. The terminal constructs the binding relationship between each identifiable object and the corresponding plot character information, the binding relationship between each identifiable object and the plot character in the film and television work, the binding relationship between each plot character and the corresponding plot character information, the association relationship between each identifiable object and each film and television clip, and the association relationship between each film and television clip and the plot information of each clip. After completing the construction of these relationships, the preset knowledge graph can be obtained.

[0144] This embodiment also includes: constructing a role association relationship between various plot characters, which may specifically be a role association relationship between various plot characters in a film or television work, each film or television clip, or each film or television picture.

[0145] In other embodiments, the preset knowledge graph also includes the binding relationship between each plot character and each candidate object, and the object information corresponding to each candidate object in the real world.

[0146] In this embodiment, each film and television clip of the film and television work and the corresponding clip plot information of each film and television clip are obtained, and for each film and television clip, the film and television screens in which the identifiable object exists in the film and television clip are determined, so that the plot character information corresponding to the identifiable object in the film and television clip can be accurately generated based on the film and television screens in which the identifiable object exists in the film and television clip and the corresponding clip plot information. Based on each film and television clip, each clip plot information, each identifiable object and the corresponding plot character information, a preset knowledge graph for the film and television work can be constructed, so that the relevant plot information of each plot character can be accurately and quickly identified when watching the film and television work later.

[0147] In one embodiment, the method further comprises:

[0148] When there are multiple identifiable objects that are plot characters in a film or TV work in a film or TV screen, role association information between the plot characters that the multiple identifiable objects respectively play in the film or TV work is displayed.

[0149] Specifically, when the playback interface in object recognition mode displays a film or TV work screen, and there are multiple identifiable objects that are plot characters in the film or TV work in the film or TV work screen, the plot character information of one or more identifiable objects in the film or TV work can be displayed.

[0150] When there are multiple identifiable objects as plot characters in a film or TV work in a film or TV screen, the terminal can obtain role association information between the plot characters played by the multiple identifiable objects in the film or TV work, thereby displaying the role association information between the multiple plot characters. The role association information includes the role association relationship between the multiple plot characters in the film or TV work.

[0151] In this embodiment, when there are multiple identifiable objects as plot characters in a film or TV work in a film or TV screen, the role association information of the plot characters played by the multiple identifiable objects in the film or TV work in the film or TV screen is displayed. That is, the role association information of multiple ruthless characters appearing in the same film or TV screen in the film or TV screen. As the plot of the film or TV work progresses, the role association information between the multiple plot characters may change, and the current role association information may be displayed according to the currently displayed film or TV screen.

[0152] In this embodiment, when there are multiple identifiable objects that serve as plot characters in a film or television work in the film or television screen, role association information between the plot characters played by the multiple identifiable objects in the film or television work is displayed, thereby showing the association between the plot characters and providing users with a better viewing experience.

[0153] In one embodiment, when a playback interface in an object recognition mode displays a video screen of a film or TV work, and there is a recognizable object as a plot character in the film or TV work in the video screen, displaying plot character information of at least one recognizable object in the film or TV work includes:

[0154] A plurality of object categories and film and television pictures of a film and television work are displayed on a playback interface in an object recognition mode; in response to a category selection event for an object category, when there is a identifiable object that is a plot character in the film and television work in the film and television picture, and the identifiable object belongs to the object category selected by the category selection event, the plot role information of the identifiable object in the film and television work is displayed.

[0155] The object category refers to the category to which the recognizable objects in the film and television images belong. The various object categories may specifically include human category, animal category, plant category, consumable resource category, etc., but are not limited thereto.

[0156] Specifically, the terminal displays multiple object categories and the video screen of the film and television work in the playback interface in the object recognition mode. The user can select the multiple object categories displayed, and the terminal responds to the category selection event of the object category to determine whether there is a recognizable object as a plot character in the film and television work in the currently displayed film and television picture. When there is a recognizable object as a plot character in the film and television work in the film and television picture, the terminal can determine whether the existing recognizable object belongs to the object category selected by the user's category selection event, and if so, obtain the plot role information of the recognizable object in the film and television work.

[0157] like Figure 7 As shown, a plurality of object categories 702 are displayed in a playback interface 700 in recognition mode, including a person category, an animal category, and a plant category. The user clicks to select the person category as the selected category. When a film and television screen 704 is displayed in the playback interface 700 in recognition mode, identifiable objects 706 belonging to the person category in the film and television screen 704 are identified, and plot character information 708 corresponding to each identifiable object 706 in the film and television work is displayed.

[0158] In this embodiment, the terminal displays the video images of the film and television work in the playback interface in the object recognition mode, and can display multiple object categories in the process of displaying the video images of the film and television work.

[0159] In this embodiment, a plurality of object categories and film and television pictures of film and television works are displayed in the playback interface in the object recognition mode to provide a plurality of recognition methods distinguished by object categories, so that the selection methods are diversified. In response to the category selection event of the object category, when there is a recognizable object as a plot character in the film and television work in the film and television picture, and the recognizable object belongs to the object category selected by the category selection event, the plot role information of the recognizable object in the film and television work is displayed, so that different types of recognizable objects in the film and television picture can be selectively recognized by object category, making object recognition in the playback scene of the film and television work more flexible.

[0160] In one embodiment, when a playback interface in an object recognition mode displays a video screen of a film or TV work, and there is a recognizable object as a plot character in the film or TV work in the video screen, displaying plot character information of at least one recognizable object in the film or TV work includes:

[0161] In the playback interface in the object recognition mode, the film and television pictures of the film and television work are displayed; when there are multiple identifiable objects that are plot characters in the film and television work in the film and television pictures, multiple object categories are displayed; in response to a category selection event for the object category, the plot role information in the film and television work of the identifiable objects belonging to the object category selected by the category selection event is displayed.

[0162] Specifically, the terminal displays the video screen of the film or television work in the playback interface in the object recognition mode, performs object detection and object recognition on the video screen, and displays multiple object categories when there are multiple identifiable objects that are plot characters in the film or television work in the video screen.

[0163] The user can select from the multiple object categories displayed. The terminal responds to the category selection event of the object category, determines the identifiable objects in the film and television screen that belong to the object category selected in the category selection event, so as to obtain the plot role information of the identifiable object in the film and television work, and the terminal displays the plot role information of the identified object in the film and television work.

[0164] In this embodiment, in the playback interface in the object recognition mode, the video screen of the film and television work is displayed. When there are multiple identifiable objects as plot characters in the film and television work in the film and television screen, multiple object categories are displayed, so that the presence of multiple plot characters in the film and television screen can be used as a display condition for the object category, and multiple recognition options are provided for the user. In response to a category selection event for the object category, the plot role information in the film and television work of the identifiable objects belonging to the object category selected by the category selection event is displayed, so that the user can selectively identify different types of identifiable objects, making object recognition in the playback scene of the film and television work more flexible.

[0165] In one embodiment, the method further comprises:

[0166] When the display time of the plot character information reaches the preset time, the display of the plot character information is canceled and the object recognition mode is exited.

[0167] Specifically, the terminal displays the plot role information of the identifiable object in the film and television work, and detects the display time of the plot role information. When the display time of the plot role information reaches a preset time, the display of the plot role information is cancelled.

[0168] Furthermore, when the display time of the plot character information reaches a preset time, the display of the plot character information is cancelled and the object recognition mode is automatically exited.

[0169] In this embodiment, when the display time of the plot character information reaches a preset time, the display of the plot character information is canceled and the object recognition mode is exited, so that the display of the plot character information can be automatically canceled and the object recognition mode can be automatically exited.

[0170] In one embodiment, the film and television work includes a plurality of film and television segments, and the plot role information includes segment role information of the plot role in each film and television segment; the method further includes:

[0171] In response to a segment jump operation triggered by the segment role information of a plot role, the video screen is jumped to the video segment to which the segment role information belongs.

[0172] Specifically, a film and television work includes a plurality of film and television clips, and the plot role information of the identifiable object in the film and television work includes the clip role information corresponding to the plot role played by the identifiable object in each film and television clip.

[0173] The user can select from the displayed multiple segment role information to jump from the video segment to which the currently displayed video screen belongs to to the video segment to which the selected segment role information belongs, thereby playing the new video segment.

[0174] In one embodiment, the method further comprises:

[0175] In response to jumping to the film and television segment to which the segment role information belongs, the object recognition mode is exited, and the film and television segment to which the segment role information belongs is played on the playback interface of the object recognition mode exited.

[0176] Specifically, in response to the user's segment jump operation, the terminal jumps to the film and television segment to which the segment role information triggered by the user belongs for playback, and automatically exits the object recognition mode to play a new film and television segment.

[0177] In this embodiment, after jumping from the currently played video clip to another video clip, the object recognition mode is automatically exited, so that the original playback mode can be automatically restored based on the jump.

[0178] In one embodiment, the method further comprises:

[0179] When the playback interface in the object recognition mode displays a video screen of a film or TV work, and there are identifiable objects that are plot characters in the film or TV work in the video screen, object information of at least one identifiable object in the real world is displayed.

[0180] The object information includes the identity information of the identifiable object in the real world and the work information corresponding to the works in which the object has participated in history. The identity information includes at least one of name, age, or body shape. The work can be at least one of a film and television work, an audio work, or a literary work.

[0181] Specifically, the terminal displays a video screen of a film or TV work in a playback interface in an object recognition mode. If there is a recognizable object as a plot character in the film or TV work in the video screen, the terminal obtains object information of at least one recognizable object in the real world.

[0182] In this embodiment, when there is a recognizable object as a plot character in the film or television picture, the object information of the recognizable object in the real world is displayed.

[0183] In this embodiment, when there are multiple identifiable objects as plot characters in a film or TV screen, object information of at least one identifiable object in the real world may be displayed. For example, if there are three identifiable objects in a film or TV screen, plot character information of one, two, or three identifiable objects in the film or TV work may be displayed, and object information of one, two, or three identifiable objects in the real world may be displayed.

[0184] like Figure 8 As shown, a film or TV work is played in a playback interface 800 to display the film or TV screen of the film or TV work. When there is a recognizable object 802 as a plot character in the film or TV screen, plot character information 804 of the recognizable object 802 in the film or TV work and object information 806 of the recognizable object 802 in the real world are displayed.

[0185] In one embodiment, when the playback interface in object recognition mode displays a video screen of a film or TV work, and there are identifiable objects in the video screen that are plot characters in the film or TV work, the object information of at least one identifiable object in the real world is displayed in the playback interface in object recognition mode.

[0186] In other embodiments, the object information of the identifiable object in the real world may be displayed in a pop-up window, a floating layer, etc. The object information may also be displayed by jumping to an information display interface.

[0187] In this embodiment, when the playback interface in the object recognition mode displays a video screen of a film or TV work, and there is a recognizable object as a plot character in the film or TV work in the video screen, the plot character information of at least one recognizable object in the film or TV work and the object information of the at least one recognizable object in the real world are displayed in the preset display area of ​​the playback interface in the object recognition mode. Figure 8 As shown, plot character information 804 is displayed in the left area of ​​the identifiable object, and object information 806 is displayed in the right area of ​​the identifiable object 802.

[0188] In this embodiment, in response to the display of the plot role information and the object information, the playing of the film and television work is temporarily stopped. Further, in response to the cancellation of the display of the plot role information and the object information, the playing of the film and television work is continued.

[0189] In this embodiment, when the playback interface in the object recognition mode displays the film and television screen of a film and television work, and there is a identifiable object that is a plot character in the film and television work in the film and television screen, the plot character information of at least one identifiable object in the film and television work and the object information in the real world are displayed, so that viewers of the film and television work can understand both the relevant plot information of the identifiable object of interest in the play and the real information in the real world, which not only improves the viewer's drama-chasing experience, but also avoids the need for users to jump to other interfaces to search for relevant information of the real actors who play the plot roles.

[0190] In this embodiment, when the playback interface in the object recognition mode displays a video screen of a film or TV work, and there is a recognizable object as a plot character in the film or TV work in the video screen, before displaying object information of at least one recognizable object in the real world, the method further includes:

[0191] Perform object detection on film and television images to obtain identifiable objects in the film and television images; search for object information of identifiable objects in the real world from the preset knowledge graph.

[0192] In one embodiment, Fig. 9 As shown, when the playback interface in the object recognition mode displays a video screen of a film or TV work, and there is an identifiable object as a plot character in the film or TV work in the film or TV work, the plot character information of at least one identifiable object in the film or TV work is displayed, including steps S902 to S904:

[0193] Step S902: When the playback interface in the object recognition mode displays a video screen of a film or TV work, and there is an identifiable object that is a plot character in the film or TV work in the video screen, a plot character display option and an object information display option are displayed.

[0194] Among them, the plot character display option is an option used to trigger the display of plot character information, and the object information display option is an option for the user to trigger the display of real information in the real world.

[0195] Specifically, the terminal displays the video screen of the film and television work in the playback interface in the object recognition mode. In the case where there is an identifiable object as a plot character in the film and television work in the video screen, a plot character display option and an object information display option are displayed. The user can trigger at least one of the plot character display option and the object information display option to display corresponding information. Triggering the plot character display option displays the plot character information; triggering the object information display option displays the object information in the real world.

[0196] Step S904 , in response to an option selection event for a plot role display option, displaying plot role information of at least one identifiable object in a film or television work.

[0197] Specifically, when the user selects the plot character display option, the terminal responds to the option selection event of the plot character display option, identifies the identifiable objects existing in the displayed film and television screen, and obtains the plot character information of at least one identifiable object in the film and television work, thereby displaying the plot character information corresponding to the at least one identifiable object.

[0198] In this embodiment, in response to the display of the plot role information, the display of the plot role display options is canceled; in response to the cancellation of the display of the plot role information, the plot role display options are displayed again.

[0199] When the playback interface in the object recognition mode displays a video screen of a film or TV work, and there is a recognizable object as a plot character in the film or TV work in the video screen, displaying object information of at least one recognizable object in the real world includes step S906:

[0200] Step S906 , in response to an option selection event for the object information display option, displaying object information of at least one identifiable object in the real world.

[0201] Specifically, when the user selects the object information display option, the terminal responds to the option selection event of the object information display option, identifies the identifiable objects existing in the displayed film and television screen, and obtains the object information of at least one identifiable object in the real world, thereby displaying the object information corresponding to the at least one identifiable object.

[0202] In this embodiment, in response to display of the object information, the display of the object information display options is canceled; in response to canceling the display of the object information, the object information display options are displayed again.

[0203] In this embodiment, plot character display options and object information display options are provided for user selection, so that the user can freely choose to display plot character information or real information. In addition, the display of the two display options is conditional, that is, when the playback interface in the object recognition mode displays the film and television screen of the film and television work, and there is a recognizable object as a plot character in the film and television work in the film and television screen, it is displayed, which means that the option is displayed only when there is a recognizable object in the played film and television screen, making the display of the two display options more flexible. The user can trigger the display of at least one of the plot character information or object information by selecting at least one of the display options, making the display of the two information more diverse and flexible.

[0204] In one embodiment, the object information includes identity information of the identifiable object in the real world and the works in which the identifiable object has participated in history; when the playback interface in the object recognition mode displays a video screen of a film or TV work, and there is an identifiable object as a plot character in the film or TV work in the video screen, the object information of at least one identifiable object in the real world is displayed, including:

[0205] When the playback interface in object recognition mode displays a film or TV work screen, and there is an identifiable object that is a plot character in the film or TV work in the film or TV work screen, the identity information of the identifiable object in the real world is displayed; the works in which the identifiable object has historically participated are classified and displayed according to preset work categories.

[0206] Among them, the preset work categories may include film and television works, audio works and literary works. Film and television works can also be further divided into movies, TV series and variety shows.

[0207] Specifically, the terminal displays the video screen of the film and television work in the playback interface in the object recognition mode. If there is a recognizable object as a plot character in the film and television work in the video screen, the terminal can obtain and display the identity information of the recognizable object in the real world.

[0208] Furthermore, the terminal may determine the works that the identifiable object has historically participated in in the real world, and classify the works that the identifiable object has historically participated in according to preset work categories, so as to classify and display them according to the preset work categories. Figure 8 In the object information 806, the works in which the identifiable object 802 has participated are displayed separately according to film and television works and variety shows. The film and television works may include film and television 1 and film and television 2, and the variety show works include variety show 1 and variety show 2.

[0209] In this embodiment, when the playback interface in the object recognition mode displays a video screen of a film or TV work, and there is an identifiable object as a plot character in the film or TV work in the video screen, the works in which the identifiable object has historically participated are classified and displayed according to the preset work categories, and the works in which the identifiable object has historically participated can be subdivided according to the work categories, which is convenient for users to view and find corresponding works. In addition, the classified display makes the display effect of the works on the interface more neat and beautiful.

[0210] In one embodiment, the method further comprises:

[0211] In response to a selection event of a work in which the identifiable object has historically participated, the video screen is jumped to a playback interface of the work selected by the selected event; and the selected work is played on the playback interface of the work selected by the selected event.

[0212] Specifically, for the displayed works, the user can select the works of interest to display the works of interest to the user. In response to the user's selection event of the works in which the identifiable object has historically participated, the terminal determines the works selected by the selected event and jumps from the film and television screen to the playback interface of the works selected by the selected event. The terminal plays the works selected by the selected event on the playback interface.

[0213] In this embodiment, each work displayed is associated with a jump play link of the work. When a user triggers a work, the work can be jumped to a corresponding interface for playback through the jump play link.

[0214] In this embodiment, in response to a selection event of a work in which an identifiable object has historically participated, the playback interface jumps from the film and television screen to the playback interface of the work selected by the selected event, so that the user can jump to the interface of the work of interest to play the selected work without having to search separately, and the operation is simple and quick.

[0215] In one embodiment, an object recognition method is provided, which is applied to a computer device and includes:

[0216] Obtaining each film and television clip of the film and television work and the corresponding clip plot information of each film and television clip; for each film and television clip, determining the film and television screen in which the identifiable object exists in the film and television clip; generating the plot role information corresponding to the identifiable object in the film and television clip according to the film and television screen in which the identifiable object exists in the film and television clip and the corresponding clip plot information;

[0217] Obtain the object information of each identifiable object in the real world, construct a knowledge graph based on the object information, each film and television clip, the plot information of each clip, each identifiable object and the corresponding plot role information, and obtain a preset knowledge graph.

[0218] Display the playback interface, which is used to play movies and TV shows;

[0219] An object recognition interactive control is displayed on the playback interface, and in response to an object recognition trigger event for the object recognition interactive control, a playback interface in an object recognition mode is displayed.

[0220] In the playback interface in the object recognition mode, the video screen of the video work is displayed;

[0221] In response to an object selection event of a identifiable object in a film or television screen, role recognition is performed on the identifiable object selected by the object selection event to obtain the plot role played by the identifiable object; the film or television work to which the plot role belongs, as well as the corresponding plot role information in the film or television work and the object information of the identifiable object in the real world are searched from a preset knowledge graph; the plot role information in the film or television work and the object information in the real world of the identifiable object selected by the object selection event are displayed.

[0222] Furthermore, when the display time of the plot character information reaches a preset time, the display of the plot character information is cancelled and the object recognition mode is exited.

[0223] Furthermore, the film or television work includes multiple film or television clips, and the plot role information includes clip role information of the plot character in each film or television clip. In response to a clip jump operation triggered by the clip role information of the plot character, the video is jumped from the film or television screen to the film or television clip to which the clip role information belongs; in response to jumping to the film or television clip to which the clip role information belongs, the object recognition mode is exited, and the film or television clip to which the clip role information belongs is played.

[0224] Furthermore, the object information includes identity information of the identifiable object in the real world and work information of the works in which the identifiable object has participated historically, and the works in which the identifiable object has participated historically are classified and displayed according to preset work categories.

[0225] Furthermore, in response to a selection event of a work in which the identifiable object has historically participated, the video screen is jumped to a playback interface of the work selected by the selected event; and the selected work is played on the playback interface of the work selected by the selected event.

[0226] In one embodiment, an application scenario of an object recognition method is provided, which is specifically applied to a video playback scenario, which may be a TV series playback scenario, a movie playback scenario, a documentary playback scenario, and the like.

[0227] Taking the playing scenario of a TV series as an example, a "Who is he" function is set in the playing interface of the playing application. If the playing application has the permission to play the TV series, the third-party retrieval application and other TV series information in the playing application are called to collect the plot characters and the actors who play the plot characters in the TV series. The collected information is updated to the original knowledge graph to form a preset knowledge graph.

[0228] When the user opens the playback interface in the playback application, the "Who is he?" control can be displayed in the playback interface. When the user triggers the "Who is he?" control, the object recognition mode is entered. During the playback of the TV series, the user can click on the characters in the TV screen displayed by the TV series. The terminal determines the character boundary corresponding to the character clicked by the user, recognizes the character, and obtains the corresponding actor name.

[0229] Next, the preset knowledge graph is called to query the plot role information corresponding to the plot role played by the actor in the TV series from the preset knowledge graph, so that the "information about who he is in the play" can be displayed on the playback interface. The plot role information may include relevant plot information of the plot role in each episode of the TV series.

[0230] It is also possible to query the actor's real information from the preset knowledge graph, so that users can understand who he is in reality. This real information includes the film and television works, variety shows and other works that the actor has participated in, and these works are displayed by category on the playback interface.

[0231] The displayed plot character information and real information can be manually selected to be canceled by the user, and can be automatically canceled after being displayed for a preset time.

[0232] Furthermore, for the plot information of each episode of the plot character, the user can choose to click on one of the plots to jump to the episode corresponding to the plot. Furthermore, the episode can be played from the last moment when the plot character appears in the episode.

[0233] For the works in which the actors are shown, the user can click on any work to jump to the play interface of the work for playback.

[0234] In this embodiment, the terminal may include at least one of a mobile terminal, a PC terminal, or a TV terminal. On the mobile terminal and the PC terminal, the user triggers the "Who is He" control in the lower right corner of the play interface, and on the TV terminal, the user triggers the "Who is He" function by pressing a button on the remote control. The remote control button is, for example, the ↑ key on the remote control.

[0235] After being triggered, users on mobile and PC terminals select the person they wish to identify by clicking, and users on TV terminals select by pressing the "←" and "→" buttons on the remote control, ultimately determining a person for "Who is He?" identification.

[0236] In one embodiment, an object recognition method is provided, including the construction of a preset knowledge graph and the recognition of identifiable objects. Fig.10 As shown, including:

[0237] Knowledge graph construction:

[0238] like Fig.11 As shown in the figure, after a new TV series is released, the knowledge graph construction will be triggered. First, the TV series title request crawling interface will be used to crawl the search website for each character image in the play, the correspondence between the character and the actor name, the character introduction information and the episode plot information.

[0239] For the actors involved in the film and television dramas, the actor knowledge graph is updated, and the actor map, actor name, actor profile, and corresponding performance works, including film and television works, variety shows, etc., are collected from various playback platforms. The ratings of related film and television works are collected, and the works are sorted from high to low by the ratings. The top 5 film and television works and variety shows are collected respectively. Short videos related to the actor are collected within the playback platform of the film and television drama, and the top 5 short videos are collected by sorting from high to low by the number of views.

[0240] Human Recognition Unit:

[0241] The character recognition unit is used to identify actors appearing in the film and television screen. Specifically, for the film and television screen triggered by the user, MTCNN (Multi-Task Cascaded Convolutional Networks) is first used to detect the face area appearing in the current screen, and then ArcFace is used to identify the actor character in each face area.

[0242] MTCNN is a deep learning algorithm for face detection and facial key point detection. It divides face detection and facial key point detection into three stages through cascading: P-Net (Proposal Network), R-Net (Refine Network) and O-Net (Output Network). Among them, Proposal Network is used to generate candidate face frames; Refine Network is used to screen candidate face frames and remove unqualified frames; it is used to detect facial key points, and finally the position and confidence of the face frame are predicted through regression model and classification model.

[0243] The ArcFace algorithm is a deep learning algorithm for face recognition. The core of the ArcFace algorithm is a new loss function that adds a cosine similarity constraint to the traditional Softmax classifier, making faces of the same category closer in the feature space and the distance between faces of different categories more obvious. This can improve the accuracy and robustness of face recognition.

[0244] There are two scenarios in this embodiment where the character recognition unit is needed: one is to identify the characters that appear at different times in the TV series when a new TV series is released, and to build an inverted index, that is, to store the time when each actor appears; the other scenario is that after the user triggers the "Who is He" function, the actor appearing in the TV series will be identified by the character recognition unit.

[0245] Identify identifiable objects such as Fig.12 As shown in Figure 1, the user triggers the “Who is he?” function and uses SAM (SegmentAnythingModel) to segment the character boundaries. SAM is a single model, such as Fig.13 As shown in , both interactive and automatic segmentation can be performed. The trained SAM can return valid segmentation masks for any cue information, where the cue information can be foreground, background points, thick boxes or masks, free-form text, or any information that can indicate what to segment in the image. The SAM model can generate segmentation boundaries for any object in any image or video, such as Fig.13 Segmentation results 1, 2 and 3 are shown in FIG.

[0246] Each character in the film and television screen is obtained by character boundary segmentation. When the user selects a character to be identified, face detection and face recognition are performed on the character selected by the user to obtain the corresponding actor name.

[0247] The preset knowledge graph returns two pieces of information corresponding to the actor’s name: “Who is he in the play” and “Who is he in reality”.

[0248] The results of "Who is he in real life" are displayed around the actor himself. Based on the identified actor's identity information, the actor's information is queried in the knowledge graph, including: actor picture, actor name, actor profile, related film and television works, related variety shows, related short videos and other information. When the user further clicks on the actor picture, actor name, actor profile and other related areas, it will jump to the corresponding detail display interface; when the user clicks on the related film and television works, related variety shows, related short videos and other areas, it will jump to the corresponding playback interface.

[0249] The display results of "Who is he in the drama" revolve around the role played by the actor in the film and television drama. Based on the identified actor identity information and the title of the currently playing drama, the corresponding plot character information is queried in the preset knowledge graph, including: character map, character name, character introduction, and information such as finding the plot of the character in the previous episode based on the currently playing episode. When the user clicks on the previous episode feature area, based on the recognition results of the character recognition unit, all the moments when the actor appeared in the drama will be queried, and then based on the episode number of the currently playing film and television screen, the moment when the actor appeared in the previous episode will be queried, and finally the moment when the actor last appeared in the previous episode will be jumped to play.

[0250] In this embodiment, users usually watch TV series in a fragmented way, that is, users watch different clips or episodes of TV series at different times and occasions, rather than watching the entire series at one time. Users may interrupt or switch to other content at any time during the viewing process. This fragmented viewing method poses new challenges to video platforms. TV series are usually very long, movies are usually measured in hours, and TV series usually have dozens of episodes, each episode is dozens of minutes. TV series often involve multiple characters, and the relationships are complex, the conflicts are strong, and the stories are dense. The above-mentioned fragmented viewing method causes many users to forget the character information that appeared before when watching, which will affect the user's viewing experience, thereby causing the user to lose interest in watching TV series.

[0251] The object recognition method of this embodiment is a further expansion and innovation of the video function, that is, it provides a "Who is Ta" function, which includes two parts: "Who is Ta in the play" and "Who is Ta in reality". Among them, "Who is Ta in reality" further distinguishes between related film and television works, related variety shows, related short videos and other categories, so that users can selectively pay attention to the content they are interested in. "Who is Ta in the Play" revolves around the role played by the actor, including: character map, character name, character profile, the character's plot in the previous episode and other information, which makes it convenient for users to quickly understand the relevant information of the role and improve the user's experience of chasing dramas.

[0252] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0253] Based on the same inventive concept, the embodiment of the present application also provides an object recognition device for implementing the object recognition method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more object recognition device embodiments provided below can refer to the limitations on the object recognition method above, and will not be repeated here.

[0254] In one embodiment, Fig.14 As shown, an object recognition device 1400 is provided, including: an interface display module 1402, an object recognition trigger module 1404 and a plot role display module 1406, wherein:

[0255] The interface display module 1402 is used to display the play interface, and the play interface is used to play the film and television works.

[0256] The object recognition trigger module 1404 is used to display a playback interface in an object recognition mode in response to an object recognition trigger event for a film or TV work.

[0257] The plot role display module 1406 is used to display the plot role information of at least one identifiable object in the film and television work when the playback interface in the object recognition mode displays the film and television work's film and television picture and there is a identifiable object as a plot role in the film and television work in the film and television picture.

[0258] In this embodiment, a playback interface is displayed, and the displayed playback interface is used to play a film or television work. In response to an object recognition trigger event for the film or television work, an object recognition mode is entered, and the playback interface in the object recognition mode is displayed, so that in the object recognition mode, it can be determined whether there is a identifiable object as a plot character in the film or television screen of the film or television work. If so, the plot role information of at least one identifiable object in the film or television work is displayed, and the corresponding plot information can be provided to the user to prompt the user of the relevant plot of the currently appearing plot character in the film or television work, so as to facilitate the user to quickly understand the relevant information of the character in the film or television work and improve the user's drama chasing experience.

[0259] In one embodiment, the object recognition trigger module 1404 is further configured to display an object recognition interactive control on the playback interface; and in response to an object recognition trigger event for the object recognition interactive control, display a playback interface in an object recognition mode.

[0260] In this embodiment, the object recognition interactive control is displayed on the playback interface, which can provide the user with a functional control for entering the object recognition mode, and the user can freely choose whether to enter the object recognition mode.

[0261] In response to an object recognition trigger event of the object recognition interactive control by the user, a playback interface in object recognition mode is displayed to identify plot characters in a film or television work, thereby obtaining relevant plot information of the plot characters in the film or television work, making it easier for users to understand the plot information related to the plot characters.

[0262] In one embodiment, the plot role display module 1406 is also used to display the film and television pictures of the film and television work in the playback interface in the object recognition mode; when there is a identifiable object that is a plot role in the film and television work in the film and television picture, in response to an object selection event for the identifiable object, the plot role information of the identifiable object selected by the object selection event in the film and television work is displayed.

[0263] In this embodiment, the video screen of the film and television work is displayed in the playback interface in the object recognition mode. When there is a recognizable object as a plot character in the film and television work in the film and television work in the film and television work displayed in the object recognition mode, the user can freely select the recognizable object to be recognized, and based on the user's free selection, the plot character information of the recognizable object selected by the user in the film and television work is displayed, making the display of the plot character information more flexible.

[0264] In one embodiment, the apparatus further comprises:

[0265] The object detection module is used to perform object detection on the film and television images to obtain identifiable objects in the film and television images.

[0266] The role recognition module is used to perform role recognition on identifiable objects in film and television images to obtain the plot roles played by the identifiable objects.

[0267] The search module is used to search for the film and television works to which the plot characters belong, as well as the corresponding plot character information in the film and television works from the preset knowledge graph.

[0268] In this embodiment, object detection is performed on the film and television screen to obtain identifiable objects existing in the film and television screen, and role recognition is performed on the identifiable objects in the film and television screen to obtain the plot role played by the identifiable object, so that the film and television work to which the plot role belongs can be accurately found from the preset knowledge graph, thereby accurately finding the corresponding plot role information in the film and television work.

[0269] In one embodiment, the role recognition module is further used to extract features of the identifiable object to obtain facial features of the identifiable object; perform role recognition on the identifiable object based on the facial features to obtain the plot role played by the identifiable object.

[0270] In this embodiment, the same object plays different plot roles in different film and television works, but the facial features are the same or highly similar, so feature extraction is performed on the identifiable object to obtain the facial features of the identifiable object, so as to accurately perform role recognition on the identifiable object based on the facial features, and the plot role played by the identifiable object can be accurately identified.

[0271] In one embodiment, the apparatus further comprises:

[0272] The acquisition module is used to acquire each film and television clip of the film and television work, and the corresponding clip plot information of each film and television clip.

[0273] The determination module is used to determine, for each video clip, video frames in which identifiable objects exist in the video clip.

[0274] The generation module is used to generate plot role information corresponding to the identifiable object in the targeted film and television segment according to the film and television pictures in which the identifiable object exists and the corresponding segment plot information.

[0275] The construction module is used to construct a knowledge graph based on each film and television clip, the plot information of each clip, each identifiable object and the corresponding plot character information to obtain a preset knowledge graph.

[0276] In this embodiment, each film and television clip of the film and television work and the corresponding clip plot information of each film and television clip are obtained, and for each film and television clip, the film and television screens in which the identifiable object exists in the film and television clip are determined, so that the plot character information corresponding to the identifiable object in the film and television clip can be accurately generated based on the film and television screens in which the identifiable object exists in the film and television clip and the corresponding clip plot information. Based on each film and television clip, each clip plot information, each identifiable object and the corresponding plot character information, a preset knowledge graph for the film and television work can be constructed, so that the relevant plot information of each plot character can be accurately and quickly identified when watching the film and television work later.

[0277] In one embodiment, the plot role display module 1406 is also used to display the role association information between the plot roles played by the multiple identifiable objects in the film and television work when there are multiple identifiable objects as plot roles in the film and television work in the film and television screen.

[0278] In this embodiment, when there are multiple identifiable objects that serve as plot characters in a film or television work in the film or television screen, role association information between the plot characters played by the multiple identifiable objects in the film or television work is displayed, thereby showing the association between the plot characters and providing users with a better viewing experience.

[0279] In one embodiment, the plot role display module 1406 is also used to display multiple object categories and film and television pictures of film and television works in the playback interface in the object recognition mode; in response to a category selection event for the object category, when there is a identifiable object as a plot role in the film and television work in the film and television picture, and the identifiable object belongs to the object category selected by the category selection event, the plot role information of the identifiable object in the film and television work is displayed.

[0280] In this embodiment, a plurality of object categories and film and television pictures of film and television works are displayed in the playback interface in the object recognition mode to provide a plurality of recognition methods distinguished by object categories, so that the selection methods are diversified. In response to the category selection event of the object category, when there is a recognizable object as a plot character in the film and television work in the film and television picture, and the recognizable object belongs to the object category selected by the category selection event, the plot role information of the recognizable object in the film and television work is displayed, so that different types of recognizable objects in the film and television picture can be selectively recognized by object category, making object recognition in the playback scene of the film and television work more flexible.

[0281] In one embodiment, the plot role display module 1406 is also used to display the film and television pictures of the film and television work in the playback interface in the object recognition mode; when there are multiple identifiable objects as plot characters in the film and television work in the film and television picture, display multiple object categories; in response to a category selection event for the object category, display the plot role information in the film and television work of the identifiable objects belonging to the object category selected by the category selection event.

[0282] In this embodiment, in the playback interface in the object recognition mode, the video screen of the film and television work is displayed. When there are multiple identifiable objects as plot characters in the film and television work in the film and television screen, multiple object categories are displayed, so that the presence of multiple plot characters in the film and television screen can be used as a display condition for the object category, and multiple recognition options are provided for the user. In response to a category selection event for the object category, the plot role information in the film and television work of the identifiable objects belonging to the object category selected by the category selection event is displayed, so that the user can selectively identify different types of identifiable objects, making object recognition in the playback scene of the film and television work more flexible.

[0283] In one embodiment, the plot role display module 1406 is further configured to cancel the display of the plot role information and exit the object recognition mode when the display duration of the plot role information reaches a preset duration.

[0284] In this embodiment, when the display time of the plot character information reaches a preset time, the display of the plot character information is canceled and the object recognition mode is exited, so that the display of the plot character information can be automatically canceled and the object recognition mode can be automatically exited.

[0285] In one embodiment, the film and television work includes multiple film and television clips, and the plot role information includes clip role information of the plot role in each film and television clip; the device also includes a jump module, which is used to jump from the film and television screen to the film and television clip to which the clip role information belongs in response to a clip jump operation triggered by the clip role information of the plot role.

[0286] In this embodiment, the user can select from the displayed multiple segment role information to jump from the video segment to which the currently displayed video screen belongs to to the video segment to which the selected segment role information belongs, thereby playing the new video segment.

[0287] In one embodiment, the jump module is further used to exit the object recognition mode in response to jumping to the film and television segment to which the segment role information belongs, and play the film and television segment to which the segment role information belongs on the playback interface of the object recognition mode exit.

[0288] In this embodiment, after jumping from the currently played video clip to another video clip, the object recognition mode is automatically exited, so that the original playback mode can be automatically restored based on the jump.

[0289] In one embodiment, the plot character display module 1406 is also used to display object information of at least one identifiable object in the real world when the playback interface in the object recognition mode displays a film and television picture of a film and television work, and there are identifiable objects in the film and television picture that are plot characters in the film and television work.

[0290] In this embodiment, when the playback interface in the object recognition mode displays the film and television screen of a film and television work, and there are identifiable objects that are plot characters in the film and television work in the film and television screen, the plot character information of at least one identifiable object in the film and television work and the object information in the real world are displayed, so that viewers of the film and television work can understand the relevant plot information of the identifiable objects of interest in the play, and understand the real information in the real world, thereby improving the viewer's experience of chasing the drama.

[0291] In one embodiment, the plot role display module 1406 is also used to display a plot role display option and an object identity display option when the playback interface in the object recognition mode displays a film and television work screen, and there is a identifiable object as a plot role in the film and television work in the film and television screen; in response to an option selection event for the plot role display option, display the plot role information of at least one identifiable object in the film and television work; in response to an option selection event for the object identity display option, display the object information of at least one identifiable object in the real world.

[0292] In this embodiment, plot character display options and object information display options are provided for user selection, so that the user can freely choose to display plot character information or real information. In addition, the display of the two display options is conditional, that is, when the playback interface in the object recognition mode displays the film and television screen of the film and television work, and there is a recognizable object as a plot character in the film and television work in the film and television screen, it is displayed, which means that the option is displayed only when there is a recognizable object in the played film and television screen, making the display of the two display options more flexible. The user can trigger the display of at least one of the plot character information or object information by selecting at least one of the display options, making the display of the two information more diverse and flexible.

[0293] In one embodiment, the object information includes the identity information of the identifiable object in the real world and the works in which the identifiable object has participated historically; the plot role display module 1406 is also used to display the identity information of the identifiable object in the real world when the playback interface in the object recognition mode displays the film and television screen of the film and television work, and there is a identifiable object as a plot character in the film and television work in the film and television screen; the works in which the identifiable object has participated historically are classified and displayed according to preset work categories.

[0294] In this embodiment, when the playback interface in the object recognition mode displays a video screen of a film or TV work, and there is an identifiable object as a plot character in the film or TV work in the video screen, the works in which the identifiable object has historically participated are classified and displayed according to the preset work categories, and the works in which the identifiable object has historically participated can be subdivided according to the work categories, which is convenient for users to view and find corresponding works. In addition, the classified display makes the display effect of the works on the interface more neat and beautiful.

[0295] In one embodiment, the device also includes a jump module; the jump module is used to jump from the film and television screen to the playback interface of the work selected by the selected event in response to the selection event of the work in which the identifiable object has historically participated; and play the selected work on the playback interface of the work selected by the selected event.

[0296] In this embodiment, in response to a selection event of a work in which an identifiable object has historically participated, the playback interface jumps from the film and television screen to the playback interface of the work selected by the selected event, so that the user can jump to the interface of the work of interest to play the selected work without having to search separately, and the operation is simple and quick.

[0297] Each module in the above object recognition device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0298] In one embodiment, a computer device is provided, which may be a terminal or a server. Taking a terminal as an example, its internal structure diagram may be as follows: Fig.15As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface, the display unit and the input device are connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, an object recognition method is implemented. The display unit of the computer device is used to form a visually visible image, and can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the computer device casing, or an external keyboard, touchpad or mouse, etc.

[0299] Those skilled in the art will understand that Fig.15 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0300] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0301] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0302] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0303] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0304] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited thereto. The processors involved in each embodiment provided in this application may be general-purpose processors, central processing units, graphics processors, digital signal processors, programmable logic devices, data processing logic devices based on quantum computing, etc., but are not limited thereto.

[0305] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0306] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An object recognition method, characterized in that: The method comprises: Displaying a playback interface, wherein the playback interface is used to play a film or TV work; In response to an object recognition trigger event for the film and television work, displaying the playback interface in the object recognition mode; When the playback interface in the object recognition mode displays a video screen of the film or television work, and there is a identifiable object as a plot character in the film or television work in the video screen, plot role information of at least one of the identifiable objects in the film or television work is displayed.

2. The method according to claim 1, characterized in that The step of displaying the playback interface in the object recognition mode in response to the object recognition triggering event for the film and television work includes: Displaying an object recognition interactive control on the playback interface; In response to an object recognition trigger event for the object recognition interactive control, the playback interface in the object recognition mode is displayed.

3. The method according to claim 1, characterized in that When the playback interface in the object recognition mode displays a video screen of the video work, and there is a recognizable object as a plot character in the video work in the video screen, displaying plot character information of at least one of the recognizable objects in the video work includes: In the playback interface in the object recognition mode, displaying the video screen of the video work; In the case where there is a recognizable object as a plot character in the film and television work in the film and television picture, in response to an object selection event for the recognizable object, plot character information in the film and television work of the recognizable object selected by the object selection event is displayed.

4. The method according to claim 3, characterized in that: In the case where there is a recognizable object as a plot character in the film and television work in the film and television picture, in response to an object selection event for the recognizable object, before displaying the plot character information of the recognizable object selected by the object selection event in the film and television work, the method further includes: Performing object detection on the video picture to obtain identifiable objects in the video picture; Performing role recognition on the identifiable object in the film and television screen to obtain the plot role played by the identifiable object; Search the film and television work to which the plot character belongs, and the plot character information corresponding to the plot character in the film and television work from the preset knowledge graph.

5. The method according to claim 4, characterized in that The step of performing role recognition on the identifiable object in the film and television screen to obtain the plot role played by the identifiable object includes: Extracting features of the identifiable object to obtain facial features of the identifiable object; The identifiable object is subjected to role recognition based on the facial features to obtain the plot role played by the identifiable object.

6. The method according to claim 4, characterized in that The preset knowledge graph is obtained through a construction step, and the construction step includes: Obtaining each film and television clip of the film and television work, and the clip plot information corresponding to each of the film and television clips; For each of the video clips, determining a video frame in the video clip in which a recognizable object exists; According to the video images with identifiable objects in the targeted video clips and the corresponding clip plot information, generating the plot role information corresponding to the identifiable objects in the targeted video clips; A knowledge graph is constructed based on each of the film and television clips, the plot information of each of the clips, each of the identifiable objects and the corresponding plot character information to obtain a preset knowledge graph.

7. The method according to claim 3, characterized in that The method further comprises: When there are a plurality of identifiable objects that are plot characters in the film and television work in the film and television picture, role association information between the plot characters respectively played by the plurality of identifiable objects in the film and television work is displayed.

8. The method according to claim 1, characterized in that When the playback interface in the object recognition mode displays a video screen of the video work, and there is a recognizable object as a plot character in the video work in the video screen, displaying plot character information of at least one of the recognizable objects in the video work includes: Displaying multiple object categories and video images of the video work on the playback interface in the object recognition mode; In response to a category selection event for the object category, when there is a identifiable object that is a plot character in the film or television work in the film or television screen, and the identifiable object belongs to the object category selected by the category selection event, the plot role information of the identifiable object in the film or television work is displayed.

9. The method according to claim 1, characterized in that: When the playback interface in the object recognition mode displays a video screen of the video work, and there is a recognizable object as a plot character in the video work in the video screen, displaying plot character information of at least one of the recognizable objects in the video work includes: In the playback interface in the object recognition mode, displaying the video screen of the video work; When there are multiple identifiable objects in the film or television work that are characters in the plot of the film or television work, multiple object categories are displayed; In response to a category selection event for the object category, plot role information of the identifiable object belonging to the object category selected by the category selection event in the film and television work is displayed.

10. The method according to claim 1, characterized in that The method further comprises: When the display time of the plot character information reaches a preset time, the display of the plot character information is canceled, and the object recognition mode is exited.

11. The method according to claim 1, characterized in that: The film and television work includes a plurality of film and television clips, and the plot role information includes the clip role information of the plot role in each of the film and television clips; the method further includes: In response to a segment jump operation triggered by the segment role information of the plot role, the video screen is jumped to the video segment to which the segment role information belongs.

12. The method according to claim 11, characterized in that The method further comprises: In response to jumping to the film and television segment to which the segment role information belongs, the object recognition mode is exited, and the film and television segment to which the segment role information belongs is played on the playback interface after exiting the object recognition mode.

13. The method according to any one of claims 1 to 12, characterized in that: The method further comprises: When the playback interface in the object recognition mode displays a video screen of the film or television work, and there are identifiable objects that are plot characters in the film or television work in the video screen, object information of at least one of the identifiable objects in the real world is displayed.

14. The method according to claim 13, characterized in that When the playback interface in the object recognition mode displays a video screen of the video work, and there is a recognizable object as a plot character in the video work in the video screen, displaying plot character information of at least one of the recognizable objects in the video work includes: When the playback interface in the object recognition mode displays a video screen of the video work, and there is an identifiable object as a plot character in the video work in the video screen, a plot character display option and an object information display option are displayed; In response to an option selection event for the plot role display option, displaying plot role information of at least one of the identifiable objects in the film and television work; When the playback interface in the object recognition mode displays a video screen of the video work, and there is a recognizable object as a plot character in the video work in the video screen, displaying object information of at least one of the recognizable objects in the real world includes: In response to an option selection event for the object information display option, object information of at least one of the identifiable objects in the real world is displayed.

15. The method according to claim 13, characterized in that The object information includes identity information of the identifiable object in the real world and the works in which the identifiable object has participated in history; when the playback interface in the object recognition mode displays a video screen of the film or television work, and there is an identifiable object as a plot character in the film or television work in the film or television work, displaying object information of at least one of the identifiable objects in the real world includes: When the playback interface in the object recognition mode displays a video screen of the video work, and there is a recognizable object as a plot character in the video work in the video screen, displaying identity information of the recognizable object in the real world; The works in which the identifiable object has historically participated are classified and displayed according to preset work categories.

16. The method according to claim 15, characterized in that The method further comprises: In response to a selection event of a work in which the identifiable object has historically participated, jumping from the film and television screen to a playback interface of the work selected by the selection event; In the playback interface of the work selected by the selected event, the selected work is played.

17. An object recognition device, characterized in that: The device comprises: An interface display module, used to display a playback interface, where the playback interface is used to play a film or TV work; An object recognition trigger module, configured to display the playback interface in an object recognition mode in response to an object recognition trigger event for the film and television work; The plot role display module is used to display the plot role information of at least one identifiable object in the film and television work when the playback interface in the object recognition mode displays the film and television work's film and television pictures and there are identifiable objects as plot characters in the film and television work in the film and television pictures.

18. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 16 are implemented.

19. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 16 are implemented.

20. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 16 are implemented.

Images