Stroma source identification method, apparatus and device, readable storage medium and program product

Abstract

The invention relates to a plot source identification method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring a plot picture, wherein the plot picture is used for searching plot source information of the plot picture; performing face extraction on the plot picture to obtain a participated role face image, and performing face recognition on the participated role face image to obtain identity information of the participated role in the plot picture; performing scene recognition on the plot picture to obtain plot scene information; and searching a plot source database according to the identity information of the participating roles in the plot picture and the plot scene information to obtain plot source information corresponding to the plot picture. By adopting the method, the plot source information can be quickly and accurately identified.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a plot source identification method and device, computer equipment, computer readable storage medium and computer program product. BACKGROUND

[0002] With the development of mobile networks, watching videos through mobile terminals has become a part of people's daily life. During the video watching process, users have the need to search for video source information. For example, when a user brushes a video clip on a video platform and wants to watch the complete video, but does not know the video name or is not familiar with the actor's name, the user needs to search for the required plot source information. For another example, when a user wants to further understand the director, cast, etc. of the video while watching the video, but does not know the video name or is not familiar with the actor's name, the user needs to search for the required plot source information.

[0003] At present, only text search is supported, such as searching for a TV series according to the TV series name and actor name, etc. Obviously, this way cannot solve the above-mentioned problems faced by users, therefore, how to quickly and accurately identify plot source information is a problem to be solved. SUMMARY

[0004] Therefore, it is necessary to provide a plot source identification method and device, computer equipment, computer readable storage medium and computer program product, which can quickly and accurately identify plot source information, in view of the above technical problems.

[0005] In a first aspect, the present application provides a plot source identification method, comprising:

[0006] obtaining a plot picture, the plot picture being used to search for plot source information of the plot picture;

[0007] performing face extraction on the plot picture to obtain a cast face image, performing face recognition on the cast face image to obtain identity information of a cast in the plot picture;

[0008] performing scene recognition on the plot picture to obtain plot scene information;

[0009] searching for a plot source database according to the identity information of the cast in the plot picture and the plot scene information to obtain plot source information corresponding to the plot picture.

[0010] In a second aspect, the present application further provides a plot source identification device, comprising:

[0011] an obtaining module configured to obtain a plot picture, the plot picture being used to search for plot source information of the plot picture;

[0012] The actor identification module is configured to perform face extraction on the plot picture to obtain actor face images, and perform face recognition on the actor face images to obtain identity information of actors in the plot picture.

[0013] The scene identification module is configured to perform scene recognition on the plot picture to obtain plot scene information.

[0014] The search module is configured to search a plot source database according to the identity information of actors in the plot picture and the plot scene information to obtain plot source information corresponding to the plot picture.

[0015] In an embodiment, the acquisition module is further configured to receive a screenshot instruction for a video playing interface, perform screenshot on the video playing interface to obtain a video screenshot, and take the video screenshot as a plot picture used for searching plot source information of a movie or television series in response to a plot source information identification instruction triggered by the video screenshot.

[0016] In an embodiment, the actor identification module is further configured to input the plot picture into an image segmentation network model, obtain semantic segmentation results about the plot picture through the image segmentation network model, and extract actor face images from the plot picture according to the semantic segmentation results.

[0017] In an embodiment, the actor identification module is further configured to compare the actor face images with actor face images stored in an actor face image library respectively to obtain comparison results, and obtain identity information of actors in the plot picture according to the comparison results.

[0018] In an embodiment, the actor identification module is further configured to extract first face features of the actor face images and second face features of the actor face images stored in the actor face image library through a face feature extraction model, calculate similarities between the first face features and the corresponding second face features of each of the actor face images, and determine comparison results according to the similarities.

[0019] In an embodiment, the actor face image library stores actor identity information and face features of actor face images corresponding to the actor identity information, the face features are obtained by feature extraction through a face feature extraction model, the actor identification module is further configured to extract face features of the actor face images through the face feature extraction model, calculate similarities between the face features of the actor face images and face features of each of the actor face images stored in the actor face image library respectively, and determine comparison results according to the similarities.

[0020] In an embodiment, the scene recognition module is further configured to input the plot picture into a trained scene recognition model; and identify plot scene information corresponding to the plot picture through the scene recognition model, the plot scene information including at least one of a plot type, a plot style, a scene location, and a scene era background.

[0021] In an embodiment, the retrieval module is further configured to retrieve, according to identity information of a participating role in the plot picture, participating personnel corresponding to each plot name in a plot source database, and take a plot name including the participating personnel and the identity information as a candidate plot name; obtain plot source information of the candidate plot name from the plot source database; and select, from the candidate plot name, a target plot name with corresponding plot source information matching the plot scene information, and take plot source information of the target plot name as plot source information corresponding to the plot picture.

[0022] In an embodiment, the apparatus further includes:

[0023] A plot source database construction module is configured to obtain plot source information corresponding to a film or television plot, the plot source information including at least a plot name; and construct a plot source database according to the plot source information, the plot source information further including at least one of participating personnel, a plot introduction, a plot type, a plot style, a scene location, and a scene era background.

[0024] In an embodiment, the apparatus further includes a jump module configured to, after obtaining plot source information corresponding to the plot picture, display, in a current video playing interface, the plot source information and a plot source jump control, the plot source information including a plot name; in response to a triggering operation on the plot source jump control, jump to a video playing client and display a playing entrance of a target video corresponding to the plot name in the plot source information; and in response to a triggering operation on the playing entrance, play the target video in the video playing client.

[0025] In an embodiment, the retrieval module is further configured to, when a face extraction on the plot picture fails to obtain a participating role face image, retrieve plot source information in the plot source database according to plot scene information corresponding to the plot picture; take a plot name corresponding to plot source information matching the plot scene information as a candidate plot name; obtain a search popularity of each candidate plot name within a preset time period; select, from the candidate plot name, a target plot name with a search popularity greater than a set threshold; and take plot source information of the target plot name as plot source information corresponding to the plot picture.

[0026] In a third aspect, the present application also provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0027] obtaining a plot picture, wherein the plot picture is used to search plot source information of the plot picture;

[0028] performing face extraction on the plot picture to obtain a face image of a participating actor, performing face recognition on the face image of the participating actor to obtain identity information of the participating actor in the plot picture;

[0029] performing scene recognition on the plot picture to obtain plot scene information;

[0030] retrieving a plot source database according to the identity information of the participating actor in the plot picture and the plot scene information to obtain plot source information corresponding to the plot picture.

[0031] In a fourth aspect, the present application also provides a computer readable storage medium, which stores a computer program, wherein the computer program is executed by a processor to implement the following steps:

[0032] obtaining a plot picture, wherein the plot picture is used to search plot source information of the plot picture;

[0033] performing face extraction on the plot picture to obtain a face image of a participating actor, performing face recognition on the face image of the participating actor to obtain identity information of the participating actor in the plot picture;

[0034] performing scene recognition on the plot picture to obtain plot scene information;

[0035] retrieving a plot source database according to the identity information of the participating actor in the plot picture and the plot scene information to obtain plot source information corresponding to the plot picture.

[0036] In a fifth aspect, the present application also provides a computer program product comprising a computer program, wherein the computer program is executed by a processor to implement the following steps:

[0037] obtaining a plot picture, wherein the plot picture is used to search plot source information of the plot picture;

[0038] performing face extraction on the plot picture to obtain a face image of a participating actor, performing face recognition on the face image of the participating actor to obtain identity information of the participating actor in the plot picture;

[0039] performing scene recognition on the plot picture to obtain plot scene information;

[0040] According to the identity information of the participating role in the plot picture and the plot scene information, a plot source database is retrieved to obtain plot source information corresponding to the plot picture.

[0041] The plot source identification method, device, computer device, computer readable storage medium, and computer program product described above obtain a plot picture used to search for plot source information, obtain a participating role face image by performing face extraction on the plot picture, obtain identity information of the participating role in the plot picture by performing face recognition on the participating role face image, and obtain plot source information corresponding to the plot picture by retrieving a plot source database according to the identity information of the participating role in the plot picture and plot scene information obtained by performing scene recognition on the plot picture. Since the identity information of the participating role and the plot scene information are combined, the accuracy of plot source screening can be improved, and thus the accuracy of plot source information identification can be improved.

[0042] In general, the identification strategy that combines the identity information of the participating role and the plot scene information can ensure that the identification process is not too complex, the data volume is not too large, and the identification efficiency is relatively high. The matching of the identity information of the participating role is based on the matching of the participating role face information, which can ensure the accuracy of the identification result. BRIEF DESCRIPTION OF DRAWINGS

[0043] To more clearly illustrate the technical solutions in the embodiments of the present application or the related art, the following will briefly introduce the drawings needed to be used in the embodiments of the present application or the related art description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other related drawings can also be obtained without creative labor based on these drawings.

[0044] Figure 1 A flowchart of a process of retrieving a video based on text in the related art;

[0045] Figure 2 An application environment diagram of the plot source identification method in an embodiment;

[0046] Figure 3 A flowchart of a process of the plot source identification method in an embodiment;

[0047] Figure 4A A schematic diagram of a plot picture in an embodiment;

[0048] Figure 4BFig. 2 is a schematic diagram of a plot picture in another embodiment;

[0049] Figure 5 Fig. 3 is a schematic diagram of a step of retrieving plot source information from a plot source database in an embodiment;

[0050] Figure 6 Fig. 4 is a schematic diagram of a detailed flow of a plot source identification method in an embodiment;

[0051] Figure 7 Fig. 5 is a schematic diagram of a flow of interaction between a terminal and a server in an embodiment;

[0052] Figure 8 Fig. 6 is a schematic diagram of a structure of a convolutional neural network in an embodiment;

[0053] Figure 9 Fig. 7 is a schematic diagram of an overall architecture of a BiSenet network in an embodiment;

[0054] Figure 10 Fig. 8 is a detailed architecture diagram of a BiSenet network in an embodiment;

[0055] Figure 11 Fig. 9 is a schematic diagram of a structure of an ARM in an embodiment;

[0056] Figure 12 Fig. 10 is a schematic diagram of a feature fusion module in an embodiment;

[0057] Figure 13 Fig. 11 is a structural block diagram of a plot source identification apparatus in an embodiment;

[0058] Figure 14 Fig. 12 is an internal structure diagram of a computer device in an embodiment;

[0059] Figure 15 Fig. 13 is an internal structure diagram of a computer device in another embodiment. DETAILED DESCRIPTION

[0060] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0061] In related art, only text retrieval is supported, for example, a television drama playing source link is retrieved according to a television drama name and an actor name, etc. When text retrieval is used, a database of a server generally stores information such as a drama name and a cast, and accurate query matching can be performed, for example, a television drama name is input, and a corresponding television drama playing source link is returned. Figure 1As shown, it is a flow diagram of video retrieval based on text in the related art. However, when the user brushes a video, but does not know the video name or is not familiar with the actor name, the text retrieval cannot be directly performed.

[0062] In some other related art, the retrieval strategy based on image recognition needs to store the plot pictures or actor photos in the movie and television dramas, and then compare the current plot picture with the stored plot pictures or actor photos one by one, that is, search the pictures in all videos, and compare the search pictures with the current plot picture frame by frame, so as to identify the plot source information of the movie and television drama from which the current plot picture is derived. However, the plot pictures or actor photos stored in the database generally contain plot information, and if a large number of plot pictures or actor photos are stored for each movie and television drama, a large amount of storage resources and computing resources will be occupied, and the efficiency of comparing frame by frame with the plot pictures is relatively low, the identification process takes a long time, and the user experience is relatively poor.

[0063] The plot source identification method provided by the embodiments of the present application can be applied to the application environment as shown. Figure 2 As shown, the application environment. The terminal 102 communicates with the server 104 through the network. The data storage system can store the data required by the server 104 to process, such as the face picture data of the cast of the movie and television drama, the plot source information of the movie and television drama, and the like. The data storage system can be separately arranged, can be integrated on the server 104, or can be placed on the cloud or other network server.

[0064] In one embodiment, the plot source identification method can be performed by the server 104 alone. For example, optionally, the terminal 102 can respond to the screenshot instruction to take a screenshot of the video playing interface to obtain a plot picture, and send the plot picture to the server 104. The server 104 obtains the plot picture, performs face extraction on the plot picture to obtain a cast face image, performs face recognition on the cast face image to obtain identity information of the cast in the plot picture; the server 104 performs scene recognition on the plot picture to obtain plot scene information, and retrieves a plot source database according to the identity information of the cast in the plot picture and the plot scene information to obtain plot source information corresponding to the plot picture. Optionally, the server 104 can send the plot source information to the terminal 102, and the terminal 102 can display the plot source information on the video playing interface. In this way, the user only needs to upload the plot picture to know the plot source information of the currently played video, solving the user's retrieval demand for the video source information, and at the same time, the identification strategy combining the identity information of the cast and the plot scene information matching can ensure that the identification process is not too complex, the data volume is not too large, and the identification efficiency is relatively high, and the identification of the identity information is based on the matching of the cast face information, which can ensure the accuracy of the identification result.

[0065] In some embodiments, the plot source identification method can also be performed by the terminal 102 in cooperation with the server 104. For example, after the terminal 102 obtains the plot picture, the terminal 102 performs face extraction on the plot picture to obtain the actor face image, performs face recognition on the actor face image to obtain the identity information of the actor in the plot picture, and performs scene recognition on the plot picture to obtain the plot scene information. The terminal 102 sends a plot source search request to the server 104 according to the identity information of the actor in the plot picture and the plot scene information, the server 104 receives the plot source search request, searches the plot source database of the server 104 according to the identity information of the actor and the plot scene information in the received plot source search request, and obtains the plot source information corresponding to the plot picture. Optionally, the server 104 can send the plot source information to the terminal 102, and the terminal 102 can display the plot source information on the video playing interface.

[0066] In some embodiments, the plot source identification method can also be performed by the terminal 102 alone. For example, the terminal 102 obtains the plot picture, performs face extraction on the plot picture to obtain the actor face image, performs face recognition on the actor face image to obtain the identity information of the actor in the plot picture, and performs scene recognition on the plot picture to obtain the plot scene information. In the case that the terminal 102 can directly access the plot source database, the terminal 102 can access the plot source database according to the identity information of the actor in the plot picture and the plot scene information, search the plot source database to obtain the plot source information corresponding to the plot picture. Optionally, the terminal 102 can display the plot source information on the video playing interface.

[0067] The plot source identification method provided by the embodiments of the present application can obtain the plot picture, perform face extraction on the plot picture to obtain the actor face image, perform face recognition on the actor face image to obtain the identity information of the actor in the plot picture, and use the actor face image for face recognition instead of directly performing face recognition based on the entire plot picture, which can reduce the amount of calculation, and also does not need to search the picture in all videos according to the plot picture, and compare the searched picture with the plot picture frame by frame, which can improve the recognition efficiency. The plot scene information is obtained by performing scene recognition on the plot picture, and thus the identity information of the actor in the plot picture and the plot scene information can be combined to search the plot source database to obtain the plot source information corresponding to the plot picture. Since the identity information of the actor and the plot scene information are combined, the accuracy of plot source screening can be improved, and thus the accuracy of plot source information identification can be improved.

[0068] The terminal 102 can be, but is not limited to, various personal computers, notebook computers, smart phones, tablet computers, Internet of Things devices, and portable wearable devices. The Internet of Things device can be a smart speaker, a smart television, a smart air conditioner, a smart vehicle device, a projection device, etc. The portable wearable device can be a smart watch, a smart bracelet, a head-mounted device, etc. The head-mounted device can be a virtual reality (VR) device, an augmented reality (AR) device, smart glasses, etc. The server 104 can be a standalone physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0069] In an exemplary embodiment, as shown in Figure 3 , a plot source identification method is provided. The method is applied to a computer device (the terminal 102 or the server 104) in Figure 2 , and includes the following steps 302 to 308. Wherein:

[0070] Step 302, obtaining a plot picture, the plot picture is used to search plot source information of the plot picture.

[0071] The plot picture is a picture including plot information. The plot picture includes a video playing screen of a certain video drama, and is used to search plot source information of the plot picture.

[0072] In an embodiment, taking the computer device as an example, the computer device can obtain the plot picture through the following steps: the computer device can receive a screenshot instruction for a video playing interface, respond to the screenshot instruction to take a screenshot of the video playing interface to obtain a video screenshot, and respond to a plot source information identification instruction triggered by the video screenshot to take the video screenshot as a plot picture used to search plot source information of the video drama.

[0073] Here, the screenshot instruction can be triggered based on the terminal itself. For example, the terminal can present an icon for triggering the screenshot instruction. When the terminal detects a triggering operation of the icon by the user, the terminal receives the screenshot instruction for the video playing interface. Alternatively, the terminal is provided with a physical button for triggering a screenshot of the content displayed on the current screen. When the user triggers the physical button, the terminal receives the screenshot instruction for the content displayed on the entire current terminal screen. Alternatively, the terminal can also receive voice content of the user, identify the voice content, and when the voice content is used to instruct to take a screenshot of the current screen, the terminal receives the screenshot instruction for the content displayed on the entire current terminal screen.

[0074] As shown in Figure 4A , a schematic diagram of the plot picture in an embodiment. Referring to Figure 4A, the terminal plays a video clip in a full-screen mode, displays a screenshot icon 402 in the video playing interface, and triggers the terminal to take a screenshot of the current video playing interface and obtain a video screenshot in response to a user operation on the screenshot icon 402. The video screenshot can be used as a plot picture.

[0075] As shown in FIG. 7, it is a schematic view of a plot picture in another embodiment. Referring to FIG. 7, the terminal receives a video stream pushed by a video platform in a video waterfall form, and plays a video clip in a vertical screen mode. The video clip has a current forwarding quantity of 232 and a comment quantity of 306. If a user triggers a screenshot button (or a screenshot icon), the terminal takes a screenshot of the content displayed on the current screen and obtains a video screenshot. The video screenshot includes a video playing picture and an area for displaying other information. The terminal can use the video screenshot as a plot picture for identifying plot source information, or process the video screenshot to obtain a plot picture in the area of the video playing picture. Figure 4B Figure 4B In some application scenarios, the plot picture can also be a picture published by a user's friend (a contact or a followee) on a social application. In this way, when a user sees a screenshot picture of a video published by a friend on a social application, if the user is also interested in watching the complete video, the user can perform plot source identification on the picture. For example, the terminal detects a long press operation on the picture, and performs plot source identification on the picture through the social application in response to the long press operation. For another example, the terminal detects a copy operation on the picture, and detects an opening operation on a video playing client after the copy operation, that is, enters the video playing client after copying the picture. At this time, the video playing client can prompt "whether to perform plot source identification on the copied picture?", and performs plot source identification on the picture in response to a user confirmation of performing plot source identification on the picture.

[0076] In some application scenarios, the plot picture can also be a picture published by a user's friend (a contact or a followee) on a social application. In this way, when a user sees a screenshot picture of a video published by a friend on a social application, if the user is also interested in watching the complete video, the user can perform plot source identification on the picture. For example, the terminal detects a long press operation on the picture, and performs plot source identification on the picture through the social application in response to the long press operation. For another example, the terminal detects a copy operation on the picture, and detects an opening operation on a video playing client after the copy operation, that is, enters the video playing client after copying the picture. At this time, the video playing client can prompt "whether to perform plot source identification on the copied picture?", and performs plot source identification on the picture in response to a user confirmation of performing plot source identification on the picture.

[0077] ​The plot source information is description information of a certain film or television drama from which the plot picture is derived, including but not limited to the name of the film or television drama, the cast, the plot synopsis, the plot type, the plot style, the scene location and the scene era background, etc. Among them, the cast can specifically include the name, gender, age, birth date, face picture and the like of the cast, and the face picture is a picture containing only the face of the actor, not a cast photo. The plot synopsis is a brief description of the plot of the film or television drama, the plot type includes modern drama, ancient drama, etc., the plot style includes legend, era, city, etc. The scene location includes indoor, outdoor, classroom, playground, road, street, office building, etc., and the scene era background includes Tang Dynasty, Song Dynasty, Qing Dynasty, modern era, etc. In addition, the plot type, plot style, scene location and scene era background can also be classified as plot scene information.

[0078] In an embodiment, a plot source database can be constructed on a server. The server can collect the plot source information corresponding to the film or television drama, and store the name of the film or television drama and the corresponding plot source information in the plot source database, so as to realize the retrieval and identification of the plot source information based on the plot picture. Specifically, when constructing the plot source database, the name and the corresponding plot source information can be stored, wherein the plot source information includes a plurality of fields and corresponding field values. For example, the field is the cast, and the field value is the name of the cast. The field is the production team, and the field value is the name of the director and the producer, etc. The field is the cast picture, and the field value is the face portrait of the cast. The field is the plot type, and the field value is modern drama or ancient drama. The field is the scene era background, and the field value is modern, Tang Dynasty or Song Dynasty, etc.

[0079] In step 304, face extraction is performed on the plot picture to obtain a cast face image, face recognition is performed on the cast face image, and identity information of the cast in the plot picture is obtained.

[0080] In the embodiments of the present application, in order to quickly and accurately identify the plot source, the computer device takes the identity information of the cast as a retrieval condition to retrieve the plot source database. Therefore, after obtaining the plot picture, the computer device performs face extraction on the plot picture to obtain a cast face image, performs face recognition on the cast face image, and obtains the identity information of the cast in the plot picture, such as the name of the cast. It should be noted that the number of cast face images obtained by face extraction from the plot picture can be at least one, or multiple. If there are multiple, the computer device can perform face recognition on each cast face image obtained by face extraction, thereby obtaining multiple identity information.

[0081] The face image of the participating actor obtained by performing face extraction on the plot picture is an image including only a face. By removing the area other than the face in the plot picture and then performing face recognition, the data amount can be reduced, the face recognition efficiency can be improved, and the accuracy of face recognition can be improved. Moreover, when performing face comparison, only the face picture (or head picture) of the actor needs to be obtained for comparison, and there is no need to search for pictures in all videos according to the plot picture, and the obtained plot picture is compared with the searched pictures frame by frame. That is, a large number of pictures of the actor do not need to be stored additionally, and only one face picture of the actor needs to be stored, so that the data storage amount can be reduced, and the recognition efficiency can be improved.

[0082] In one embodiment, the face image of the participating actor obtained by performing face extraction on the plot picture includes: inputting the plot picture into an image segmentation network model; obtaining a semantic segmentation result of the plot picture by using the image segmentation network model; and extracting the face image of the participating actor from the plot picture according to the semantic segmentation result.

[0083] The image segmentation network model is a semantic image segmentation model based on a convolutional neural network, and can be a BiSeNet (Bilateral Segmentation Network) for real-time semantic segmentation. The computer device can perform semantic segmentation on the plot picture by using the image segmentation network model to obtain the semantic segmentation result. That is, the image segmentation network model can classify the pixel points in the plot picture into face region pixel points and non-face region pixel points, so that the face region (or head region) and the non-face region in the plot picture can be divided according to the semantic segmentation result, and the face image of the participating actor in the plot picture can be obtained according to the face region.

[0084] For the obtained face image of the participating actor, the computer device can perform face recognition on the face image of the participating actor to obtain the identity information of the participating actor in the plot picture. Face recognition is a kind of biometric technology for identity recognition based on face feature information. After obtaining the face image of the participating actor, the computer device can compare the face image of the participating actor with the actor face image stored in the actor face image library based on a face recognition model to determine the identity information of the participating actor in the plot picture.

[0085] Step 306: performing scene recognition on the plot picture to obtain plot scene information.

[0086] The plot scene information is information for describing a scene in a plot picture, including information for describing characters, scenes, and buildings in the plot picture, including but not limited to a plot type, a plot style, a scene location, and a scene time background, etc.

[0087] The computer device can perform scene recognition on the plot picture through a scene recognition model to obtain the plot scene information, where the scene recognition model is trained by using some general scene photos. For example, the scene recognition model is constructed based on a model suitable for a multi-classification task, and the scene recognition model is used to identify the plot type, the plot style, the scene time background, etc. corresponding to the plot picture.

[0088] The computer device can also perform scene recognition on the plot picture through a large language model to obtain the plot scene information. For example, a prompt text “please describe the plot type, style, location, and scene time background of the input picture” is set, the plot picture is taken as an input picture, and the large language model is called. Based on the prompt text and the input picture, the large language model outputs the plot scene information corresponding to the plot picture.

[0089] In some embodiments, in a case where it is necessary to know that the plot scene information corresponding to the plot picture is a plot type (for example, an ancient drama, a modern city drama, a period drama, etc.), the computer device can obtain general pictures corresponding to various plot types, compare the plot picture with the general pictures corresponding to each plot type, for example, can calculate the similarity of image features, the similarity of image styles, the similarity of image colors, etc., and determine the general picture similar to the plot picture to determine the plot type corresponding to the plot picture. In a case where the plot type is identified as an ancient drama, the computer device can further obtain general pictures corresponding to various different scene time backgrounds (for example, typical costume pictures and typical makeup pictures of different dynasties, etc.), and compare the plot picture with the general pictures corresponding to each different scene time background to determine the scene time background corresponding to the plot picture, that is, which dynasty, so as to improve the accuracy of subsequent retrieval.

[0090] In step 308, according to the identity information of the actor in the plot picture and the plot scene information, the plot source database is searched to obtain the plot source information corresponding to the plot picture.

[0091] Specifically, the names of the plots and the corresponding plot source information are stored in the plot source database, the plot source information includes the plot scene information, the plot scene information includes multiple fields, including at least one of a plot type, a plot style, a scene location, and a scene time background, and the computer device searches the plot source database according to the obtained identity information of the actor and the identified plot scene information to obtain the plot source information corresponding to the plot picture.

[0092] In one embodiment, as shown in Figure 5 According to the identity information of the participating role in the plot picture and the plot scene information, the plot source database is searched to obtain the plot source information corresponding to the plot picture, including:

[0093] In step 502, according to the identity information of the participating role in the plot picture, the participating personnel corresponding to each drama name in the plot source database is searched, and the drama name including the identity information of the participating personnel is taken as a candidate drama name.

[0094] In step 504, the plot source information of the candidate drama name is obtained from the plot source database.

[0095] In step 506, the target drama name is selected from the candidate drama name according to the matching of the corresponding plot source information and the plot scene information, and the plot source information of the target drama name is taken as the plot source information corresponding to the plot picture.

[0096] In this embodiment, the candidate drama name is searched according to the identity information of the participating role, and then the target drama name is selected from the candidate drama name according to the plot scene information. This strategy of mainly matching the identity information of the participating role and secondarily matching the plot scene information can ensure that the searching process is not too complex and the searching efficiency is relatively high. Moreover, only the participating personnel related information (actor face image and identity information) of the film and television drama needs to be stored, and the amount of data to be stored is relatively small, without the need to store a large number of plot pictures. At the same time, the identity information of the participating role and the plot scene information are combined for searching, so that the target drama name obtained finally is basically correct.

[0097] The above plot source identification method obtains the plot picture used for searching the plot source information, obtains the identity information of the participating role in the plot picture through face extraction on the plot picture to obtain the face image of the participating role, and obtains the identity information of the participating role in the plot picture through face recognition on the face image of the participating role. Instead of directly performing face recognition based on the entire plot picture, the amount of calculation can be reduced, and moreover, it is not necessary to search the picture from all videos according to the plot picture, and the search picture is compared with the plot picture frame by frame, so that the identification efficiency can be improved. The plot scene information is obtained through scene recognition on the plot picture, so that the plot source database can be searched according to the identity information of the participating role in the plot picture and the plot scene information to obtain the plot source information corresponding to the plot picture. Since the identity information of the participating role and the plot scene information are combined, the accuracy of the plot source screening can be improved, and thus the accuracy of the plot source information identification can be improved.

[0098] As shown in Figure 6 is a detailed flowchart of the plot source identification method in one embodiment. Taking the computer device (terminal 102 or server 104) in Figure 2 as an example for description, refer toFigure 6 comprising steps 602 to 614:

[0099] Step 602, receiving a user uploaded plot picture;

[0100] Step 604, image feature extraction by a face detection model, detecting the face region of the plot picture according to the image feature, and obtaining the actor face image;

[0101] Step 606, image feature extraction of the plot picture by a scene recognition model, scene recognition according to the image feature, and obtaining the plot scene information;

[0102] Step 608, identifying the identity information of the actor face image;

[0103] Step 610, retrieving the drama name from the plot source database according to the identity information, and screening out the candidate drama name;

[0104] Step 612, finding the matching target drama name from the candidate drama name according to the plot scene information;

[0105] Step 614, returning the plot source information of the matching target drama name to the terminal used by the user.

[0106] Overall, this recognition strategy combined with the identity information of the actor and the plot scene information matching can ensure that the recognition process is not too complex, the data volume is not too large, and the recognition efficiency is relatively high. Among them, the recognition of the identity information is matched with the actor face information, which can ensure the accuracy of the recognition result.

[0107] In an exemplary embodiment, the actor face image is subjected to face recognition to obtain the identity information of the actor in the plot picture, comprising: comparing the actor face image with the actor face image stored in the actor face image library respectively to obtain a comparison result; and obtaining the identity information of the actor in the plot picture according to the comparison result.

[0108] In an example embodiment, the actor face image is compared with the actor face images stored in the actor face library respectively to obtain a comparison result, including: the first face feature of the actor face image and the second face feature of the actor face images stored in the actor face library are extracted respectively by the face feature extraction model, the similarity between the first face feature and the corresponding second face feature of each actor face image is calculated, and the comparison result is determined according to the similarity. For example, when the similarity between the first face feature of the plot picture and the corresponding second face feature of the actor Actor1 is greater than a set threshold, it can be considered that the face in the actor face image is the face of the actor Actor1. In this embodiment, the actor face library stores the actor face images, only one image is needed, and the plot pictures of the actors from different TV dramas do not need to be stored, which can reduce the data amount and the storage space occupied by the stored images.

[0109] In an example embodiment, the actor face library stores the actor identity information and the face feature of the actor face image corresponding to the actor identity information, and the face feature is obtained by feature extraction of the face feature extraction model; the actor face image is compared with the actor face images stored in the actor face library respectively to obtain a comparison result, including: the face feature of the actor face image is extracted by the face feature extraction model; the similarity between the face feature of the actor face image and the face feature of each actor face image stored in the actor face library is calculated, and the comparison result is determined according to the similarity. In this embodiment, the actor face library stores the face feature of the actor face image, and the face feature has uniqueness, only one face feature needs to be stored, even the face image does not need to be stored, and the plot pictures of the actors from different TV dramas do not need to be stored, which can reduce the data amount and the storage space occupied by the stored images.

[0110] As shown in FIG. 1, it is a flowchart of the interaction between the terminal and the server in an embodiment. As shown in FIG. 2, after the client running on the terminal obtains the plot picture, the client sends a plot source identification request containing the plot picture to the interface machine layer of the server, the interface machine layer forwards the request to the image processing server according to the command protocol information of the plot source identification request to obtain the identity information and the plot scene information of the actor in the plot picture, the image processing server sends the identity information and the plot scene information of the actor in the plot picture to the query server, the query server retrieves the plot source database according to the identity information and the plot scene information of the actor to obtain the plot source information corresponding to the plot picture, and returns the obtained plot source information to the interface machine layer, and the interface machine layer returns to the client running on the terminal. Figure 7 Figure 7

[0111] ​​For example, the plot source database stores plot source information of a drama A, the corresponding actors of which include actor 1, actor 2, actor 3, and actor 4, and the corresponding plot introduction describes a plot occurring in the Zhenguan period of the Tang Dynasty. The plot source database also stores plot source information of a drama B, the corresponding actors of which include actor 2, actor 4, actor 5, and actor 6, and the corresponding plot introduction describes a plot occurring in the Kangxi period of the Qing Dynasty. The plot source database also stores plot source information of a drama C, the corresponding actors of which include actor 5, actor 6, actor 7, and actor 8, and the corresponding plot introduction describes a plot occurring in a modern city. The identity information of the actors obtained by the query server includes actor 2 and actor 4, and the plot scene information obtained is a story occurring in the Qing Dynasty. Therefore, the query server matches the drama A and the drama B in which actor 2 and actor 4 participate together when querying the plot source information, further filters the matched dramas according to the plot scene information, and obtains the drama B as the matched drama. Then, the query server returns the plot source information of the drama B to the user.

[0112] In the above embodiment, the identity information of the actors in the plot picture is obtained by performing face recognition on the face images of the actors, so that the actors participating in the video drama from which the plot picture originates is determined. This facilitates subsequent retrieval of plot source information based on the actors, and can achieve the effect of quickly and accurately identifying the plot source.

[0113] In one embodiment, the method further includes: when the face extraction on the plot picture fails to obtain the face images of the actors, retrieving plot source information in the plot source database according to the plot scene information corresponding to the plot picture; taking the drama name corresponding to the plot source information matched with the plot scene information as a candidate drama name; obtaining the search popularity of each candidate drama name in a preset time period, and selecting a target drama name from the candidate drama names according to a condition that the search popularity of the target drama name is greater than a set threshold, and taking the plot source information of the target drama name as the plot source information corresponding to the plot picture.

[0114] In this embodiment, when the plot picture does not contain a face, the computer device can retrieve the plot source database based only on the plot scene information corresponding to the plot picture. In this case, the number of drama names corresponding to the plot source information matched with the plot scene information corresponding to the plot picture, i.e., the number of candidate drama names, can be relatively large. The computer device further combines other indicators, such as search popularity (which can also be video link access popularity, play popularity, etc.), to filter the candidate drama names and obtain the final target drama name. For example, the computer device can select the candidate drama name with the highest search popularity as the “possible plot source information” and present it to the user. For another example, the number of target drama names can be multiple. The computer device can sort the multiple target drama names in descending order of search popularity and present them to the user as “possible plot source information”.

[0115] In one embodiment, after obtaining the plot source information corresponding to the plot picture, the method further comprises: displaying the plot source information and a plot source jump control in the current video playing interface, the plot source information comprising a drama name; in response to a triggering operation on the plot source jump control, jumping to a video playing client and displaying a playing entrance of a target video corresponding to the drama name in the plot source information; and in response to a triggering operation on the playing entrance, playing the target video in the video playing client.

[0116] Optionally, when multiple drama names and their corresponding plot source information are identified, the multiple plot source information can be displayed in a card style, with one drama name corresponding plot source information displayed in each card element. Optionally, the multiple drama names can also be displayed in order from high to low (or from left to right) according to their respective heat (such as search heat or play heat, etc.).

[0117] Taking a specific scenario as an example, a user watches a pushed video stream in a video platform, and currently watches a video segment. When the user wants to watch the complete video, a screenshot operation can be triggered on the current video playing interface to obtain a video screenshot. Then, a plot source identification operation is triggered on the video screenshot. The terminal will send the video screenshot to the server as a plot picture in response to a plot source information identification instruction. After the server identifies the corresponding plot source information, the plot source information is returned to the terminal. The terminal displays the plot source information and a plot source jump control to the user in the current video playing interface. The user triggers an operation on the plot source jump control. The terminal will jump from the current video platform to a video playing client installed on the terminal. The video playing client displays a playing entrance of a target video corresponding to the drama name in the plot source information. In response to a triggering operation on the playing entrance, the terminal can play the target video in the video playing client.

[0118] In this embodiment, after identifying and displaying the plot source information, the user does not need to manually switch to other clients, manually input text, and then search for the target video. The target video can be directly and automatically played in the video playing client from the current video playing interface, which is very convenient and improves the user experience.

[0119] In one embodiment, the image segmentation network model is implemented based on a Convolutional Neural Network (CNN) structure. The convolutional neural network is specially designed to process "grid-like" structured data, such as images (2D pixel grid) and time series data (time grid). Its basic structure is shown in Figure 8 A typical CNN is composed of convolutional layers, pooling layers, and fully connected layers.

[0120] Among them, the convolutional layer is the core of CNN, and the convolutional layer is used to complete the convolution operation. The values in the convolution kernel are the weights of the neural network, which are the parameters to be learned. Multiply the weights by the values at the corresponding input position (if it is the input layer, it is the pixel value, if it is the intermediate layer, it is the activation value of the intermediate layer neuron), and then add the bias, and then pass through the activation function to get the corresponding output.

[0121] The pooling layer is followed by the convolutional layer, which is a down-sampling operation that can effectively reduce the data dimension and thus reduce the model parameter quantity; while ensuring the spatial invariance of the features. Common pooling operations include max pooling and average pooling. The role of the pooling layer is to gradually reduce the spatial size of the data body, which can reduce the number of parameters in the network, reduce the consumption of computing resources, and effectively control overfitting. Max pooling uses the MAX operation to independently operate on each depth slice of the input data body, changing its spatial size. The most common form is that max pooling uses a filter with a size of 2x2 to down-sample each depth slice with a step of 2, discarding 75% of the activation information. Each MAX operation takes the maximum value from 4 numbers (that is, in a certain 2x2 region in the depth slice), and the depth remains unchanged.

[0122] The fully connected layer is the last part of the CNN. After multiple convolution + pooling layers, the input data (image) is processed into feature maps, which need to be flattened into a one-dimensional vector to input into the fully connected layer to complete the next specific task (such as classification task).

[0123] In one embodiment, a BiSenet network based on a convolutional neural network is used to detect the face area of the entire plot picture. Figure 9 is the overall architecture diagram of the BiSenet network, which mainly includes two parts, spatial path and context path.

[0124] The spatial path is to solve the problem of spatial information loss after channel reduction, and it also has a certain degree of down-sampling to reduce the size of the internal processing features, thereby improving the overall calculation speed and reducing the time consumption. The extraction of spatial information and the size of the receptive field have a great influence on the effect of semantic segmentation, and the spatial path aims to better extract spatial information. The context path solves the problem of the image content receptive field, and there is also a certain degree of down-sampling in it. The BiSenet network is based on the lightweight model Xception, and a global average pooling is added after the last output layer, which can provide the maximum receptive field containing global context information.

[0125] Figure 10 is the detailed architecture diagram of the BiSenet network. As shown in Figure 10 , the spatial path compresses the feature map to 1 / 8 of the input image size through 3 convolution layers, each of which is composed of 1 convolution with a stride of 2, 1 BN (Batch Normalization) layer and 1 ReLU (Rectified Linear Unit) activation function. Because the size is only compressed to 1 / 8, the size is relatively large, so the spatial information is relatively rich.

[0126] The ARM (Attention Refinement Module) is in the context path, and the spatial path and the context path are fused together through a feature fusion module (FFM) to form a fused feature for further upsampling calculation. The ARM captures the context information through global average pooling and learns an attention vector based on it to guide feature learning. That is, an additional branch is added to learn a weight vector to represent the weight of each channel of the input feature map.

[0127] In the semantic segmentation task, the receptive field of the model is crucial to the performance. To increase the receptive field, global pooling is used to improve the receptive field while reducing the number of pooling branches. The context path module in the model adopts a U-shaped architecture. The context path first uses downsampling to reduce the size of the input features. Because the spatial information is already addressed by the spatial path, the downsampling is directly used instead of a convolution kernel pooling. ARM is added to the 1 / 16 and 1 / 32 branches to improve the receptive field of the overall model.

[0128] As shown in Figure 11 and Figure 12 , they are the schematic diagrams of the ARM and the feature fusion module, respectively. Figure 11is the structural diagram of ARM, which is divided into two branches, which are feature-related branch and attention-related branch. The upper branch is constructed to calculate attention. The input is the feature after size adjustment. Then it is passed through a global pooling, a 1x1 convolution layer, a BN layer, and finally a sigmoid activation function to normalize all the calculated feature pixel values. Then the attention-related degree is calculated and multiplied with the original size feature map. At the level of feature representation, the semantic features of the two branches are not the same, so these features cannot be simply weighted. The spatial information captured by the spatial path encodes most of the rich detail information. The output features of the context path mainly encode context information. In other words, the output features of the spatial path are low-level, and the output features of the context path are high-level. A feature fusion module is needed to fuse the output features of different modules.

[0129] As shown in Figure 12 , it is a detailed architecture diagram of the feature fusion module in an embodiment. First, the two features are spliced on the image channel. Then the convolution layer + batch normalization + linear rectifier unit is used to fuse the features on different channels, which can fully fuse the low-level features and high-level features to generate a unified feature. Then the attention-related degree of the feature is calculated, and then multiplied between the matrices. Finally, it is added with the original feature. After feature fusion, the BiSenet network has extracted the detection information of each semantic element in the whole input image, and can up-sample the fused feature according to the size of the input image to keep the size of the output feature consistent with the input image. After up-sampling, each pixel point can be judged specifically. After aggregating the pixel points of the same class, the distribution of different semantic elements in the whole image can be calculated to locate the boundaries between different elements, so that the required semantic elements can be extracted. In this way, the cast face part in the picture can be recognized.

[0130] In a detailed embodiment, the plot source identification method comprises the following steps:

[0131] 1. The server obtains the plot source information corresponding to the film and television drama, and the plot source information at least includes the title of the drama;

[0132] 2. The server constructs a plot source database according to the plot source information, and the plot source information further includes at least one of the cast, the plot introduction, the plot type, the plot style, the scene location and the scene era background;

[0133] 3. The terminal receives a screenshot instruction for the video playing interface;

[0134] 4. The terminal responds to the screenshot instruction to take a screenshot of the video playing interface to obtain a video screenshot;

[0135] 5. The terminal responds to a plot source information identification instruction triggered by the video screenshot to take the video screenshot as a plot picture used to search for a plot source of a movie or TV series;

[0136] 6. The terminal sends the plot picture to a server, and the server receives the plot picture;

[0137] 7. The server inputs the plot picture into an image segmentation network model to obtain a semantic segmentation result about the plot picture through the image segmentation network model;

[0138] 8. The server extracts a cast member face image from the plot picture according to the semantic segmentation result;

[0139] 9. The server extracts a first face feature of the cast member face image and a second face feature of an actor face image stored in an actor face image library through a face feature extraction model, calculates a similarity between the first face feature and the corresponding second face feature of each actor face image, and determines a comparison result according to the similarity;

[0140] 10. The server obtains identity information of a cast member in the plot picture according to the comparison result;

[0141] 11. The server performs scene recognition on the plot picture to obtain plot scene information;

[0142] 12. The server retrieves cast members corresponding to each title of a plot source database according to the identity information of the cast member in the plot picture, and takes the title including the identity information of the cast member as a candidate title;

[0143] 13. The server obtains plot source information of the candidate title from the plot source database;

[0144] 14. The server screens out a target title from the candidate title, wherein the target title matches the plot scene information and the corresponding plot source information, and takes plot source information of the target title as plot source information corresponding to the plot picture;

[0145] 15. The server returns the plot source information to the terminal;

[0146] 16. The terminal displays the plot source information and a plot source jump control in a current video playing interface, and the plot source information includes a title;

[0147] 17. The terminal jumps to the video playing client and displays a playing portal of the target video corresponding to the drama name in the drama source information in response to a triggering operation on the drama source jump control;

[0148] 18. The terminal plays the target video in the video playing client in response to a triggering operation on the playing portal.

[0149] The drama source identification method provided by the embodiments of the present application can return drama source information according to the drama pictures uploaded by the user, in a fast manner without occupying a large amount of storage resources and computing resources. Firstly, compared with the traditional drama picture matching performed frame by frame, the matching strategy based on the identity information of the actors and actresses and the scene information of the drama can ensure that the search process occupies less computing resources and storage resources, is high in efficiency and short in time delay.

[0150] It should be understood that, although each step in the flowchart involved in the above embodiments is displayed in sequence according to the arrow, these steps are not necessarily executed in sequence according to the arrow. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, at least part of the steps in the flowchart involved in the above embodiments can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or steps or stages in other steps.

[0151] Based on the same inventive concept, the embodiments of the present application also provide a drama source identification device for implementing the drama source identification method described above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, and therefore the specific limitations in one or more drama source identification device embodiments provided below can refer to the limitations of the drama source identification method described above, which will not be described here again.

[0152] In one exemplary embodiment, as shown in Figure 13 a drama source identification device 1300 is provided, which includes an acquisition module 1302, an actor identification module 1304, a scene identification module 1306 and a retrieval module 1308, wherein:

[0153] The acquisition module 1302 is configured to acquire a drama picture, and the drama picture is used to search for drama source information of the drama picture.

[0154] The participating actor identification module 1304 is configured to perform face extraction on the plot picture to obtain a participating actor face image, perform face recognition on the participating actor face image, and obtain identity information of the participating actor in the plot picture.

[0155] The scene identification module 1306 is configured to perform scene identification on the plot picture to obtain plot scene information.

[0156] The searching module 1308 is configured to search a plot source database according to the identity information of the participating actor in the plot picture and the plot scene information, and obtain plot source information corresponding to the plot picture.

[0157] In an embodiment, the obtaining module 1302 is further configured to receive a screenshot instruction for the video playing interface, perform screenshot on the video playing interface to obtain a video screenshot, and trigger the plot source information identification instruction in response to the video screenshot, and take the video screenshot as the plot picture used to search the plot source information of the movie or television series.

[0158] In an embodiment, the participating actor identification module 1304 is further configured to input the plot picture into an image segmentation network model, obtain a semantic segmentation result about the plot picture through the image segmentation network model, and extract the participating actor face image from the plot picture according to the semantic segmentation result.

[0159] In an embodiment, the participating actor identification module 1304 is further configured to compare the participating actor face image with actor face images stored in an actor face image library respectively to obtain a comparison result, and obtain the identity information of the participating actor in the plot picture according to the comparison result.

[0160] In an embodiment, the participating actor identification module 1304 is further configured to extract a first face feature of the participating actor face image and a second face feature of the actor face images stored in the actor face image library through a face feature extraction model, calculate a similarity between the first face feature and the corresponding second face feature of each actor face image, and determine the comparison result according to the similarity.

[0161] In an embodiment, the actor face image library stores actor identity information and face features of the actor face images corresponding to the actor identity information, and the face features are obtained through feature extraction by the face feature extraction model; the participating actor identification module 1304 is further configured to extract a face feature of the participating actor face image through the face feature extraction model, and calculate a similarity between the face feature of the participating actor face image and face features of each actor face image stored in the actor face image library respectively, and determine the comparison result according to the similarity.

[0162] In an embodiment, the scene recognition module 1306 is further configured to input the plot picture into the trained scene recognition model; and identify plot scene information corresponding to the plot picture through the scene recognition model, the plot scene information including at least one of a plot type, a plot style, a scene location, and a scene era background.

[0163] In an embodiment, the retrieval module 1308 is further configured to retrieve, according to identity information of a participating role in the plot picture, participating personnel corresponding to each plot name in the plot source database, and take a plot name including the participating personnel and the identity information as a candidate plot name; obtain plot source information of the candidate plot name from the plot source database; and select, from the candidate plot names, a target plot name whose corresponding plot source information matches the plot scene information, and take plot source information of the target plot name as plot source information corresponding to the plot picture.

[0164] In an embodiment, the apparatus further includes:

[0165] The plot source database construction module is configured to obtain plot source information corresponding to a film or television plot, the plot source information including at least a plot name; and construct a plot source database according to the plot source information, the plot source information further including at least one of participating personnel, a plot introduction, a plot type, a plot style, a scene location, and a scene era background.

[0166] In an embodiment, the apparatus further includes a jump module configured to, after obtaining plot source information corresponding to the plot picture, display, in a current video playing interface, the plot source information and a plot source jump control, the plot source information including a plot name; in response to a triggering operation on the plot source jump control, jump to a video playing client and display a playing entrance of a target video corresponding to the plot name in the plot source information; and in response to a triggering operation on the playing entrance, play the target video in the video playing client.

[0167] In an embodiment, the retrieval module 1308 is further configured to, when a face extraction on the plot picture fails to obtain a participating role face image, retrieve plot source information in the plot source database according to plot scene information corresponding to the plot picture; select, from the plot source information, a plot name corresponding to plot source information matching the plot scene information as a candidate plot name; obtain a search popularity of each candidate plot name within a preset time period; select, from the candidate plot names, a target plot name whose search popularity is greater than a set threshold; and take plot source information of the target plot name as plot source information corresponding to the plot picture.

[0168] The plot source identification device 1300 obtains a plot picture used for searching plot source information, obtains a performer face image by performing face extraction on the plot picture, obtains identity information of the performer in the plot picture by performing face recognition on the performer face image, and extracts the performer face image for face recognition instead of directly performing face recognition based on the entire plot picture, so that the calculation amount can be reduced, and the search efficiency can be improved by comparing the search picture frame by frame with the plot picture without searching the picture in all videos according to the plot picture. The plot source identification device 1300 obtains plot scene information by performing scene recognition on the plot picture. In this way, the plot source database can be searched by combining the identity information of the performer in the plot picture and the plot scene information, so as to obtain the plot source information corresponding to the plot picture. Since the identity information of the performer and the plot scene information are combined, the accuracy of plot source screening can be improved, and the accuracy of plot source information identification can be improved. In general, the identification strategy of combining the identity information of the performer and the plot scene information can ensure that the identification process is not too complex, the data amount is not too large, and the identification efficiency is relatively high. The identification of the identity information is based on the matching of the performer face information, so that the accuracy of the identification result can be ensured.

[0169] The modules in the plot source identification device 1300 can be implemented by software, hardware, or a combination thereof. The modules can be embedded in or independent of a processor in a computer device in a hardware form, or stored in a memory in a computer device in a software form, so as to be called and executed by a processor to perform the operations of the modules.

[0170] In an exemplary embodiment, a computer device, which can be a server, is provided, and an internal structure diagram of the computer device can be as shown in Figure 14 The computer device includes a processor, a memory, an input / output interface, and a communication interface. The processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The database of the computer device is configured to store plot source data. The input / output interface of the computer device is configured to exchange information between the processor and external devices. The communication interface of the computer device is configured to communicate with external terminals through a network connection. The computer program is executed by the processor to implement a plot source identification method.

[0171] In an example embodiment, a computer device, which can be a terminal, is provided, and an internal structure diagram of the computer device can be as shown in FIG. 1. Figure 15 The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is configured to exchange information between the processor and external devices. The communication interface of the computer device is configured to perform wired or wireless communication with external terminals. The wireless communication can be achieved through WIFI, mobile cellular network, Near Field Communication (NFC) or other technologies. The computer program is executed by the processor to implement a plot source identification method. The display unit of the computer device is configured to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or a key, a trackball or a touchpad arranged on the shell of the computer device, or an external keyboard, a touchpad or a mouse, etc.

[0172] Those skilled in the art can understand that Figure 13 , Figure 14 the structure shown in FIG. 1 is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0173] In an example embodiment, a computer device is provided, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the plot source identification method provided in the embodiments of the present application.

[0174] In an example embodiment, a computer readable storage medium is provided, which stores a computer program. The computer program is executed by a processor to implement the steps of the plot source identification method provided in the embodiments of the present application.

[0175] In an embodiment, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the steps of the plot source identification method provided in the embodiments of the present application.

[0176] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties, and the collection, use and processing of related data need to comply with relevant regulations.

[0177] It can be understood by those skilled in the art that all or part of the processes in the above-mentioned embodiment methods can be completed by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments of each method. In the embodiments of the present application, any reference to memory, database or other medium can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive random access memory (Resistive Random Access Memory, ReRAM), magnetoresistive random access memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric memory (Ferroelectric Random Access Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc. The database involved in the embodiments of the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments of the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (Artificial Intelligence, AI) processor, etc., without being limited thereto.

[0178] The technical features of the above embodiments can be combined in any manner. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described, however, as long as the combinations of the technical features do not contradict each other, they should be considered to be within the scope of the present application.

[0179] The above embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it should not be understood as a limitation on the patent scope of the present application. It should be pointed out that, for ordinary skilled persons in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A scenario source identification method characterized by, The method comprises: acquiring a plot picture, the plot picture being used to search plot source information of the plot picture; performing face extraction on the plot picture to obtain a cast face image, and performing face recognition on the cast face image to obtain identity information of a cast in the plot picture; performing scene recognition on the plot picture to obtain plot scene information; retrieving a plot source database according to the identity information of the cast in the plot picture and the plot scene information to obtain plot source information corresponding to the plot picture.

2. The method of claim 1, wherein, The acquiring of the plot picture comprises: receiving a screenshot instruction for a video playing interface; in response to the screenshot instruction, performing screenshot on the video playing interface to obtain a video screenshot; in response to a plot source information recognition instruction triggered by the video screenshot, taking the video screenshot as a plot picture used to search plot source information of a movie or television series.

3. The method of claim 1, wherein, The performing of face extraction on the plot picture to obtain a cast face image comprises: inputting the plot picture into an image segmentation network model; obtaining semantic segmentation results about the plot picture through the image segmentation network model; extracting a cast face image from the plot picture according to the semantic segmentation results.

4. The method of claim 1, wherein, The performing of face recognition on the cast face image to obtain identity information of a cast in the plot picture comprises: comparing the cast face image with actor face images stored in an actor face image library respectively to obtain comparison results; obtaining the identity information of the cast in the plot picture according to the comparison results.

5. The method of claim 4, wherein, The comparing of the cast face image with actor face images stored in an actor face image library respectively to obtain comparison results comprises: extracting first face features of the cast face image and second face features of the actor face images stored in the actor face image library respectively through a face feature extraction model, calculating similarities between the first face features and corresponding second face features of each of the actor face images, and determining comparison results according to the similarities.

6. The method of claim 4, wherein, The actor face image library stores actor identity information and face features of actor face images corresponding to the actor identity information, and the face features are obtained through feature extraction by a face feature extraction model; The comparing of the cast face image with actor face images stored in an actor face image library respectively to obtain comparison results comprises: extracting face features of the cast face image through the face feature extraction model; calculating similarities between the face features of the cast face image and face features of each of the actor face images stored in the actor face image library respectively, and determining comparison results according to the similarities.

7. The method of claim 1, wherein, The performing of scene recognition on the plot picture to obtain plot scene information comprises: inputting the plot picture into a trained scene recognition model; identifying plot scene information corresponding to the plot picture through the scene recognition model, the plot scene information comprising at least one of a plot type, a plot style, a scene location, and a scene era background.

8. The method according to any one of claims 1 to 7, characterized in that, The method further comprises: obtaining the plot source information corresponding to the movie and television drama, wherein the plot source information at least comprises a title; constructing a plot source database according to the plot source information, wherein the plot source information further comprises at least one of the following: a cast, a plot introduction, a plot type, a plot style, a scene location, and a scene era background. After obtaining the plot source information corresponding to the plot picture, the method further comprises:

9. The method of claim 1, wherein, displaying the plot source information and a plot source jump control in a current video playing interface, wherein the plot source information comprises a title; in response to a triggering operation on the plot source jump control, jumping to a video playing client and displaying a playing entrance of a target video corresponding to the title in the plot source information; in response to a triggering operation on the playing entrance, playing the target video in the video playing client.

10. The method of claim 1, wherein, The method further comprises: when the face extraction on the plot picture fails to obtain a face image of a participating role, retrieving plot source information in the plot source database according to plot scene information corresponding to the plot picture; selecting a title corresponding to plot source information matching the plot scene information as a candidate title; obtaining a search popularity of each candidate title in a preset period of time, selecting a target title with a search popularity greater than a set threshold from the candidate titles, and taking plot source information of the target title as plot source information corresponding to the plot picture.

11. The method according to any one of claims 1 to 10, characterized in that, The device comprises: an obtaining module configured to obtain a plot picture, wherein the plot picture is used to search for plot source information of the plot picture; a participating role identification module configured to perform face extraction on the plot picture to obtain a face image of a participating role, and perform face recognition on the face image of the participating role to obtain identity information of the participating role in the plot picture; a scene identification module configured to perform scene recognition on the plot picture to obtain plot scene information; 12. A scenario source identification apparatus characterized by comprising: a retrieving module configured to retrieve a plot source database according to the identity information of the participating role in the plot picture and the plot scene information to obtain plot source information corresponding to the plot picture. The processor implements the steps of the method of any one of claims 1 to 11 when executing the computer program. The computer program implements the steps of the method of any one of claims 1 to 11 when executed by the processor. The computer program implements the steps of the method of any one of claims 1 to 11 when executed by the processor. ​ 13. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, ​ 14. A computer readable storage medium having stored thereon a computer program, characterized in that, ​ 15. A computer program product comprising a computer program, characterized in that, ​

Images