Recommendation content generation method and device, equipment and storage medium

Abstract

The invention provides a recommendation content generation method and device, equipment and a storage medium, which are used for solving the problem of low generation efficiency of generation of recommendation content. The method at least comprises the steps that at least one piece of plot description information is combined with object attribute information, and an object recommendation task is established and executed; for each piece of plot description information, the object recommendation task is executed in the following mode: extracting a corresponding scene corpus from each plot scene described by the corresponding plot description information; extracting attribute corpora respectively matched with the at least one obtained scene corpus from the object attribute information; respectively fusing the at least one scene corpus and the attribute corpora matched with the scene corpora to generate corresponding candidate recommendation contents; and selecting at least one target recommendation content from the obtained candidate recommendation contents. The mode of generating the recommended content in combination with the plot description information and the object attribute information does not depend on response operation, and the generation efficiency of the recommended content is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device, equipment and storage medium for generating recommended content. Background Art

[0002] With the continuous development of technology, more and more terminal devices can present recommended content to viewers during the process of playing audio and video. Through the presented recommended content, the terminal device can introduce the recommended object to the viewer, so that the viewer can view, comment, forward or purchase the recommended object.

[0003] For example, when playing a TV series, the terminal device can present recommended content such as pictures, animations, and introductions for the recommended objects in certain playback screens without blocking the presentation of the plot. Thus, when watching these playback screens in the TV series, the audience can also watch the recommended content for the recommended objects.

[0004] Under the related art, the method for generating recommended content is usually: in response to a content input operation triggered for a recommendation object, multiple candidate recommended content for the recommendation object is obtained; in response to a location selection operation triggered for a film or TV series, a target location in the film or TV series is determined; in response to a content selection operation triggered for multiple candidate recommended content, a target recommended content is selected from the multiple candidate recommended content; and the target recommended content is added to the above-mentioned target location.

[0005] However, with the prevalence of the promotion method of presenting recommended content during the playback of audio and video, the demand for generating recommended content has increased significantly. The recommended content generation method under the related technology that relies heavily on response operations will cause the terminal device to be in a state of waiting for response for a long time, reducing the efficiency of generating recommended content.

[0006] It can be seen that the generation efficiency of recommended content under related technologies is low. Summary of the invention

[0007] The embodiments of the present application provide a method, apparatus, device and storage medium for generating recommended content, which are used to solve the problem of low generation efficiency and stability in accurately generating recommended content.

[0008] In a first aspect, a method for generating recommended content is provided, comprising:

[0009] Obtaining object attribute information of at least one plot segment and a recommended object contained in the video to be processed;

[0010] Extracting the plot description information of the at least one plot segment respectively; wherein each plot description information is used to summarize the plot scene presented by the corresponding plot segment from at least one description angle;

[0011] The at least one plot description information obtained is combined with the object attribute information to establish and execute an object recommendation task; wherein, for each plot description information, the object recommendation task is executed in the following manner: extracting corresponding scene corpus from each plot scene described by the corresponding plot description information; extracting attribute corpus that matches the at least one scene corpus obtained from the object attribute information; and fusing the at least one scene corpus and its matching attribute corpus to generate corresponding candidate recommendation content;

[0012] At least one target recommended content is selected from the obtained candidate recommended contents.

[0013] In a second aspect, a recommended content generating device is provided, comprising:

[0014] An acquisition module: used to acquire object attribute information of at least one plot segment and a recommended object contained in the video to be processed;

[0015] Processing module: used to extract the plot description information of the at least one plot segment respectively; wherein each plot description information is used to summarize the plot scene presented by the corresponding plot segment from at least one description angle;

[0016] The processing module is further used to: combine the obtained at least one plot description information with the object attribute information respectively, establish and execute the object recommendation task; wherein, for each plot description information, the object recommendation task is executed in the following manner: extract corresponding scene corpus from each plot scene described by the corresponding plot description information; extract attribute corpus that matches the obtained at least one scene corpus respectively from the object attribute information; respectively fuse the at least one scene corpus and its respectively matched attribute corpus to generate corresponding candidate recommendation content;

[0017] The processing module is further configured to select at least one target recommended content from the obtained candidate recommended content.

[0018] Optionally, the acquisition module is specifically used for:

[0019] Get the video to be processed;

[0020] Performing storyboard processing on the video to be processed to obtain a plurality of storyboard segments, wherein different storyboard segments have different viewing angles;

[0021] Extracting plot features of each of the plurality of storyboard segments respectively;

[0022] Based on the obtained plot features, clustering processing is performed on multiple groups of shot segments whose plot similarities meet the similarity condition in the multiple storyboard segments to obtain at least one plot segment.

[0023] Optionally, the acquisition module is specifically used for:

[0024] Based on the obtained plot features, determine the temporal correlation and spatial correlation between every two storyboard segments in the plurality of storyboard segments; wherein the temporal correlation represents: the degree of continuity of the plot between the corresponding two storyboard segments; and the spatial correlation represents: the degree of matching between the spatial layouts of the plot elements contained in the corresponding two storyboard segments;

[0025] Determine the plot similarity between every two storyboard segments in the plurality of storyboard segments based on the obtained temporal correlation and spatial correlation between every two storyboard segments;

[0026] Clustering is performed on multiple groups of storyboard segments whose plot similarities satisfy a similarity condition in the multiple storyboard segments to obtain at least one plot segment.

[0027] Optionally, when the at least one description angle includes an audio line angle, the processing module is specifically configured to:

[0028] Performing audio separation processing on the video to be processed to obtain an audio file;

[0029] Based on the audio file, for the at least one plot segment, the following operations are performed respectively:

[0030] Extracting acoustic features of the audio segment corresponding to the plot segment from the audio file;

[0031] Filtering the sub-segments of the audio segment whose acoustic features do not meet the voice feature screening condition to obtain the voice segment corresponding to the plot segment;

[0032] The plot lines contained in the plot segment extracted from the voice segment are used as a plot scene presented by the plot segment summarized from the perspective of the audio lines to generate plot description information of the plot segment.

[0033] Optionally, when the at least one description angle includes a picture text angle, the processing module is specifically configured to:

[0034] For the at least one plot segment, the following operations are performed respectively:

[0035] Performing text detection on a plurality of video frames included in the plot segment respectively to determine text regions containing text in corresponding video frames;

[0036] Performing text recognition on the text regions of the multiple video frames respectively to obtain the screen texts contained in the multiple video frames respectively;

[0037] The obtained screen texts are used as a plot scene presented by the plot segment summarized from the perspective of the screen texts to generate plot description information of the plot segment.

[0038] Optionally, when the at least one description angle includes a character movement angle, the processing module is specifically configured to:

[0039] For the at least one plot segment, the following operations are performed respectively:

[0040] Performing face recognition on a plurality of video frames included in the plot segment respectively to determine the characters to which the faces included in the corresponding video frames belong;

[0041] Respectively determining the relative position between each character included in the plot segment and the corresponding video frame;

[0042] Respectively counting the relative time at which each character included in the plot segment appears in the plot segment and the relative position in the corresponding video frame to obtain the character movement;

[0043] The obtained character movements are used as a plot scene presented by the plot segment summarized from the perspective of the character movements, and plot description information of the plot segment is generated.

[0044] Optionally, when the at least one description angle includes a content description angle, the processing module is specifically configured to:

[0045] For the at least one plot segment, the following operations are performed respectively:

[0046] Extracting visual features and audio features of the plot segment;

[0047] Using a trained multimodal recognition model, performing text recognition on the fusion features of the visual features and the audio features to obtain a content description of the plot segment;

[0048] The obtained content description is used as a plot scene summarized from the perspective of the content description, and plot description information of the plot segment is generated.

[0049] Optionally, the processing module is specifically used for:

[0050] Obtaining a pre-stored task template; wherein the task template at least indicates: object attribute information and the location where the information of at least one plot description information is added, and content constraints of the candidate recommended content; the content constraints are at least used to constrain the text form and semantic sentiment tendency of the candidate recommended content;

[0051] According to the information adding position indicated by the task template, the at least one plot description information and the object attribute information are respectively added to the task template to establish an object recommendation task.

[0052] Optionally, the task template further indicates a location where information with topic information is added; and the processing module is specifically configured to:

[0053] Respectively obtain topic information of a corresponding initial topic set from a plurality of preset network resources;

[0054] Based on the topic keywords in the topic information of each of the obtained initial topic sets, multiple target topic sets matching the video to be processed are selected from the initial topic sets, and at least one target topic matching the video to be processed is selected from each of the target topic sets;

[0055] According to the information adding position indicated by the task template, the obtained target topics, the at least one plot description information and the object attribute information are added to the task template respectively to establish an object recommendation task.

[0056] Optionally, the processing module is specifically used for:

[0057] Extracting topic keywords of each initial topic contained in each initial topic set based on the obtained topic information of each initial topic set;

[0058] Based on the obtained topic keywords, from the initial topic sets, an intermediate topic set whose corresponding topic keyword is included in the at least one plot description information is selected to obtain a plurality of intermediate topic sets, and from each of the intermediate topic sets, an intermediate topic whose corresponding topic keyword is included in the at least one plot description information is selected to obtain at least one intermediate topic;

[0059] Based on the topic discussion heat of at least one intermediate topic in each of the intermediate topic sets quantified from the respective topic information of the multiple intermediate topic sets, the intermediate topic whose topic discussion heat is greater than a preset heat threshold is taken as the target topic, and the intermediate topic set containing the target topic is taken as the target topic set, to obtain the multiple target topic sets and the at least one target topic in each of the target topic sets.

[0060] Optionally, the processing module is specifically used for:

[0061] For each plot description information in the established object recommendation task, perform the following operations respectively:

[0062] The large model is used to perform the following operations: extracting corresponding scene corpora from each plot scene described in the plot description information; extracting attribute corpora that match the at least one scene corpus obtained from the object attribute information; and fusing the at least one scene corpus and its matching attribute corpora to generate corresponding candidate recommendation content.

[0063] The plot description information, the preset content constraint conditions and the object attribute information are combined to establish a content review task; wherein the content constraint conditions are at least used to constrain the text form and semantic sentiment tendency of the candidate recommended content;

[0064] The content audit task is performed using the audit big model to generate content audit results; wherein the content audit results indicate whether the candidate recommended content generated by the generation big model needs to be modified.

[0065] Optionally, the processing module is specifically used for:

[0066] Use the audit model to do the following:

[0067] Based on the content constraint condition, perform content evaluation on each of the candidate recommended contents to obtain a corresponding evaluation result; wherein the evaluation result is used to indicate the sub-content in the corresponding candidate recommended content that does not meet the content constraint condition;

[0068] When it is determined that the sub-content indicated by the obtained evaluation result is not a null value, a content modification instruction is generated based on the evaluation result to obtain a content review result; wherein the content modification instruction is used to instruct the generated large model to re-execute the object recommendation task to modify the corresponding sub-content;

[0069] When it is determined that the sub-content indicated by the obtained evaluation result is a null value, a content review result is generated, indicating that the corresponding candidate recommended content meets the content constraint condition.

[0070] Optionally, the processing module is specifically used for:

[0071] For each candidate recommendation content, the following operations are performed respectively:

[0072] Semantically identifying the candidate recommended content and the corresponding plot description information to obtain content semantic features and description information semantic features;

[0073] Determining the plot relevance between the candidate recommended content and the corresponding plot description information based on the feature similarity between the content semantic feature and the description information semantic feature;

[0074] When it is determined that the plot relevance is greater than a preset relevance threshold, the candidate recommended content is used as the target recommended content.

[0075] According to a third aspect, a computer program product is provided, comprising a computer program, which implements the method according to the first aspect when executed by a processor.

[0076] According to a fourth aspect, a computer device is provided, comprising:

[0077] A memory for storing program instructions;

[0078] The processor is used to call the program instructions stored in the memory and execute the method as described in the first aspect according to the obtained program instructions.

[0079] According to a fifth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the method described in the first aspect.

[0080] In the embodiment of the present application, considering that a complete video may contain at least one plot, the video to be processed is divided into at least one plot segment, which is combined with the object attribute information of the recommendation object as the basis for generating candidate recommendation content. On the one hand, the amount of data processing when generating each candidate recommendation content can be reduced, thereby improving the efficiency of generating recommended content; on the other hand, each candidate recommendation content is generated in a targeted manner, so that the target recommendation content is more integrated with a certain plot segment, which is more attractive to the audience and helps to improve the recommendation effect.

[0081] Furthermore, the corresponding plot description information extracted from the plot clip is a summary of the corresponding plot clip from at least one description angle. On the one hand, this allows for a clear description angle reference when subsequently generating candidate recommendation content, so as to generate candidate recommendation content that better matches the plot clip. On the other hand, this allows the generated candidate recommendation content to be highly matched with the corresponding plot clip in at least one description angle. For viewers who focus on at least one description angle, the generated candidate recommendation content is more attractive, which helps to improve the recommendation effect.

[0082] Furthermore, when performing the object recommendation task, the corresponding scene corpus is extracted from each plot scene described in the plot description information as the basis for extracting the attribute corpus from the object attribute information. On the one hand, the recommended content can be automatically generated to improve the efficiency of generating the recommended content; on the other hand, the recommended object can be recommended by immersing in each plot scene, which improves the audience's acceptance of the recommended content and helps to improve the recommendation effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] Figure 1A A schematic diagram of an application field of the method for generating recommended content provided in an embodiment of the present application;

[0084] Figure 1B A schematic diagram of a principle of a method for generating recommended content in related technology;

[0085] Figure 1C An application scenario of the method for generating recommended content provided in the embodiment of the present application;

[0086] Figure 2 A schematic diagram of a flow chart of a method for generating recommended content provided in an embodiment of the present application;

[0087] Figure 3A A schematic diagram of a method for generating recommended content provided in an embodiment of the present application;

[0088] Figure 3B A schematic diagram of a method for generating recommended content provided in an embodiment of the present application Figure 2 ;

[0089] Figure 4A A third schematic diagram of a principle of a method for generating recommended content provided in an embodiment of the present application;

[0090] Figure 4B A fourth schematic diagram of a principle of a method for generating recommended content provided in an embodiment of the present application;

[0091] Figure 5A A fifth schematic diagram of a principle of a method for generating recommended content provided in an embodiment of the present application;

[0092] Figure 5B A sixth schematic diagram of a principle of a method for generating recommended content provided in an embodiment of the present application;

[0093] Fig. 6A A schematic diagram of a method for generating recommended content according to an embodiment of the present application is shown in FIG7;

[0094] Figure 6B A schematic diagram of a method for generating recommended content provided in an embodiment of the present application Figure 8 ;

[0095] Figure 6C A schematic diagram of a method for generating recommended content provided in an embodiment of the present application Figure 9 ;

[0096] Fig. 7A A schematic diagram of a principle of a method for generating recommended content provided in an embodiment of the present application is shown in Figure 10;

[0097] Figure 7B A schematic diagram 11 of a principle of a method for generating recommended content provided in an embodiment of the present application;

[0098] Figure 7C A schematic diagram 12 of a principle of a method for generating recommended content provided in an embodiment of the present application;

[0099] Fig.7D 13 is a schematic diagram of a principle of a method for generating recommended content provided in an embodiment of the present application;

[0100] Figure 8 A first structural diagram of a device for generating recommended content provided in an embodiment of the present application;

[0101] Fig. 9 A structural diagram of a recommended content generation device provided in an embodiment of the present application Figure 2 . DETAILED DESCRIPTION

[0102] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0103] Some of the terms used in the embodiments of the present application are explained below to facilitate understanding by those skilled in the art.

[0104] (1) Band-Aid form:

[0105] The Band-Aid format is to find the recommended content placement that matches the tone of the recommended object in the video screen, accurately combine the elements in the video screen with the recommended object, and create a recommendation format that appears in the video screen in a floating, non-interruptive form.

[0106] The embodiments of the present application relate to artificial intelligence (AI) technology and cloud computing. Artificial intelligence technology and cloud computing can be applied to multiple fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned driving, automatic driving, smart transportation, smart maps, assisted driving, vehicle terminals, aircraft, digital twins, virtual humans, robots, artificial intelligence generated content (AIGC), conversational interaction, smart medical care, smart customer service, game AI, etc. It is believed that with the development of technology, artificial intelligence technology and cloud computing will be applied in more fields and play an increasingly important role.

[0107] It should be noted that the embodiments of the present application involve operations to obtain data such as video to be processed or object attribute information. When the above embodiments of the present application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0108] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0109] The following is a brief introduction to the application fields of the recommended content generation method provided in the embodiments of the present application.

[0110] With the continuous development of technology, more and more terminal devices can present recommended content to viewers during the process of playing audio and video. Through the presented recommended content, the terminal device can introduce the recommended object to the viewer, so that the viewer can view, comment, forward or purchase the recommended object.

[0111] For example, when playing a TV series, the terminal device can present recommended content such as pictures, animations, and introductions for the recommended objects in certain playback screens without blocking the presentation of the plot. Thus, when watching these playback screens in the TV series, the audience can also watch the recommended content for the recommended objects.

[0112] For example, if the recommended object is a drug, please refer to Figure 1A, which is a playback screen in a TV series being played, and contains a character. In the plot of the playback screen, the character is shown as a character who keeps drinking because of a bad mood. Then, in the lower left corner of the playback screen, the recommended content of the drug can be presented in the form of a floating layer or non-interrupted patch, including a drug picture and a recommendation. The recommendation is "Drinking a glass makes me feel happy, but my stomach hurts until dawn." The recommendation matches the plot of the playback screen without any sense of disobedience, which can increase the audience's acceptance and favorability of the recommended content, thereby improving the conversion rate.

[0113] Under the related art, the method for generating recommended content is usually: in response to a content input operation triggered for a recommendation object, multiple candidate recommended content for the recommendation object is obtained; in response to a location selection operation triggered for a film or TV series, a target location in the film or TV series is determined; in response to a content selection operation triggered for multiple candidate recommended content, a target recommended content is selected from the multiple candidate recommended content; and the target recommended content is added to the above-mentioned target location.

[0114] Please refer to Figure 1B , a video frame sequence of the video to be processed is presented in the interface, and in response to a selection operation on a target video frame, the target video frame is presented in the interface. For example, a selection operation is triggered at a position where the picture changes slowly or the plot rhythm is tense, and a video frame is selected as a target video frame for adding recommended content.

[0115] The interface may also present the recommendation points of the recommendation object provided by the recommendation object provider in the form of a list, such as recommendation point_1 and recommendation point_2. In response to the content input operation triggered for the recommendation object, candidate recommendation content may be written, such as candidate recommendation content_3. The interface may also display the obtained candidate recommendation content, such as candidate recommendation content_1 and candidate recommendation content_2.

[0116] Therefore, from among the candidate recommended contents of the recommendation object, the candidate recommended contents that match the target video frame, are unique in creativity and have strong appeal can be selected as the target recommended contents and added to the target video frame.

[0117] However, with the prevalence of the promotion method of presenting recommended content during the playback of audio and video, the demand for generating recommended content has increased significantly. The recommended content generation method under the related technology that relies heavily on response operations will cause the terminal device to be in a state of waiting for response for a long time, reducing the efficiency of generating recommended content.

[0118] It can be seen that the generation efficiency of recommended content under related technologies is low.

[0119] In order to solve the problem of low efficiency in generating recommended content, the present application proposes a method for generating recommended content. In the method, object attribute information of at least one plot segment and a recommended object contained in a video to be processed is obtained. Plot description information of at least one plot segment is extracted respectively. Each plot description information is used to summarize the plot scene presented by the corresponding plot segment from at least one description angle.

[0120] The obtained at least one plot description information is combined with the object attribute information, an object recommendation task is established and executed, and at least one target recommendation content is selected from each obtained candidate recommendation content.

[0121] Among them, for each plot description information, the object recommendation task is performed in the following way:

[0122] Extract corresponding scene corpora from each plot scene described in the corresponding plot description information. Extract attribute corpora that match the at least one scene corpus obtained from the object attribute information. Fuse the at least one scene corpus and its matching attribute corpora respectively to generate corresponding candidate recommendation content.

[0123] In the embodiment of the present application, considering that a complete video may contain at least one plot, the video to be processed is divided into at least one plot segment, which is combined with the object attribute information of the recommendation object as the basis for generating candidate recommendation content. On the one hand, the amount of data processing when generating each candidate recommendation content can be reduced, thereby improving the efficiency of generating recommended content; on the other hand, each candidate recommendation content is generated in a targeted manner, so that the target recommendation content is more integrated with a certain plot segment, which is more attractive to the audience and helps to improve the recommendation effect.

[0124] Furthermore, the corresponding plot description information extracted from the plot clip is a summary of the corresponding plot clip from at least one description angle. On the one hand, this allows for a clear description angle reference when subsequently generating candidate recommendation content, so as to generate candidate recommendation content that better matches the plot clip. On the other hand, this allows the generated candidate recommendation content to be highly matched with the corresponding plot clip in at least one description angle. For viewers who focus on at least one description angle, the generated candidate recommendation content is more attractive, which helps to improve the recommendation effect.

[0125] Furthermore, when performing the object recommendation task, the corresponding scene corpus is extracted from each plot scene described in the plot description information as the basis for extracting the attribute corpus from the object attribute information. On the one hand, the recommended content can be automatically generated to improve the efficiency of generating the recommended content; on the other hand, the recommended object can be recommended by immersing in each plot scene, which improves the audience's acceptance of the recommended content and helps to improve the recommendation effect.

[0126] The following describes the application scenarios of the recommended content generation method provided by this application.

[0127] Please refer to Figure 1C , is a schematic diagram of an application scenario of the recommendation content generation method provided by the present application. The application scenario includes a server 101 and a client 102, and the server 101 and the client 102 can communicate with each other, and the communication method can be to use wired communication technology for communication, for example, by connecting a network cable or a serial port line for communication; or to use wireless communication technology for communication, for example, by using Bluetooth or wireless fidelity (wireless fidelity, WIFI) and other technologies for communication, and there is no specific limitation.

[0128] The client 102 generally refers to a device that can present a video to be processed or candidate recommended content, such as a terminal device, a third-party application program accessible to the terminal device, or a web page accessible to the terminal device, etc. The server 101 generally refers to a device that can generate recommended content, such as a terminal device or a server, etc.

[0129] Terminal devices include but are not limited to mobile phones, computers, smart medical devices, smart home appliances, vehicle-mounted terminals or aircraft, etc. Servers include but are not limited to cloud servers, local servers or associated third-party servers, etc. Both the client 101 and the server 102 can use cloud computing to reduce the occupation of local computing resources; cloud storage can also be used to reduce the occupation of local storage resources.

[0130] As an embodiment, the server 101 and the client 102 may be the same device, or may be different devices, or may be different devices with some modules in common, etc., without specific limitation.

[0131] The following is based on Figure 1C , the recommended content generation method provided in the embodiment of the present application is specifically introduced. Please refer to Figure 2 , which is a flow chart of a method for generating recommended content provided in an embodiment of the present application.

[0132] S201, obtaining object attribute information of at least one plot segment and a recommended object included in a video to be processed.

[0133] The plot presented in the same plot segment is coherent, while the plot presented in different plot segments is incoherent. For example, the same plot segment interprets the cause, process and result of an event; for example, the same plot segment interprets the relationship between the characters in a family; for example, the same plot segment interprets the development of the plot with one character as the main character within a period of time, etc., without specific restrictions.

[0134] At least one plot clip can be downloaded from a network resource, such as at least one edited plot clip uploaded by an editor on other video platforms; it can also be sent by other receiving devices, such as a plot clip obtained by a mouse selection operation; it can also be obtained by segmenting the video to be processed, without specific limitation.

[0135] The recommended objects are the objects promoted in the video to be processed, such as items, services, stores, other audio-visual works, applications or news messages, etc., without specific restrictions.

[0136] The object attribute information is used to introduce the provider of the recommended object and the recommended object, for example, including the public information of the provider of the recommended object and the recommended object in the network resources, such as using the trained text recognition model to perform semantic recognition on each public information, and reorganize the language based on the recognized semantics, construct a text description, and output the object attribute information; it also includes the purpose of recommendation, the main recommendation points of the recommended object, the target recommendation effect of the provider of the recommended object, the application scenario of the recommended object, the development history of the provider of the recommended object, etc., without specific restrictions. For example, when the recommended object is clothing, the object attribute information includes clothing brand, brand story, brand spokesperson, clothing material, clothing style, clothing color, clothing matching plan, clothing wearing season, clothing characteristics, clothing design ideas, clothing audience groups and clothing purchase evaluations, etc.

[0137] As an embodiment, the following describes a process of obtaining at least one plot segment by taking a method of segmenting a video to be processed as an example.

[0138] After obtaining the video to be processed, the video to be processed is storyboarded to obtain multiple storyboard segments. Different storyboard segments have different viewing angles. Different storyboard segments may also have different shooting scenes, narrative time, character actions, light and shadow colors, or character positions, etc., without specific restrictions.

[0139] Please refer to Figure 3A After obtaining the video to be processed, the video to be processed can be preprocessed, such as adjusting the resolution or frame rate of the video to be processed, so as to facilitate efficient and accurate storyboard processing.

[0140] From the video frame sequence formed by each video frame type contained in the preprocessed video to be processed, key video frames are extracted, such as selecting a video frame as a key video frame at every preset number of frames, or using an adjacent frame comparison method to select a video frame with representative content as a key video frame, etc., without specific limitation.

[0141] For each key video frame obtained, the frame difference between every two adjacent key video frames can be determined according to measurement indicators such as the sum of absolute pixel differences (Sum of Absolute Differences, SAD) or the structural similarity index (Structural Simil.arity Index, SSIM).

[0142] According to the obtained frame differences, it is determined that the segments of the key video frames whose frame differences are less than the preset differences are one storyboard segment, and the segments of the key video frames whose frame differences are not less than the preset differences are different storyboard segments. For example, the threshold method (Threshold) and the content-sensitive threshold method (Content) provided by the PySceneDetect library are used to determine the start position and end position of each storyboard segment based on the obtained frame differences. In this way, multiple storyboard segments after the storyboard processing of the video to be processed can be obtained.

[0143] After obtaining each storyboard segment, extract the plot features of each of the multiple storyboard segments respectively, and based on the obtained plot features, perform clustering processing on multiple groups of storyboard segments whose plot similarities meet the similarity condition in the multiple storyboard segments respectively, to obtain at least one plot segment. Satisfying the similarity condition indicates that the corresponding storyboard segments have plot coherence, and not satisfying the similarity condition indicates that the corresponding storyboard segments have plot incoherence.

[0144] After clustering processing, multiple plot clips can be obtained. Then, all of the multiple plot clips can be used to generate candidate recommended content, or at least one plot clip can be selected from the multiple plot clips to generate candidate recommended content, for example, at least one plot clip containing preset interest points is selected, and the interest points are the locations of the clips where the audience is more interested, or the locations of the clips with richer storylines in the video to be processed, etc., without specific restrictions.

[0145] Please refer to Figure 3B In (1), a trained feature extraction model is used to extract at least one of the visual features and audio features of each storyboard segment as a plot feature. A clustering algorithm (such as K-means, DBSCAN, etc.) or a trained plot recognition model (such as a recurrent neural network RNN ​​or a long short-term memory LSTM network) is used to cluster the extracted plot features of each storyboard segment, and at least one storyboard segment whose plot similarity meets the similarity condition is divided into a group of storyboard segments to obtain multiple groups of storyboard segments. Please refer to Figure 3B In (2), the starting position and ending position of the corresponding plot segment are determined according to the starting position and ending position of each storyboard segment in each group of storyboard segments in the video to be processed, so that the multiple divided storyboard segments can be obtained.

[0146] First, the video to be processed is divided into multiple storyboard segments, so that each storyboard segment is shot from the same perspective. On this basis, the storyboard segments are clustered according to the plot characteristics of the storyboard segments, so that each plot segment obtained is a plot-coherent segment, avoiding the situation where the plot is fragmented in a plot segment, and improving the accuracy of generating recommended content based on plot-coherent plot segments.

[0147] As an example, when determining whether the plot similarity satisfies the similarity condition, it can be measured from two perspectives: time and space. For example, the plot between two storyboard segments describes what happened in a coherent time. Please refer to Figure 4A (1) is from the storyboard clip with the plot time at 9 pm. Figure 4A (2) in the figure comes from the storyboard with the plot time at 9:05 p.m., then it can be said to some extent that the two storyboards are coherent; for another example, the positions of the objects contained in the two storyboards are similar, please refer to Figure 4B (1) and (2) in the figure are from two different storyboards, and the two tea cups, tables, two chairs, dolls, etc. contained therein are in the same or similar positions. This can indicate to a certain extent that the two storyboards are coherent, etc., without any specific restrictions. By converting the plot into quantifiable data such as time and space layout, it is possible to determine the similarity of the plot and determine whether the plot similarity meets the similarity conditions.

[0148] Based on the obtained plot features, the temporal correlation and spatial correlation between each two storyboard segments in the multiple storyboard segments are determined. The temporal correlation represents: the degree of continuity of the plot between the corresponding two storyboard segments. The spatial correlation represents: the degree of matching between the spatial layouts of the plot elements contained in the corresponding two storyboard segments.

[0149] Based on the obtained temporal correlation and spatial correlation between each two storyboard segments, the plot similarity between each two storyboard segments in the multiple storyboard segments is determined. Thus, the multiple groups of storyboard segments whose plot similarities meet the similarity condition in the multiple storyboard segments can be clustered to obtain at least one plot segment.

[0150] For example, the weighted sum of the temporal correlation and the spatial correlation is used as the plot similarity between the corresponding two storyboard segments; for another example, the minimum value of the temporal correlation and the spatial correlation is used as the plot similarity between the corresponding two storyboard segments; for another example, the maximum value of the temporal correlation and the spatial correlation is used as the plot similarity between the corresponding two storyboard segments, etc., without specific limitation.

[0151] S202: extracting plot description information of at least one plot segment respectively.

[0152] After obtaining each plot segment, the plot description information of at least one plot segment can be extracted respectively. Each plot description information is used to summarize the plot scene presented by the corresponding plot segment from at least one description angle. The description angle includes, for example, at least one of the audio dialogue angle, the screen text angle, the character movement angle, and the content description angle. Each plot description information can also summarize the plot scene presented by the corresponding plot segment from other angles, which are not limited here.

[0153] The following introduces the process of extracting plot description information from the perspectives of audio lines, screen text, character movement, and content description.

[0154] Audio dialogue angle:

[0155] Perform audio separation on the video to be processed to obtain an audio file. Based on the audio file, perform the following operations on at least one plot segment:

[0156] Extract the acoustic features of the audio clip corresponding to the plot clip from the audio file. Filter the sub-segments in the audio clip whose acoustic features do not meet the voice feature screening conditions to obtain the voice clip corresponding to the plot clip. Use the plot lines contained in the plot clip extracted from the voice clip as a plot scene presented by the plot clip from the perspective of the audio lines to generate plot description information of the plot clip.

[0157] Based on the plot lines, plot description information that better matches the content of the plot clips can be generated, which helps to generate recommended content that better matches the plot clips and improves the accuracy of generating recommended content.

[0158] Please refer to Figure 5A , separate the audio track from the video file of the video to be processed to obtain the audio file. After obtaining the audio file, the audio file can be preprocessed, for example, including operations such as removing noise and normalizing audio amplitude, so as to facilitate subsequent efficient and accurate processing.

[0159] For the preprocessed audio files, acoustic features such as energy, zero-crossing rate (frequency of signal changing from positive to negative or from negative to positive), spectral features, Mel-frequency cepstral coefficients (MFCCs) and short-time Fourier transform (STFT) of the audio segments corresponding to the plot segments can be extracted from them. Since the energy of speech signals is usually higher than that of non-speech signals, the sub-segments of the audio segments whose acoustic features do not meet the speech feature screening conditions can be filtered from the perspective of energy; since the zero-crossing rate of speech signals is different from that of non-speech signals, the sub-segments of the audio segments whose acoustic features do not meet the speech feature screening conditions can be filtered from the perspective of zero-crossing rate, etc.

[0160] After obtaining the voice clips corresponding to the plot clips, an automatic speech recognition (ASR) model can be used to convert the voice clips into text form to obtain the plot lines.

[0161] The plot lines can be directly used as the plot description information of the plot fragment, or the plot lines can be fused with texts obtained from other angles to obtain the plot description information of the plot fragment, etc., without specific limitation.

[0162] Screen text angle:

[0163] For at least one plot segment, the following operations are performed respectively: text detection is performed on multiple video frames included in the plot segment to determine a text region containing text in the corresponding video frame. Text recognition is performed on the text regions of each of the multiple video frames to obtain screen texts contained in each of the multiple video frames. The obtained screen texts are used as a plot scene presented by the plot segment from the perspective of the screen text to generate plot description information of the plot segment.

[0164] By identifying the text in the video frames, the generated plot description information will not miss the plot in the plot clips that the audience needs to understand, which helps to generate recommended content that better matches the plot clips and improves the accuracy of the generation of recommended content.

[0165] For example, the multiple video frames contained in the plot clip may be preprocessed, including scaling, grayscale, binarization and other operations, to enhance the image quality. Then, text detection is performed on each preprocessed video frame to determine whether the corresponding video frame contains text, and if so, the text region of the text in the corresponding video frame is determined.

[0166] An optical character recognition (OCR) strategy is adopted to respectively recognize characters in the text regions of the plurality of video frames, and the recognized characters are converted into text form to obtain the screen texts contained in the plurality of video frames.

[0167] The screen text may be directly used as the plot description information of the plot segment, or the screen text may be fused with text obtained from other angles to obtain the plot description information of the plot segment, etc., without specific limitation.

[0168] Please refer to Figure 5BFor example, a video frame includes a hospital name. Then, the text region 501 where the hospital name is located in the video frame can be determined first, and then the characters in the text region can be recognized to obtain "XXX Hospital". Plot description information can be generated based on "XXX Hospital", and then, recommended content related to the medical field can be generated for the plot segment.

[0169] Character movement angle:

[0170] For at least one plot segment, the following operations are performed respectively: face recognition is performed on multiple video frames contained in the plot segment respectively, and the characters to which the faces contained in the corresponding video frames belong are determined. The relative positions between each character contained in the plot segment and the corresponding video frames are determined respectively. The relative time when each character contained in the plot segment appears in the plot segment and the relative position in the corresponding video frame are counted respectively to obtain the character movement. The obtained character movement is used as a plot scene presented in the plot segment from the perspective of character movement to generate plot description information of the plot segment.

[0171] By analyzing the movements of characters in a plot clip, we can understand the plot conveyed by the characters' actions, making the generated plot description information more consistent with the plot clip, which helps to generate recommended content that better matches the plot clip and improves the accuracy of generating recommended content.

[0172] For example, a trained convolutional neural network is used to identify faces appearing in multiple video frames contained in a plot segment, and multiple face regions are obtained. A trained object detection network is used to determine faces belonging to the same person among the faces appearing in the plot segment, and the same person identifier is added to the face regions belonging to the same person. The person identifier is used to uniquely characterize the corresponding person. A trained object detection network is used to determine the relative position between each face region and the corresponding video frame. The relative time of each character appearing in the plot segment is counted. For example, if character A appears in the plot segment at the 5th second and leaves at the 15th second, the relative time can be recorded as [5, 15].

[0173] By using the relative time at which each character appears in the plot clip, the multiple video frames associated with each character can be determined, and then based on the determined relative positions, the position change of each character in the multiple video frames associated with it can be analyzed to obtain the corresponding character movement. For example, the character movement is "character A starts from the side of character B, goes around the table from the left hand side, and walks to the opposite side of character B".

[0174] The character may be a main character determined by obtaining the cast list of the video to be processed, or may be any character appearing in the video to be processed, etc., without specific limitation.

[0175] Content description angle:

[0176] For at least one story segment, perform the following operations:

[0177] Extract the visual features and audio features of the plot clip. Use the trained multimodal recognition model to perform text recognition on the fusion features of the visual features and audio features to obtain the content description of the plot clip. Use the obtained content description as a summary of the plot scene presented in the plot clip from the perspective of content description to generate the plot description information of the plot clip. For example, the content description is "Character A has tears in his eyes. At the moment when the tears fall, character A walks towards character B and hugs character B. The two hug each other and cry bitterly..."

[0178] By analyzing the content, background music, and character voice and intonation presented in the plot clips, we can understand the scene layout and the plot performed by the characters, making the generated plot description information more in line with the plot clips, which helps to generate recommended content that better matches the plot clips and improves the accuracy of the generated recommended content.

[0179] As an embodiment, the plot description information may include a plot scene summarized from one of the above-mentioned description angles, or may include plot scenes summarized from multiple description angles, or may include plot scenes summarized from other angles, etc., without specific limitation. When including plot scenes summarized from multiple description angles, taking the plot scenes summarized from the above-mentioned four description angles as an example, the four described plot scenes may be arranged in sequence as the plot description information, or the four plot scenes may be semantically fused to obtain the plot description information, etc., without specific limitation. For example, a trained text recognition large model may be used to perform semantic recognition on the four obtained plot scenes, and based on the recognized semantics, the language may be reorganized to construct a text description, so that the corresponding plot description information may be output.

[0180] S203: Combine the obtained at least one plot description information with the object attribute information to establish and execute an object recommendation task.

[0181] After obtaining at least one plot description information and object attribute information, each plot description information can be combined with the object attribute information to establish and execute a corresponding object recommendation task; or at least one plot description information can be combined with the object attribute information to establish and execute an object recommendation task, etc., without specific limitation.

[0182] There are many ways to combine plot description information and object attribute information to establish an object recommendation task. For example, the plot description information and object attribute information are combined in the form of key-value pairs to obtain an object recommendation task. For another example, the plot description information and object attribute information are added to a preset task template to obtain an object recommendation task, etc., without specific limitation.

[0183] The following introduces the process of establishing an object recommendation task by taking the method of adding plot description information and object attribute information to a preset task template to obtain an object recommendation task as an example.

[0184] A pre-stored task template is obtained, where the task template at least indicates: object attribute information and at least one plot description information adding position, and content constraint conditions of candidate recommended content. The content constraint conditions are used to constrain at least the text form and semantic sentiment tendency of the candidate recommended content.

[0185] According to the information adding position indicated by the task template, at least one plot description information and object attribute information are added to the task template respectively to establish an object recommendation task.

[0186] Therefore, based on the object recommendation task, the text form and semantic sentiment tendency of the candidate recommendation content to be generated can be clarified, avoiding the generation of candidate recommendation content with exaggerated text form, non-compliant semantics, or semantically negative sentiment tendency, thereby improving the accuracy of the generated candidate recommendation content.

[0187] The task template may also indicate a role that performs the object recommendation task, so as to more clearly display the task goal of the object recommendation task, for example, the role is a "copy planner" or the like.

[0188] The task template can also indicate the execution capabilities of the above roles, so as to more clearly demonstrate the task background of the object recommendation task. For example, the execution capability of the copywriter role is "good at writing attractive recommendation content for the recommended objects, able to write recommendation content based on a full understanding of the delivery scenarios of the recommended content, the object attribute information of the recommended objects and the video to be processed, and the recommended content is presented in the form of a patch in the smallest angular area during the specified playback period of the video to be processed."

[0189] The task template can also indicate the task details, so as to more clearly display the task content of the object recommendation task, for example, the task details are "I will provide the plot description information of a plot clip and the object attribute information of the recommended object. It is necessary to combine the plot description information and the object attribute information to write recommended content that meets the content constraints, so as to attract viewers who are watching the video to be processed to trigger clicks, forwarding, comments or purchases on the recommended content. The recommended content written needs to be fully integrated into the plot scenes presented in the plot clips to improve the acceptance of the viewers."

[0190] Content constraints can also be used to constrain the generation format of candidate recommended content, for example, to generate candidate recommended content in the generation format of "candidate recommended content_1: XXXX". If multiple content types are preset, such as target demand scenario type, hot topic type, and demand scenario extension type, then when generating candidate recommended content, one content type can be selected from the preset multiple content types, and the content type can be used as a content constraint to generate candidate recommended content that meets the content type; each content type can also be used as a content constraint to sequentially generate multiple candidate recommended content that meets multiple content types.

[0191] For example, the recommendation object is cold medicine, and the plot clip presents a snowy scene, then "snowing" is the target demand scene, and the candidate recommendation content of the target demand scene type that can be generated is "It is easy to catch a cold when the temperature drops due to snow, so cold medicine should be kept on hand."

[0192] "Office" is an extension of the demand scenario, and the candidate recommendation content of the demand scenario extension type that can be generated is "You can also keep cold medicine in the office."

[0193] If the hot topics include topics about character A, then "character A" is the hot topic, and candidate recommendation content of the hot topic type may be generated as "character A is also warm and caring."

[0194] As an embodiment, when establishing an object recommendation task, the object attribute information can be combined with the topic information of the hot topic in the current network resources to obtain combined information, and then the plot description information is combined with the combined information to establish a corresponding object recommendation task. This can improve the attractiveness of the generated recommended content and improve the conversion rate of the recommended content.

[0195] The following takes the combination method of adding topic information of hot topics in the task template as an example to introduce the process of establishing an object recommendation task.

[0196] The task template can also indicate the location where information with topic information is added. In this case, the topic information of the corresponding initial topic set can be obtained from the preset multiple network resources respectively. For example, the first topic information of the first initial topic set can be obtained from the first network resource, and the second topic information of the second initial topic set can be obtained from the second network resource, etc., so that the hot topics in multiple network resources can be comprehensively considered to improve the accuracy of the determined hot topics.

[0197] Please refer to Fig. 6A , which is a schematic diagram of topic information of an initial topic set, including the top 11 initial topics in terms of discussion popularity, and the end of each initial topic is marked with a discussion popularity value, etc.

[0198] Based on the topic keywords in the topic information of each of the obtained initial topic sets, multiple target topic sets matching the video to be processed are selected from each of the initial topic sets, and at least one target topic matching the video to be processed is selected from each of the target topic sets. The topic information includes relevant information of each initial topic in the corresponding initial topic set. For example, the topic information includes the discussion heat ranking, like heat ranking, comment heat ranking, celebrity discussion heat ranking, positive review heat ranking, etc. of each initial topic in the corresponding initial topic set, without specific limitation.

[0199] The topic keywords may include the topic keywords contained in each initial topic in the corresponding initial topic set. The initial topics are, for example, character topics, plot topics, network meme topics, etc., without specific limitation.

[0200] For example, if there is at least one plot description information among the plot description information of the video to be processed that contains a topic keyword in a topic information, then it can be determined that the initial topic set corresponding to the topic information matches the video to be processed; for another example, if the number of plot description information containing a topic keyword in a topic information among the plot description information of the video to be processed accounts for more than a preset proportion of the total number of plot description information, then it can be determined that the initial topic set corresponding to the topic information matches the video to be processed; for another example, if there is one plot description information among the plot description information of the video to be processed, such that when the number of appearances of a topic keyword in a topic information in the plot description information reaches a preset number, then it can be determined that the initial topic set corresponding to the topic information matches the video to be processed, etc., without specific limitation.

[0201] Then, the obtained target topics, at least one plot description information and object attribute information can be added to the task template according to the information adding position indicated by the task template to establish the corresponding object recommendation task.

[0202] As an embodiment, when selecting at least one target topic matching the video to be processed from each target topic set based on the topic keywords in the topic information of each of the obtained initial topic sets, the topic keywords of each initial topic contained in each initial topic set can be extracted based on the topic information of each of the obtained initial topic sets. Based on the obtained topic keywords, an intermediate topic set whose corresponding topic keywords are contained in at least one plot description information is selected from each initial topic set to obtain multiple intermediate topic sets, and an intermediate topic whose corresponding topic keywords are contained in at least one plot description information is selected from each intermediate topic set to obtain at least one intermediate topic.

[0203] Based on the topic discussion heat of at least one intermediate topic in each intermediate topic set quantified from the respective topic information of multiple intermediate topic sets, the intermediate topics with topic discussion heat greater than a preset heat threshold are taken as target topics, and the intermediate topic set containing the target topics is taken as the target topic set, to obtain multiple target topic sets and at least one target topic in each target topic set.

[0204] If the topic information includes the topic discussion heat of each intermediate topic, then the topic discussion heat can be directly obtained; if the topic information includes the number of likes, forwardings and comments of each intermediate topic, then the weighted sum of the number of likes, forwardings and comments can be performed to obtain the corresponding topic discussion heat; if the topic information includes the comment content of each intermediate topic, then the content difference between each comment content can be determined based on each comment content. The greater the content difference and the greater the number of comment contents, the higher the topic discussion heat; the smaller the content difference or the smaller the number of comment contents, the lower the topic discussion heat. Therefore, based on the content difference and the number of comment contents, the topic information can be quantified to obtain the corresponding topic discussion heat.

[0205] By generating recommended content through target topics that are hotly discussed and match the video to be processed, the generated recommended content can not only fit the plot, but also fit the current hot spots, further improving the attractiveness of the generated recommended content.

[0206] Please refer to Figure 6B , each initial topic set is initial topic set_1, initial topic set_2, ... and initial topic set_N. Each initial topic set contains the top 11 initial topics in the topic ranking obtained from a network platform. The topic ranking also contains the topic discussion heat of each initial topic (such as the number at the end of the topic in the figure). Then the topic information of each initial topic set contains the corresponding initial topics and their topic discussion heat.

[0207] Taking an initial topic set_1 as an example, the topic keywords of each initial topic included in the initial topic set_1 are extracted from the topic information of the initial topic set_1. For example, one topic keyword is extracted from each initial topic, then there are 11 topic keywords, namely "XXXX_1", "XXXX_2", ... and "XXXX_11".

[0208] If there is a topic keyword among these 11 topic keywords included in each plot description information, then the initial topic set_1 is used as the intermediate topic set_1; or, when the number of words of the topic keywords included in each plot description information among these 11 topic keywords is greater than the preset word number threshold, the initial topic set_1 is used as the intermediate topic set_1; or, when the ratio between the number of words of the topic keywords included in each plot description information among these 11 topic keywords and the total number 11 of these 11 topic keywords is greater than the preset ratio, the initial topic set_1 is used as the intermediate topic set_1, etc. There is no specific restriction, and the process of selecting the intermediate topic is similar, which will not be repeated here.

[0209] If the initial topic set_1 is the intermediate topic set_1, then from the intermediate topics included in the intermediate topic set_1, the intermediate topics whose corresponding topic keywords are included in each plot description information are determined to obtain at least one intermediate topic, for example, including "intermediate topic_2".

[0210] Based on the topic information of the intermediate topic set_1, the topic discussion heat of at least one intermediate topic is determined, for example, the topic discussion heat of the 11 intermediate topics includes "1152408", "894063", ... and "452252". If the preset heat threshold is "500000", then the topic discussion heat "894063" of "intermediate topic_2" is greater than the preset heat threshold "500000", therefore, "intermediate topic_2" can be determined as the target topic.

[0211] As an embodiment, after the object recommendation task is established, the object recommendation task can be executed for each plot description information. When there are multiple plot description information, the object recommendation task can be executed for multiple plot description information in parallel, or it can be executed in sequence according to the order of the plot segments corresponding to the plot description information in the video to be processed, without specific limitation. By executing the object recommendation task, at least one candidate recommendation content that is suitable for each plot segment can be generated. Compared with the method of directly generating candidate recommendation content based on the complete video to be processed, or the method of directly filling in vocabulary based on the content template, the generation method provided by the embodiment of the present application is more flexible, and the generated candidate recommendation content is more suitable for the plot of its placement location, thereby improving the attractiveness of the recommended content.

[0212] The following is an example of performing an object recommendation task for a plot description information, please refer to S2031 to S2033. The execution process of other object recommendation tasks is similar and will not be repeated here.

[0213] S2031, extracting corresponding scene corpus from each plot scene described by the corresponding plot description information.

[0214] For example, if a plot scene is described as "On a snowy day, character A rides a horse, rushes to location A, and catches a cold on the way", then the scene corpus extracted for this plot scene may include "snowy day", "riding a horse to location A", etc.

[0215] S2032: extracting attribute corpora that match the at least one obtained scene corpus from the object attribute information.

[0216] For example, if the object attribute information includes "cold medicine", "treatment of colds, fatigue, runny nose, etc. caused by wind and cold", etc., then the extracted attribute corpus may include "cold" and "cold medicine", etc.

[0217] S2033: Fuse at least one scene corpus and its respective matching attribute corpus to generate corresponding candidate recommendation content.

[0218] For example, a scene corpus includes "snowy day" and "riding a horse to location A", and the attribute corpus includes "cold" and "cold medicine". Then the attribute corpus matching the scene corpus is "cold" and "cold medicine". By combining the scene corpus and the attribute corpus matching the scene corpus to form a sentence, we can obtain the candidate recommendation content "It is easy to catch a cold when the temperature drops due to snow, so cold medicine should always be on hand."

[0219] As an embodiment, a large model may be generated to execute the above steps S2031 to S2033, so that the generation process of candidate recommendation content is more intelligent. By utilizing the semantic understanding ability of the large model, richer and more accurate candidate recommendation content can be obtained.

[0220] Please refer to Figure 6C After the generation big model outputs the candidate recommendation content, the audit big model can be used to conduct content audit on the candidate recommendation content to check whether the candidate recommendation content meets the content constraints, whether it has a good recommendation effect, and whether it matches the plot description information and object attribute information, so as to ensure the accuracy of the final output candidate recommendation content.

[0221] Combine the plot description information, preset content constraints and object attribute information to establish a content review task. The content constraints are used to constrain at least the text form and semantic sentiment tendency of the candidate recommended content. Use the review big model to perform the content review task and generate the content review result. The content review result indicates whether the candidate recommended content generated by the big model needs to be modified.

[0222] If modification is required, the content review results can be fed back to the generated large model to instruct the generated large model to regenerate the corresponding candidate recommendation content; if modification is not required, the current candidate recommendation content can be used as the final output candidate recommendation content.

[0223] As an embodiment, when instructing the generation of a large model to regenerate corresponding candidate recommended content, only the portion that needs to be modified may be instructed to be regenerated, or all the candidate recommended content may be instructed to be regenerated, without specific limitation.

[0224] The following is an example of the process of using the audit model to perform content audit tasks. By only modifying the sub-content that does not meet the content constraints, the amount of data calculation can be reduced, the generation efficiency can be improved, the generation process of duplicate content can be simplified, and unnecessary computing resources can be avoided.

[0225] Based on the content constraint, each candidate recommended content is evaluated to obtain the corresponding evaluation result. The evaluation result is used to indicate the sub-content in the corresponding candidate recommended content that does not meet the content constraint. For example, the content constraint includes that the number of characters in the candidate recommended content is between 8 and 12 characters, and the candidate recommended content contains 13 characters, then the sub-content that does not meet the content constraint is the candidate recommended content itself. For another example, the content constraint includes that the candidate recommended content cannot contain words such as "smoking" and "cigarettes", and the candidate recommended content is "It is easy to catch a cold when it snows and the temperature drops. Cold medicine is more useful than cigarettes during the cold period", then the sub-content that does not meet the content constraint is "Cold medicine is more useful than cigarettes during the cold period"; for another example, the candidate recommended content does not contain sub-content that does not meet the content constraint, then the sub-content that does not meet the content constraint is an empty value, such as "".

[0226] When it is determined that the sub-content indicated by the obtained evaluation result is not a null value, a content modification instruction is generated based on the evaluation result to obtain a content review result. The content modification instruction is used to instruct the generated large model to re-execute the object recommendation task to modify the corresponding sub-content. When it is determined that the sub-content indicated by the obtained evaluation result is a null value, a content review result is generated that indicates that the corresponding candidate recommended content meets the content constraint condition.

[0227] The generation of the big model and the review of the big model can be implemented using the trained multimodal big model. The trained multimodal big model can be fine-tuned through the collected sample data in the recommendation field so that it can be more adapted to the recommendation field. For example, the sample data is the brand information, recommended content and delivery video of the recommended content collected in the past. The trained multimodal big model is used as the generation big model to infer the content type of the recommended content and analyze the writing logic of the recommended content, so that the trained multimodal big model can classify and logically deduce the written recommended content. After the sample data is sorted out, the trained multimodal big model can be fine-tuned with the manually written recommended content as the target, so as to improve the understanding ability of the trained multimodal big model in the recommendation field.

[0228] S204: Select at least one target recommended content from the obtained candidate recommended content.

[0229] After obtaining each candidate recommended content, at least one target recommended content may be selected from the obtained candidate recommended content. For example, a target recommended content may be selected for each plot segment; another example is to select a target recommended content for the video to be processed; another example is to select a candidate recommended content whose plot relevance is greater than a preset relevance threshold as the target recommended content based on the plot relevance between the candidate recommended content and the corresponding plot segment, etc., without specific limitation.

[0230] As an embodiment, the process of determining whether a candidate recommended content can be used as a target recommended content is introduced as an example. The process of determining other candidate recommended content is similar and will not be repeated here.

[0231] Semantic recognition is performed on the candidate recommended content and the corresponding plot description information respectively to obtain content semantic features and description information semantic features. Based on the feature similarity between the content semantic features and the description information semantic features, the plot relevance between the candidate recommended content and the corresponding plot description information is determined. When it is determined that the plot relevance is greater than a preset relevance threshold, the candidate recommended content is used as the target recommended content; when it is determined that the plot relevance is not greater than the preset relevance threshold, the candidate recommended content is deleted.

[0232] Therefore, not only can the target recommended content be automatically generated, but also the appropriate placement position, i.e. the corresponding plot segment, can be selected for the target recommended content. The placement of recommended content can accurately lock on a specific scene, stimulate strong emotional resonance of the audience, and implant the recommended object into people's hearts.

[0233] The following is an example of the method for generating recommended content provided in the embodiment of the present application, in which the recommended object in the advertising scenario is an advertising product, the recommended content is an advertising slogan, the object attribute information includes the product functional characteristics, product usage scenario, product brand story and advertising appeal of the advertising product, and the content constraint conditions include advertising-related regulations and the word count range of the advertising slogan. Please refer to Fig. 7A .

[0234] After obtaining the video to be processed, the video frame sequence of the video to be processed is obtained, and the PySceneDetect technology is used to perform storyboard processing on the video frame sequence to obtain multiple storyboard segments. The multiple storyboard segments are clustered to obtain multiple plot segments.

[0235] The ASR speech recognition model is used to extract the speech segments of each plot segment, and the plot lines of the plot segment are recognized based on the speech segments.

[0236] The OCR text recognition technology is used to perform text detection on each video frame in each plot segment to obtain the text area contained in each video frame, and then perform text recognition on the text area to obtain the screen text contained in the plot segment.

[0237] By using face recognition technology, we can identify the characters that appear in each plot segment, their relative positions in the corresponding video frames, the relative time at which they appear in the plot segment, and their movements.

[0238] The trained multimodal large model is used to identify each storyboard segment and each plot segment to obtain the content description of the corresponding storyboard segment and the content description of the corresponding plot segment.

[0239] Thus, the plot lines, screen texts, characters, relative positions, relative time, character movements, content descriptions of each plot segment, and the content descriptions of each storyboard segment can be input into the trained text recognition large model, and the trained text recognition large model is used to recognize the corresponding semantics, and the language is reorganized to generate plot description information for each plot segment. Each plot description information includes the corresponding storyboard plot descriptions of each storyboard segment and the fragment plot descriptions of the corresponding plot segment.

[0240] For example, the video to be processed contains N plot segments, namely, plot segment_1, plot segment_2, ..., and plot segment_N. Plot segment_1 includes n1 storyboard segments, namely, storyboard segment_11, storyboard segment_12, ..., and storyboard segment_1n1; plot segment_2 includes n2 storyboard segments, namely, storyboard segment_21, storyboard segment_22, ..., and storyboard segment_2n1; and so on. Then, the storyboard description of plot segment_1 is storyboard description_1, and so on. The storyboard description of storyboard segment_11 is storyboard description_11, and so on.

[0241] Get the object attribute information of the recommended object and get at least one target topic. Add the object attribute information of the advertised product, such as product features, product usage scenarios, product brand stories, and advertising appeals, and at least one target topic to the task template, and add the plot description information to the task template to establish the object recommendation tasks corresponding to each plot segment. Please refer to Figure 7B , a schematic diagram of an object recommendation task.

[0242] Input each object recommendation task into the trained generative model to obtain at least one candidate recommendation content output by the trained generative model for each object recommendation task. Figure 7C , which is an output schematic diagram of at least one candidate recommendation content.

[0243] After at least one candidate recommendation is output, a content review task is established based on at least one candidate recommendation, the corresponding plot description information and object attribute information. Please refer to Fig.7D , which is a schematic diagram of a content review task.

[0244] The content review task is input into the trained review model to obtain the content review result output by the trained review model. When the content review result indicates that a candidate recommendation content needs to be modified, the content review result also includes modification suggestions, and the content review result is fed back to the trained generation model, so that the trained generation model regenerates the candidate recommendation content based on the modification suggestions.

[0245] After obtaining at least one candidate recommended content corresponding to each plot segment, based on the plot relevance output by the trained large generation model, when the plot relevance is greater than a preset relevance threshold, the candidate recommended content is used as the target recommended content.

[0246] In the embodiment of the present application, the whole process automatically generates target recommendation content, and finds suitable plot clips to add adapted target recommendation content. Through the plot description information, the plot of the TV series can be understood at a high level, and more time periods suitable for the release of recommended content can be intelligently excavated. The plot scene and the recommended object are organically connected through the target recommendation content, reducing the sense of disobedience of the target recommendation content and improving the acceptance and effect of the target recommendation content. Through object attribute information and hot topics, etc., the right target recommendation content is automatically created, accurately locking in a specific plot scene, stimulating strong emotional resonance among the audience, and implanting the recommended object and effect into people's hearts.

[0247] Based on the same inventive concept, the present application embodiment provides a recommended content generation device, which can realize the functions corresponding to the aforementioned recommended content generation method. Figure 8 , the device includes an acquisition module 801 and a processing module 802, wherein:

[0248] Acquisition module 801: used to acquire object attribute information of at least one plot segment and a recommended object contained in a video to be processed;

[0249] Processing module 802: used to extract plot description information of at least one plot segment respectively; wherein each plot description information is used to summarize the plot scene presented by the corresponding plot segment from at least one description angle;

[0250] The processing module 802 is further used to: combine the obtained at least one plot description information with the object attribute information respectively, establish and execute the object recommendation task; wherein, for each plot description information, the object recommendation task is executed in the following manner: extracting the corresponding scene corpus from each plot scene described by the corresponding plot description information; extracting the attribute corpus that matches the obtained at least one scene corpus respectively from the object attribute information; respectively fusing the at least one scene corpus and its respectively matched attribute corpus to generate the corresponding candidate recommendation content;

[0251] The processing module 802 is further configured to select at least one target recommended content from the obtained candidate recommended content.

[0252] In a possible embodiment, the acquisition module 801 is specifically used for:

[0253] Get the video to be processed;

[0254] Performing storyboard processing on the video to be processed to obtain a plurality of storyboard segments, wherein different storyboard segments have different viewing angles;

[0255] Extract the plot features of multiple storyboard clips respectively;

[0256] Based on the obtained plot features, clustering processing is performed on multiple groups of storyboard segments whose plot similarities meet similarity conditions in the multiple storyboard segments to obtain at least one plot segment.

[0257] In a possible embodiment, the acquisition module 801 is specifically used for:

[0258] Based on the obtained plot features, determine the temporal correlation and spatial correlation between every two storyboard segments in the plurality of storyboard segments; wherein the temporal correlation represents: the degree of continuity of the plot between the corresponding two storyboard segments; and the spatial correlation represents: the degree of matching between the spatial layouts of the plot elements contained in the corresponding two storyboard segments;

[0259] Determine the plot similarity between every two storyboard segments in the plurality of storyboard segments based on the obtained temporal correlation and spatial correlation between every two storyboard segments;

[0260] Clustering is performed on multiple groups of storyboard segments whose plot similarities satisfy similarity conditions in the multiple storyboard segments to obtain at least one plot segment.

[0261] In a possible embodiment, when the at least one description angle includes an audio line angle, the processing module 802 is specifically configured to:

[0262] Perform audio separation processing on the video to be processed to obtain an audio file;

[0263] Based on the audio file, perform the following operations for at least one plot segment:

[0264] Extracting acoustic features of the audio clip corresponding to the plot clip from the audio file;

[0265] Filter the sub-segments in the audio segment whose acoustic features do not meet the speech feature screening conditions to obtain the speech segment corresponding to the plot segment;

[0266] The plot lines contained in the plot segment extracted from the voice segment are used as a plot scene presented by the plot segment summarized from the perspective of the audio lines to generate plot description information of the plot segment.

[0267] In a possible embodiment, when the at least one description angle includes a picture text angle, the processing module 802 is specifically configured to:

[0268] For at least one story segment, perform the following operations:

[0269] Performing text detection on multiple video frames contained in the plot segment respectively, and determining text regions containing text in corresponding video frames;

[0270] Performing text recognition on the text regions of the multiple video frames respectively to obtain the screen texts contained in the multiple video frames;

[0271] The obtained screen texts are used as a plot scene presented by the plot segment summarized from the perspective of the screen texts to generate plot description information of the plot segment.

[0272] In a possible embodiment, when the at least one description angle includes a character movement angle, the processing module 802 is specifically configured to:

[0273] For at least one story segment, perform the following operations:

[0274] Performing face recognition on multiple video frames contained in the plot clip respectively to determine the person to whom the face contained in the corresponding video frame belongs;

[0275] Determine the relative position between each character included in the plot segment and the corresponding video frame;

[0276] The relative time when each character in the plot segment appears in the plot segment and the relative position in the corresponding video frame are counted respectively to obtain the character movement position;

[0277] The obtained character movements are used as a plot scene presented by the plot segment summarized from the perspective of the character movements, and plot description information of the plot segment is generated.

[0278] In a possible embodiment, when the at least one description angle includes a content description angle, the processing module 802 is specifically configured to:

[0279] For at least one story segment, perform the following operations:

[0280] Extract visual and audio features of plot clips;

[0281] Using the trained multimodal recognition model, the fusion features of visual features and audio features are used for text recognition to obtain the content description of the plot clips;

[0282] The obtained content description is used as a plot scene presented by the plot segment summarized from the perspective of content description, and plot description information of the plot segment is generated.

[0283] In a possible embodiment, the processing module 802 is specifically configured to:

[0284] Obtaining a pre-stored task template; wherein the task template at least indicates: object attribute information and the location where the information of at least one plot description information is added, and content constraints of the candidate recommended content; the content constraints are at least used to constrain the text form and semantic sentiment tendency of the candidate recommended content;

[0285] According to the information adding position indicated by the task template, at least one plot description information and object attribute information are added to the task template respectively to establish an object recommendation task.

[0286] In a possible embodiment, the task template further indicates the location where the information of the topic information is added; the processing module 802 is specifically used for:

[0287] Respectively obtain topic information of a corresponding initial topic set from a plurality of preset network resources;

[0288] Based on the topic keywords in the topic information of each of the obtained initial topic sets, multiple target topic sets matching the video to be processed are selected from each of the initial topic sets, and at least one target topic matching the video to be processed is selected from each of the target topic sets;

[0289] According to the information adding position indicated by the task template, each target topic, at least one plot description information and object attribute information obtained are added to the task template to establish an object recommendation task.

[0290] In a possible embodiment, the processing module 802 is specifically configured to:

[0291] Extracting topic keywords of each initial topic contained in each initial topic set based on the obtained topic information of each initial topic set;

[0292] Based on the obtained topic keywords, from each initial topic set, an intermediate topic set whose corresponding topic keyword is included in at least one plot description information is selected to obtain a plurality of intermediate topic sets, and from each intermediate topic set, an intermediate topic whose corresponding topic keyword is included in at least one plot description information is selected to obtain at least one intermediate topic;

[0293] Based on the topic discussion heat of at least one intermediate topic in each intermediate topic set quantified from the respective topic information of multiple intermediate topic sets, the intermediate topics with topic discussion heat greater than a preset heat threshold are taken as target topics, and the intermediate topic set containing the target topics is taken as the target topic set, to obtain multiple target topic sets and at least one target topic in each target topic set.

[0294] In a possible embodiment, the processing module 802 is specifically configured to:

[0295] For each plot description information in the established object recommendation task, perform the following operations respectively:

[0296] The large model is used to perform the following operations: extracting corresponding scene corpora from each plot scene described in the plot description information; extracting attribute corpora that match the at least one scene corpus obtained from the object attribute information; fusing the at least one scene corpus and its matching attribute corpora to generate corresponding candidate recommendation content;

[0297] The plot description information, the preset content constraint conditions and the object attribute information are combined to establish a content review task; wherein the content constraint conditions are at least used to constrain the text form and semantic sentiment tendency of the candidate recommended content;

[0298] The audit big model is used to perform content audit tasks and generate content audit results; wherein the content audit results indicate whether the candidate recommended content generated by the big model needs to be modified.

[0299] In a possible embodiment, the processing module 802 is specifically configured to:

[0300] Use the audit model to do the following:

[0301] Based on the content constraint condition, each candidate recommended content is evaluated to obtain a corresponding evaluation result; wherein the evaluation result is used to indicate the sub-content in the corresponding candidate recommended content that does not meet the content constraint condition;

[0302] When it is determined that the sub-content indicated by the obtained evaluation result is not a null value, a content modification instruction is generated based on the evaluation result to obtain a content review result; wherein the content modification instruction is used to instruct the generated large model to re-execute the object recommendation task to modify the corresponding sub-content;

[0303] When it is determined that the sub-content indicated by the obtained evaluation result is a null value, a content review result is generated that indicates that the corresponding candidate recommended content meets the content constraint condition.

[0304] In a possible embodiment, the processing module 802 is specifically configured to:

[0305] For each candidate recommendation content, perform the following operations:

[0306] Semantic recognition is performed on candidate recommendation content and corresponding plot description information to obtain content semantic features and description information semantic features;

[0307] Determine the plot relevance between the candidate recommended content and the corresponding plot description information based on the feature similarity between the content semantic features and the description information semantic features;

[0308] When it is determined that the plot relevance is greater than a preset relevance threshold, the candidate recommended content is used as the target recommended content.

[0309] Please refer to Fig. 9 , is a computer device 900 provided in an embodiment of the present application, and the computer device 900 can be, for example, Figure 1C The client 101 or the server 102 in the data storage program. The current version and the historical version of the data storage program and the application software corresponding to the data storage program can be installed on the computer device 900, and the computer device 900 includes a processor 980 and a memory 920. In some embodiments, the computer device 900 may include a display unit 940, and the display unit 940 includes a display panel 941 for displaying a user interactive operation interface, etc.

[0310] In a possible embodiment, the display panel 941 may be configured in the form of a liquid crystal display (LCD) or an organic light-emitting diode (OLED).

[0311] The processor 980 is used to read the computer program and then execute the method defined by the computer program. For example, the processor 980 reads the data storage program or file, so as to run the data storage program on the computer device 900 and display the corresponding interface on the display unit 940. The processor 980 may include one or more general-purpose processors and may also include one or more DSPs (Digital Signal Processors) to perform related operations to implement the technical solutions provided in the embodiments of the present application.

[0312] The memory 920 generally includes internal memory and external memory, and the internal memory may be a random access memory (RAM), a read-only memory (ROM), and a cache (CACHE), etc. The external memory may be a hard disk, an optical disk, a USB disk, a floppy disk, or a tape drive, etc. The memory 920 is used to store computer programs and other data, and the computer program includes applications corresponding to each client, etc. Other data may include data generated after the operating system or application is run, and the data includes system data (such as configuration parameters of the operating system) and user data. In the embodiment of the present application, the computer program is stored in the memory 920, and the processor 980 executes the computer program in the memory 920 to implement any of the methods discussed in the previous figure.

[0313] The display unit 940 is used to receive input digital information, character information or contact touch operation / contactless gesture, and generate signal input related to user settings and function control of the computer device 900. Specifically, in the embodiment of the present application, the display unit 940 may include a display panel 941. The display panel 941 is, for example, a touch screen, which can collect user touch operations on or near it (such as operations performed by the user using fingers, stylus, or any other suitable object or accessory on or on the display panel 941), and drive corresponding connection devices according to a pre-set program.

[0314] In a possible embodiment, the display panel 941 may include two parts: a touch detection device and a touch controller. The touch detection device detects the touch position of the player, detects the signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts it into the touch point coordinates, and then sends it to the processor 980, and can receive and execute the command sent by the processor 980.

[0315] The display panel 941 may be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the display unit 940, in some embodiments, the computer device 900 may further include an input unit 930, which may include an image input device 931 and other input devices 932, wherein the other input devices may include, but are not limited to, one or more of a physical keyboard, a function key (such as a volume control key, a switch key, etc.), a trackball, a mouse, a joystick, and the like.

[0316] In addition to the above, the computer device 900 may also include a power supply 990 for supplying power to other modules, an audio circuit 960, a near field communication module 970, and an RF circuit 910. The computer device 900 may also include one or more sensors 950, such as an acceleration sensor, a light sensor, a pressure sensor, etc. The audio circuit 960 specifically includes a speaker 961 and a microphone 962, etc. For example, the computer device 900 can collect the user's voice through the microphone 962 to perform corresponding operations, etc.

[0317] As an embodiment, the number of the processors 980 may be one or more, and the processor 980 and the memory 920 may be coupled or relatively independently configured.

[0318] As an example, Fig. 9 The processor 980 in the embodiment can be used to implement the following Figure 8 The functions of the acquisition module 801 and the processing module 802 in .

[0319] As an example, Fig. 9 The processor 980 in can be used to implement the corresponding functions of the server or terminal device discussed above.

[0320] A person of ordinary skill in the art can understand that: all or part of the steps of implementing the above method embodiment can be completed by a computer program. The aforementioned computer program can be stored in a computer-readable storage medium. When the computer program is executed, it executes the steps of the above method embodiment; and the aforementioned storage medium includes: a mobile storage device, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0321] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product, for example, through a computer program product, which is stored in a storage medium, including a computer program for a computer device to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0322] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for generating recommended content, characterized in that: include: Obtaining object attribute information of at least one plot segment and a recommended object contained in the video to be processed; Extracting the plot description information of the at least one plot segment respectively; wherein each plot description information is used to summarize the plot scene presented by the corresponding plot segment from at least one description angle; The at least one plot description information obtained is combined with the object attribute information to establish and execute an object recommendation task; wherein, for each plot description information, the object recommendation task is executed in the following manner: extracting corresponding scene corpus from each plot scene described by the corresponding plot description information; extracting attribute corpus that matches the at least one scene corpus obtained from the object attribute information; and fusing the at least one scene corpus and its matching attribute corpus to generate corresponding candidate recommendation content; At least one target recommended content is selected from the obtained candidate recommended contents.

2. The method according to claim 1, characterized in that The step of obtaining at least one plot segment contained in the video to be processed includes: Get the video to be processed; Performing storyboard processing on the video to be processed to obtain a plurality of storyboard segments, wherein different storyboard segments have different viewing angles; Extracting plot features of each of the plurality of storyboard segments respectively; Based on the obtained plot features, clustering processing is performed on multiple groups of shot segments whose plot similarities meet the similarity condition in the multiple storyboard segments to obtain at least one plot segment.

3. The method according to claim 2, characterized in that The step of clustering the multiple groups of shot segments whose plot similarities satisfy the similarity condition in the multiple shot segments based on the obtained plot features to obtain at least one plot segment comprises: Based on the obtained plot features, determine the temporal correlation and spatial correlation between every two storyboard segments in the plurality of storyboard segments; wherein the temporal correlation represents: the degree of continuity of the plot between the corresponding two storyboard segments; and the spatial correlation represents: the degree of matching between the spatial layouts of the plot elements contained in the corresponding two storyboard segments; Determine the plot similarity between every two storyboard segments in the plurality of storyboard segments based on the obtained temporal correlation and spatial correlation between every two storyboard segments; Clustering is performed on multiple groups of storyboard segments whose plot similarities satisfy a similarity condition in the multiple storyboard segments to obtain at least one plot segment.

4. The method according to claim 1, characterized in that: When the at least one description angle includes an audio dialogue angle, respectively extracting the plot description information of the at least one plot segment includes: Performing audio separation processing on the video to be processed to obtain an audio file; Based on the audio file, for the at least one plot segment, the following operations are performed respectively: Extracting acoustic features of the audio segment corresponding to the plot segment from the audio file; Filtering the sub-segments of the audio segment whose acoustic features do not meet the voice feature screening condition to obtain the voice segment corresponding to the plot segment; The plot lines contained in the plot segment extracted from the voice segment are used as a plot scene presented by the plot segment summarized from the perspective of the audio lines to generate plot description information of the plot segment.

5. The method according to claim 1, characterized in that When the at least one description angle includes a screen text angle, respectively extracting the plot description information of the at least one plot segment includes: For the at least one plot segment, the following operations are performed respectively: Performing text detection on a plurality of video frames included in the plot segment respectively to determine text regions containing text in corresponding video frames; Performing text recognition on the text regions of the multiple video frames respectively to obtain the screen texts contained in the multiple video frames respectively; The obtained screen texts are used as a plot scene presented by the plot segment summarized from the perspective of the screen texts to generate plot description information of the plot segment.

6. The method according to claim 1, characterized in that When the at least one description angle includes a character movement angle, respectively extracting the plot description information of the at least one plot segment includes: For the at least one plot segment, the following operations are performed respectively: Performing face recognition on a plurality of video frames included in the plot segment respectively to determine the characters to which the faces included in the corresponding video frames belong; Respectively determining the relative position between each character included in the plot segment and the corresponding video frame; Respectively counting the relative time at which each character included in the plot segment appears in the plot segment and the relative position in the corresponding video frame to obtain the character movement; The obtained character movements are used as a plot scene presented by the plot segment summarized from the perspective of the character movements, and plot description information of the plot segment is generated.

7. The method according to claim 1, characterized in that When the at least one description angle includes a content description angle, respectively extracting the plot description information of the at least one plot segment includes: For the at least one plot segment, the following operations are performed respectively: Extracting visual features and audio features of the plot segment; Using a trained multimodal recognition model, performing text recognition on the fusion features of the visual features and the audio features to obtain a content description of the plot segment; The obtained content description is used as a plot scene summarized from the perspective of the content description, and plot description information of the plot segment is generated.

8. The method according to any one of claims 1 to 7, characterized in that: The step of combining the obtained at least one plot description information with the object attribute information to establish an object recommendation task includes: Obtaining a pre-stored task template; wherein the task template at least indicates: object attribute information and the location where the information of at least one plot description information is added, and content constraints of the candidate recommended content; the content constraints are at least used to constrain the text form and semantic sentiment tendency of the candidate recommended content; According to the information adding position indicated by the task template, the at least one plot description information and the object attribute information are respectively added to the task template to establish an object recommendation task.

9. The method according to claim 8, characterized in that The task template also indicates the location where information with topic information is added; The step of adding the at least one plot description information and the object attribute information to the task template according to the information adding position indicated by the task template, and establishing the object recommendation task, comprises: Acquire topic information of a corresponding initial topic set from a plurality of preset network resources respectively; Based on the topic keywords in the topic information of each of the obtained initial topic sets, multiple target topic sets matching the video to be processed are selected from the initial topic sets, and at least one target topic matching the video to be processed is selected from each of the target topic sets; According to the information adding position indicated by the task template, the obtained target topics, the at least one plot description information and the object attribute information are added to the task template respectively to establish an object recommendation task.

10. The method according to claim 8, characterized in that The step of selecting, based on topic keywords in topic information of each of the obtained initial topic sets, a plurality of target topic sets matching the video to be processed from the initial topic sets, and selecting, from each of the target topic sets, at least one target topic matching the video to be processed, comprises: Extracting topic keywords of each initial topic contained in each initial topic set based on the obtained topic information of each initial topic set; Based on the obtained topic keywords, from the initial topic sets, an intermediate topic set whose corresponding topic keyword is included in the at least one plot description information is selected to obtain a plurality of intermediate topic sets, and from each of the intermediate topic sets, an intermediate topic whose corresponding topic keyword is included in the at least one plot description information is selected to obtain at least one intermediate topic; Based on the topic discussion heat of at least one intermediate topic in each of the intermediate topic sets quantified from the respective topic information of the multiple intermediate topic sets, the intermediate topic whose topic discussion heat is greater than a preset heat threshold is taken as the target topic, and the intermediate topic set containing the target topic is taken as the target topic set, to obtain the multiple target topic sets and the at least one target topic in each of the target topic sets.

11. The method according to any one of claims 1 to 7, characterized in that: The performing of the object recommendation task includes: For each plot description information in the established object recommendation task, perform the following operations: The large model is used to perform the following operations: extracting corresponding scene corpora from each plot scene described in the plot description information; extracting attribute corpora that match the at least one scene corpus obtained from the object attribute information; and fusing the at least one scene corpus and its matching attribute corpora to generate corresponding candidate recommendation content. The plot description information, the preset content constraint conditions and the object attribute information are combined to establish a content review task; wherein the content constraint conditions are at least used to constrain the text form and semantic sentiment tendency of the candidate recommended content; The content audit task is performed using the audit big model to generate content audit results; wherein the content audit results indicate whether the candidate recommended content generated by the generation big model needs to be modified.

12. The method according to claim 11, characterized in that The use of the audit big model to perform the content audit task and generate content audit results includes: Use the audit model to do the following: Based on the content constraint condition, perform content evaluation on each of the candidate recommended contents to obtain a corresponding evaluation result; wherein the evaluation result is used to indicate the sub-content in the corresponding candidate recommended content that does not meet the content constraint condition; When it is determined that the sub-content indicated by the obtained evaluation result is not a null value, a content modification instruction is generated based on the evaluation result to obtain a content review result; wherein the content modification instruction is used to instruct the generated large model to re-execute the object recommendation task to modify the corresponding sub-content; When it is determined that the sub-content indicated by the obtained evaluation result is a null value, a content review result is generated, indicating that the corresponding candidate recommended content meets the content constraint condition.

13. The method according to claim 11, characterized in that The step of selecting at least one target recommended content from the obtained candidate recommended content includes: For each candidate recommendation content, the following operations are performed respectively: Semantically identifying the candidate recommended content and the corresponding plot description information to obtain content semantic features and description information semantic features; Determining the plot relevance between the candidate recommended content and the corresponding plot description information based on the feature similarity between the content semantic feature and the description information semantic feature; When it is determined that the plot relevance is greater than a preset relevance threshold, the candidate recommended content is used as the target recommended content.

14. A recommended content generating device, characterized in that: include: An acquisition module: used to acquire object attribute information of at least one plot segment and a recommended object contained in the video to be processed; Processing module: used to extract the plot description information of the at least one plot segment respectively; wherein each plot description information is used to summarize the plot scene presented by the corresponding plot segment from at least one description angle; The processing module is further used to: combine the obtained at least one plot description information with the object attribute information respectively, establish and execute the object recommendation task; wherein, for each plot description information, the object recommendation task is executed in the following manner: extract corresponding scene corpus from each plot scene described by the corresponding plot description information; extract attribute corpus that matches the obtained at least one scene corpus respectively from the object attribute information; respectively fuse the at least one scene corpus and its respectively matched attribute corpus to generate corresponding candidate recommendation content; The processing module is further configured to select at least one target recommended content from the obtained candidate recommended content.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 13 is implemented.

16. A computer device, characterized in that: include: A memory for storing program instructions; The processor is used to call the program instructions stored in the memory, and execute the method according to any one of claims 1 to 13 according to the obtained program instructions.

17. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the method according to any one of claims 1 to 13.

Images