Data processing method, device and equipment and readable storage medium

Abstract

The invention discloses a data processing method, apparatus and device, and a readable storage medium. The method comprises the steps of obtaining video data; performing script analysis on the video data to obtain a video script; the video script comprises video clip scripts corresponding to a plurality of video clips respectively; obtaining a bullet screen special effect material set, and generating a prompt word containing the bullet screen special effect material set; performing analysis matching on each video clip script and each bullet screen special effect material in a bullet screen special effect material set according to a bullet screen analysis instruction in the cue word to obtain bullet screen configuration suggestion information with a suggestion output format indicated by the cue word; the bullet screen configuration suggestion information comprises a bullet screen special effect material suggested and configured for a target video clip in the plurality of video clips and suggestion reason information. According to the invention, the configuration efficiency of the bullet screen special effect and the configuration effect of the bullet screen special effect can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data processing method, apparatus, device, and readable storage medium. Background Art

[0002] Operators can first watch videos to find scenes that require barrage effects. Based on existing barrage effects resources, they can then select the ones that match the scenes and configure them on the barrage management platform. However, if operators configure barrage effects that don't closely match the scenes, the effects' effectiveness may be reduced. Furthermore, since the entire configuration process relies heavily on manual operations, this can lead to low efficiency. Summary of the Invention

[0003] The embodiments of the present application provide a data processing method, device, computer, storage medium and program product, which can improve the configuration efficiency and configuration effect of barrage special effects.

[0004] On the one hand, an embodiment of the present application provides a data processing method, the method comprising:

[0005] Get video data;

[0006] Perform script analysis on the video data to obtain a video script; the video script includes video clip scripts corresponding to multiple video clips;

[0007] Obtain a collection of barrage special effects materials and generate prompt words containing the collection of barrage special effects materials;

[0008] According to the barrage analysis instructions in the prompt words, each video clip script is analyzed and matched with each barrage special effects material in the barrage special effects material set to obtain barrage configuration recommendation information with the recommended output format indicated by the prompt words; the barrage configuration recommendation information includes the barrage special effects materials recommended for the target video clip in multiple video clips and the reason information for the recommendation.

[0009] Among them, also include:

[0010] In response to a triggering operation on a configuration suggestion control in a bullet chat management page, multiple video set identification information is displayed; the multiple video set identification information includes target video set identification information corresponding to the video set to which the video data belongs;

[0011] In response to a trigger operation for the target video set identification information, multiple video set identification information associated with the video set to which the video data belongs is displayed on the bullet chat management page; the multiple video set identification information includes the target video set identification information corresponding to the video data;

[0012] In response to the triggering operation for the target video episode identification information, the barrage configuration suggestion information corresponding to the video data is displayed in the barrage management page; the barrage configuration suggestion information includes the special effect material attribute information and suggestion reason information of the barrage special effect material.

[0013] Among them, also include:

[0014] In response to the quality scoring operation for the barrage configuration suggestion information, the barrage configuration suggestion information and the input scoring result are displayed in association; the scoring result is used to indicate whether the barrage special effect material indicated by the barrage configuration suggestion information is configured for the video data.

[0015] The video data is parsed to obtain the video script, including:

[0016] The video frame in the video data is subjected to feature extraction through the video frame feature encoding layer of the multimodal model to obtain a video frame feature vector;

[0017] Perform text recognition on the video frames to obtain the text information in the video data, and extract the features of the text information through the text feature encoding layer of the multimodal model to obtain the text feature vector;

[0018] The audio information in the video data is extracted through the audio feature encoding layer of the multimodal model to obtain an audio feature vector;

[0019] Performing feature vector fusion on the text feature vector, the video frame feature vector, and the audio feature vector associated with the same video frame in the video data to obtain a fusion vector corresponding to each video frame in the video data;

[0020] The fusion vectors corresponding to each video frame are concatenated to obtain a multimodal feature vector sequence, and a video script for the video data is generated based on the multimodal feature vector sequence.

[0021] The video script for the video data is generated based on the multimodal feature vector sequence, including:

[0022] Identify scene change frames in video data through multimodal models;

[0023] Divide the multimodal feature vector sequence based on the scene change frames to obtain multimodal feature vectors corresponding to multiple video clips;

[0024] Decoding the multimodal feature vectors corresponding to the plurality of video clips to obtain video clip scripts corresponding to the plurality of video clips;

[0025] The video clip scripts corresponding to the multiple video clips are combined into a video script of the video data.

[0026] The video script also includes the theme information of the video set to which the video data belongs; according to the barrage analysis instruction in the prompt word, each video clip script is analyzed and matched with each barrage special effect material in the barrage special effect material set, and barrage configuration suggestion information with the recommended output format indicated by the prompt word is obtained, including:

[0027] In the large language model, according to the barrage analysis instruction in the prompt word, a first barrage special effect material set associated with the theme information is selected from the barrage special effect material set;

[0028] Perform feature extraction on each video clip script according to the barrage analysis instruction to obtain a script feature vector corresponding to each video clip script, and perform feature extraction on each barrage special effect material in the first barrage special effect material set to obtain a special effect feature vector corresponding to each barrage special effect material;

[0029] The script attention vector sequence of the script feature vector is constructed through the large language model, and the script attention vector sequence and the special effect feature vector are matched for similarity to obtain the matching result;

[0030] Output the bullet screen configuration suggestion information in the suggested output format indicated by the prompt word according to the matching result.

[0031] The large language model includes an attention processing layer; the script attention vector sequence of the script feature vector constructed by the large language model includes:

[0032] Perform a dot product operation on the query parameter matrix in the attention processing layer and the script feature vector to obtain the query vector sequence of the self-attention function;

[0033] Perform a dot product operation on the key parameter matrix in the attention processing layer and the script feature vector to obtain the key vector sequence of the self-attention function;

[0034] Perform a dot product operation on the value parameter matrix in the attention processing layer and the script feature vector to obtain the value vector sequence of the self-attention function; the query parameter matrix, key parameter matrix, and value parameter matrix are all matrices composed of the learnable parameters in the attention processing layer;

[0035] A script attention vector sequence for the script feature vector is generated based on the query vector sequence, the key vector sequence, and the value vector sequence.

[0036] Outputting the bullet screen configuration suggestion information in the suggested output format indicated by the prompt word according to the matching result includes:

[0037] Obtain target barrage special effects materials that match the target video clip script in the video script according to the matching results; the barrage special effects material set includes the target barrage special effects materials;

[0038] Obtain target special effect material attribute information and target suggestion reason information of the target barrage special effect material according to the recommended output format;

[0039] Based on the target special effect material attribute information and the target suggestion reason information, the barrage configuration suggestion information having the suggested output format indicated by the prompt word is generated.

[0040] Among them, also include:

[0041] Obtain historical video data, historical barrage special effects materials, and historical barrage configuration suggestion information; historical barrage configuration suggestion information is generated based on the usage records of historical barrage special effects materials based on historical video data;

[0042] Input historical video data and historical barrage special effects materials into the initial large language model, and output predicted barrage configuration suggestion information through the initial large language model;

[0043] A training loss value is generated based on historical barrage configuration suggestion information and predicted barrage configuration suggestion information, and the model parameters of the initial large language model are adjusted based on the training loss value to obtain a large language model for generating barrage configuration suggestion information.

[0044] Among them, also include:

[0045] Get a set of suggested information; the suggested information set contains multiple suggested information, including bullet screen configuration suggested information;

[0046] Performing a quality score on the set of recommended information to obtain a score set; the score set contains the score results corresponding to each piece of recommended information, and the score results are positive or negative.

[0047] If the proportion of positive scoring results in the scoring set is greater than the proportion threshold, an automatic configuration service is set up in the barrage management platform; the automatic configuration service is used to instruct the barrage management platform to configure the barrage special effect materials indicated by the new barrage configuration suggestion information to the new video clip when the large language model outputs new barrage configuration suggestion information for the new video clip.

[0048] Among them, also include:

[0049] If the proportion of negative scoring results in the scoring set is greater than the proportion threshold, the positive scoring results in the scoring set are determined as positive sample data, and the negative scoring results in the scoring set are determined as negative sample data;

[0050] Re-fine-tune the large language model based on positive and negative sample data.

[0051] The acquisition of video data includes:

[0052] Get a video comment parsing request; the video comment parsing request includes video episode identification information;

[0053] Obtaining video set identification information in the video single set identification information according to the video barrage analysis request, and obtaining a video library; the video library includes multiple single-set videos respectively contained in multiple video sets;

[0054] A target video set corresponding to the video set identification information is searched in the video library, and a single-episode video corresponding to the video single-episode identification information among multiple single-episode videos associated with the target video set is determined as video data.

[0055] In one aspect, an embodiment of the present application provides a data processing device, the device comprising:

[0056] A transceiver module for acquiring video data;

[0057] The video script generation module is used to perform script analysis on the video data to obtain a video script; the video script includes video clip scripts corresponding to multiple video clips;

[0058] A prompt word generation module is used to obtain a set of barrage special effect materials and generate prompt words containing the set of barrage special effect materials;

[0059] The model processing module is used to analyze and match each video clip script with each barrage special effect material in the barrage special effect material set according to the barrage analysis instructions in the prompt word, and obtain barrage configuration recommendation information with the recommended output format indicated by the prompt word; the barrage configuration recommendation information includes the barrage special effect materials recommended for the target video clip in multiple video clips and the reason information for the recommendation.

[0060] In a possible implementation, the data processing device further includes a trigger display module, which is specifically configured to perform the following operations:

[0061] In response to a triggering operation on a configuration suggestion control in a bullet chat management page, multiple video set identification information is displayed; the multiple video set identification information includes target video set identification information corresponding to the video set to which the video data belongs;

[0062] In response to a trigger operation for the target video set identification information, multiple video set identification information associated with the video set to which the video data belongs is displayed on the bullet chat management page; the multiple video set identification information includes the target video set identification information corresponding to the video data;

[0063] In response to the triggering operation for the target video episode identification information, the barrage configuration suggestion information corresponding to the video data is displayed in the barrage management page; the barrage configuration suggestion information includes the special effect material attribute information and suggestion reason information of the barrage special effect material.

[0064] In a possible implementation, the trigger display module is further configured to perform the following operations:

[0065] In response to the quality scoring operation for the barrage configuration suggestion information, the barrage configuration suggestion information and the input scoring result are displayed in association; the scoring result is used to indicate whether the barrage special effect material indicated by the barrage configuration suggestion information is configured for the video data.

[0066] In one possible implementation, the video script generation module is used to perform script parsing on the video data. When the video script is obtained, the module is specifically used to perform the following operations:

[0067] The video frame in the video data is subjected to feature extraction through the video frame feature encoding layer of the multimodal model to obtain a video frame feature vector;

[0068] Perform text recognition on the video frames to obtain the text information in the video data, and extract the features of the text information through the text feature encoding layer of the multimodal model to obtain the text feature vector;

[0069] The audio information in the video data is extracted through the audio feature encoding layer of the multimodal model to obtain an audio feature vector;

[0070] Performing feature vector fusion on the text feature vector, the video frame feature vector, and the audio feature vector associated with the same video frame in the video data to obtain a fusion vector corresponding to each video frame in the video data;

[0071] The fusion vectors corresponding to each video frame are concatenated to obtain a multimodal feature vector sequence, and a video script for the video data is generated based on the multimodal feature vector sequence.

[0072] In one possible implementation, when the video script generation module is used to generate a video script for video data based on a multimodal feature vector sequence, it is specifically used to perform the following operations:

[0073] Identify scene change frames in video data through multimodal models;

[0074] Divide the multimodal feature vector sequence based on the scene change frames to obtain multimodal feature vectors corresponding to multiple video clips;

[0075] Decoding the multimodal feature vectors corresponding to the plurality of video clips to obtain video clip scripts corresponding to the plurality of video clips;

[0076] The video clip scripts corresponding to the multiple video clips are combined into a video script of the video data.

[0077] In one possible implementation, the video script also includes theme information of the video set to which the video data belongs; the model processing module is configured to analyze and match each video clip script with each barrage special effect material in the barrage special effect material set according to the barrage analysis instruction in the prompt word, and when obtaining barrage configuration suggestion information having a suggested output format indicated by the prompt word, specifically perform the following operations:

[0078] In the large language model, according to the barrage analysis instruction in the prompt word, a first barrage special effect material set associated with the theme information is selected from the barrage special effect material set;

[0079] Perform feature extraction on each video clip script according to the barrage analysis instruction to obtain a script feature vector corresponding to each video clip script, and perform feature extraction on each barrage special effect material in the first barrage special effect material set to obtain a special effect feature vector corresponding to each barrage special effect material;

[0080] The script attention vector sequence of the script feature vector is constructed through the large language model, and the script attention vector sequence and the special effect feature vector are matched for similarity to obtain the matching result;

[0081] Output the bullet screen configuration suggestion information in the suggested output format indicated by the prompt word according to the matching result.

[0082] In one possible implementation, the large language model includes an attention processing layer; when the model processing module is used to construct a script attention vector sequence of the script feature vector using the large language model, it is specifically used to perform the following operations:

[0083] Perform a dot product operation on the query parameter matrix in the attention processing layer and the script feature vector to obtain the query vector sequence of the self-attention function;

[0084] Perform a dot product operation on the key parameter matrix in the attention processing layer and the script feature vector to obtain the key vector sequence of the self-attention function;

[0085] Perform a dot product operation on the value parameter matrix in the attention processing layer and the script feature vector to obtain the value vector sequence of the self-attention function; the query parameter matrix, key parameter matrix, and value parameter matrix are all matrices composed of the learnable parameters in the attention processing layer;

[0086] A script attention vector sequence for the script feature vector is generated based on the query vector sequence, the key vector sequence, and the value vector sequence.

[0087] In one possible implementation, when the model processing module is configured to output the bullet chat configuration suggestion information having the suggested output format indicated by the prompt word according to the matching result, it is specifically configured to perform the following operations:

[0088] Obtain target barrage special effects materials that match the target video clip script in the video script according to the matching results; the barrage special effects material set includes the target barrage special effects materials;

[0089] Obtain target special effect material attribute information and target suggestion reason information of the target barrage special effect material according to the recommended output format;

[0090] Based on the target special effect material attribute information and the target suggestion reason information, the barrage configuration suggestion information having the suggested output format indicated by the prompt word is generated.

[0091] In a possible implementation, the model processing module is further configured to perform the following operations:

[0092] Obtain historical video data, historical barrage special effects materials, and historical barrage configuration suggestion information; historical barrage configuration suggestion information is generated based on the usage records of historical barrage special effects materials based on historical video data;

[0093] Input historical video data and historical barrage special effects materials into the initial large language model, and output predicted barrage configuration suggestion information through the initial large language model;

[0094] A training loss value is generated based on historical barrage configuration suggestion information and predicted barrage configuration suggestion information, and the model parameters of the initial large language model are adjusted based on the training loss value to obtain a large language model for generating barrage configuration suggestion information.

[0095] In a possible implementation, the model processing module is further configured to perform the following operations:

[0096] Get a set of suggested information; the suggested information set contains multiple suggested information, including bullet screen configuration suggested information;

[0097] Performing a quality score on the set of recommended information to obtain a score set; the score set contains the score results corresponding to each piece of recommended information, and the score results are positive or negative.

[0098] If the proportion of positive scoring results in the scoring set is greater than the proportion threshold, an automatic configuration service is set up in the barrage management platform; the automatic configuration service is used to instruct the barrage management platform to configure the barrage special effect materials indicated by the new barrage configuration suggestion information to the new video clip when the large language model outputs new barrage configuration suggestion information for the new video clip.

[0099] In a possible implementation, the model processing module is further configured to perform the following operations:

[0100] If the proportion of negative scoring results in the scoring set is greater than the proportion threshold, the positive scoring results in the scoring set are determined as positive sample data, and the negative scoring results in the scoring set are determined as negative sample data;

[0101] Re-fine-tune the large language model based on positive and negative sample data.

[0102] In one possible implementation, when the transceiver module is used to obtain video data, it is specifically used to perform the following operations:

[0103] Get a video comment parsing request; the video comment parsing request includes video episode identification information;

[0104] Obtaining video set identification information in the video single set identification information according to the video barrage analysis request, and obtaining a video library; the video library includes multiple single-set videos respectively contained in multiple video sets;

[0105] A target video set corresponding to the video set identification information is searched in the video library, and a single-episode video corresponding to the video single-episode identification information among multiple single-episode videos associated with the target video set is determined as video data.

[0106] On the one hand, an embodiment of the present application provides a computer device, including a processor, a memory, and an input and output interface;

[0107] The processor is connected to the memory and the input and output interface respectively, wherein the input and output interface is used to receive and output data, the memory is used to store the computer program, and the processor is used to call the computer program so that the computer device including the processor executes the method in one aspect of the embodiment of the present application.

[0108] On one hand, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. The computer program is suitable for being loaded and executed by a processor so that a computer device having the processor executes the method in one aspect of the embodiment of the present application.

[0109] In one aspect, an embodiment of the present application provides a computer program product, comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the methods provided in various optional embodiments of the present application. In other words, when the computer program is executed by the processor, the methods provided in various optional embodiments of the present application are implemented.

[0110] The embodiment of the present application obtains video data and performs script parsing on the video data to obtain a video script containing multiple video clip scripts. After generating a prompt word containing a set of bullet screen special effects materials, each video clip script can be analyzed and matched with each bullet screen special effects material in the bullet screen special effects material set according to the bullet screen analysis instruction in the prompt word, and bullet screen configuration suggestion information with a recommended output format indicated by the prompt word is obtained, wherein the bullet screen configuration suggestion information includes the bullet screen special effects material recommended for the target video clip in the multiple video clips and the reason for the suggestion. By parsing the video script and then analyzing and matching the video script and the bullet screen special effects material, the entire configuration process can be simplified and the configuration efficiency of the bullet screen special effects can be improved. Moreover, analyzing and matching each video clip script with each bullet screen special effects material in the bullet screen special effects material set can effectively reduce the possibility that the configured bullet screen special effects have a low match with the plot, making the configured bullet screen special effects material and the video script more matched, so that the configuration effect of configuring the bullet screen special effects material based on the bullet screen configuration suggestion information is better. BRIEF DESCRIPTION OF THE DRAWINGS

[0111] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0112] Figure 1 This is a network interaction architecture diagram provided by an embodiment of the present application;

[0113] Figure 2 This is a scenario diagram of a data processing method provided in an embodiment of the present application. Figure 1 ;

[0114] Figure 3 This is a flow diagram of a data processing method provided in an embodiment of the present application. Figure 1 ;

[0115] Figure 4 This is a flow diagram of a data processing method provided in an embodiment of the present application. Figure 2 ;

[0116] Figure 5 This is a system architecture diagram of data processing provided by an embodiment of the present application;

[0117] Figure 6 This is a schematic diagram of a model structure of a data processing method provided in an embodiment of the present application;

[0118] Figure 7This is a scenario diagram of a data processing method provided in an embodiment of the present application. Figure 2 ;

[0119] Figure 8 is a schematic diagram of a data processing device provided in an embodiment of the present application;

[0120] Figure 9 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0121] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0122] Among them, if it is necessary to collect object (such as user, etc.) data in this application, a prompt interface or pop-up window will be displayed before or during the collection. The prompt interface or pop-up window is used to remind the user that certain data is currently being collected. Only after the user confirms the prompt interface or pop-up window, the relevant steps for data acquisition will be started, otherwise the process will end. Moreover, the acquired user data will be used in reasonable and legal scenarios or purposes. Optionally, in some scenarios where user data needs to be used but has not been authorized by the user, authorization can be requested from the user, and the user data can be used when the authorization is passed.

[0123] It is understandable that in the specific implementation of this application, the user data involved, when the following embodiments of this application are applied to specific products or technologies, needs to obtain user permission or consent, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of the relevant regions.

[0124] In the examples of this application, see Figure 1 , Figure 1 This is a network interaction architecture diagram provided by an embodiment of the present application, such as Figure 1 As shown, the network interaction architecture diagram may include a business server 100 and a terminal device 200. The terminal device 200 may have a communication connection with the business server 100, wherein the above-mentioned communication connection does not limit the connection method, and may be directly or indirectly connected through a wired communication method, or directly or indirectly connected through a wireless communication method, or through other methods, which are not limited in this application.

[0125] It should be understood that Figure 1The terminal device 200 shown in the figure can be installed with an application client having a data processing function. When the application client runs on the terminal device 200, it can be used with the above-mentioned Figure 1 The business servers 100 shown in the figure interact with each other so that the business server 100 can receive business data from the terminal device 200. Among them, the application client can be a social application, instant messaging application, live broadcast application, short video application, video application, music application, shopping application, novel application, browser, etc., which has the function of displaying data information such as text, images, audio and video. Among them, the application client can be an independent client or an embedded sub-client (for example, an application applet, browser web page access, etc.) integrated in a client (for example, an instant messaging client, a social client, a video client, etc.), which is not limited here.

[0126] like Figure 1 As shown, the business server 100 can obtain video data and a set of barrage special effects materials. By performing script parsing on the video data, a video script containing multiple video clip scripts can be obtained. By analyzing and matching each video clip script with each barrage special effects material in the set of barrage special effects materials, barrage configuration suggestion information for the video data can be generated, wherein the barrage special effects refer to the color changes, motion trajectory changes or other animation effects triggered under specific rules (time points, keywords) during the scrolling of the barrage. The barrage configuration suggestion information includes multiple video clip scripts that can be configured with barrage special effects materials, and any video clip script included in the barrage configuration suggestion information can be associated with a matching barrage special effects material.

[0127] After configuring the barrage special effect material based on the barrage configuration suggestion information, the barrage special effect material can be displayed in the video data, and the business server 100 can send the video data containing the barrage special effect material to the terminal device 200.

[0128] The object can view the video data containing barrage special effects materials through the terminal device 200, wherein the triggering rule of the barrage special effects materials can be a time period special effects trigger, such as when the video data is played to the 2 minute and 45 second character description of the plot, the petal special effects can be triggered; the triggering rule of the barrage special effects can also be a keyword special effects trigger, such as when the object sends the barrage "XXX (such as beautiful)", the petal special effects can be triggered, and there is no restriction here.

[0129] See Figure 2 , Figure 2 This is a scenario diagram of a data processing method provided in an embodiment of the present application. Figure 1 .like Figure 2As shown, the business server 100 can obtain video data from the video library, as well as the theme information of the video set to which the video data belongs, wherein the theme information may include the theme name of the video set, the theme type of the video set (science fiction, suspense, etc.) and the author of the video set.

[0130] After the business server 100 inputs the video data into the multimodal model, the multimodal model can be used to perform script analysis on the video data, that is, the video data containing subtitles (text information), voice (audio information) and images (video frames) is analyzed and processed, and the video plot corresponding to each time period (or each video clip) is analyzed to obtain a video script, wherein the video script is structured script content, including the time point of the video data, plot content, plot scene, character and other information, and can include video clip scripts corresponding to multiple video clips, such as "Clip position: 2 minutes 45 to 3 minutes 15; in the room, character A picks up a small object to interact with character B; character A: XXX; character B: XXX;..."

[0131] The business server 100 can obtain a barrage special effect material set containing multiple barrage special effect materials from the barrage management platform through a special effect calculation service (a backend microservice in the business server 100 used to forward data and call models), and generate a prompt word containing the barrage special effect material set, a barrage analysis instruction, and a suggested output format, such as "barrage special effect material set: barrage special effect material 1, barrage special effect material 2, ...; barrage analysis instruction: find barrage special effect materials that match the video script in the barrage special effect material set; suggested output format: whether to recommend configuring barrage special effect material X; special effect material attribute information of barrage special effect material X (such as barrage special effect name, special effect display duration, and other barrage special effect rules); reason information for recommending barrage special effect material X; other suggestion information (such as providing that new barrage special effect materials can be introduced if special effects are not currently met)". The business server 100 can call the large language model through the special effect calculation service, and input the theme information, prompt word, and video script as input data into the large language model.

[0132] In the large language model, the theme information, each video clip script and each barrage special effect material in the barrage special effect material set can be analyzed and matched separately according to the barrage analysis instructions in the prompt word. For example, multiple barrage special effect materials associated with the theme information can be filtered out from the barrage special effect material set. For example, the theme information may include "Theme name: The battle between character A and character B; Theme type: Action", then the business server 100 can filter out flame special effect materials, lightning special effect materials, etc. associated with the action theme from the barrage special effect material set.

[0133] By analyzing and matching each video clip script in the video script with each barrage special effect material in the screened multiple barrage special effect materials, barrage configuration suggestion information with the recommended output format indicated by the prompt word can be generated. Among them, the barrage configuration suggestion information includes the barrage special effect materials (barrage special effect name, barrage special effect rules) recommended for the target video clip in the multiple video clips and the reason information for the suggestion. For example, in the barrage configuration suggestion information, the segment suggestion information with the recommended output format for the target video clip can be "recommended configuration of barrage special effect material 1; special effect material attribute information of barrage special effect material 1 (such as barrage special effect name or barrage special effect ID (Identification), special effect display duration and other barrage special effect rules); recommended reason information of barrage special effect material 1; other suggestion information (such as special effects that are not currently met, giving the possibility of introducing new barrage special effect materials) etc." The business server 100 can send multiple segment suggestion information (i.e., suggestion information respectively given for multiple video segment scripts, such as segment suggestion information 1, segment suggestion information 2, ...) under the target video single episode identification information (used to uniquely identify the above-mentioned video data) to the terminal device 200, and the operation object can view the multiple segment suggestion information through the terminal device 200. Among them, the barrage configuration suggestion information includes multiple video segment scripts that can be configured with barrage special effect materials, and any video segment script included in the barrage configuration suggestion information can be associated with a matching barrage special effect material.

[0134] The embodiment of the present application can obtain a video script containing multiple video clip scripts by acquiring video data and performing script parsing on the video data according to a multimodal model. After generating a prompt word containing a set of barrage special effects materials, the prompt word and the video script can be input into a large language model, so that each video clip script can be analyzed and matched with each barrage special effects material in the set of barrage special effects materials according to the barrage analysis instructions in the prompt word in the large language model, and barrage configuration suggestion information with a recommended output format indicated by the prompt word is obtained, wherein the barrage configuration suggestion information includes the barrage special effects materials recommended for the target video clip in the multiple video clips and the reason for the suggestion. By parsing the video script through a multimodal model and then analyzing and matching the video script and the barrage special effects materials according to the large language model, the entire configuration process can be simplified and the configuration efficiency of the barrage special effects can be improved. Moreover, by using a large language model to analyze and match each video clip script with each barrage special effect material in the barrage special effect material set, the possibility of the configured barrage special effect having a low match with the plot can be effectively reduced, making the configured barrage special effect material and the video script more matched. Then, configuring the barrage special effect material based on the barrage configuration suggestion information will have a better configuration effect.

[0135] Further, see Figure 3, Figure 3 This is a flow diagram of a data processing method provided in an embodiment of the present application. Figure 1 The data processing method can be executed by a computer device, which can be Figure 1 The service server 100 or terminal device 200 shown in the figure will be described below by taking the data processing method executed by a computer device as an example. The data processing method may include at least the following steps S101 to S104:

[0136] Step S101, obtaining video data;

[0137] In an embodiment of the present application, a computer device can obtain video data and subject information of the video set to which the video data belongs from a video library. Each video set can correspond to a unique video set identification information (Content Identifier, CID), and the subject information can be an introduction to the video set identification information, such as information such as the theme name of the video set, the theme type of the video set (science fiction, suspense, etc.), and the author of the video set. Each video data under each video set can correspond to a unique video single set identification information (Video Identifier, VID). The computer device can obtain the video data through the specified VID information, and then obtain the CID information of the video set corresponding to the video data, and then obtain the subject information of the video set to which the video data belongs. Optionally, the computer device can also first obtain CID information containing multiple VID information, and then determine the single set video corresponding to the VID information specified in the CID information as the video data.

[0138] Step S102: parsing the video data to obtain a video script; the video script includes video clip scripts corresponding to the plurality of video clips;

[0139] In an embodiment of the present application, a computer device can perform script parsing on the video data corresponding to the specified VID information through a multimodal model to obtain a video script. For example, the video data can be retrieved based on the specified VID information through RAG (Retrieval-Augmented Generation) technology, and then the video script corresponding to the video data can be obtained. Optionally, the computer device can also parse the video scripts corresponding to multiple single-episode videos through a multimodal model, and then obtain the video script corresponding to the specified single-episode video according to the specified VID information. The embodiment of the present application does not limit this.

[0140] Specifically, the computer device can extract the video frame feature vector, text feature vector and audio feature vector associated with each video frame in the video data through the multimodal model. By performing feature vector fusion on the text feature vector, video frame feature vector and audio feature vector associated with the video frame, the fusion vector corresponding to each video frame can be obtained. The fusion vector corresponding to each video frame is then vector-spliced to obtain a multimodal feature vector sequence. The multimodal model can also be used to identify scene change frames in the video data (for example, if video frame F in the video data is an indoor scene and the previous frame of video frame F is an outdoor scene, then video frame F can be determined as a scene change frame). By dividing the multimodal feature vector sequence based on the scene change frames, the plot of each video clip can be kept coherent and clear, and multimodal feature vectors corresponding to multiple video clips can be obtained. The multimodal feature vectors corresponding to the multiple video clips are decoded to obtain video clip scripts corresponding to the multiple video clips. Then, the video clip scripts corresponding to the multiple video clips can be combined into a video script for the video data.

[0141] Step S103, obtaining a set of barrage special effect materials, and generating prompt words containing the set of barrage special effect materials;

[0142] In an embodiment of the present application, a computer device can obtain a configured barrage special effect material set from a barrage management platform, and then generate a prompt word containing the barrage special effect material set, thereby inputting the theme information, prompt word and video script into the large language model. For example, the special effect material attribute information (such as barrage special effect name, special effect display duration and other barrage special effect rules) of each barrage special effect material in the barrage special effect material set, barrage analysis instructions (such as "find barrage special effect materials that match the video script in the barrage special effect material set") and the recommended output format of the large language model (whether it is recommended to configure barrage special effect material X; special effect material attribute information of barrage special effect material X (such as barrage special effect name, special effect display duration and other barrage special effect rules); recommended reason information of barrage special effect material X; other recommended information (such as special effects that are not currently met, giving the possibility of introducing new barrage special effect materials) etc.). By obtaining a prompt word template, filling the special effect material attribute information, barrage analysis instructions and recommended output format of each barrage special effect material into the prompt word template, a prompt word can be obtained. The computer device can input the above-mentioned topic information, prompt words and video script into the large language model as input data.

[0143] In step S104, each video clip script is analyzed and matched with each barrage special effect material in the barrage special effect material set according to the barrage analysis instruction in the prompt word, and barrage configuration suggestion information with the recommended output format indicated by the prompt word is obtained; the barrage configuration suggestion information includes the barrage special effect material recommended for the target video clip in multiple video clips and the reason information for the suggestion.

[0144] In an embodiment of the present application, the computer device can analyze and match each video clip script with each barrage special effects material in the barrage special effects material set in a large language model according to the barrage analysis instructions in the prompt words used to match video clip scripts and barrage special effects materials.

[0145] Specifically, the computer device can perform feature extraction on each video clip script according to the barrage analysis instructions for feature extraction indicated in the prompt words, obtain the script feature vector corresponding to each video clip script, and perform feature extraction on each barrage special effect material in the barrage special effect material set, and obtain the special effect feature vector corresponding to each barrage special effect material; in the large language model, a script attention vector sequence of the script feature vector can be constructed based on the self-attention mechanism, and a matching result can be obtained by performing similarity matching on the script attention vector sequence and the special effect feature vector, wherein each script attention vector in the script attention vector sequence can be used to characterize the script features of the video clip script that has been associated with contextual script information.

[0146] For example, by calculating the similarity (such as cosine similarity) between a script attention vector and the special effect feature vector of each barrage special effect material, multiple similarity calculation results can be obtained. If the largest similarity calculation result is greater than the similarity threshold, the barrage special effect material associated with the special effect feature vector corresponding to the largest similarity calculation result is associated with the video clip corresponding to the script attention vector, and then segment suggestion information for the video clip script can be generated based on the association relationship. The barrage configuration suggestion information can be composed of segment suggestion information corresponding to multiple video clip scripts, one of which can be "suggested configuration of barrage special effect material 1; special effect material attribute information of barrage special effect material 1 (such as barrage special effect name or barrage special effect ID (Identification), special effect display duration and other barrage special effect rules); recommended reason information for barrage special effect material 1; other suggestion information (such as special effects that are not currently met, providing new barrage special effect materials that can be introduced)". The computer device can submit the barrage configuration suggestion information to the operation object, so that the operation object can configure the barrage special effect material based on the barrage configuration suggestion information.

[0147] The embodiment of the present application can obtain a video script containing multiple video clip scripts by acquiring video data and performing script parsing on the video data according to a multimodal model. After generating a prompt word containing a set of barrage special effects materials, the prompt word and the video script can be input into a large language model, so that each video clip script can be analyzed and matched with each barrage special effects material in the set of barrage special effects materials according to the barrage analysis instructions in the prompt word in the large language model, and barrage configuration suggestion information with a recommended output format indicated by the prompt word is obtained, wherein the barrage configuration suggestion information includes the barrage special effects materials recommended for the target video clip in the multiple video clips and the reason for the suggestion. By parsing the video script through a multimodal model and then analyzing and matching the video script and the barrage special effects materials according to the large language model, the entire configuration process can be simplified and the configuration efficiency of the barrage special effects can be improved. Moreover, by using a large language model to analyze and match each video clip script with each barrage special effect material in the barrage special effect material set, the possibility of the configured barrage special effect having a low match with the plot can be effectively reduced, making the configured barrage special effect material and the video script more matched. Then, configuring the barrage special effect material based on the barrage configuration suggestion information will have a better configuration effect.

[0148] Further, see Figure 4 , Figure 4 This is a flow diagram of a data processing method provided in an embodiment of the present application. Figure 2 The data processing method can be executed by a computer device, which can be Figure 1 The business server 100 or terminal device 200 shown in FIG. 1 is shown. The following description will be made by taking the data processing method executed by a computer device as an example. The data processing method may include at least the following steps S201 to S207:

[0149] Step S201: Obtain a video comment parsing request; obtain video set identification information in the video episode identification information according to the video comment parsing request, and obtain a video library; search the video library for a target video set corresponding to the video set identification information, and determine the single episode video corresponding to the video episode identification information among multiple single episode videos associated with the target video set as video data;

[0150] In the examples of this application, please refer to Figure 5 , Figure 5 This is a system architecture diagram of data processing provided by an embodiment of the present application, such as Figure 5As shown, the computer device can obtain a video barrage parsing request including video episode identification information, and then obtain the video set identification information in the video episode identification information according to the video barrage parsing request. For example, the video episode identification information can be 10_01, and the video set identification information 10 can be obtained according to the video episode identification information 10_01, wherein field 10 is used to identify a single video episode 10, and field 01 is used to identify a single video episode 01 under the video episode.

[0151] The computer device can obtain a video library containing multiple video sets from the video media platform, such as video set 30, video set 40, etc., and multiple video sets can respectively contain multiple single-episode videos (such as video set 30 can contain single-episode video 30_01, single-episode video 30_02,...). The corresponding target video set can be queried in the video library through the video set identification information. For example, the target video set 10 can be queried according to the video set identification information 10, so that the single-episode video 10_01 corresponding to the video single-episode identification information 10_01 among the multiple single-episode videos (single-episode video 10_01, single-episode video 10_02...) associated with the target video set 10 can be determined as video data.

[0152] Step S202: extracting features from video frames in the video data using the video frame feature coding layer of the multimodal model to obtain a video frame feature vector; performing text recognition on the video frames to obtain text information in the video data, and extracting features from the text information using the text feature coding layer of the multimodal model to obtain a text feature vector; and extracting features from the audio information in the video data using the audio feature coding layer of the multimodal model to obtain an audio feature vector.

[0153] In an embodiment of the present application, a multimodal model may include a video frame feature coding layer, a text feature coding layer, and an audio feature coding layer. The computer device performs script parsing on the video data through the multimodal model, and the specific process of obtaining the video script may be: extracting features of the video frames in the video data through the video frame feature coding layer to obtain a video frame feature vector; performing text recognition on the video frames to obtain text information in the video data, and extracting features of the text information through the text feature coding layer to obtain a text feature vector; extracting features of the audio information in the video data through the audio feature coding layer to obtain an audio feature vector; performing feature vector fusion on the text feature vectors, video frame feature vectors, and audio feature vectors associated with the same video frame in the video data to obtain a fusion vector corresponding to each video frame in the video data; performing vector splicing on the fusion vectors corresponding to each video frame to obtain a multimodal feature vector sequence, and generating a video script for the video data based on the multimodal feature vector sequence. Among them, the multimodal model is an artificial intelligence system that can simultaneously process and understand multiple data modalities (such as text, images, audio, video, etc.). Through cross-modal alignment and fusion technology (such as CLIP (Contrastive Language-Image Pre-Training, image text encoding model) architecture), it maps information of different modalities into a unified semantic space to achieve complex tasks such as image-text mutual generation, video semantic analysis, and multimodal retrieval.

[0154] Specifically, the video frame feature coding layer can be a network model such as a CNN (Convolutional Neural Network) model and a ViT (Vision Transformer) model. The computer device can preprocess the video frames in the video data (normalization processing, data enhancement, etc.) through the video frame feature coding layer to obtain a standard image corresponding to each video frame. By performing image recognition on the standard image through the network model (the process is forward propagation and feature transformation), a multi-dimensional feature vector can be obtained. The multi-dimensional feature vector is further subjected to feature vector pooling processing to obtain a video frame feature vector.

[0155] The computer device can use deep learning methods (such as CTPN (Connectionist Text Proposal Network)) to perform text recognition on the lines and subtitles in the video frames to obtain text information in the video data, and extract features from the text information through the text feature encoding layer to obtain a text feature vector. The text feature encoding layer can be a BERT model (Bidirectional Encoder Representations from Transformers). In the text feature encoding layer, the text information can be a word sequence obtained by splitting and encoding the text information through a Tokenizer. The text splitting can be achieved based on words in the text information, or based on subwords or characters. There is no limitation here.

[0156] A lemma in a lemma sequence is the smallest basic unit obtained after a text message is split. Its structure can include [CLS], lemma 1, lemma 2, and [SEP]. [CLS] can be added to the beginning of the Tokenizer segmentation result for classification tasks, and [SEP] can be added to the end of the Tokenizer segmentation result for sentence segmentation. All lemmas in the text message are embedded into vectors of the same size in each encoding layer of the text feature encoding layer. In the BERT model, this can include lemma embedding, paragraph embedding, and position embedding.

[0157] Among them, word embedding can embed text information into E [CLS] 、E 词元1 、E 词元2 and E [SEP] , thereby mapping the text information into a high-dimensional space to capture the context and semantic information of the text information, where E [CLS] It can be used to represent the global semantics of the entire text information. Paragraph embedding can be used to distinguish different paragraphs and sentences in text information. If the embedding of each word is the same continuously, it is the same sentence. For example, if the paragraph embedding of the text information is E C 、E C 、E D and E D , then E C and E C Part represents a sentence, E D and E DPartially represents another sentence. Positional embedding can embed text information into E1, E2, E3, and E4, capturing the positional order of tokens in the text and understanding the semantic structure. By concatenating the results of token embedding, paragraph embedding, and positional embedding, a text feature vector can be obtained.

[0158] Computer devices can pre-process the audio information in the video data (such as format unification, frame processing, and normalization) through an audio feature encoding layer (such as the VGGish model (an audio feature extractor similar to VGG)), and then perform time-frequency conversion on each frame of audio based on methods such as the Short-Time Fourier Transform (STFT) to obtain a spectrum, and map the spectrum into a spectrogram through a filter. After forward propagation and feature extraction of the spectrogram in the VGGish model, the audio features corresponding to each frame of audio are obtained. The audio features corresponding to each frame of audio are then subjected to frame-level feature splicing and global pooling processing to obtain an audio feature vector. Among them, the Short-Time Fourier Transform is a tool for analyzing the frequency components of a signal in different time segments. It divides the signal into multiple short time windows and performs Fourier transform on the signal in each window to simultaneously obtain the time and frequency information of the signal. This method is suitable for non-stationary signals (i.e., signals whose frequency components change over time), such as audio signals.

[0159] Step S203: performing feature vector fusion on the text feature vectors, video frame feature vectors, and audio feature vectors associated with the same video frame in the video data to obtain a fusion vector corresponding to each video frame in the video data; performing vector concatenation on the fusion vectors corresponding to each video frame to obtain a multimodal feature vector sequence, and generating a video script for the video data based on the multimodal feature vector sequence;

[0160] In an embodiment of the present application, the computer device can further align and fuse the text feature vectors, video frame feature vectors and audio feature vectors associated with the same video frame in the video data, such as inputting the text feature vectors, video frame feature vectors and audio feature vectors associated with the same video frame into a transformer model for learning fusion features, and obtaining a fusion vector corresponding to each video frame, such as the sequence embedding output by the encoding layer of the transformer model. The sequence embedding can also include the embeddings corresponding to the text feature vector, the video frame feature vector and the audio feature vector, respectively, so as to characterize the global semantics of the text feature vector, the video frame feature vector and the audio feature vector, and retain more original global features and local features, which is suitable for subsequent feature interaction learning.

[0161] The computer device can perform vector splicing on the fusion vectors corresponding to each video frame to obtain a multimodal feature vector sequence, and a video script for the video data can be generated based on the multimodal feature vector sequence. The specific process can be: identifying scene change frames in the video data through a multimodal model; dividing the multimodal feature vector sequence based on the scene change frames to obtain multimodal feature vectors corresponding to multiple video clips; decoding the multimodal feature vectors corresponding to the multiple video clips to obtain video clip scripts corresponding to the multiple video clips; and combining the video clip scripts corresponding to the multiple video clips into a video script for the video data.

[0162] For example, video frame F and video frame G are two consecutive video frames. A computer device can use a multimodal model to identify shot switching (such as changes in screen color or motion), audio mutations (such as spectrum shifts from low frequency to high frequency), and text information changes (such as the transformation of topic description words or scene description words) from video frame F to video frame G. If video frame F in the video data is an indoor scene and video frame G is an outdoor scene, it can be determined that a scene change has occurred from video frame F to video frame G, and therefore video frame G can be determined as a scene change frame in the video data. The scene change frame can be used to divide the multimodal feature vector sequence into multimodal feature vectors corresponding to multiple video clips (i.e., multimodal feature vectors corresponding to multiple identical video frames and multimodal feature vectors corresponding to multiple scene change frames), thereby keeping the plot of each video clip coherent and clear.

[0163] The decoder decodes the multimodal feature vectors corresponding to multiple video clips based on factors such as time period, plot, and characters. This can generate corresponding video clip scripts for each of the video clips, such as "Clip position: 2:45 to 3:15; Character A picks up a small object in the room and interacts with Character B; Character A: XXX; Character B: XXX; ...," and "Clip position: 3:15 to 3:45; Character B stands in the room, holding a handrail and interacting with Character A; Character B: XXX; Character A: XXX; ...." The computer device can combine the video clip scripts corresponding to the multiple video clips into a video script corresponding to the video data.

[0164] Step S204: obtaining a set of barrage special effects materials and generating prompt words containing the set of barrage special effects materials;

[0165] In an embodiment of the present application, a computer device can obtain a barrage special effect material set containing multiple barrage special effect materials from a barrage management platform. For example, it can first generate a material acquisition request (such as an HTTP (HyperText TransferProtocol, Hypertext Transfer Protocol) request), and then obtain a barrage special effect material set containing multiple barrage special effect materials based on the material acquisition request in the barrage management platform, and then generate a prompt word based on the barrage special effect material set, barrage analysis instructions and the recommended output format of the large language model.

[0166] For example, you can first obtain a prompt word template containing the areas to be filled (the first area to be filled, the second area to be filled, and the third area to be filled), and then parse the special effect material attribute information of each barrage special effect material in the barrage special effect material set (such as the barrage special effect name or barrage special effect ID, special effect display duration, and other barrage special effect rules), and obtain the recommended output format of the large language model (such as whether it is recommended to configure barrage special effect material X; the special effect material attribute information of barrage special effect material X; the reason information for recommending barrage special effect material X; other recommendation information (such as barrage special effect materials that are not currently met, and new barrage special effect materials that can be introduced) etc.). The computer device can fill the special effect material attribute information of each barrage special effect material into the first to-be-filled area of the prompt word template, fill the barrage analysis instruction into the second to-be-filled area, and fill the recommended output format into the third to-be-filled area to obtain prompt words, such as "barrage special effect material collection: barrage special effect material 1 (including special effect material attribute information), barrage special effect material 2,...; barrage analysis instruction: find barrage special effect materials that match the video script in the barrage special effect material collection; recommended output format: whether to recommend configuring barrage special effect material X; special effect material attribute information of barrage special effect material X; recommended reason information of barrage special effect material X; other recommended information".

[0167] like Figure 5 As shown, a computer device can call a large language model through a special effects computing service, thereby inputting the subject information, prompt words, and video script of the video set to which the video data belongs as input data into the large language model. Among them, a large language model (LLM) refers to a computer model that can process and generate natural language; it represents a major advancement in the field of artificial intelligence and is expected to change the field through acquired knowledge. LLM can predict the next word or sentence by learning the statistical laws and semantic information of language data. As the input data set and parameter space continue to expand, the ability of LLM will also increase accordingly. It is used in various application fields, such as machine learning and image processing, and is therefore also called a multimodal large language model.

[0168] In an embodiment of the present application, the process of training a large language model for generating barrage configuration suggestion information can be: obtaining historical video data, historical barrage special effects materials and historical barrage configuration suggestion information; the historical barrage configuration suggestion information is generated based on the usage record of historical video data for historical barrage special effects materials; the historical video data and historical barrage special effects materials are input into the initial large language model, and the predicted barrage configuration suggestion information is output through the initial large language model; a training loss value is generated according to the historical barrage configuration suggestion information and the predicted barrage configuration suggestion information, and the model parameters of the initial large language model are adjusted based on the training loss value to obtain a large language model for generating barrage configuration suggestion information.

[0169] Specifically, the computer device can obtain historical video data from the video library, and obtain historical barrage special effects materials and historical barrage configuration suggestion information from the barrage management platform, wherein the historical barrage configuration suggestion information is generated based on the usage records of historical video data for historical barrage special effects materials, that is, the historical barrage configuration suggestion information is integrated and generated based on barrage special effects cases with good feedback effects. Historical barrage special effects materials include historical barrage special effects material names, historical barrage special effects material IDs, effect displays of historical barrage special effects materials (sufficient text notes describing their effects so that the large language model can better understand the usage scenarios of barrage special effects materials), barrage special effects rules of historical barrage special effects materials, and other special effects material attribute information. The historical barrage configuration suggestion information includes the operation strategy of the barrage special effects material: the operation object shall supplement the explanation of the reasons and suggestions for using the barrage special effects material, as well as some relevant precautions, such as how long the display time of the barrage special effects material should not exceed, and how many barrage special effects rules should not be used in a single episode of video.

[0170] Please also see Figure 6 , Figure 6 A schematic diagram of a model structure of a data processing method provided in an embodiment of the present application is as follows: Figure 6 As shown, the computer device can calculate the model loss based on the historical barrage configuration suggestion information and the predicted barrage configuration suggestion information output by the initial large language model to obtain a training loss value, and then fine-tune the model parameters of the initial large language model based on the training loss value to obtain a large language model for generating barrage configuration suggestion information. For example, taking the stochastic gradient descent method to adjust the model parameters of the initial large language model as an example, the basic idea is to update the model parameters along the negative gradient direction of the loss function so that the training loss value gradually decreases. For the model parameter θ, the method of updating the model parameter can be defined as follows: Where α is the learning rate, The training loss is the gradient of the training loss with respect to the model parameters. By continuously iteratively updating the model parameters θ in the opposite direction of the gradient, the training loss is continuously reduced. When the training loss falls below a preset training loss threshold, the computer device determines that the model parameters θ have reached convergence. Thus, a large language model for generating bullet comment configuration suggestions can be obtained.

[0171] Step S205: In the large language model, based on the barrage analysis instruction in the prompt word, a first barrage special effect material set associated with the theme information is selected from the barrage special effect material set; feature extraction is performed on each video clip script based on the barrage analysis instruction to obtain a script feature vector corresponding to each video clip script; feature extraction is performed on each barrage special effect material in the first barrage special effect material set to obtain a special effect feature vector corresponding to each barrage special effect material;

[0172] In an embodiment of the present application, the computer device can select a first barrage special effects material set associated with the theme information from the barrage special effects material set based on the barrage analysis instructions in the prompt word (such as selecting barrage special effects materials associated with the theme information). For example, if the theme information may be "This is a comedy story between character A and character B", then the barrage special effects materials such as fireworks special effects materials and funny expression special effects materials associated with comedy can be determined as the first barrage special effects material set in the barrage special effects material set.

[0173] Furthermore, the computer device can perform feature extraction on the video clip script in the large language model according to the barrage analysis instruction (such as performing feature extraction on the video clip script and the barrage special effects material respectively), and obtain a script feature vector corresponding to each video clip script, such as performing text feature extraction on the character description text of the video clip script to obtain a character feature vector, performing text feature extraction on the line description text of the video clip script to obtain a line feature vector, etc. Feature extraction is performed on each barrage special effects material in the first barrage special effects material set through the large language model, such as performing feature extraction based on the keywords or material style (such as comedy style, combat style, etc.) of the barrage special effects material, to obtain a special effects feature vector corresponding to each barrage special effects material.

[0174] Step S206: constructing a script attention vector sequence of the script feature vector through the large language model, performing similarity matching on the script attention vector sequence and the special effect feature vector to obtain a matching result; outputting barrage configuration suggestion information having a suggested output format indicated by the prompt word according to the matching result;

[0175] In an embodiment of the present application, the large language model may include an attention processing layer, and the specific process of the computer device constructing the script attention vector sequence of the script feature vector through the large language model may be: performing a point multiplication operation on the query parameter matrix in the attention processing layer and the script feature vector to obtain a query vector sequence of the self-attention function; performing a point multiplication operation on the key parameter matrix in the attention processing layer and the script feature vector to obtain a key vector sequence of the self-attention function; performing a point multiplication operation on the value parameter matrix in the attention processing layer and the script feature vector to obtain a value vector sequence of the self-attention function; the query parameter matrix, the key parameter matrix and the value parameter matrix are all matrices composed of learnable parameters in the attention processing layer; based on the query vector sequence, the key vector sequence and the value vector sequence, a script attention vector sequence for the script feature vector is generated.

[0176] Specifically, the computer device can obtain the query parameter matrix W in the attention processing layer Q , key parameter matrix W K Sum parameter matrix W V , the script feature vector L and the query parameter matrix W Q By performing a dot multiplication operation, we can obtain the query vector sequence Q of the self-attention function, which is defined as Q = W Q L; combine the script feature vector L with the key parameter matrix W K By performing a dot multiplication operation, we can get the query vector sequence K of the self-attention function, which is defined as K = W K L; combine the script feature vector L with the key parameter matrix W V By performing a dot multiplication operation, we can obtain the query vector sequence V of the self-attention function, which is defined as V = W V ·L. Among them, the query parameter matrix W Q , key parameter matrix W K Sum parameter matrix W V Both are matrices composed of learnable parameters in the attention processing layer, which are used to map the script feature vector to the representation space through linear transformation.

[0177] The computer device can generate a script attention vector sequence for the script feature vector through the query vector sequence Q, the key vector sequence K, and the value vector sequence V. For example, the key vector sequence K can be transposed to obtain the first fusion parameter K T , the query vector sequence Q and the first fusion parameter K T The result of the dot multiplication operation is determined as the second fusion parameter QK T , based on the second fusion parameter QK T The number of dimensions d corresponding to the script feature vector k Generate the third fusion parameters Normalize the third fusion parameter to obtain the normalized parameter Normalize the parameters The result of the dot product operation with the value vector V is determined as the script attention vector sequence Attention(Q,K,V). The process can be shown as the following formula (1):

[0178]

[0179] The computer device can perform similarity matching between the script attention vector in the above-mentioned script attention vector sequence and the special effect feature vector of each barrage special effect material to obtain a matching result. For example, the dot product result of the script attention vector and the special effect feature vector of each barrage special effect material can be calculated by cosine similarity. After normalizing the modulus of the script attention vector and the modulus of the special effect feature vector of each barrage special effect material, the product of the modulus of the script attention vector and the modulus of the special effect feature vector of each barrage special effect material is calculated, thereby calculating the ratio of the dot product result to the product of the modulus, and using the ratio as the cosine similarity between the script attention vector and the special effect feature vector of each barrage special effect material, that is, the matching result. If the maximum cosine similarity is greater than the similarity threshold, the barrage special effect material corresponding to the maximum cosine similarity is determined as the barrage special effect material recommended for the target video clip in the barrage configuration recommendation information.

[0180] The computer device can output barrage configuration suggestion information with the suggested output format indicated by the prompt word based on the matching result, and the specific process can be: obtaining the target barrage special effect material that matches the target video clip script in the video script based on the matching result; the barrage special effect material set includes the target barrage special effect material; obtaining the target special effect material attribute information and target suggestion reason information of the target barrage special effect material according to the suggested output format; generating the barrage configuration suggestion information with the suggested output format indicated by the prompt word based on the target special effect material attribute information and the target suggestion reason information.

[0181] Specifically, the computer device can determine the barrage special effects material corresponding to the special effects feature vector whose cosine similarity is greater than the similarity threshold as the target barrage special effects material in the barrage special effects material set that matches the target video clip script in the video script, and according to the above-mentioned recommended output format, the target special effects material attribute information (barrage special effects name of the target barrage special effects material, display duration of the target barrage special effects material, and other barrage special effects rules) and target recommended reason information (recommended reasons for using the target barrage special effects material, etc.) of the target barrage special effects material can be obtained. Based on the target special effects material attribute information, the target recommended reason information and the barrage analysis instruction, the barrage configuration recommendation information with the recommended output format indicated by the prompt word can be generated.

[0182] Among them, the barrage configuration suggestion information includes multiple video clip scripts that can be configured with barrage special effect materials, and any video clip script included in the barrage configuration suggestion information can be associated with a matching barrage special effect material. For example, the barrage configuration suggestion information can be composed of segment suggestion information corresponding to multiple video clip scripts, one of which can be "suggested configuration of barrage special effect material 1; special effect material attribute information of barrage special effect material 1 (such as barrage special effect name or barrage special effect ID (Identification), special effect display duration and other barrage special effect rules); reason information for the suggestion of barrage special effect material 1; other suggestion information (such as special effects that are not currently met, giving the possibility of introducing new barrage special effect materials) etc."

[0183] Step S207, obtain a set of suggestion information; the suggestion information set includes multiple suggestion information, and the multiple suggestion information includes barrage configuration suggestion information; perform quality scoring on the suggestion information set to obtain a scoring set; the scoring set includes a scoring result corresponding to each suggestion information, and the scoring result is a positive scoring result or a negative scoring result.

[0184] In the embodiments of this application, Figure 6 As shown, the computer device can obtain the set of suggestion information output by the large language model for multiple single-episode videos corresponding to multiple video single-episode identification information, and send the suggestion information set containing the barrage configuration suggestion information to the barrage management platform through the special effects computing service, and call the write interface of the barrage management platform to record the suggestion information set. In the barrage management platform, by performing quality scoring on the suggestion information set (such as quality scoring through operation objects), a scoring set containing the scoring results corresponding to each piece of suggestion information can be obtained, where the scoring results can be divided into positive scoring results (such as excellent, good) or negative scoring results (such as poor).

[0185] The computer device can count the proportion of positive and negative scoring results in the scoring set. If the proportion of positive scoring results is greater than a ratio threshold (such as 0.7), it can be determined that the large language model has been iterated, that is, it can provide accurate recommendation information, so that an automated configuration service can be set up in the barrage management platform. The automated configuration service is used to instruct the barrage management platform to configure the barrage special effect material indicated by the new barrage configuration recommendation information to the new video clip when the large language model outputs new barrage configuration recommendation information for the new video clip. For example, the new barrage configuration recommendation information of the large language model for the new video clip can be: "Recommended configuration of barrage special effect material; recommended barrage special effect configuration: barrage special effect material 3, special effect display duration 1 minute, triggered by keyword; recommended reason information: the barrage special effect material type is in line with the current clip style." Then, based on the automated configuration service, the computer device can configure the barrage special effect material 3 indicated by the new barrage configuration recommendation information on the barrage management platform, and display the configured barrage special effect material 3 on the new video clip.

[0186] If the proportion of negative scoring results in the scoring set is greater than the proportion threshold, it can be determined that the large language model has not been iterated, that is, it cannot provide accurate recommendation information. The computer device can then use the RLHF (Reinforcement Learning from Human Feedback) method to determine the positive scoring results in the scoring set as positive sample data, and the negative scoring results in the scoring set as negative sample data, and perform reinforcement learning on the large language model based on the positive sample data and the negative sample data.

[0187] For example, the large language model can generate the predicted probability that the scoring result of the new video data is a positive scoring result, as well as the true probability that the scoring result is a positive scoring result. Based on the predicted probability and the true probability, the fine-tuning training loss value for the large language model can be generated. The cross-entropy loss function is used as an example to illustrate the loss function for generating the fine-tuning training loss value. The process of calculating the model loss value R can be shown as formula (2):

[0188]

[0189] in, is the predicted probability generated by the large language model, y represents the true probability generated by the large language model, which can correspond to 0 or 1 in the binary classification. When the scoring result is a positive scoring result, y = 1, and the loss value is This means that when the predicted probability The closer it is to 1, the closer the model loss value R is to 0, and the more accurate the prediction of the large language model is; when the scoring result is a negative scoring result, y = 0, and the loss value is Then when the predicted probability The closer it is to 0, the closer the resulting model loss value R is to 0, and the more accurate the large language model's predictions are. The computer device can calculate the average of the model loss values R corresponding to all scoring results to obtain a fine-tuning training loss value, which reflects the large language model's prediction accuracy for the predicted probability under the current parameters. Under the stochastic gradient descent method, the large language model can be reinforced with learning based on the fine-tuning training loss value to obtain a reinforcement learning model for the barrage domain, that is, a large language model that can align the configuration preferences of operational objects in the barrage domain. The process can be referred to the process of adjusting the model parameters of the initial large language model using the stochastic gradient descent method described above, and will not be repeated here.

[0190] Further, please also refer to Figure 7 , Figure 7 This is a scenario diagram of a data processing method provided in an embodiment of the present application. Figure 2 ,like Figure 7 As shown, the computer device can create a configuration suggestion control in the bullet chat management page of the bullet chat management platform, and can respond to the trigger operation of the configuration suggestion control in the bullet chat management page to perform CID aggregation display, that is, display multiple video set identification information, such as video set identification information 1, video set identification information 2, ..., and the multiple video set identification information includes the target video set identification information corresponding to the video set to which the video data belongs. It is understandable that the multiple video set identification information can also be displayed in the form of a control in the bullet chat management page.

[0191] The computer device can respond to the trigger operation for the target video set identification information, filter the vid list in the barrage management page, and display multiple vid information under one cid information, that is, display multiple video episode identification information associated with the video set to which the video data belongs, such as video episode identification information 1, video episode identification information 2,..., and the multiple video episode identification information includes the target video episode identification information corresponding to the video data. It can be understood that the multiple video episode identification information can also be displayed in the barrage management page in the form of controls.

[0192] The computer device can respond to the trigger operation for the target video episode identification information and display multiple fragment suggestion information corresponding to the video data on the barrage management page, such as fragment suggestion information 1, fragment suggestion information 2, ..., and the multiple fragment suggestion information can respectively correspond to multiple video fragments in the video data. Each fragment suggestion information includes the name of the barrage special effect, the barrage special effect rules (special effect display duration, special effect display time period, etc.) and the reason for the suggestion. It is understandable that multiple fragment suggestion information can also be displayed in the form of controls on the barrage management page.

[0193] The computer device can respond to the quality scoring operation for the segment suggestion information (including scoring options such as excellent, good, and poor), and display the segment suggestion information and the input scoring result in association; the scoring result is used to indicate whether the video segment is configured with the barrage special effects material indicated by the segment suggestion information, that is, before reinforcement learning is performed on the large language model, the scoring results corresponding to multiple segment suggestion information can be used to guide the operation object to configure the barrage special effects material. When the number of segment suggestion information is greater than the quantity threshold, the computer device can select the above-mentioned positive and negative sample data based on the scoring results corresponding to the multiple segment suggestion information.

[0194] The embodiment of the present application can obtain a video script containing multiple video clip scripts by acquiring video data and performing script parsing on the video data according to a multimodal model. After generating a prompt word containing a set of barrage special effects materials, the prompt word and the video script can be input into a large language model, so that each video clip script can be analyzed and matched with each barrage special effects material in the set of barrage special effects materials according to the barrage analysis instructions in the prompt word in the large language model, and barrage configuration suggestion information with a recommended output format indicated by the prompt word is obtained, wherein the barrage configuration suggestion information includes the barrage special effects materials recommended for the target video clip in the multiple video clips and the reason for the suggestion. By parsing the video script through a multimodal model and then analyzing and matching the video script and the barrage special effects materials according to the large language model, the entire configuration process can be simplified and the configuration efficiency of the barrage special effects can be improved. Moreover, by using a large language model to analyze and match each video clip script with each barrage special effect material in the barrage special effect material set, the possibility of the configured barrage special effect having a low match with the plot can be effectively reduced, making the configured barrage special effect material and the video script more matched. Then, configuring the barrage special effect material based on the barrage configuration suggestion information will have a better configuration effect.

[0195] Further, see Figure 8 , Figure 8 Schematic diagram of a data processing device provided in an embodiment of the present application. Figure 8 As shown, the data processing device may include a transceiver module 1100, a video script generation module 1200, a prompt word generation module 1300 and a model processing module 1400.

[0196] The transceiver module 1100 is used to obtain video data;

[0197] The video script generation module 1200 is used to perform script analysis on the video data to obtain a video script; the video script includes video clip scripts corresponding to multiple video clips;

[0198] The prompt word generation module 1300 is used to obtain a set of barrage special effect materials and generate prompt words containing the set of barrage special effect materials;

[0199] The model processing module 1400 is used to analyze and match each video clip script with each barrage special effect material in the barrage special effect material set according to the barrage analysis instructions in the prompt word, and obtain barrage configuration recommendation information with the recommended output format indicated by the prompt word; the barrage configuration recommendation information includes the barrage special effect materials recommended for the target video clip in multiple video clips and the reason information for the recommendation.

[0200] In a possible implementation, the data processing device further includes a trigger display module 1500, which is specifically configured to perform the following operations:

[0201] In response to a triggering operation on a configuration suggestion control in a bullet chat management page, multiple video set identification information is displayed; the multiple video set identification information includes target video set identification information corresponding to the video set to which the video data belongs;

[0202] In response to a trigger operation for the target video set identification information, multiple video set identification information associated with the video set to which the video data belongs is displayed on the bullet chat management page; the multiple video set identification information includes the target video set identification information corresponding to the video data;

[0203] In response to the triggering operation for the target video episode identification information, the barrage configuration suggestion information corresponding to the video data is displayed in the barrage management page; the barrage configuration suggestion information includes the special effect material attribute information and suggestion reason information of the barrage special effect material.

[0204] In a possible implementation, the trigger display module 1500 is further configured to perform the following operations:

[0205] In response to the quality scoring operation for the barrage configuration suggestion information, the barrage configuration suggestion information and the input scoring result are displayed in association; the scoring result is used to indicate whether the barrage special effect material indicated by the barrage configuration suggestion information is configured for the video data.

[0206] In one possible implementation, the video script generation module 1200 is used to perform script parsing on the video data, and when the video script is obtained, specifically to perform the following operations:

[0207] The video frame in the video data is subjected to feature extraction through the video frame feature encoding layer of the multimodal model to obtain a video frame feature vector;

[0208] Perform text recognition on the video frames to obtain the text information in the video data, and extract the features of the text information through the text feature encoding layer of the multimodal model to obtain the text feature vector;

[0209] The audio information in the video data is extracted through the audio feature encoding layer of the multimodal model to obtain an audio feature vector;

[0210] Performing feature vector fusion on the text feature vector, the video frame feature vector, and the audio feature vector associated with the same video frame in the video data to obtain a fusion vector corresponding to each video frame in the video data;

[0211] The fusion vectors corresponding to each video frame are concatenated to obtain a multimodal feature vector sequence, and a video script for the video data is generated based on the multimodal feature vector sequence.

[0212] In one possible implementation, when the video script generation module 1200 is used to generate a video script for video data based on a multimodal feature vector sequence, it is specifically used to perform the following operations:

[0213] Identify scene change frames in video data through multimodal models;

[0214] Divide the multimodal feature vector sequence based on the scene change frames to obtain multimodal feature vectors corresponding to multiple video clips;

[0215] Decoding the multimodal feature vectors corresponding to the plurality of video clips to obtain video clip scripts corresponding to the plurality of video clips;

[0216] The video clip scripts corresponding to the multiple video clips are combined into a video script of the video data.

[0217] In one possible implementation, the video script also includes theme information of the video set to which the video data belongs. The model processing module 1400 is configured to analyze and match each video clip script with each barrage special effect material in the barrage special effect material set according to the barrage analysis instruction in the prompt word, and to obtain barrage configuration suggestion information having a suggested output format indicated by the prompt word, specifically to perform the following operations:

[0218] In the large language model, according to the barrage analysis instruction in the prompt word, a first barrage special effect material set associated with the theme information is selected from the barrage special effect material set;

[0219] Perform feature extraction on each video clip script according to the barrage analysis instruction to obtain a script feature vector corresponding to each video clip script, and perform feature extraction on each barrage special effect material in the first barrage special effect material set to obtain a special effect feature vector corresponding to each barrage special effect material;

[0220] The script attention vector sequence of the script feature vector is constructed through the large language model, and the script attention vector sequence and the special effect feature vector are matched for similarity to obtain the matching result;

[0221] Output the bullet screen configuration suggestion information in the suggested output format indicated by the prompt word according to the matching result.

[0222] In one possible implementation, the large language model includes an attention processing layer; when the model processing module 1400 is used to construct a script attention vector sequence of the script feature vector using the large language model, it is specifically used to perform the following operations:

[0223] Perform a dot product operation on the query parameter matrix in the attention processing layer and the script feature vector to obtain the query vector sequence of the self-attention function;

[0224] Perform a dot product operation on the key parameter matrix in the attention processing layer and the script feature vector to obtain the key vector sequence of the self-attention function;

[0225] Perform a dot product operation on the value parameter matrix in the attention processing layer and the script feature vector to obtain the value vector sequence of the self-attention function; the query parameter matrix, key parameter matrix, and value parameter matrix are all matrices composed of the learnable parameters in the attention processing layer;

[0226] A script attention vector sequence for the script feature vector is generated based on the query vector sequence, the key vector sequence, and the value vector sequence.

[0227] In one possible implementation, when the model processing module 1400 is configured to output the bullet comment configuration suggestion information having the suggested output format indicated by the prompt word according to the matching result, it is specifically configured to perform the following operations:

[0228] Obtain target barrage special effects materials that match the target video clip script in the video script according to the matching results; the barrage special effects material set includes the target barrage special effects materials;

[0229] Obtain target special effect material attribute information and target suggestion reason information of the target barrage special effect material according to the recommended output format;

[0230] Based on the target special effect material attribute information and the target suggestion reason information, the barrage configuration suggestion information having the suggested output format indicated by the prompt word is generated.

[0231] In a possible implementation, the model processing module 1400 is further configured to perform the following operations:

[0232] Obtain historical video data, historical barrage special effects materials, and historical barrage configuration suggestion information; historical barrage configuration suggestion information is generated based on the usage records of historical barrage special effects materials based on historical video data;

[0233] Input historical video data and historical barrage special effects materials into the initial large language model, and output predicted barrage configuration suggestion information through the initial large language model;

[0234] A training loss value is generated based on historical barrage configuration suggestion information and predicted barrage configuration suggestion information, and the model parameters of the initial large language model are adjusted based on the training loss value to obtain a large language model for generating barrage configuration suggestion information.

[0235] In a possible implementation, the model processing module 1400 is further configured to perform the following operations:

[0236] Get a set of suggested information; the suggested information set contains multiple suggested information, including bullet screen configuration suggested information;

[0237] Performing a quality score on the set of recommended information to obtain a score set; the score set contains the score results corresponding to each piece of recommended information, and the score results are positive or negative.

[0238] If the proportion of positive scoring results in the scoring set is greater than the proportion threshold, an automatic configuration service is set up in the barrage management platform; the automatic configuration service is used to instruct the barrage management platform to configure the barrage special effect materials indicated by the new barrage configuration suggestion information to the new video clip when the large language model outputs new barrage configuration suggestion information for the new video clip.

[0239] In a possible implementation, the model processing module 1400 is further configured to perform the following operations:

[0240] If the proportion of negative scoring results in the scoring set is greater than the proportion threshold, the positive scoring results in the scoring set are determined as positive sample data, and the negative scoring results in the scoring set are determined as negative sample data;

[0241] Re-fine-tune the large language model based on positive and negative sample data.

[0242] In one possible implementation, when the transceiver module 1100 is used to obtain video data, it is specifically used to perform the following operations:

[0243] Get a video comment parsing request; the video comment parsing request includes video episode identification information;

[0244] Obtaining video set identification information in the video single set identification information according to the video barrage analysis request, and obtaining a video library; the video library includes multiple single-set videos respectively contained in multiple video sets;

[0245] A target video set corresponding to the video set identification information is searched in the video library, and a single-episode video corresponding to the video single-episode identification information among multiple single-episode videos associated with the target video set is determined as video data.

[0246] The embodiment of the present application can obtain a video script containing multiple video clip scripts by acquiring video data and performing script parsing on the video data according to a multimodal model. After generating a prompt word containing a set of barrage special effects materials, the prompt word and the video script can be input into a large language model, so that each video clip script can be analyzed and matched with each barrage special effects material in the set of barrage special effects materials according to the barrage analysis instructions in the prompt word in the large language model, and barrage configuration suggestion information with a recommended output format indicated by the prompt word is obtained, wherein the barrage configuration suggestion information includes the barrage special effects materials recommended for the target video clip in the multiple video clips and the reason for the suggestion. By parsing the video script through a multimodal model and then analyzing and matching the video script and the barrage special effects materials according to the large language model, the entire configuration process can be simplified and the configuration efficiency of the barrage special effects can be improved. Moreover, by using a large language model to analyze and match each video clip script with each barrage special effect material in the barrage special effect material set, the possibility of the configured barrage special effect having a low match with the plot can be effectively reduced, making the configured barrage special effect material and the video script more matched. Then, configuring the barrage special effect material based on the barrage configuration suggestion information will have a better configuration effect.

[0247] Further, see Figure 9 , Figure 9 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. Figure 9As shown, the computer device 1000 in the embodiment of the present application may include: a processor 1001, a network interface 1004 and a memory 1005. In addition, the above-mentioned computer device 1000 may also include: an object interface 1003, and at least one communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. Among them, the object interface 1003 may include a display screen (Display), a keyboard (Keyboard), and the object interface 1003 may optionally include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or it may be a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 1005 may optionally also be at least one storage device located away from the aforementioned processor 1001. As Figure 9 As shown, the memory 1005 as a computer-readable storage medium may include an operating system, a network communication module, an object interface module, and a computer program.

[0248] The network interface 1004 can provide a network communication network element; the object interface 1003 is mainly used to provide an input interface for the object; and the processor 1001 can be used to call the computer program stored in the memory 1005 to execute the steps in each embodiment of the present application.

[0249] In addition, it should be noted that: the embodiment of the present application also provides a computer-readable storage medium, the computer-readable storage medium stores a computer program, the computer program is suitable for being loaded and executed by the processor Figure 3 or Figure 4 For details on the methods provided in each step, please refer to the Figure 3 or Figure 4 The implementation methods provided in each step will not be repeated here. In addition, the description of the beneficial effects of adopting the same method will not be repeated. For technical details not disclosed in the computer-readable storage medium embodiment involved in this application, please refer to the description of the method embodiment of this application. As an example, the computer program can be deployed to be executed on one computer device, or on multiple computer devices located in one location, or on multiple computer devices distributed in multiple locations and interconnected by a communication network.

[0250] The computer-readable storage medium may be the device provided in any of the aforementioned embodiments or the internal storage unit of the computer device, such as the hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the computer device. Furthermore, the computer-readable storage medium may also include both the internal storage unit of the computer device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0251] The present application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium, and the processor executes the computer program, so that the computer device performs Figure 3 or Figure 4 The methods provided in the various optional methods are therefore not described in detail here.

[0252] The terms "first", "second", etc. in the description, claims, and drawings of the embodiments of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "comprising" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product, or device comprising a series of steps or units is not limited to the listed steps or modules, but may optionally include steps or modules not listed, or may optionally include other step units inherent to these processes, methods, apparatuses, products, or devices.

[0253] In the embodiments of the present application, the term "module" or "unit" refers to a computer program or a part of a computer program that has a predetermined function and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be part of an overall module or unit that includes the function of the module or unit.

[0254] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in this description according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0255] The methods and related devices provided in the embodiments of the present application are described with reference to the method flow charts and / or structural diagrams provided in the embodiments of the present application. Specifically, each process and / or block in the method flow charts and / or structural diagrams, as well as the combination of processes and / or blocks in the flow charts and / or block diagrams, can be implemented by computer programs. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable device to generate a machine, so that the computer program executed by the processor of the computer or other programmable device generates a program for implementing the process. Figure 1 Schematic diagram of one or more processes and / or structures Figure 1 The computer programs can also be stored in a computer-readable memory that can direct a computer or other programmable device to work in a specific manner, so that the computer program stored in the computer-readable memory produces a manufactured product including the program device, or is transmitted through a computer-readable storage medium. The computer program can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (for example, coaxial cable, optical fiber, digital line (DSL)) or wireless (for example, infrared, wireless, microwave, etc.). The program device is implemented in the process Figure 1 Schematic diagram of one or more processes and / or structures Figure 1 These computer programs can also be loaded onto computers or other programmable devices so that a series of operating steps are executed on the computer or other programmable devices to produce computer-implemented processing, thereby providing a computer program executed on the computer or other programmable device for implementing the functions specified in the process. Figure 1 The flow or flows and / or structures illustrate the steps of the functions specified in one block or multiple blocks.

[0256] The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs.

[0257] The modules in the device of the embodiment of the present application can be merged, divided and deleted according to actual needs.

[0258] The above disclosure is only a preferred embodiment of the present application, and certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A data processing method, characterized in that: The method comprises: Get video data; Performing script analysis on the video data to obtain a video script; the video script includes video clip scripts corresponding to the plurality of video clips; Obtaining a set of barrage special effects materials, and generating prompt words containing the set of barrage special effects materials; According to the barrage analysis instructions in the prompt word, each video clip script is analyzed and matched with each barrage special effects material in the barrage special effects material set to obtain barrage configuration recommendation information with the recommended output format indicated by the prompt word; the barrage configuration recommendation information includes the barrage special effects material recommended for the target video clip among the multiple video clips and the reason for the recommendation.

2. The method according to claim 1, characterized in that The method further comprises: In response to a triggering operation on a configuration suggestion control in a bullet screen management page, multiple video set identification information is displayed; the multiple video set identification information includes target video set identification information corresponding to the video set to which the video data belongs; In response to a triggering operation on the target video set identification information, displaying multiple video episode identification information associated with the video set to which the video data belongs on the bullet comment management page; the multiple video episode identification information includes the target video episode identification information corresponding to the video data; In response to the triggering operation for the target video episode identification information, the barrage configuration suggestion information corresponding to the video data is displayed in the barrage management page; the barrage configuration suggestion information includes the special effect material attribute information of the barrage special effect material and the suggestion reason information.

3. The method according to claim 2, characterized in that The method further comprises: In response to a quality scoring operation for the barrage configuration suggestion information, the barrage configuration suggestion information and the input scoring result are displayed in association; the scoring result is used to indicate whether the barrage special effect material indicated by the barrage configuration suggestion information is configured for the video data.

4. The method according to claim 1, wherein The script parsing of the video data to obtain the video script includes: Extracting features from the video frames in the video data through a video frame feature encoding layer of a multimodal model to obtain a video frame feature vector; Performing text recognition on the video frame to obtain text information in the video data, and performing feature extraction on the text information through the text feature encoding layer of the multimodal model to obtain a text feature vector; Extracting features of the audio information in the video data through the audio feature encoding layer of the multimodal model to obtain an audio feature vector; Performing feature vector fusion on the text feature vector, the video frame feature vector, and the audio feature vector associated with the same video frame in the video data to obtain a fusion vector corresponding to each video frame in the video data; The fusion vectors corresponding to each video frame are vector-concatenated to obtain a multimodal feature vector sequence, and a video script for the video data is generated based on the multimodal feature vector sequence.

5. The method according to claim 4, characterized in that Generating a video script for the video data based on the multimodal feature vector sequence includes: identifying scene change frames in the video data using the multimodal model; Dividing the multimodal feature vector sequence based on the scene change frame to obtain multimodal feature vectors corresponding to a plurality of video clips; Decoding the multimodal feature vectors corresponding to the plurality of video clips to obtain video clip scripts corresponding to the plurality of video clips; The video clip scripts corresponding to the multiple video clips are combined into the video script of the video data.

6. The method according to claim 1, characterized in that The video script also includes theme information of the video set to which the video data belongs; and the step of analyzing and matching each video clip script with each barrage special effect material in the barrage special effect material set according to the barrage analysis instruction in the prompt word to obtain barrage configuration suggestion information having a suggested output format indicated by the prompt word, including: In the large language model, according to the barrage analysis instruction in the prompt word, a first barrage special effect material set associated with the theme information is selected from the barrage special effect material set; Performing feature extraction on each video clip script according to the barrage analysis instruction to obtain a script feature vector corresponding to each video clip script, and performing feature extraction on each barrage special effect material in the first barrage special effect material set to obtain a special effect feature vector corresponding to each barrage special effect material; Constructing a script attention vector sequence of the script feature vector using the large language model, performing similarity matching on the script attention vector sequence and the special effect feature vector to obtain a matching result; Output the bullet screen configuration suggestion information having the suggested output format indicated by the prompt word according to the matching result.

7. The method according to claim 6, characterized in that The large language model includes an attention processing layer; the script attention vector sequence for constructing the script feature vector through the large language model includes: Performing a dot product operation on the query parameter matrix in the attention processing layer and the script feature vector to obtain a query vector sequence of the self-attention function; Performing a dot product operation on the key parameter matrix in the attention processing layer and the script feature vector to obtain a key vector sequence of the self-attention function; Performing a dot product operation on the value parameter matrix in the attention processing layer and the script feature vector to obtain a value vector sequence of the self-attention function; the query parameter matrix, the key parameter matrix, and the value parameter matrix are all matrices composed of learnable parameters in the attention processing layer; A script attention vector sequence for the script feature vector is generated based on the query vector sequence, the key vector sequence, and the value vector sequence.

8. The method according to claim 6, characterized in that Outputting the bullet screen configuration suggestion information having the suggested output format indicated by the prompt word according to the matching result includes: Acquire a target barrage special effect material that matches the target video clip script in the video script according to the matching result; the barrage special effect material set includes the target barrage special effect material; Obtain target special effect material attribute information and target suggestion reason information of the target barrage special effect material according to the suggested output format; Based on the target special effect material attribute information and the target suggestion reason information, barrage configuration suggestion information having a suggested output format indicated by the prompt word is generated.

9. The method according to claim 1, characterized in that The method further comprises: Obtaining historical video data, historical barrage special effect materials, and historical barrage configuration suggestion information; the historical barrage configuration suggestion information is generated based on the usage record of the historical video data for the historical barrage special effect materials; Inputting the historical video data and the historical barrage special effect materials into an initial large language model, and outputting predicted barrage configuration suggestion information through the initial large language model; A training loss value is generated according to the historical barrage configuration suggestion information and the predicted barrage configuration suggestion information, and the model parameters of the initial large language model are adjusted based on the training loss value to obtain a large language model for generating the barrage configuration suggestion information.

10. The method according to claim 1, characterized in that The method further comprises: Obtain a set of suggestion information; the set of suggestion information includes a plurality of suggestion information, and the plurality of suggestion information includes the barrage configuration suggestion information; Performing a quality score on the set of suggestion information to obtain a score set; the score set includes a score result corresponding to each piece of suggestion information, and the score result is a positive score result or a negative score result; If the proportion of positive scoring results in the scoring set is greater than the proportion threshold, an automatic configuration service is set up in the barrage management platform; the automatic configuration service is used to instruct the barrage management platform to configure the barrage special effect materials indicated by the new barrage configuration suggestion information to the new video clip when the large language model outputs new barrage configuration suggestion information for the new video clip.

11. The method according to claim 9, characterized in that The method further comprises: If the proportion of negative scoring results in the scoring set is greater than the proportion threshold, the positive scoring results in the scoring set are determined as positive sample data, and the negative scoring results in the scoring set are determined as negative sample data; The large language model is re-fine-tuned based on the positive sample data and the negative sample data.

12. The method according to claim 1, characterized in that The obtaining of video data includes: Obtain a video comment parsing request; the video comment parsing request includes video episode identification information; Obtaining the video set identification information in the video single set identification information according to the video barrage analysis request, and obtaining a video library; the video library includes multiple single-set videos respectively included in multiple video sets; A target video set corresponding to the video set identification information is searched in the video library, and a single-episode video corresponding to the video single-episode identification information among a plurality of single-episode videos associated with the target video set is determined as video data.

13. A data processing device, characterized in that: The device comprises: A transceiver module for acquiring video data; A video script generation module is used to perform script analysis on the video data to obtain a video script; the video script includes video clip scripts corresponding to multiple video clips; A prompt word generation module is used to obtain a set of barrage special effect materials and generate prompt words containing the set of barrage special effect materials; The model processing module is used to analyze and match each video clip script with each barrage special effect material in the barrage special effect material set according to the barrage analysis instructions in the prompt word, and obtain barrage configuration recommendation information with the recommended output format indicated by the prompt word; the barrage configuration recommendation information includes the barrage special effect material recommended for the target video clip among the multiple video clips and the reason for the recommendation.

14. A computer device, characterized in that: Includes processor, memory, input and output interfaces; The processor is connected to the memory and the input / output interface respectively, wherein the input / output interface is used to receive and output data, the memory is used to store a computer program, and the processor is used to call the computer program so that the computer device executes the method according to any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which is suitable for being loaded and executed by a processor, so that a computer device having the processor executes the method according to any one of claims 1 to 12.

16. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 12 is implemented.

Images