Data processing method, device, apparatus and readable storage medium

Abstract

The application discloses a data processing method, device and equipment and a readable storage medium. The method comprises the following steps: acquiring video data; performing script analysis on the video data to obtain a video script; the video script comprises video segment scripts corresponding to a plurality of video segments; acquiring a barrage special effect material set, generating a prompt word comprising the barrage special effect material set; performing analysis matching on each video segment script and each barrage special effect material in the barrage special effect material set according to a barrage analysis instruction in the prompt word, obtaining barrage configuration suggestion information with a suggested output format indicated by the prompt word; the barrage configuration suggestion information comprises barrage special effect materials and suggestion reason information suggested for a target video segment in the plurality of video segments. According to the application, the configuration efficiency of the barrage special effect and the configuration effect of the barrage special effect can be improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data processing method, apparatus, device, and readable storage medium. Background Technology

[0002] Users can first identify the storylines requiring bullet screen effects by watching videos, then select bullet screen effects that match the storyline from existing bullet screen effect materials, and finally configure the matching bullet screen effects on the bullet screen management platform. If users configure bullet screen effects with low matching to the storyline, it may reduce the effectiveness of the configured bullet screen effects. Furthermore, because the entire configuration process relies heavily on manual operation, it may result in low configuration efficiency. Summary of the Invention

[0003] This application provides a data processing method, apparatus, computer, storage medium, and program product that can improve the configuration efficiency and effect of bullet screen effects.

[0004] One embodiment of this application provides a data processing method, the method comprising:

[0005] Acquire video data;

[0006] The video data is parsed to obtain the video script; the video script contains multiple video segment scripts corresponding to each video segment.

[0007] Obtain a set of bullet screen effect materials and generate prompts containing the bullet screen effect materials set;

[0008] Based on the bullet screen analysis instructions in the prompt, each video segment script is analyzed and matched with each bullet screen effect material in the bullet screen effect material set to obtain bullet screen configuration suggestion information with the suggested output format indicated by the prompt; the bullet screen configuration suggestion information includes the bullet screen effect materials suggested for the target video segment in multiple video segments and the reason for the suggestion.

[0009] This also includes:

[0010] In response to the triggered operation of the configuration suggestion control on the bullet screen management page, multiple video set identification information is displayed; the multiple video set identification information includes the target video set identification information corresponding to the video set to which the video data belongs;

[0011] In response to a trigger operation targeting the identification information of the target video set, the bullet screen management page displays the identification information of multiple video sets associated with the video set to which the video data belongs; the identification information of multiple video sets includes the identification information of the target video set corresponding to the video data;

[0012] In response to the triggered operation targeting the identifier information of the target video episode, the bullet screen management page displays the bullet screen configuration suggestion information corresponding to the video data; the bullet screen configuration suggestion information includes the special effect material attribute information and the reason for the suggestion.

[0013] This also includes:

[0014] In response to the quality rating operation of the bullet screen configuration suggestion information, the bullet screen configuration suggestion information and the input rating result are displayed together; the rating result is used to indicate whether the bullet screen effect material indicated by the bullet screen configuration suggestion information is configured in the video data configuration.

[0015] This includes parsing the video data to obtain the video script, which includes:

[0016] The video frame feature encoding layer of the multimodal model extracts features from the video frames in the video data to obtain the video frame feature vector;

[0017] Text recognition is performed on video frames to obtain text information in the video data. The text information is then used to extract features from the text feature encoding layer of a multimodal model to obtain text feature vectors.

[0018] Audio feature vectors are obtained by extracting audio information from video data through the audio feature coding layer of a multimodal model.

[0019] The text feature vector, video frame feature vector, and audio feature vector associated with the same video frame in the video data are fused to obtain the fused vector corresponding to each video frame in the video data.

[0020] The fusion vectors corresponding to each video frame are concatenated to obtain a multimodal feature vector sequence, and a video script for the video data is generated based on the multimodal feature vector sequence.

[0021] Among them, the generation of video scripts for video data based on multimodal feature vector sequences includes:

[0022] Identify scene change frames in video data using a multimodal model;

[0023] The multimodal feature vector sequence is divided based on the scene change frames to obtain multimodal feature vectors corresponding to multiple video segments respectively;

[0024] Decode the multimodal feature vectors corresponding to multiple video segments to obtain the video segment scripts corresponding to each video segment.

[0025] The video scripts corresponding to multiple video segments are combined into a video script of video data.

[0026] The video script also includes the theme information of the video set to which the video data belongs; based on the bullet screen analysis instructions in the prompts, each video segment script is analyzed and matched with each bullet screen effect material in the bullet screen effect material set to obtain bullet screen configuration suggestion information with the suggested output format indicated by the prompts, including:

[0027] In the large language model, based on the bullet screen analysis instructions in the prompt words, the first set of bullet screen effect materials related to the theme information is selected from the set of bullet screen effect materials.

[0028] Based on the barrage analysis instructions, feature extraction is performed on the script of each video segment to obtain the script feature vector corresponding to each video segment script. Feature extraction is also performed on each barrage effect material in the first barrage effect material set to obtain the effect feature vector corresponding to each barrage effect material.

[0029] The script attention vector sequence of script feature vectors is constructed by using a large language model. The script attention vector sequence and the special effects feature vector are then matched for similarity to obtain the matching results.

[0030] Based on the matching results, output suggested bullet screen configuration information with the suggested output format indicated by the prompt words.

[0031] The large language model includes an attention processing layer; the script attention vector sequence, which constructs script feature vectors through the large language model, includes:

[0032] Perform a dot product operation on the query parameter matrix and script feature vector in the attention processing layer to obtain the query vector sequence of the self-attention function;

[0033] Perform a dot product operation between the key parameter matrix in the attention processing layer and the script feature vector to obtain the key vector sequence of the self-attention function;

[0034] The value parameter matrix and script feature vector in the attention processing layer are multiplied by a dot product to obtain the value vector sequence of the self-attention function; the query parameter matrix, key parameter matrix and value parameter matrix are all matrices composed of learnable parameters in the attention processing layer;

[0035] Generate a script attention vector sequence based on the query vector sequence, key vector sequence, and value vector sequence, targeting the script feature vector.

[0036] The system outputs suggested bullet screen configuration information with the suggested output format indicated by the prompt words, based on the matching results. This includes:

[0037] Based on the matching results, target bullet screen effect materials that match the target video segment script in the video script are obtained; the bullet screen effect material set includes the target bullet screen effect materials;

[0038] Obtain the target bullet screen effect material's attribute information and the reasoning behind the target suggestion information according to the suggested output format;

[0039] Based on the target special effects material attribute information and the target suggestion reason information, generate bullet screen configuration suggestion information with the suggested output format indicated by the prompt words.

[0040] This also includes:

[0041] It retrieves historical video data, historical bullet screen effect materials, and historical bullet screen configuration suggestions; the historical bullet screen configuration suggestions are generated based on the usage records of historical bullet screen effect materials from historical video data.

[0042] Historical video data and historical bullet screen effect materials are input into the initial large language model, and the initial large language model outputs prediction bullet screen configuration suggestions;

[0043] Training loss values ​​are generated based on historical and predicted bullet comment configuration suggestions. The model parameters of the initial large language model are then adjusted based on the training loss values ​​to obtain the large language model used to generate bullet comment configuration suggestions.

[0044] This also includes:

[0045] Retrieve a set of suggestion information; the set of suggestion information contains multiple suggestion information, including suggestion information for bullet screen configuration.

[0046] The set of suggested information is scored to obtain a score set; the score set contains the score result corresponding to each suggested information, and the score result is either a positive score result or a negative score result;

[0047] If the proportion of positive rating results in the rating set is greater than the proportion threshold, then an automated configuration service is set in the bullet screen management platform. The automated configuration service is used to instruct the bullet screen management platform to configure the bullet screen effect materials indicated by the new bullet screen configuration suggestion information to the new video clip when the big language model outputs new bullet screen configuration suggestion information for the new video clip.

[0048] This also includes:

[0049] If the proportion of negative ratings in the rating set is greater than the proportion threshold, then the positive ratings in the rating set are determined as positive sample data, and the negative ratings in the rating set are determined as negative sample data.

[0050] The large language model is retrained using both positive and negative sample data.

[0051] Acquiring video data includes:

[0052] Get the video bullet comment parsing request; the video bullet comment parsing request includes video episode identifier information;

[0053] Based on the video bullet screen parsing request, obtain the video set identifier information from the video set identifier information, and obtain the video library; the video library includes multiple video sets, each containing multiple individual video sets;

[0054] Query the target video set corresponding to the video set identifier information in the video library, and identify the single video set corresponding to the single video set identifier information among the multiple single video sets associated with the target video set as video data.

[0055] One embodiment of this application provides a data processing apparatus, the apparatus comprising:

[0056] The transceiver module is used to acquire video data;

[0057] The video script generation module is used to parse video data to obtain video scripts; the video script contains multiple video segment scripts corresponding to each video segment.

[0058] The prompt word generation module is used to obtain a set of bullet screen effect materials and generate prompt words containing the set of bullet screen effect materials;

[0059] The model processing module is used to analyze and match the script of each video segment with each bullet screen effect material in the bullet screen effect material set according to the bullet screen analysis instructions in the prompt words, and obtain bullet screen configuration suggestion information with the suggested output format indicated by the prompt words; the bullet screen configuration suggestion information includes the bullet screen effect materials suggested for the target video segment in multiple video segments and the reason for the suggestion.

[0060] In one possible implementation, the data processing device further includes a trigger display module, which is specifically used to perform the following operations:

[0061] In response to the triggered operation of the configuration suggestion control on the bullet screen management page, multiple video set identification information is displayed; the multiple video set identification information includes the target video set identification information corresponding to the video set to which the video data belongs;

[0062] In response to a trigger operation targeting the identification information of the target video set, the bullet screen management page displays the identification information of multiple video sets associated with the video set to which the video data belongs; the identification information of multiple video sets includes the identification information of the target video set corresponding to the video data;

[0063] In response to the triggered operation targeting the identifier information of the target video episode, the bullet screen management page displays the bullet screen configuration suggestion information corresponding to the video data; the bullet screen configuration suggestion information includes the special effect material attribute information and the reason for the suggestion.

[0064] In one possible implementation, the trigger display module is also specifically used to perform the following operations:

[0065] In response to the quality rating operation of the bullet screen configuration suggestion information, the bullet screen configuration suggestion information and the input rating result are displayed together; the rating result is used to indicate whether the bullet screen effect material indicated by the bullet screen configuration suggestion information is configured in the video data configuration.

[0066] In one possible implementation, the video script generation module is used to parse the video data into a script. When the video script is obtained, it is specifically used to perform the following operations:

[0067] The video frame feature encoding layer of the multimodal model extracts features from the video frames in the video data to obtain the video frame feature vector;

[0068] Text recognition is performed on video frames to obtain text information in the video data. The text information is then used to extract features from the text feature encoding layer of a multimodal model to obtain text feature vectors.

[0069] Audio feature vectors are obtained by extracting audio information from video data through the audio feature coding layer of a multimodal model.

[0070] The text feature vector, video frame feature vector, and audio feature vector associated with the same video frame in the video data are fused to obtain the fused vector corresponding to each video frame in the video data.

[0071] The fusion vectors corresponding to each video frame are concatenated to obtain a multimodal feature vector sequence, and a video script for the video data is generated based on the multimodal feature vector sequence.

[0072] In one possible implementation, when the video script generation module generates a video script for video data based on a multimodal feature vector sequence, it specifically performs the following operations:

[0073] Identify scene change frames in video data using a multimodal model;

[0074] The multimodal feature vector sequence is divided based on the scene change frames to obtain multimodal feature vectors corresponding to multiple video segments respectively;

[0075] Decode the multimodal feature vectors corresponding to multiple video segments to obtain the video segment scripts corresponding to each video segment.

[0076] The video scripts corresponding to multiple video segments are combined into a video script of video data.

[0077] In one possible implementation, the video script also includes the theme information of the video set to which the video data belongs; the model processing module is used to analyze and match each video segment script with each bullet screen effect material in the bullet screen effect material set according to the bullet screen analysis instructions in the prompt words, and when obtaining bullet screen configuration suggestion information with the suggested output format indicated by the prompt words, it is specifically used to perform the following operations:

[0078] In the large language model, based on the bullet screen analysis instructions in the prompt words, the first set of bullet screen effect materials related to the theme information is selected from the set of bullet screen effect materials.

[0079] Based on the barrage analysis instructions, feature extraction is performed on the script of each video segment to obtain the script feature vector corresponding to each video segment script. Feature extraction is also performed on each barrage effect material in the first barrage effect material set to obtain the effect feature vector corresponding to each barrage effect material.

[0080] The script attention vector sequence of script feature vectors is constructed by using a large language model. The script attention vector sequence and the special effects feature vector are then matched for similarity to obtain the matching results.

[0081] Based on the matching results, output suggested bullet screen configuration information with the suggested output format indicated by the prompt words.

[0082] In one possible implementation, the large language model includes an attention processing layer; the model processing module, when constructing a script attention vector sequence of script feature vectors through the large language model, specifically performs the following operations:

[0083] Perform a dot product operation on the query parameter matrix and script feature vector in the attention processing layer to obtain the query vector sequence of the self-attention function;

[0084] Perform a dot product operation between the key parameter matrix in the attention processing layer and the script feature vector to obtain the key vector sequence of the self-attention function;

[0085] The value parameter matrix and script feature vector in the attention processing layer are multiplied by a dot product to obtain the value vector sequence of the self-attention function; the query parameter matrix, key parameter matrix and value parameter matrix are all matrices composed of learnable parameters in the attention processing layer;

[0086] Generate a script attention vector sequence based on the query vector sequence, key vector sequence, and value vector sequence, targeting the script feature vector.

[0087] In one possible implementation, when the model processing module outputs bullet screen configuration suggestion information with the suggested output format indicated by the prompt words based on the matching results, it specifically performs the following operations:

[0088] Based on the matching results, target bullet screen effect materials that match the target video segment script in the video script are obtained; the bullet screen effect material set includes the target bullet screen effect materials;

[0089] Obtain the target bullet screen effect material's attribute information and the reasoning behind the target suggestion information according to the suggested output format;

[0090] Based on the target special effects material attribute information and the target suggestion reason information, generate bullet screen configuration suggestion information with the suggested output format indicated by the prompt words.

[0091] In one possible implementation, the model processing module is also specifically used to perform the following operations:

[0092] It retrieves historical video data, historical bullet screen effect materials, and historical bullet screen configuration suggestions; the historical bullet screen configuration suggestions are generated based on the usage records of historical bullet screen effect materials from historical video data.

[0093] Historical video data and historical bullet screen effect materials are input into the initial large language model, and the initial large language model outputs prediction bullet screen configuration suggestions;

[0094] Training loss values ​​are generated based on historical and predicted bullet comment configuration suggestions. The model parameters of the initial large language model are then adjusted based on the training loss values ​​to obtain the large language model used to generate bullet comment configuration suggestions.

[0095] In one possible implementation, the model processing module is also specifically used to perform the following operations:

[0096] Retrieve a set of suggestion information; the set of suggestion information contains multiple suggestion information, including suggestion information for bullet screen configuration.

[0097] The set of suggested information is scored to obtain a score set; the score set contains the score result corresponding to each suggested information, and the score result is either a positive score result or a negative score result;

[0098] If the proportion of positive rating results in the rating set is greater than the proportion threshold, then an automated configuration service is set in the bullet screen management platform. The automated configuration service is used to instruct the bullet screen management platform to configure the bullet screen effect materials indicated by the new bullet screen configuration suggestion information to the new video clip when the big language model outputs new bullet screen configuration suggestion information for the new video clip.

[0099] In one possible implementation, the model processing module is also specifically used to perform the following operations:

[0100] If the proportion of negative ratings in the rating set is greater than the proportion threshold, then the positive ratings in the rating set are determined as positive sample data, and the negative ratings in the rating set are determined as negative sample data.

[0101] The large language model is retrained using both positive and negative sample data.

[0102] In one possible implementation, when the transceiver module acquires video data, it specifically performs the following operations:

[0103] Get the video bullet comment parsing request; the video bullet comment parsing request includes video episode identifier information;

[0104] Based on the video bullet screen parsing request, obtain the video set identifier information from the video set identifier information, and obtain the video library; the video library includes multiple video sets, each containing multiple individual video sets;

[0105] Query the target video set corresponding to the video set identifier information in the video library, and identify the single video set corresponding to the single video set identifier information among the multiple single video sets associated with the target video set as video data.

[0106] One embodiment of this application provides a computer device, including a processor, a memory, and an input / output interface;

[0107] The processor is connected to a memory and an input / output interface, respectively. The input / output interface is used to receive and output data, the memory is used to store computer programs, and the processor is used to call the computer programs so that the computer device containing the processor executes the method in one aspect of the embodiments of this application.

[0108] One aspect of this application provides a computer-readable storage medium storing a computer program adapted to be loaded and executed by a processor, so that a computer device having the processor performs the method of one aspect of this application.

[0109] One aspect of this application provides a computer program product comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the methods provided in various optional embodiments of this application. In other words, when executed by the processor, the computer program implements the methods provided in various optional embodiments of this application.

[0110] This application embodiment obtains a video script containing multiple video segment scripts by acquiring video data and parsing the video data. After generating a prompt word containing a set of bullet screen effect materials, each video segment script can be analyzed and matched with each bullet screen effect material in the bullet screen effect material set according to the bullet screen analysis instructions in the prompt word. This yields bullet screen configuration suggestion information with the suggested output format indicated by the prompt word. The bullet screen configuration suggestion information includes the bullet screen effect materials suggested for the target video segment among the multiple video segments, as well as the reason for the suggestion. By parsing the video script and then analyzing and matching the video script and bullet screen effect materials, the entire configuration process can be simplified, and the configuration efficiency of bullet screen effects can be improved. Moreover, analyzing and matching each video segment script with each bullet screen effect material in the bullet screen effect material set separately can effectively reduce the possibility of low matching degree between the configured bullet screen effects and the plot, making the configured bullet screen effect materials and the video script more matched. Therefore, the configuration effect of configuring bullet screen effect materials based on bullet screen configuration suggestion information is better. Attached Figure Description

[0111] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0112] Figure 1 This is a network interaction architecture diagram provided in an embodiment of this application;

[0113] Figure 2 This is a scenario illustration of a data processing method provided in an embodiment of this application. Figure 1 ;

[0114] Figure 3 This is a flowchart illustrating a data processing method provided in an embodiment of this application. Figure 1 ;

[0115] Figure 4 This is a flowchart illustrating a data processing method provided in an embodiment of this application. Figure 2 ;

[0116] Figure 5 This is a system architecture diagram for data processing provided in an embodiment of this application;

[0117] Figure 6 This is a schematic diagram of the model structure of a data processing method provided in an embodiment of this application;

[0118] Figure 7This is a scenario illustration of a data processing method provided in an embodiment of this application. Figure 2 ;

[0119] Figure 8 This is a schematic diagram of a data processing device provided in an embodiment of this application;

[0120] Figure 9 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0121] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0122] If this application requires the collection of object data (such as user data), a prompt interface or pop-up window will be displayed before and during the collection process. This prompt interface or pop-up window is used to inform the user that certain data is being collected. The data acquisition steps will only begin after the user confirms the prompt interface or pop-up window; otherwise, the process will end. Furthermore, the acquired user data will be used in reasonable and legal scenarios or for legitimate purposes. Optionally, in scenarios where user data needs to be used but user authorization has not been obtained, authorization can be requested from the user, and the user data can only be used after authorization is granted.

[0123] It is understood that, in the specific embodiments of this application, the user data involved requires user permission or consent when the following embodiments of this application are applied to specific products or technologies, and the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant regions.

[0124] In the embodiments of this application, please refer to Figure 1 , Figure 1 This is a network interaction architecture diagram provided in an embodiment of this application, such as... Figure 1 As shown, the network interaction architecture diagram may include a service server 100 and a terminal device 200. The terminal device 200 may have a communication connection with the service server 100. The communication connection is not limited to a specific method. It may be directly or indirectly connected via wired communication, or directly or indirectly connected via wireless communication, or in other ways. This application does not impose any restrictions on this method.

[0125] It should be understood that, such as Figure 1The terminal device 200 shown can be equipped with an application client that has data processing capabilities. When the application client runs on the terminal device 200, it can interact with the aforementioned... Figure 1 The business servers 100 shown interact with each other, enabling each business server 100 to receive business data from the terminal device 200. The application client can be a social application, instant messaging application, live streaming application, short video application, video application, music application, shopping application, novel application, browser, or other application client capable of displaying text, images, audio, and video data. This application client can be a standalone client or an embedded sub-client integrated into another client (such as an instant messaging client, social client, video client, etc.) (e.g., an app, browser webpage access, etc.), and is not limited here.

[0126] like Figure 1 As shown, the business server 100 can obtain video data and a set of bullet screen effect materials. By parsing the video data, it can obtain a video script containing multiple video segment scripts. By analyzing and matching each video segment script with each bullet screen effect material in the bullet screen effect material set, it can generate bullet screen configuration suggestion information for the video data. Bullet screen effects refer to color changes, motion trajectory changes, or other animation effects triggered under specific rules (time points, keywords) during bullet screen scrolling. The bullet screen configuration suggestion information includes multiple video segment scripts that can be configured with bullet screen effect materials, and any video segment script included in the bullet screen configuration suggestion information can be associated with a matching bullet screen effect material.

[0127] After configuring the bullet screen effect materials based on the bullet screen configuration suggestions, the bullet screen effect materials can be displayed in the video data. The business server 100 can send the video data containing the bullet screen effect materials to the terminal device 200.

[0128] The user can view video data containing bullet screen effects through terminal device 200. The triggering rules for bullet screen effects can be time-based effects, such as triggering the petal effect when the video data plays to the 2-minute and 45-second mark where the characters describe the plot. The triggering rules for bullet screen effects can also be keyword-based effects, such as triggering the petal effect when the user sends the bullet screen "XXX (e.g., beautiful)". There are no restrictions on this.

[0129] Please see Figure 2 , Figure 2 This is a scenario illustration of a data processing method provided in an embodiment of this application. Figure 1 .like Figure 2As shown, the business server 100 can obtain video data from the video library, as well as the theme information of the video set to which the video data belongs. The theme information may include the theme name of the video set, the theme type of the video set (science fiction, suspense, etc.), and the author of the video set.

[0130] After the business server 100 inputs the video data into the multimodal model, it can perform script parsing on the video data. That is, it analyzes and processes the video data containing subtitles (text information), voice (audio information), and images (video frames), and parses the video plot corresponding to each time period (or each video segment) to obtain the video script. The video script is a structured script content, including the time points of the video data, plot content, plot scenes, characters, and other information. It can also contain video segment scripts corresponding to multiple video segments, such as "Segment position: 2 minutes 45 to 3 minutes 15; In the room, character A picks up a small object and interacts with character B; Character A: XXX; Character B: XXX; ...".

[0131] Business server 100 can obtain a set of bullet screen effect materials containing multiple bullet screen effect materials from the bullet screen management platform through the special effects calculation service (a background microservice in business server 100 used for forwarding data and calling models). It can also generate prompts containing the bullet screen effect material set, bullet screen analysis instructions, and suggested output formats, such as: "Bullet screen effect material set: Bullet screen effect material 1, Bullet screen effect material 2, ...; Bullet screen analysis instructions: Find the bullet screen effect material that matches the video script in the bullet screen effect material set; Suggested output format: Is it recommended to configure bullet screen effect material X; Special effects material attribute information of bullet screen effect material X (such as bullet screen effect name, effect display duration, and other bullet screen effect rules); Suggestion reason information for bullet screen effect material X; Other suggestion information (such as if there is currently no satisfied effect, suggesting the introduction of new bullet screen effect materials) etc." Business server 100 can call the large language model through the special effects calculation service and input the theme information, prompts, and video script as input data into the large language model.

[0132] In the large language model, based on the bullet screen analysis instructions in the prompt words, the theme information, the script of each video segment, and each bullet screen effect material in the bullet screen effect material set can be analyzed and matched separately. For example, multiple bullet screen effect materials related to the theme information can be filtered from the bullet screen effect material set. For example, the theme information may include "Theme name: Battle between character A and character B; Theme type: Action", then the business server 100 can filter out flame effect materials, lightning effect materials, etc., related to the action theme from the bullet screen effect material set.

[0133] By analyzing and matching each video segment script in the video script with each bullet screen effect material from a selection of multiple bullet screen effect materials, bullet screen configuration suggestion information with suggested output format indicated by prompt words can be generated. This bullet screen configuration suggestion information includes suggested bullet screen effect materials (bullet screen effect name, bullet screen effect rules) for the target video segment among multiple video segments, as well as the rationale for the suggestion. For example, the suggested output format for the target video segment in the bullet screen configuration suggestion information could be: "Suggested configuration of bullet screen effect material 1; attribute information of bullet screen effect material 1 (such as bullet screen effect name or bullet screen effect ID (Identification), effect display duration, and other bullet screen effect rules); rationale for suggesting bullet screen effect material 1; other suggestion information (such as suggesting the introduction of new bullet screen effect materials if no effect is currently satisfied)." The business server 100 can send multiple segment suggestion information (i.e., suggestion information given for multiple video segment scripts, such as segment suggestion information 1, segment suggestion information 2, ...) under the target video single episode identification information (used to uniquely identify the aforementioned video data) to the terminal device 200. The operating object can view multiple segment suggestion information through the terminal device 200. Among them, the bullet screen configuration suggestion information includes multiple video segment scripts that can be configured with bullet screen special effects materials, and any video segment script included in the bullet screen configuration suggestion information can be associated with a matching bullet screen special effects material.

[0134] This application embodiment obtains a video script containing multiple video segment scripts by acquiring video data and performing script parsing on the video data according to a multimodal model. After generating prompts containing a set of bullet screen effect materials, the prompts and video scripts can be input into a large language model. The large language model then analyzes and matches each video segment script with each bullet screen effect material in the bullet screen effect material set according to the bullet screen analysis instructions in the prompts, obtaining bullet screen configuration suggestion information with the suggested output format indicated by the prompts. The bullet screen configuration suggestion information includes the suggested bullet screen effect materials for the target video segment among the multiple video segments, along with the rationale for the suggestion. Parsing the video script using a multimodal model and then analyzing and matching the video script and bullet screen effect materials according to a large language model simplifies the entire configuration process and improves the efficiency of bullet screen effect configuration. Furthermore, by analyzing and matching each video segment script with each bullet screen effect material in the bullet screen effect material set through a large language model, the possibility of low matching degree between the configured bullet screen effects and the plot can be effectively reduced, making the configured bullet screen effect materials and video scripts more compatible. Therefore, configuring bullet screen effect materials based on bullet screen configuration suggestion information will have a better configuration effect.

[0135] Further, please see Figure 3, Figure 3 This is a flowchart illustrating a data processing method provided in an embodiment of this application. Figure 1 This data processing method can be executed by a computer device, which can be, for example, Figure 1 The business server 100 or terminal device 200 shown below will be described using the example of this data processing method being executed by a computer device. This data processing method will at least include the following steps S101-S104:

[0136] Step S101: Acquire video data;

[0137] In this embodiment, the computer device can obtain video data and the theme information of the video set to which the video data belongs from a video library. Each video set can correspond to a unique content identifier (CID), and the theme information can be a brief description of the video set identifier, such as the video set's theme name, theme type (science fiction, suspense, etc.), and author information. Each video data within each video set can correspond to a unique video episode identifier (VID). The computer device can obtain video data using the specified VID information, then obtain the CID information of the corresponding video set, and finally obtain the theme information of the video set to which the video data belongs. Optionally, the computer device can also first obtain CID information containing multiple VIDs, and then determine the video episode corresponding to the specified VID information in the CID information as the video data.

[0138] Step S102: Script parsing is performed on the video data to obtain the video script; the video script contains video segment scripts corresponding to multiple video segments.

[0139] In this embodiment, a computer device can use a multimodal model to parse the video data corresponding to specified VID information to obtain a video script. For example, video data can be retrieved based on specified VID information using RAG (Retrieval-Augmented Generation) technology, and then the video script corresponding to the video data can be obtained. Optionally, the computer device can also use a multimodal model to parse the video scripts corresponding to multiple single-episode videos, and then obtain the video script corresponding to a specified single-episode video based on the specified VID information. This embodiment does not impose any limitations on this.

[0140] Specifically, computer equipment can extract video frame feature vectors, text feature vectors, and audio feature vectors associated with each video frame in video data through a multimodal model. By fusing the text feature vectors, video frame feature vectors, and audio feature vectors associated with each video frame, a fused vector corresponding to each video frame can be obtained. Then, the fused vectors corresponding to each video frame are concatenated to obtain a multimodal feature vector sequence. The multimodal model can also identify scene change frames in video data (e.g., if video frame F in the video data is an indoor scene, and the previous frame of video frame F is an outdoor scene, then video frame F can be identified as a scene change frame). Based on the scene change frames, the multimodal feature vector sequence can be divided to keep the plot of each video segment coherent and clear, and to obtain the multimodal feature vectors corresponding to multiple video segments. Decoding the multimodal feature vectors corresponding to multiple video segments yields the video segment scripts corresponding to multiple video segments. Finally, the video segment scripts corresponding to multiple video segments can be combined into a video script for the video data.

[0141] Step S103: Obtain the set of bullet screen effect materials and generate prompts containing the set of bullet screen effect materials;

[0142] In this embodiment, the computer device can obtain a configured set of bullet screen effect materials from the bullet screen management platform, and then generate prompt words containing the set of bullet screen effect materials, thereby inputting the theme information, prompt words, and video script into the large language model. For example, it can obtain the effect material attribute information (such as bullet screen effect name, effect display duration, and other bullet screen effect rules), bullet screen analysis instructions (such as "find the bullet screen effect material that matches the video script in the bullet screen effect material set") and the suggested output format of the large language model (whether it is recommended to configure bullet screen effect material X; the effect material attribute information of bullet screen effect material X (such as bullet screen effect name, effect display duration, and other bullet screen effect rules); the reason for recommending bullet screen effect material X; and other suggested information (such as if there is no effect that is currently satisfied, suggesting that a new bullet screen effect material can be introduced). By obtaining the prompt word template and filling the prompt word template with the effect material attribute information, bullet screen analysis instructions, and suggested output format of each bullet screen effect material, the prompt words can be obtained. Computer devices can input the aforementioned topic information, prompts, and video scripts as input data into a large language model.

[0143] Step S104: Based on the bullet screen analysis instructions in the prompt, analyze and match each video segment script with each bullet screen effect material in the bullet screen effect material set to obtain bullet screen configuration suggestion information with the suggested output format indicated by the prompt; the bullet screen configuration suggestion information includes the bullet screen effect materials suggested for the target video segment in multiple video segments and the reason for the suggestion.

[0144] In this embodiment of the application, the computer device can analyze and match each video segment script with each bullet screen effect material in the bullet screen effect material set according to the bullet screen analysis instructions in the prompt words used to match the video segment script and bullet screen effect material in the large language model.

[0145] Specifically, the computer device can perform feature extraction on each video segment script according to the bullet screen analysis instructions in the prompt words, obtaining the script feature vector corresponding to each video segment script. It can also perform feature extraction on each bullet screen effect material in the bullet screen effect material set, obtaining the effect feature vector corresponding to each bullet screen effect material. In the large language model, a script attention vector sequence of script feature vectors can be constructed based on the self-attention mechanism. By performing similarity matching between the script attention vector sequence and the effect feature vectors, the matching result can be obtained. Each script attention vector in the script attention vector sequence can be used to represent the script features of the video segment script with associated context script information.

[0146] For example, by calculating the similarity (e.g., cosine similarity) between a script attention vector and the effect feature vector of each bullet screen effect material, multiple similarity calculation results can be obtained. If the largest similarity calculation result is greater than the similarity threshold, the bullet screen effect material associated with the effect feature vector corresponding to the largest similarity calculation result is associated with the video segment corresponding to the script attention vector. Based on this association, segment suggestion information for the script of that video segment can be generated. The bullet screen configuration suggestion information can consist of segment suggestion information corresponding to multiple video segment scripts. One segment suggestion information can be: "Suggest configuring bullet screen effect material 1; effect material attribute information of bullet screen effect material 1 (such as bullet screen effect name or bullet screen effect ID (Identification), effect display duration, and other bullet screen effect rules); the reason for suggesting bullet screen effect material 1; other suggestion information (such as suggesting the introduction of new bullet screen effect materials if no effect is currently satisfied)." Computer devices can submit bullet screen configuration suggestions to the operation entity, enabling the operation entity to configure bullet screen special effects materials based on the bullet screen configuration suggestions.

[0147] This application embodiment obtains a video script containing multiple video segment scripts by acquiring video data and performing script parsing on the video data according to a multimodal model. After generating prompts containing a set of bullet screen effect materials, the prompts and video scripts can be input into a large language model. The large language model then analyzes and matches each video segment script with each bullet screen effect material in the bullet screen effect material set according to the bullet screen analysis instructions in the prompts, obtaining bullet screen configuration suggestion information with the suggested output format indicated by the prompts. The bullet screen configuration suggestion information includes the suggested bullet screen effect materials for the target video segment among the multiple video segments, along with the rationale for the suggestion. Parsing the video script using a multimodal model and then analyzing and matching the video script and bullet screen effect materials according to a large language model simplifies the entire configuration process and improves the efficiency of bullet screen effect configuration. Furthermore, by analyzing and matching each video segment script with each bullet screen effect material in the bullet screen effect material set through a large language model, the possibility of low matching degree between the configured bullet screen effects and the plot can be effectively reduced, making the configured bullet screen effect materials and video scripts more compatible. Therefore, configuring bullet screen effect materials based on bullet screen configuration suggestion information will have a better configuration effect.

[0148] Further, please see Figure 4 , Figure 4 This is a flowchart illustrating a data processing method provided in an embodiment of this application. Figure 2 This data processing method can be executed by a computer device, which can be, for example, Figure 1 The business server 100 or terminal device 200 shown. The following description will use the example of this data processing method being executed by a computer device. This data processing method may include at least the following steps S201-S207:

[0149] Step S201: Obtain video bullet screen parsing request; obtain video set identifier information from video set identifier information according to video bullet screen parsing request, and obtain video library; query the target video set corresponding to the video set identifier information in the video library, and determine the single set video corresponding to the video set identifier information among the multiple single set videos associated with the target video set as video data;

[0150] In the embodiments of this application, please refer to the following: Figure 5 , Figure 5 This is a system architecture diagram for data processing provided in an embodiment of this application, such as... Figure 5As shown, the computer device can obtain a video bullet comment parsing request containing video episode identifier information, and then obtain the video set identifier information from the video episode identifier information based on the video bullet comment parsing request. For example, the video episode identifier information could be 10_01, then based on the video episode identifier information 10_01, the video set identifier information 10 can be obtained. Here, field 10 is used to identify a single video set 10, and field 01 is used to identify a single episode video 01 within that video set.

[0151] Computer equipment can obtain a video library containing multiple video sets from a video media platform, such as video set 30, video set 40, etc. Each video set can contain multiple individual videos (e.g., video set 30 can contain individual videos 30_01, 30_02, etc.). The corresponding target video set can be queried in the video library through the video set identification information. For example, the target video set 10 can be queried based on the video set identification information 10. In this way, the individual video 10_01 corresponding to the video individual set identification information 10_01 among the multiple individual videos (individual video 10_01, individual video 10_02, etc.) associated with the target video set 10 can be identified as video data.

[0152] Step S202: Extract features from video frames in video data using the video frame feature encoding layer of the multimodal model to obtain video frame feature vectors; perform text recognition on video frames to obtain text information in video data, extract features from the text information using the text feature encoding layer of the multimodal model to obtain text feature vectors; extract features from audio information in video data using the audio feature encoding layer of the multimodal model to obtain audio feature vectors.

[0153] In this embodiment, the multimodal model may include a video frame feature coding layer, a text feature coding layer, and an audio feature coding layer. The specific process by which the computer device parses the video data using the multimodal model to obtain the video script may be as follows: Features are extracted from the video frames in the video data through the video frame feature coding layer to obtain video frame feature vectors; text recognition is performed on the video frames to obtain text information in the video data; features are extracted from the text information through the text feature coding layer to obtain text feature vectors; features are extracted from the audio information in the video data through the audio feature coding layer to obtain audio feature vectors; the text feature vectors, video frame feature vectors, and audio feature vectors associated with the same video frames in the video data are fused to obtain a fused vector corresponding to each video frame in the video data; the fused vectors corresponding to each video frame are concatenated to obtain a multimodal feature vector sequence; and a video script for the video data is generated based on the multimodal feature vector sequence. Among them, multimodal models are artificial intelligence systems that can simultaneously process and understand multiple data modalities (such as text, images, audio, video, etc.). Through cross-modal alignment and fusion technologies (such as CLIP (Contrastive Language-Image Pre-Training) architecture), information from different modalities is mapped to a unified semantic space to achieve complex tasks such as image-text mutual generation, video semantic analysis, and multimodal retrieval.

[0154] Specifically, the video frame feature encoding layer can be a CNN (Convolutional Neural Network) model, a ViT (Vision Transformer) model, or other network models. Computer devices can preprocess the video frames in the video data (normalization, data augmentation, etc.) through the video frame feature encoding layer to obtain a standard image corresponding to each video frame. Then, by performing image recognition on the standard image through the network model (the process is forward propagation and feature transformation), a multi-dimensional feature vector can be obtained. Further feature vector pooling processing of the multi-dimensional feature vector can yield the video frame feature vector.

[0155] Computer devices can use deep learning methods (such as CTPN (Connectionist Text Proposal Network)) to perform text recognition on dialogue and subtitles in video frames, obtaining text information from the video data. Then, by extracting features from the text information through a text feature encoding layer, a text feature vector can be obtained. This text feature encoding layer can be a BERT model (Bidirectional Encoder Representations from Transformers). In this layer, the text information can be a sequence of tokens obtained by splitting and encoding the text information using a tokenizer. The text splitting can be based on words, subwords, or characters; there are no restrictions on this.

[0156] A token in a token sequence refers to the smallest basic unit obtained after splitting text information. Its structure can include [CLS], token 1, token 2, and [SEP]. [CLS] can be added to the beginning of the tokenizer segmentation result for classification tasks, and [SEP] can be added to the end of the tokenizer segmentation result for sentence segmentation. All tokens in the text information are embedded into vectors of the same size in each encoding layer of the text feature encoding layer. In the BERT model, this can include token embedding, paragraph embedding, and position embedding.

[0157] Among them, lexical embedding can embed text information into E [CLS] E 词元1 E 词元2 and E [SEP] This maps textual information to a high-dimensional space, capturing the context and semantic information of the text, where E [CLS] It can be used to represent the global semantics of the entire text information. Paragraph embedding can be used to distinguish different paragraphs and sentences in text information. If the embedding of each word is consecutively the same, it is the same sentence. For example, if the paragraph embedding of the text information is E C E C E D and E D Then E C and E C Partially representing a sentence, E D and E DPartially representing another sentence. Positional embedding can embed text information into E1, E2, E3, and E4 to capture the positional order of words in the text and understand semantic structure. By concatenating the results of word embedding, paragraph embedding, and positional embedding, a text feature vector can be obtained.

[0158] Computer devices can preprocess audio information in video data using audio feature coding layers (such as the VGGish model, which is similar to VGG's audio feature extractor) (e.g., format unification, frame segmentation, and normalization). Then, based on methods like Short-Time Fourier Transform (STFT), each frame of audio undergoes time-frequency conversion to obtain its spectrum. The spectrum is then mapped to a spectrogram using filters. In the VGGish model, the spectrogram undergoes forward propagation and feature extraction to obtain the audio features corresponding to each frame. Finally, the audio features corresponding to each frame are concatenated at the frame level and global pooled to obtain the audio feature vector. The Short-Time Fourier Transform is a tool used to analyze the frequency components of a signal at different time segments. It divides the signal into multiple short time windows and performs a Fourier transform on the signal within each window, simultaneously obtaining both time and frequency information. This method is suitable for non-stationary signals (i.e., signals whose frequency components change over time), such as audio signals.

[0159] Step S203: The text feature vector, video frame feature vector and audio feature vector associated with the same video frame in the video data are fused to obtain the fused vector corresponding to each video frame in the video data; the fused vectors corresponding to each video frame are concatenated to obtain a multimodal feature vector sequence; and a video script for the video data is generated based on the multimodal feature vector sequence.

[0160] In this embodiment, the computer device can further align and fuse the text feature vectors, video frame feature vectors, and audio feature vectors associated with the same video frame in the video data. For example, the text feature vectors, video frame feature vectors, and audio feature vectors associated with the same video frame can be input into a transformer model to learn the fused features, thereby obtaining the fused vector corresponding to each video frame. For example, the sequence embedding output by the encoding layer of the transformer model can also include the embeddings corresponding to the text feature vector, video frame feature vector, and audio feature vector respectively, so as to characterize the global semantics of the text feature vector, video frame feature vector, and audio feature vector, and retain the more original global and local features, which are suitable for subsequent feature interaction learning.

[0161] Computer equipment can concatenate the fusion vectors corresponding to each video frame to obtain a multimodal feature vector sequence. Based on the multimodal feature vector sequence, a video script for the video data can be generated. The specific process can be as follows: identify scene change frames in the video data through a multimodal model; divide the multimodal feature vector sequence based on the scene change frames to obtain multimodal feature vectors corresponding to multiple video segments; decode the multimodal feature vectors corresponding to multiple video segments to obtain video segment scripts corresponding to multiple video segments; and combine the video segment scripts corresponding to multiple video segments into a video script for the video data.

[0162] For example, video frames F and G are two consecutive video frames. Computer devices can use multimodal models to identify scene transitions (such as changes in image color or motion), audio abrupt changes (such as a shift in the frequency spectrum from low to high frequencies), and textual information changes (such as changes in topic or scene descriptors) between video frames F and G. If video frame F in the video data represents an indoor scene, while video frame G represents an outdoor scene, then it can be determined that a scene change has occurred between video frames F and G. Therefore, video frame G can be identified as the scene change frame in the video data. Using scene change frames, the multimodal feature vector sequence can be divided into multimodal feature vectors corresponding to multiple video segments (i.e., multimodal feature vectors corresponding to multiple identical video frames and multimodal feature vectors corresponding to multiple scene change frames), thus maintaining the coherence and clarity of the plot in each video segment.

[0163] By decoding the multimodal feature vectors corresponding to multiple video segments based on factors such as time period, plot, and characters, a video segment script corresponding to each video segment can be obtained. For example, "Segment location: 2 minutes 45 to 3 minutes 15; In the room, character A picks up a small object and interacts with character B; Character A: XXX; Character B: XXX; ...", or "Segment location: 3 minutes 15 to 3 minutes 45; In the room, character B stands and holds onto a handrail, interacting with character A; Character B: XXX; Character A: XXX; ...". Computer equipment can combine the video segment scripts corresponding to multiple video segments into a video script corresponding to the video data.

[0164] Step S204: Obtain the set of bullet screen effect materials and generate prompts containing the set of bullet screen effect materials;

[0165] In this embodiment, a computer device can obtain a set of bullet screen effect materials containing multiple bullet screen effect materials from a bullet screen management platform. For example, it can first generate a material acquisition request (such as an HTTP (HyperText Transfer Protocol) request), then obtain the set of bullet screen effect materials containing multiple bullet screen effect materials from the bullet screen management platform based on the material acquisition request, and then generate prompt words based on the set of bullet screen effect materials, bullet screen analysis instructions, and the suggested output format of the large language model.

[0166] For example, you can first obtain the prompt word template containing the areas to be filled (the first area to be filled, the second area to be filled, and the third area to be filled), then parse out the effect material attribute information of each bullet effect material in the bullet effect material set (such as bullet effect name or bullet effect ID, effect display duration, and other bullet effect rules), and obtain the suggested output format of the large language model (such as whether it is recommended to configure bullet effect material X; the effect material attribute information of bullet effect material X; the reason for the suggestion of bullet effect material X; other suggested information (such as if there is no bullet effect material that meets the requirements, and suggest that new bullet effect materials can be introduced) etc.). The computer device can fill the first area to be filled in the prompt word template with the effect material attribute information of each bullet screen effect material, fill the second area to be filled in the bullet screen analysis command, and fill the third area to be filled in the suggested output format, to obtain the prompt word, such as "Bullet screen effect material set: bullet screen effect material 1 (including effect material attribute information), bullet screen effect material 2, ...; Bullet screen analysis command: find the bullet screen effect material that matches the video script in the bullet screen effect material set; Suggested output format: whether it is recommended to configure bullet screen effect material X; effect material attribute information of bullet screen effect material X; reason information for recommending bullet screen effect material X; other suggested information".

[0167] like Figure 5 As shown, computer devices can invoke large language models through special effects computing services, inputting thematic information, cue words, and video scripts of the video set to which the video data belongs as input data. A large language model (LLM) is a computer model capable of processing and generating natural language; it represents a significant advancement in the field of artificial intelligence and holds the promise of transforming the field through acquired knowledge. LLMs can predict the next word or sentence by learning the statistical patterns and semantic information of language data. As the input dataset and parameter space expand, the capabilities of LLMs also increase. It is used in various application areas, such as machine learning and image processing, and is therefore also known as a multimodal large language model.

[0168] In this embodiment, the process of training a large language model for generating bullet screen configuration suggestion information may be as follows: acquiring historical video data, historical bullet screen effect materials, and historical bullet screen configuration suggestion information; the historical bullet screen configuration suggestion information is generated based on the usage records of historical bullet screen effect materials in the historical video data; inputting the historical video data and historical bullet screen effect materials into an initial large language model, and outputting predicted bullet screen configuration suggestion information through the initial large language model; generating a training loss value based on the historical bullet screen configuration suggestion information and the predicted bullet screen configuration suggestion information, and adjusting the model parameters of the initial large language model based on the training loss value to obtain a large language model for generating bullet screen configuration suggestion information.

[0169] Specifically, computer devices can obtain historical video data from video libraries and historical bullet comment effect materials and configuration suggestions from the bullet comment management platform. The historical bullet comment configuration suggestions are generated based on the usage records of historical bullet comment effect materials from historical video data; that is, they are generated by integrating bullet comment effect cases with good feedback. Historical bullet comment effect materials include the material name, ID, effect description (with sufficient textual annotations to describe the effect so that the large language model can better understand the usage scenario), and the bullet comment effect rules, among other attribute information. The historical bullet comment configuration suggestions include the operational strategy for the bullet comment effect materials: the operational entity provides supplementary explanations and suggestions for using the bullet comment effect materials, as well as some related precautions, such as the maximum display duration of the bullet comment effect materials and the maximum number of bullet comment effect rules that should not be used in a single video episode.

[0170] Please see also Figure 6 , Figure 6 This application provides a schematic diagram of the model structure of a data processing method, as shown in the embodiments. Figure 6 As shown, the computer device can calculate the model loss based on historical bullet screen configuration suggestion information and the predicted bullet screen configuration suggestion information output by the initial large language model, obtaining the training loss value. Based on this training loss value, the model parameters of the initial large language model are fine-tuned to obtain the large language model used to generate bullet screen configuration suggestion information. For example, taking the stochastic gradient descent method to adjust the model parameters of the initial large language model as an example, the basic idea is to update the model parameters along the negative gradient direction of the loss function, so that the training loss value gradually decreases. For the model parameter θ, the method of updating the model parameters can be defined as follows: Where α is the learning rate. It is the gradient of the training loss value with respect to the model parameters. By iteratively updating the model parameters θ along the opposite direction of the gradient, the training loss value is continuously reduced. When the training loss value is lower than a preset training loss threshold, the computer can determine that the model parameters θ have converged. Thus, a large language model for generating bullet screen configuration suggestions can be obtained.

[0171] Step S205: In the large language model, according to the bullet screen analysis instructions in the prompt words, select the first set of bullet screen effect materials related to the theme information from the set of bullet screen effect materials; extract features from the script of each video segment according to the bullet screen analysis instructions to obtain the script feature vector corresponding to each video segment script; extract features from each bullet screen effect material in the first set of bullet screen effect materials to obtain the effect feature vector corresponding to each bullet screen effect material.

[0172] In this embodiment of the application, the computer device can select the first set of bullet screen effect materials associated with the theme information from the bullet screen effect material set according to the bullet screen analysis instructions in the prompt words (such as selecting bullet screen effect materials associated with the theme information). For example, if the theme information is "This is a comedy story about character A and character B", then bullet screen effect materials such as fireworks effect materials and funny expression effect materials associated with comedy can be determined as the first set of bullet screen effect materials.

[0173] Furthermore, the computer equipment can extract features from the video clip script and bullet screen effect materials in a large language model based on bullet screen analysis instructions (such as extracting features from the video clip script and bullet screen effect materials separately), obtaining script feature vectors corresponding to each video clip script. For example, it can extract text features from the character description text of the video clip script to obtain character feature vectors, and extract text features from the dialogue description text of the video clip script to obtain dialogue feature vectors, etc. By extracting features from each bullet screen effect material in the first bullet screen effect material set using a large language model, such as extracting features based on the keywords or material style (such as comedy style, fighting style, etc.) of the bullet screen effect material, it can obtain effect feature vectors corresponding to each bullet screen effect material.

[0174] Step S206: Construct a script attention vector sequence of script feature vectors through a large language model; perform similarity matching between the script attention vector sequence and the special effects feature vectors to obtain the matching result; output bullet screen configuration suggestion information with the suggested output format indicated by the prompt words based on the matching result.

[0175] In this embodiment, the large language model may include an attention processing layer. The specific process by which the computer device constructs a script attention vector sequence for the script feature vector through the large language model may be as follows: performing a dot product operation on the query parameter matrix and the script feature vector in the attention processing layer to obtain a query vector sequence for the self-attention function; performing a dot product operation on the key parameter matrix and the script feature vector in the attention processing layer to obtain a key vector sequence for the self-attention function; performing a dot product operation on the value parameter matrix and the script feature vector in the attention processing layer to obtain a value vector sequence for the self-attention function; the query parameter matrix, key parameter matrix, and value parameter matrix are all matrices composed of learnable parameters in the attention processing layer; and generating a script attention vector sequence for the script feature vector based on the query vector sequence, key vector sequence, and value vector sequence.

[0176] Specifically, the computer device can obtain the query parameter matrix W in the attention processing layer. Q Key parameter matrix W K Sum parameter matrix W V The script feature vector L and the query parameter matrix W are combined. Q Performing a dot product operation yields the query vector sequence Q of the self-attention function, defined as Q = W. Q •L; Combine the script feature vector L with the key parameter matrix W K Performing a dot product operation yields the query vector sequence K of the self-attention function, defined as K = W. K •L; Combine the script feature vector L with the key parameter matrix W V Performing a dot product operation yields the query vector sequence V of the self-attention function, defined as V = W. V ·L. Wherein, the query parameter matrix W Q Key parameter matrix W K Sum parameter matrix W V These are matrices composed of learnable parameters from the attention processing layer, used to map script feature vectors to the representation space through linear transformation.

[0177] Computer equipment can generate a script attention vector sequence based on script feature vectors by querying the vector sequence Q, the key vector sequence K, and the value vector sequence V. For example, the key vector sequence K can be transposed to obtain the first fusion parameter K. T The query vector sequence Q is combined with the first fusion parameter K. T The result of the dot product operation is determined as the second fusion parameter QK. T Based on the second fusion parameter QK T The number of dimensions d corresponding to the script feature vector k Generate third fusion parameters The third fusion parameter is normalized to obtain the normalized parameter. normalized parameters The result of the dot product operation with the value vector V is determined as the script attention vector sequence Attention(Q,K,V), and the process can be shown in the following formula (1):

[0178]

[0179] The computer device can perform similarity matching between the script attention vector in the above script attention vector sequence and the effect feature vector of each bullet screen effect material to obtain the matching result. For example, the dot product of the script attention vector and the effect feature vector of each bullet screen effect material can be calculated using cosine similarity. After normalizing the magnitude of the script attention vector and the magnitude of the effect feature vector of each bullet screen effect material, the product of the magnitude of the script attention vector and the magnitude of the effect feature vector of each bullet screen effect material is calculated. The ratio of the dot product to the product of the magnitudes is then calculated, and this ratio is used as the cosine similarity between the script attention vector and the effect feature vector of each bullet screen effect material, i.e., the matching result. If the maximum cosine similarity is greater than the similarity threshold, then the bullet screen effect material corresponding to the maximum cosine similarity is determined as the bullet screen effect material recommended for the target video segment in the bullet screen configuration suggestion information.

[0180] Computer devices can output suggested bullet screen configuration information in the suggested output format indicated by prompt words based on the matching results. The specific process can be as follows: obtain target bullet screen effect materials that match the target video segment script in the video script based on the matching results; the bullet screen effect material set includes the target bullet screen effect materials; obtain the target effect material attribute information and target suggestion reason information of the target bullet screen effect materials according to the suggested output format; generate suggested bullet screen configuration information in the suggested output format indicated by prompt words based on the target effect material attribute information and target suggestion reason information.

[0181] Specifically, the computer device can identify the bullet screen effect material corresponding to the effect feature vector with a cosine similarity greater than the similarity threshold as the target bullet screen effect material in the bullet screen effect material set that matches the target video segment script in the video script. In accordance with the above-mentioned suggested output format, the target bullet screen effect material attribute information (bullet screen effect name, display duration, and other bullet screen effect rules) and target suggestion reason information (suggested reasons for using the target bullet screen effect material, etc.) can be obtained. Based on the target effect material attribute information, target suggestion reason information, and bullet screen analysis instructions, bullet screen configuration suggestion information with the suggested output format indicated by the prompt words can be generated.

[0182] The bullet screen configuration suggestion information includes multiple video clip scripts that can be configured with bullet screen effect materials, and any video clip script included in the bullet screen configuration suggestion information can be associated with a matching bullet screen effect material. For example, the bullet screen configuration suggestion information can be composed of segment suggestion information corresponding to multiple video clip scripts. One segment suggestion information can be "suggest configuring bullet screen effect material 1; the effect material attribute information of bullet screen effect material 1 (such as bullet screen effect name or bullet screen effect ID (Identification), effect display duration, and other bullet screen effect rules); the reason for suggesting bullet screen effect material 1; other suggestion information (such as if there is no effect that is currently satisfied, suggesting the introduction of new bullet screen effect materials), etc."

[0183] Step S207: Obtain a set of suggestion information; the set of suggestion information contains multiple suggestions, including suggestions for bullet screen configuration; score the set of suggestion information to obtain a score set; the score set contains the score result corresponding to each suggestion, and the score result is either a positive score result or a negative score result.

[0184] In the embodiments of this application, such as Figure 6 As shown, the computer device can obtain the suggestion information set output by the large language model for multiple video episodes corresponding to the identification information of multiple video episodes, and send the suggestion information set containing the bullet screen configuration suggestion information to the bullet screen management platform through the special effects calculation service, and call the write interface of the bullet screen management platform to record the suggestion information set. In the bullet screen management platform, by performing quality scoring on the suggestion information set (such as quality scoring by operation object), a score set containing the score result corresponding to each suggestion information can be obtained, where the score result can be divided into positive score result (such as excellent, good) or negative score result (such as poor).

[0185] Computer devices can statistically analyze the proportion of positive and negative ratings in a rating set. If the proportion of positive ratings exceeds a threshold (e.g., 0.7), it can be determined that the large language model has completed its iteration and can provide accurate suggestions. This allows for the setup of an automated configuration service in the bullet comment management platform. The automated configuration service instructs the platform to apply the suggested bullet comment effects to a new video clip when the large language model outputs new bullet comment configuration suggestions. For example, the large language model's suggested bullet comment configuration for a new video clip might be: "Suggested configuration of bullet comment effects; Suggested bullet comment effect configuration: Bullet comment effect material 3, effect display duration 1 minute, triggered by keyword; Reason for suggestion: This bullet comment effect material type matches the current clip style." Based on this, the computer device can then configure the suggested bullet comment effect material 3 on the bullet comment management platform using the automated configuration service, and display the configured bullet comment effect material 3 on the new video clip.

[0186] If the proportion of negative ratings in the rating set is greater than a threshold, it can be determined that the large language model has not completed its iteration, meaning it cannot provide accurate suggestions. In this case, the computer device can use the RLHF (Reinforcement Learning from Human Feedback) method to identify positive ratings in the rating set as positive sample data and negative ratings as negative sample data, and then perform reinforcement learning on the large language model based on the positive and negative sample data.

[0187] For example, a large language model can generate the predicted probability that the rating result of a new video data is a positive rating result, as well as the true probability that the rating result is a positive rating result. Based on the predicted probability and the true probability, a fine-tuning training loss value for the large language model can be generated. Taking the cross-entropy loss function as the loss function for generating the fine-tuning training loss value as an example, the process of calculating the model loss value R can be shown in formula (2):

[0188]

[0189] in, The loss value is the predicted probability generated by the large language model, where y represents the true probability generated by the large language model. In binary classification, y can correspond to 0 or 1. When the rating result is a positive rating result, y = 1, and the loss value is... This means when predicting probability The closer the value is to 1, the closer the model loss value R is to 0, and the more accurate the prediction of the large language model; when the rating result is negative, y = 0, and the loss value is... Then when the predicted probability The closer the value of the model loss R is to 0, the closer it is to 0, and the more accurate the prediction of the large language model. Computer equipment can calculate the average of the model loss R corresponding to all rating results to obtain the fine-tuned training loss value, thus reflecting the accuracy of the large language model's prediction of probabilities under the current parameters. Furthermore, under the stochastic gradient descent method, reinforcement learning can be performed on the large language model based on the fine-tuned training loss value to obtain a reinforcement learning model for the bullet screen domain. This model aligns with the configuration preferences of the target audience in the bullet screen domain. The process is similar to the process of adjusting the model parameters of the initial large language model using stochastic gradient descent, and will not be repeated here.

[0190] Furthermore, please also refer to Figure 7 , Figure 7 This is a scenario illustration of a data processing method provided in an embodiment of this application. Figure 2 ,like Figure 7 As shown, computer devices can create configuration suggestion controls on the bullet comment management page of the bullet comment management platform, and can respond to trigger operations on the configuration suggestion controls on the bullet comment management page to perform CID aggregation display, that is, display multiple video set identifier information, such as video set identifier information 1, video set identifier information 2, ..., and the multiple video set identifier information includes the target video set identifier information corresponding to the video set to which the video data belongs. It can be understood that multiple video set identifier information can also be displayed in the form of controls on the bullet comment management page.

[0191] The computer device can respond to the trigger operation for the target video set identification information, filter the vid list on the bullet screen management page, and display multiple vid information under one cid information. That is, it can display multiple video set identification information associated with the video set to which the video data belongs, such as video set identification information 1, video set identification information 2, ..., and the multiple video set identification information includes the target video set identification information corresponding to the video data. It can be understood that the multiple video set identification information can also be displayed in the form of controls on the bullet screen management page.

[0192] The computer device can respond to trigger operations targeting the identifier information of a single episode of a target video, displaying multiple segment suggestion information corresponding to the video data on the bullet comment management page, such as segment suggestion information 1, segment suggestion information 2, ... These multiple segment suggestion information can each correspond to multiple video segments in the video data. Each segment suggestion information includes the bullet comment effect name, bullet comment effect rules (effect display duration, effect display time period, etc.), and the reason for the suggestion. It can be understood that multiple segment suggestion information can also be displayed on the bullet comment management page in the form of controls.

[0193] The computer device can respond to the quality scoring operation of the segment suggestion information (including scoring options such as excellent, good, and poor), and display the segment suggestion information and the input scoring results in association. The scoring results are used to indicate whether to configure the bullet screen effect material indicated by the segment suggestion information for the video segment. That is, before performing reinforcement learning on the large language model, the scoring results corresponding to multiple segment suggestion information can guide the operation object to configure the bullet screen effect material. When the number of segment suggestion information is greater than the number threshold, the computer device can select the above positive and negative sample data based on the scoring results corresponding to multiple segment suggestion information.

[0194] This application embodiment obtains a video script containing multiple video segment scripts by acquiring video data and performing script parsing on the video data according to a multimodal model. After generating prompts containing a set of bullet screen effect materials, the prompts and video scripts can be input into a large language model. The large language model then analyzes and matches each video segment script with each bullet screen effect material in the bullet screen effect material set according to the bullet screen analysis instructions in the prompts, obtaining bullet screen configuration suggestion information with the suggested output format indicated by the prompts. The bullet screen configuration suggestion information includes the suggested bullet screen effect materials for the target video segment among the multiple video segments, along with the rationale for the suggestion. Parsing the video script using a multimodal model and then analyzing and matching the video script and bullet screen effect materials according to a large language model simplifies the entire configuration process and improves the efficiency of bullet screen effect configuration. Furthermore, by analyzing and matching each video segment script with each bullet screen effect material in the bullet screen effect material set through a large language model, the possibility of low matching degree between the configured bullet screen effects and the plot can be effectively reduced, making the configured bullet screen effect materials and video scripts more compatible. Therefore, configuring bullet screen effect materials based on bullet screen configuration suggestion information will have a better configuration effect.

[0195] Further, please see Figure 8 , Figure 8 This is a schematic diagram of a data processing device provided in an embodiment of this application. Figure 8 As shown, the data processing device may include a transceiver module 1100, a video script generation module 1200, a prompt word generation module 1300, and a model processing module 1400.

[0196] The transceiver module 1100 is used to acquire video data;

[0197] The video script generation module 1200 is used to parse video data to obtain a video script; the video script contains multiple video segment scripts corresponding to each video segment.

[0198] The prompt word generation module 1300 is used to obtain a set of bullet screen effect materials and generate prompt words containing the set of bullet screen effect materials;

[0199] The model processing module 1400 is used to analyze and match each video segment script with each bullet screen effect material in the bullet screen effect material set according to the bullet screen analysis instructions in the prompt words, and obtain bullet screen configuration suggestion information with the suggested output format indicated by the prompt words; the bullet screen configuration suggestion information includes the bullet screen effect materials suggested for the target video segment in multiple video segments and the reason for the suggestion.

[0200] In one possible implementation, the data processing device further includes a trigger display module 1500, which is specifically used to perform the following operations:

[0201] In response to the triggered operation of the configuration suggestion control on the bullet screen management page, multiple video set identification information is displayed; the multiple video set identification information includes the target video set identification information corresponding to the video set to which the video data belongs;

[0202] In response to a trigger operation targeting the identification information of the target video set, the bullet screen management page displays the identification information of multiple video sets associated with the video set to which the video data belongs; the identification information of multiple video sets includes the identification information of the target video set corresponding to the video data;

[0203] In response to the triggered operation targeting the identifier information of the target video episode, the bullet screen management page displays the bullet screen configuration suggestion information corresponding to the video data; the bullet screen configuration suggestion information includes the special effect material attribute information and the reason for the suggestion.

[0204] In one possible implementation, the trigger display module 1500 is also specifically used to perform the following operations:

[0205] In response to the quality rating operation of the bullet screen configuration suggestion information, the bullet screen configuration suggestion information and the input rating result are displayed together; the rating result is used to indicate whether the bullet screen effect material indicated by the bullet screen configuration suggestion information is configured in the video data configuration.

[0206] In one possible implementation, the video script generation module 1200 is used to parse the video data into a script. When the video script is obtained, it is specifically used to perform the following operations:

[0207] The video frame feature encoding layer of the multimodal model extracts features from the video frames in the video data to obtain the video frame feature vector;

[0208] Text recognition is performed on video frames to obtain text information in the video data. The text information is then used to extract features from the text feature encoding layer of a multimodal model to obtain text feature vectors.

[0209] Audio feature vectors are obtained by extracting audio information from video data through the audio feature coding layer of a multimodal model.

[0210] The text feature vector, video frame feature vector, and audio feature vector associated with the same video frame in the video data are fused to obtain the fused vector corresponding to each video frame in the video data.

[0211] The fusion vectors corresponding to each video frame are concatenated to obtain a multimodal feature vector sequence, and a video script for the video data is generated based on the multimodal feature vector sequence.

[0212] In one possible implementation, when the video script generation module 1200 generates a video script for video data based on a multimodal feature vector sequence, it specifically performs the following operations:

[0213] Identify scene change frames in video data using a multimodal model;

[0214] The multimodal feature vector sequence is divided based on the scene change frames to obtain multimodal feature vectors corresponding to multiple video segments respectively;

[0215] Decode the multimodal feature vectors corresponding to multiple video segments to obtain the video segment scripts corresponding to each video segment.

[0216] The video scripts corresponding to multiple video segments are combined into a video script of video data.

[0217] In one possible implementation, the video script also includes the theme information of the video set to which the video data belongs; when the model processing module 1400 analyzes and matches each video segment script with each bullet screen effect material in the bullet screen effect material set according to the bullet screen analysis instructions in the prompt words, and obtains bullet screen configuration suggestion information with the suggested output format indicated by the prompt words, it is specifically used to perform the following operations:

[0218] In the large language model, based on the bullet screen analysis instructions in the prompt words, the first set of bullet screen effect materials related to the theme information is selected from the set of bullet screen effect materials.

[0219] Based on the barrage analysis instructions, feature extraction is performed on the script of each video segment to obtain the script feature vector corresponding to each video segment script. Feature extraction is also performed on each barrage effect material in the first barrage effect material set to obtain the effect feature vector corresponding to each barrage effect material.

[0220] The script attention vector sequence of script feature vectors is constructed by using a large language model. The script attention vector sequence and the special effects feature vector are then matched for similarity to obtain the matching results.

[0221] Based on the matching results, output suggested bullet screen configuration information with the suggested output format indicated by the prompt words.

[0222] In one possible implementation, the large language model includes an attention processing layer; the model processing module 1400, when constructing a script attention vector sequence of script feature vectors through the large language model, specifically performs the following operations:

[0223] Perform a dot product operation on the query parameter matrix and script feature vector in the attention processing layer to obtain the query vector sequence of the self-attention function;

[0224] Perform a dot product operation between the key parameter matrix in the attention processing layer and the script feature vector to obtain the key vector sequence of the self-attention function;

[0225] The value parameter matrix and script feature vector in the attention processing layer are multiplied by a dot product to obtain the value vector sequence of the self-attention function; the query parameter matrix, key parameter matrix and value parameter matrix are all matrices composed of learnable parameters in the attention processing layer;

[0226] Generate a script attention vector sequence based on the query vector sequence, key vector sequence, and value vector sequence, targeting the script feature vector.

[0227] In one possible implementation, when the model processing module 1400 outputs bullet screen configuration suggestion information with the suggested output format indicated by the prompt words based on the matching results, it is specifically used to perform the following operations:

[0228] Based on the matching results, target bullet screen effect materials that match the target video segment script in the video script are obtained; the bullet screen effect material set includes the target bullet screen effect materials;

[0229] Obtain the target bullet screen effect material's attribute information and the reasoning behind the target suggestion information according to the suggested output format;

[0230] Based on the target special effects material attribute information and the target suggestion reason information, generate bullet screen configuration suggestion information with the suggested output format indicated by the prompt words.

[0231] In one possible implementation, the model processing module 1400 is further configured to perform the following operations:

[0232] It retrieves historical video data, historical bullet screen effect materials, and historical bullet screen configuration suggestions; the historical bullet screen configuration suggestions are generated based on the usage records of historical bullet screen effect materials from historical video data.

[0233] Historical video data and historical bullet screen effect materials are input into the initial large language model, and the initial large language model outputs prediction bullet screen configuration suggestions;

[0234] Training loss values ​​are generated based on historical and predicted bullet comment configuration suggestions. The model parameters of the initial large language model are then adjusted based on the training loss values ​​to obtain the large language model used to generate bullet comment configuration suggestions.

[0235] In one possible implementation, the model processing module 1400 is further configured to perform the following operations:

[0236] Retrieve a set of suggestion information; the set of suggestion information contains multiple suggestion information, including suggestion information for bullet screen configuration.

[0237] The set of suggested information is scored to obtain a score set; the score set contains the score result corresponding to each suggested information, and the score result is either a positive score result or a negative score result;

[0238] If the proportion of positive rating results in the rating set is greater than the proportion threshold, then an automated configuration service is set in the bullet screen management platform. The automated configuration service is used to instruct the bullet screen management platform to configure the bullet screen effect materials indicated by the new bullet screen configuration suggestion information to the new video clip when the big language model outputs new bullet screen configuration suggestion information for the new video clip.

[0239] In one possible implementation, the model processing module 1400 is further configured to perform the following operations:

[0240] If the proportion of negative ratings in the rating set is greater than the proportion threshold, then the positive ratings in the rating set are determined as positive sample data, and the negative ratings in the rating set are determined as negative sample data.

[0241] The large language model is retrained using both positive and negative sample data.

[0242] In one possible implementation, when the transceiver module 1100 acquires video data, it specifically performs the following operations:

[0243] Get the video bullet comment parsing request; the video bullet comment parsing request includes video episode identifier information;

[0244] Based on the video bullet screen parsing request, obtain the video set identifier information from the video set identifier information, and obtain the video library; the video library includes multiple video sets, each containing multiple individual video sets;

[0245] Query the target video set corresponding to the video set identifier information in the video library, and identify the single video set corresponding to the single video set identifier information among the multiple single video sets associated with the target video set as video data.

[0246] This application embodiment obtains a video script containing multiple video segment scripts by acquiring video data and performing script parsing on the video data according to a multimodal model. After generating prompts containing a set of bullet screen effect materials, the prompts and video scripts can be input into a large language model. The large language model then analyzes and matches each video segment script with each bullet screen effect material in the bullet screen effect material set according to the bullet screen analysis instructions in the prompts, obtaining bullet screen configuration suggestion information with the suggested output format indicated by the prompts. The bullet screen configuration suggestion information includes the suggested bullet screen effect materials for the target video segment among the multiple video segments, along with the rationale for the suggestion. Parsing the video script using a multimodal model and then analyzing and matching the video script and bullet screen effect materials according to a large language model simplifies the entire configuration process and improves the efficiency of bullet screen effect configuration. Furthermore, by analyzing and matching each video segment script with each bullet screen effect material in the bullet screen effect material set through a large language model, the possibility of low matching degree between the configured bullet screen effects and the plot can be effectively reduced, making the configured bullet screen effect materials and video scripts more compatible. Therefore, configuring bullet screen effect materials based on bullet screen configuration suggestion information will have a better configuration effect.

[0247] Further, please see Figure 9 , Figure 9 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Figure 9As shown, the computer device 1000 in this embodiment may include: a processor 1001, a network interface 1004, and a memory 1005. Furthermore, the computer device 1000 may also include: an object interface 1003, and at least one communication bus 1002. The communication bus 1002 is used to implement communication between these components. The object interface 1003 may include a display screen and a keyboard; optionally, the object interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as at least one disk storage device. Optionally, the memory 1005 may also be at least one storage device located remotely from the processor 1001. Figure 9 As shown, the memory 1005, which is a computer-readable storage medium, may include an operating system, a network communication module, an object interface module, and a computer program.

[0248] The network interface 1004 can provide network communication elements; the object interface 1003 is mainly used to provide an input interface for objects; and the processor 1001 can be used to call the computer program stored in the memory 1005 to execute the steps in the various embodiments of this application.

[0249] Furthermore, it should be noted that embodiments of this application also provide a computer-readable storage medium storing a computer program adapted to be loaded and executed by the processor. Figure 3 or Figure 4 For details on the methods provided in each step, please refer to the document. Figure 3 or Figure 4 The implementation methods provided for each step are not repeated here. Furthermore, the beneficial effects of using the same method are also not repeated. For technical details not disclosed in the computer-readable storage medium embodiments involved in this application, please refer to the description of the method embodiments of this application. As an example, a computer program may be deployed to execute on a single computer device, or on multiple computer devices located in one location, or on multiple computer devices distributed across multiple locations and interconnected via a communication network.

[0250] The computer-readable storage medium can be the apparatus provided in any of the foregoing embodiments or the internal storage unit of the computer device, such as the hard disk or memory of the computer device. The computer-readable storage medium can also be an external storage device of the computer device, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., provided on the computer device. Furthermore, the computer-readable storage medium can include both internal storage units and external storage devices of the computer device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0251] This application also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform... Figure 3 or Figure 4 The methods provided are among the various optional methods available in the code, so they will not be elaborated upon here.

[0252] The terms "first," "second," etc., in the specification, claims, and drawings of this application are used to distinguish different objects, not to describe a specific order. Furthermore, the term "comprising," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or device that includes a series of steps or units is not limited to the listed steps or modules, but may optionally include steps or modules not listed, or may optionally include other step units inherent to these processes, methods, apparatuses, products, or devices.

[0253] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0254] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been generally described in terms of functionality. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0255] The methods and related apparatus provided in this application are described with reference to the method flowcharts and / or structural diagrams provided in this application. Specifically, each block of the method flowcharts and / or structural diagrams, as well as combinations of blocks in the flowcharts and / or block diagrams, can be implemented by a computer program. These computer programs can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable device to create a machine, such that the computer program, executed by the processor of the computer or other programmable device, produces a mechanism for implementing the process... Figure 1 A schematic diagram of one or more processes and / or structures. Figure 1 The computer program may be a means for performing the functions specified in one or more boxes. These computer programs may also be stored in a computer-readable storage medium that can direct a computer or other programmable device to function in a particular manner, causing the computer program stored in the computer-readable storage medium to produce an article of manufacture including the program means, or to be transmitted via a computer-readable storage medium. The computer program can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The program means is implemented in the process. Figure 1 A schematic diagram of one or more processes and / or structures. Figure 1 The functions specified in one or more boxes. These computer programs may also be loaded onto a computer or other programmable device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby providing the computer program executing on the computer or other programmable device with the means to implement the process. Figure 1 A process or multiple processes and / or structures illustrate the steps of the functions specified in one or more boxes.

[0256] The steps in the method of this application embodiment can be adjusted, combined, or deleted according to actual needs.

[0257] The modules in the device of this application embodiment can be merged, divided, and deleted according to actual needs.

[0258] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. A data processing method, characterized in that, The method includes: Acquire video data; The video data is parsed to obtain a video script; the video script contains multiple video segment scripts corresponding to each video segment. Obtain a set of bullet screen effect materials and generate prompts containing the set of bullet screen effect materials; Based on the bullet screen analysis instructions in the prompt words, each video segment script is analyzed and matched with each bullet screen effect material in the bullet screen effect material set to obtain bullet screen configuration suggestion information with the suggested output format indicated by the prompt words; the bullet screen configuration suggestion information includes the bullet screen effect material suggested for the target video segment among the multiple video segments and the suggestion reason information; the bullet screen effect material includes the bullet screen effect name and bullet screen effect rules.

2. The method according to claim 1, characterized in that, The method further includes: In response to a trigger operation on the configuration suggestion control in the bullet screen management page, multiple video set identification information is displayed; the multiple video set identification information includes the target video set identification information corresponding to the video set to which the video data belongs; In response to a trigger operation targeting the target video set identifier information, multiple video set identifier information associated with the video set to which the video data belongs is displayed on the bullet screen management page; the multiple video set identifier information includes the target video set identifier information corresponding to the video data; In response to a trigger operation targeting the single-episode identifier information of the target video, the bullet screen configuration suggestion information corresponding to the video data is displayed on the bullet screen management page; the bullet screen configuration suggestion information includes the special effect material attribute information of the bullet screen special effect material and the suggestion reason information.

3. The method according to claim 2, characterized in that, The method further includes: In response to the quality rating operation of the bullet screen configuration suggestion information, the bullet screen configuration suggestion information and the input rating result are displayed together; the rating result is used to indicate whether the bullet screen effect material indicated by the bullet screen configuration suggestion information is configured for the video data.

4. The method according to claim 1, characterized in that, The step of parsing the video data to obtain a video script includes: The video frame feature encoding layer of the multimodal model is used to extract features from the video frames in the video data to obtain video frame feature vectors; The video frame is subjected to text recognition to obtain text information in the video data. The text information is then subjected to feature extraction through the text feature encoding layer of the multimodal model to obtain a text feature vector. The audio feature vector is obtained by extracting audio information from the video data through the audio feature coding layer of the multimodal model. The text feature vector, the video frame feature vector, and the audio feature vector associated with the same video frame in the video data are fused to obtain a fused vector corresponding to each video frame in the video data. The fusion vectors corresponding to each video frame are concatenated to obtain a multimodal feature vector sequence, and a video script for the video data is generated based on the multimodal feature vector sequence.

5. The method according to claim 4, characterized in that, The process of generating a video script for the video data based on the multimodal feature vector sequence includes: The multimodal model is used to identify scene change frames in the video data; The multimodal feature vector sequence is divided based on the scene change frames to obtain multimodal feature vectors corresponding to multiple video segments respectively; Decode the multimodal feature vectors corresponding to the multiple video segments to obtain the video segment scripts corresponding to the multiple video segments; The video segment scripts corresponding to the multiple video segments are combined into a video script for the video data.

6. The method according to claim 1, characterized in that, The video script also includes the theme information of the video set to which the video data belongs; the step of analyzing and matching each video segment script with each bullet screen effect material in the bullet screen effect material set according to the bullet screen analysis instructions in the prompt words to obtain bullet screen configuration suggestion information with the suggested output format indicated by the prompt words, including: In the large language model, based on the bullet screen analysis instructions in the prompt words, a first set of bullet screen effect materials associated with the topic information is selected from the set of bullet screen effect materials; Based on the barrage analysis instructions, feature extraction is performed on the script of each video segment to obtain the script feature vector corresponding to each video segment script. Feature extraction is also performed on each barrage effect material in the first barrage effect material set to obtain the effect feature vector corresponding to each barrage effect material. The script attention vector sequence of the script feature vector is constructed by the large language model, and the script attention vector sequence and the special effects feature vector are matched for similarity to obtain the matching result. Based on the matching results, output bullet screen configuration suggestions with the suggested output format indicated by the prompt words.

7. The method according to claim 6, characterized in that, The large language model includes an attention processing layer; the construction of the script attention vector sequence of the script feature vector through the large language model includes: Perform a dot product operation between the query parameter matrix in the attention processing layer and the script feature vector to obtain the query vector sequence of the self-attention function; Perform a dot product operation between the key parameter matrix in the attention processing layer and the script feature vector to obtain the key vector sequence of the self-attention function; The value parameter matrix in the attention processing layer is multiplied by the script feature vector to obtain the value vector sequence of the self-attention function; the query parameter matrix, the key parameter matrix, and the value parameter matrix are all matrices composed of learnable parameters in the attention processing layer; Based on the query vector sequence, the key vector sequence, and the value vector sequence, a script attention vector sequence is generated for the script feature vector.

8. The method according to claim 6, characterized in that, The step of outputting bullet screen configuration suggestion information with the suggested output format indicated by the prompt word based on the matching result includes: Based on the matching result, target bullet screen effect materials that match the target video segment script in the video script are obtained; the bullet screen effect material set includes the target bullet screen effect materials. According to the suggested output format, obtain the target special effects material attribute information and the target suggestion reason information of the target bullet screen special effects material; Based on the target special effects material attribute information and the target suggestion reason information, generate bullet screen configuration suggestion information with the suggested output format indicated by the prompt words.

9. The method according to claim 1, characterized in that, The method further includes: Acquire historical video data, historical bullet screen effect materials, and historical bullet screen configuration suggestion information; the historical bullet screen configuration suggestion information is generated based on the usage records of the historical bullet screen effect materials in the historical video data; The historical video data and the historical bullet screen effect materials are input into the initial large language model, and the initial large language model outputs the predicted bullet screen configuration suggestion information. A training loss value is generated based on the historical barrage configuration suggestion information and the predicted barrage configuration suggestion information. The model parameters of the initial large language model are adjusted based on the training loss value to obtain a large language model used to generate the barrage configuration suggestion information.

10. The method according to claim 6, characterized in that, The method further includes: Obtain a set of suggestion information; the set of suggestion information includes multiple suggestion information, including the bullet screen configuration suggestion information; The set of suggested information is given a quality score to obtain a score set; the score set contains the score result corresponding to each piece of suggested information, and the score result is either a positive score result or a negative score result; If the proportion of positive rating results in the rating set is greater than the proportion threshold, an automated configuration service is set in the bullet screen management platform. The automated configuration service is used to instruct the bullet screen management platform to configure the bullet screen effect materials indicated by the new bullet screen configuration suggestion information onto the new video segment when the large language model outputs new bullet screen configuration suggestion information for the new video segment.

11. The method according to claim 10, characterized in that, The method further includes: If the proportion of negative rating results in the rating set is greater than the proportion threshold, then the positive rating results in the rating set are determined as positive sample data, and the negative rating results in the rating set are determined as negative sample data. The large language model is retrained and fine-tuned based on the positive and negative sample data.

12. The method according to claim 1, characterized in that, The acquisition of video data includes: Obtain the video bullet comment parsing request; the video bullet comment parsing request includes video episode identifier information; Based on the video bullet screen parsing request, the video set identifier information in the video set identifier information is obtained, and a video library is obtained; the video library includes multiple video sets, each containing multiple set videos; The target video set corresponding to the video set identifier information is queried in the video library, and the single video set corresponding to the single video set identifier information among the multiple single video sets associated with the target video set is identified as video data.

13. A data processing apparatus, characterized in that, The device includes: The transceiver module is used to acquire video data; The video script generation module is used to parse the video data to obtain a video script; the video script contains multiple video segment scripts corresponding to each video segment. The prompt word generation module is used to obtain a set of bullet screen effect materials and generate prompt words containing the set of bullet screen effect materials; The model processing module is used to analyze and match each video segment script with each bullet screen effect material in the bullet screen effect material set according to the bullet screen analysis instructions in the prompt words, and obtain bullet screen configuration suggestion information with the suggested output format indicated by the prompt words; the bullet screen configuration suggestion information includes the bullet screen effect material suggested for the target video segment among the multiple video segments and the suggestion reason information; the bullet screen effect material includes the bullet screen effect name and bullet screen effect rules.

14. A computer device, characterized in that, Includes processor, memory, and input / output interfaces; The processor is connected to the memory and the input / output interface respectively, wherein the input / output interface is used to receive data and output data, the memory is used to store computer programs, and the processor is used to call the computer programs so that the computer device executes the method according to any one of claims 1-12.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted to be loaded and executed by a processor to cause a computer device having the processor to perform the method of any one of claims 1-12.

16. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1-12.

Images