Game commentary generation method and device
By capturing game videos and using pre-trained models to generate commentary text, the problem of existing technologies being unable to provide real-time commentary for all games has been solved, achieving efficient and accurate commentary generation for all games.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-01-14
- Publication Date
- 2026-04-21
AI Technical Summary
Existing technology struggles to generate real-time commentary in all games because not all games support extracting real-time data via APIs, making it difficult for non-core players or novice viewers to understand key events in complex games.
By collecting game videos, a pre-trained game commentary generation model is used to identify game scenes, generate commentary text, and split it into subtitles to be added to video frames. The model is trained using a pre-trained language processing model and game character data samples, and high- and low-quality texts are separated and trained in multiple rounds to improve generation accuracy and efficiency.
It enables the generation of commentary for all types of games, improving the accuracy and efficiency of commentary generation. It can identify content other than game information, such as player information, and generate richer commentary text.
Smart Images
Figure CN121908085A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of game information processing technology, and more specifically, to a method and apparatus for generating game commentary. Background Technology
[0002] With the continuous development of the gaming and live streaming industries, more and more people are using game live streams and videos as a form of daily leisure and entertainment. Because many games involve highly complex strategies, rapidly changing battlefield situations, and rich game elements, non-core players or novice viewers often find it difficult to accurately understand the key events unfolding on screen. Therefore, it is necessary to add game commentary for novice viewers during live streams and video playback.
[0003] Existing technologies typically use game data during gameplay to generate real-time commentary. This method requires prior access to the game's interface to extract real-time data. However, not all games support extracting real-time data through interfaces, so existing game commentary generation technologies are not applicable to all games. Summary of the Invention
[0004] This invention provides a method and apparatus for generating game commentary, which can effectively generate various types of game commentary.
[0005] According to a first aspect of this application, a method for generating game commentary is provided, the method comprising: Collect game videos; The game video is input into a pre-trained game commentary generation model to obtain game commentary text; The game commentary text is split according to each game video frame in the game video to obtain the commentary text segment corresponding to each game video frame; The narration text fragments are added as subtitles to the corresponding game video frames.
[0006] Optionally, the pre-training of the game commentary generation model includes: Obtain game character data samples corresponding to game training video samples; The game training video samples and the game character data samples are input into a pre-trained language processing model for processing to obtain the game commentary reference text corresponding to the game training video samples. The game commentary generation model is trained using the game training video samples and the corresponding game commentary reference text to obtain the trained game commentary generation model.
[0007] Optionally, the step of training the game commentary generation model to be trained based on the game training video samples and the corresponding game commentary reference text to obtain the trained game commentary generation model includes: The game commentary reference texts are classified according to a preset text quality evaluation strategy to obtain a first commentary reference text and a second commentary reference text; the text quality of the second commentary reference text is higher than that of the first commentary reference text. The game commentary generation model to be trained is trained based on the first commentary reference text and the corresponding game training video samples to obtain the intermediate game commentary generation model to be trained. The intermediate game commentary generation model to be trained is trained based on the second commentary reference text and the corresponding game training video samples to obtain the trained game commentary generation model.
[0008] Optionally, the step of classifying the game commentary reference text according to a preset text quality evaluation strategy to obtain a first commentary reference text and a second commentary reference text includes: The game commentary reference text is scored according to a preset text quality evaluation strategy, and game commentary reference texts with scores higher than a preset score threshold are selected as candidate texts. The second explanatory reference text is selected from the candidate texts according to the preset event tags, and the game explanatory reference text other than the second explanatory reference text is used as the first explanatory reference text.
[0009] Optionally, training the intermediate game commentary generation model based on the second commentary reference text and the corresponding game training video samples includes: Obtain the interfering text of the second explanatory reference text; Using the game training video sample corresponding to the second commentary reference text as input, the second commentary reference text as the output target, and the interference text as the output counterexample, the intermediate game commentary generation model to be trained is trained, and the training loss is calculated according to the preset loss function. The parameters of the intermediate game commentary generation model to be trained are updated according to the training loss.
[0010] Optionally, the loss function includes preference loss; The calculation of the preference loss includes: Obtain the first preference probability of the intermediate game commentary generation model to be trained to generate the second commentary reference text, and the first deviation probability of generating the interference text; Obtain the second preference probability of the intermediate game commentary generation model in the current training to generate the second commentary reference text, and the second deviation probability of generating the interference text; Calculate the first ratio between the first preference probability and the second preference probability, and calculate the second ratio between the first deviation probability and the second deviation probability; The preference loss is calculated based on the first difference between the first ratio and the second ratio.
[0011] Optionally, the loss function may further include one or more of quality loss and generation loss; The calculation of the mass loss includes: The mass loss is calculated based on the second difference between the first ratio and the preset ratio threshold, and the third difference between the preset ratio threshold and the first ratio; The generation loss is the cross-entropy constructed based on the second preference probability.
[0012] Optionally, each of the game training video samples includes a preset number of consecutive game video frames; The acquisition of game character data samples corresponding to game training video samples includes: Image recognition is performed on each game video frame in the game video to extract game character attribute information and game character position information from each game video frame; Construct a game character information sequence by combining the game character attribute information and game character position information of all the game video frames according to the time order of the game video frames; The sequence of game character information is used as the game character data sample corresponding to the game training video sample.
[0013] Optionally, obtaining the game character data samples corresponding to the game training video samples further includes: Image recognition is performed on each game video frame in the game video. When the game video frame includes game event prompt information, the game event prompt information is added to the game character information sequence according to the time of the game video frame.
[0014] According to a second aspect of this application, a training apparatus for a game commentary generation model is provided, the apparatus comprising: The data acquisition module is used to capture game videos; The commentary text acquisition module is used to input the game video into a pre-trained game commentary generation model to obtain the game commentary text; The text processing module is used to split the game commentary text according to each game video frame in the game video to obtain the commentary text segment corresponding to each game video frame; The subtitle adding module is used to add the narration text fragments as subtitles to the corresponding game video frames.
[0015] According to a third aspect of this application, an electronic device is provided, comprising: Memory, used to store one or more computer programs; A processor, when the one or more computer programs are executed by the processor, implements the game commentary generation method described in the first aspect above.
[0016] According to a fourth aspect of this application, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the game commentary generation method described in the first aspect above.
[0017] Based on any of the above aspects, the game commentary generation method, apparatus, electronic device, and computer storage medium provided in this application embodiment can process real-time acquired game videos to obtain game commentary text through a pre-trained game commentary generation model. It does not require the collection of data during game execution, and can capture game information through game screens in game videos, thus effectively obtaining game commentary text for various types of games.
[0018] Meanwhile, the game commentary generation method, apparatus, electronic device, and computer storage medium provided in this application embodiment can generate high-quality game commentary reference text for training in batches by using a pre-trained language processing model to process game training video samples, thereby effectively improving the generation efficiency and accuracy of the game commentary generation model.
[0019] Furthermore, the game commentary generation method, apparatus, electronic device, and computer storage medium provided in this application divide the game commentary reference text according to text instructions into a low-quality first commentary reference text and a high-quality second commentary reference text. The low-quality first commentary reference text is used to initially train the game commentary generation model, enabling the model to learn image recognition and game-related knowledge while maintaining its general capabilities. Based on this, the high-quality second commentary reference text is used to reinforce the intermediate game commentary generation model, thereby effectively improving the reasoning ability of the game commentary generation model. Attached Figure Description
[0020] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0021] Figure 1This is an illustrative application scenario diagram of the game commentary generation method provided in this embodiment.
[0022] Figure 2 This is a flowchart illustrating the steps of the game commentary generation method provided in this embodiment.
[0023] Figure 3 This is a schematic diagram of the pre-training process of the game commentary generation model provided in this embodiment.
[0024] Figure 4 This is a flowchart illustrating the steps involved in obtaining game character data samples for this embodiment.
[0025] Figure 5 This is a schematic diagram of a sub-step of step S3 provided in this embodiment.
[0026] Figure 6 This is a schematic diagram of the functional modules of the game commentary generation device provided in this embodiment.
[0027] Figure 7 This embodiment provides a schematic diagram of the electronic device. Detailed Implementation
[0028] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this application. To better illustrate the following embodiments, some components in the drawings may be omitted, enlarged, or reduced, and do not represent the actual dimensions of the product; it is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.
[0029] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.
[0030] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.
[0031] With the continuous development of the gaming and live streaming industries, more and more people are using game live streams and videos as a form of daily leisure and entertainment. Because many games involve highly complex strategies, rapidly changing battlefield situations, and rich game elements, non-core players or novice viewers often find it difficult to accurately understand the key events unfolding on screen. Therefore, it is necessary to add game commentary for novice viewers during live streams and video playback.
[0032] Existing technologies typically use game data during gameplay to generate real-time commentary. This method requires prior access to the game's interface to extract real-time data. However, not all games support extracting real-time data through interfaces, so existing game commentary generation technologies are not applicable to all games.
[0033] This embodiment provides a technical solution that can solve the above problems. The specific implementation of this application will be described in detail below with reference to the accompanying drawings.
[0034] An exemplary illustration of an application scenario for a game commentary generation method provided in this text application embodiment. For example... Figure 1 As shown, the application scenario includes at least a server 100 and a terminal 200 that can communicate with the server 100.
[0035] Understandably, the server 100 can be an independent electronic device or a cluster of multiple electronic devices; the terminal 200 can be a smartphone terminal, a personal computer, a tablet computer, a vehicle terminal, etc., but is not limited to these.
[0036] In one possible implementation, server 100 and terminal 200 may respectively execute the game commentary generation method provided in the embodiments of this application, or, optionally, the game commentary generation method provided in the embodiments of this application may be partially executed in server 100 and partially executed in terminal 200.
[0037] like Figure 2 As shown, this embodiment provides a method for generating game commentary, which may include the following steps: A1: Capture game videos; A2: Input the game video into the game commentary generation model to obtain the game commentary text; A3: The game commentary text is split according to each game video frame in the game video to obtain the commentary text segment corresponding to each game video frame; A4: Add the narration text segment as a subtitle to the corresponding game video frame.
[0038] Understandably, the game commentary text contains descriptions corresponding to all game progress information in the game video. Then, the game commentary text is split according to the timestamps in the game commentary text. The timestamps can guide the output of the game commentary generation model during training. The timestamps are matched with the game training video samples or game video frames in the game video. The commentary text segments with added timestamps can obtain game video frames that match the commentary text segments according to the timestamps. Finally, the commentary text segments are added as subtitles to the game video frames, allowing viewers to understand the game progress by watching the subtitles in the game video.
[0039] In this embodiment, the game commentary generation model can identify the current game screen displayed in a game video and generate corresponding game commentary text. It does not require real-time game data and is applicable to all types of games. Furthermore, by recognizing the game screen in the game video, it can effectively identify information beyond the game's own data. For example, in game tournament videos, it can also extract information about the players, thereby generating richer game commentary text.
[0040] In this embodiment, as Figure 3 As shown, the pre-training of the game commentary generation model may include the following steps: S1: Obtain game character data samples corresponding to the game training video samples; In this embodiment, the game training video samples may include a preset number of consecutive game video frames. In one implementation, a preset number of consecutive game video frames may be extracted from the game recording video or game live stream video as the game training video samples. Preferably, the game training video samples may be obtained by collecting event videos or event live streams of the corresponding game. Event videos and event live streams usually contain more in-game information. Therefore, the game training video samples collected by collecting event videos or event live streams can contain more game information for subsequent language processing models to learn.
[0041] In one implementation, such as Figure 4 As shown, step S1 may include the following steps: S11: Perform image recognition on each game video frame in the game video, and extract the game character attribute information and game character position information from each game video frame; S12: Construct a game character information sequence by taking the game character attribute information and game character position information of all the game video frames according to the time order of the game video frames; S13: Use the game character information sequence as the game character data sample corresponding to the game training video sample.
[0042] In this embodiment, the game character attribute information may include game character image, game character avatar, and character attributes, etc., related to the game character. Further, the character attributes may include character HP (HitPoints), character MP (Magic Points), and character-related icon states, such as buff states, abnormal states, and skill states. In one implementation, the game character attribute information may also include player identity information, primarily used in multiplayer games, such as competitive games, to distinguish different game characters and the players operating them.
[0043] It is understood that the game character attribute information may include various attributes of the corresponding game character in the game. The game character and its attributes are an important part of the game content and can help players and viewers understand the state changes of the game character in the game process.
[0044] In this embodiment, in addition to obtaining the game character attribute information, game character position information is also obtained. The game character position information is the relative coordinates of the corresponding game character in the game screen shown in the game video frame. It can be understood that if the game screen contains multiple game characters, the game character position information of all game characters in the game screen can be obtained, and the game character position information can be associated with the game character attribute information of the corresponding game character.
[0045] By obtaining the game character's location information, it is possible to determine the game character's or player's operational intentions and the interactive relationships between the game characters, which can help players and viewers understand the game character's trajectory.
[0046] In one implementation, the game character's position information is obtained by identifying the character image or player identity information in the game screen and based on the mapping position of the character image or player identity information in the game screen.
[0047] Understandably, in this embodiment, by recognizing and processing each game video frame, the game character attribute information and game character position information corresponding to the game video frame are obtained, and a game character information sequence is constructed according to the time order, so that the subsequent language processing model can better understand the relevant information of the game content, and thus generate more accurate explanatory text for reference.
[0048] In one embodiment, step S1 may further include the following steps: Image recognition is performed on each game video frame in the game video. When the game video frame includes game event prompt information, the game event prompt information is added to the game character information sequence according to the time of the game video frame.
[0049] Understandably, in games, events may be displayed through text or icons. For example, in competitive games, defeat information might be displayed as a highlighted icon, while in role-playing games, quest progress might be indicated through text prompts. These game event prompts better showcase the game events occurring within the corresponding game video frames, thus better assisting the language processing model in understanding these events. Therefore, the game video frames can be identified to determine if they contain such game event prompts. If they do, they are added to the game character information sequence so that the subsequent language processing model can understand the game's events. S2: Input the game training video samples and the game character data samples into a pre-trained language processing model for processing to obtain the game commentary reference text corresponding to the game training video samples; In this embodiment, the language processing model can be a model with question-and-answer logic capabilities. It is understood that although the language processing model can recognize images, it does not contain game-related knowledge information. Therefore, the language processing model cannot understand game visuals. Thus, the game character data samples extracted in step S1 are needed to help the language processing model learn from the game training video samples, enabling it to generate game commentary reference text about the game training video samples. This game commentary reference text is used for subsequent training of a game-specific commentary generation model.
[0050] Understandably, through the processing in steps S1 and S2, a large number of game training video samples and the corresponding game commentary reference texts can be obtained, providing a large amount of training data for the pre-training of the game commentary generation model. Compared with manually generated game commentary reference texts, the language processing model can generate game commentary reference texts more quickly, and the generated game commentary reference texts are also more accurate and standardized, thus ensuring that the subsequent training of the game commentary generation model is more accurate and that more accurate game commentary texts are obtained.
[0051] S3: Train the game commentary generation model to be trained based on the game training video samples and the corresponding game commentary reference text to obtain the trained game commentary generation model.
[0052] In this embodiment, the game commentary generation model to be trained can be a pre-trained question-and-answer model. It is understood that the game commentary generation model to be trained has basic logical reasoning and question-and-answer capabilities, but lacks game-related knowledge and the ability to understand game visuals. Therefore, it is necessary to train the game commentary generation model using the game training video samples and the game commentary reference text so that the game commentary generation model can accurately identify game videos and generate game commentary text.
[0053] In this embodiment, as Figure 5 As shown, step S3 may include the following sub-steps: S31: Classify the game commentary reference text according to the preset text quality evaluation strategy to obtain the first commentary reference text and the second commentary reference text; S32: Train the game commentary generation model to be trained based on the first commentary reference text and the corresponding game training video samples to obtain the intermediate game commentary generation model to be trained; S33: Train the intermediate game commentary generation model to be trained based on the second commentary reference text and the corresponding game training video samples to obtain the trained game commentary generation model.
[0054] In this embodiment, the text quality of the second commentary reference text is higher than that of the first commentary reference text. The first commentary reference text can be understood as text with many grammatically incorrect, incomplete, and repetitive sentences, as well as many sentences unrelated to the game content. This data is noise data, which will lead to a decrease in text quality. In contrast, the first commentary reference text can be understood as text with few or no grammatically incorrect, incomplete, and repetitive sentences, as well as few or no sentences unrelated to the game content. This type of data has high readability and can be used to train the intermediate game commentary generation model.
[0055] In this embodiment, step S31 may include the following steps: The game commentary reference text is scored according to a preset text quality evaluation strategy. Game commentary reference texts with scores higher than a preset score threshold are selected as candidate texts. The second commentary reference text is selected from the candidate texts according to preset event tags, and the game commentary reference texts other than the second commentary reference text are selected as the first commentary reference text.
[0056] The game commentary reference text can be scored using a pre-trained scoring model based on the text quality evaluation strategy. The scoring model can assess the reference text based on the number of incomplete sentences, grammatically incorrect sentences, repetitive sentences, and sentences irrelevant to the game. An initial screening using the scoring threshold can eliminate a large number of low-quality texts. Further screening using event tags yields high-quality second commentary reference texts. The event tags can be set according to the specific game type or content. For example, for competitive games, the event tags can be set to text or icons displayed on the game screen, such as "defeat" or "consecutive defeats." If the second commentary reference text contains the event tags, it indicates that an important game event has occurred on the game screen, potentially containing significant game progression changes and game content information. Therefore, using the second commentary reference text allows the game commentary generation model to learn richer game knowledge, improving the generation accuracy of the game commentary generation model.
[0057] As mentioned above, although the game commentary generation model to be trained has reasoning ability, it lacks game-related knowledge. Therefore, if the game commentary generation model is directly trained using high-quality second commentary reference text, it is easy to cause overfitting. It may also cause the trained game commentary generation model to lose its original basic logical reasoning ability. Therefore, in this embodiment, the game commentary generation model to be trained is further pre-trained using low-quality first commentary reference text. This allows the game commentary generation model to learn game-related knowledge and game screen recognition using the low-quality text in the first commentary reference text. This enables the generated intermediate game commentary generation model to have preliminary game knowledge for further reinforcement learning.
[0058] In this embodiment, step S32 may include the following steps: Based on the first commentary reference text, text prompts are generated. The text prompts are used to guide the game commentary generation model to be trained to generate the corresponding first commentary reference text. Then, the game training video sample corresponding to the first commentary reference text is used as the input of the game commentary generation model to be trained, and the first commentary reference text is used as the target output of the game commentary generation model to be trained. The game commentary generation model to be trained is trained to obtain the intermediate game commentary generation model to be trained.
[0059] In one embodiment, the game commentary generation model may include an image feature extraction module, a multilayer perceptron module, and a text prediction processing module. The image feature extraction module is used to extract image features from each game video frame in the input game training video sample. The multilayer perceptron module is used to align the image features with the text features corresponding to the text prompts. The text prediction processing module is used to make predictions based on the image features and the text features to obtain the predicted commentary text.
[0060] The training step in step S32 may include: By freezing the network parameters of the text prediction processing module and activating and updating the network parameters of the image feature extraction module and the multilayer perception module during training, the image features and text knowledge of the game screen can be injected into the game commentary generation model, so that the game commentary generation model does not lose its general reasoning and understanding ability.
[0061] Then, the intermediate game commentary model to be trained can be reinforced using the high-quality second commentary reference text.
[0062] Understandably, since the game commentary generation model to be trained is a pre-trained question-answering model, the reinforcement training of the intermediate game commentary generation model can be understood as a fine-tuning process of instructions for the intermediate game commentary generation model. Instruction fine-tuning achieves model training through standardized output. However, in complex scenarios, standardized output can easily lead to deviations in model training and inference, causing inference errors to gradually worsen and resulting in problems such as hallucinations. Therefore, in this embodiment, a hybrid preference optimization approach is used to reinforce the intermediate game commentary generation model. Step S33 can then include the following steps: Obtain the interference text of the second commentary reference text; take the game training video sample corresponding to the second commentary reference text as input, the second commentary reference text as output target, and the interference text as output counterexample to train the intermediate game commentary generation model to be trained, calculate the training loss according to the preset loss function, and update the parameters of the intermediate game commentary generation model to be trained according to the training loss.
[0063] In this embodiment, the second explanatory reference text can be understood as a positive output target, and the interference text can be understood as a negative output counterexample. During the reinforcement training process, the intermediate game commentary generation model will make the output result align with the second explanatory reference text while keeping the output result away from the interference text. By training through the hybrid preference optimization method, the output of the game commentary generation model is made more in line with human preferences, and the illusion problem caused by a large deviation between game knowledge and general knowledge is avoided.
[0064] In one implementation, the loss function includes preference loss; The calculation of the preference loss may include: Obtain the first preference probability of the intermediate game commentary generation model to be trained to generate the second commentary reference text, and the first deviation probability of generating the interference text; Obtain the second preference probability of the intermediate game commentary generation model in the current training to generate the second commentary reference text, and the second deviation probability of generating the interference text; Calculate the first ratio between the first preference probability and the second preference probability, and calculate the second ratio between the first deviation probability and the second deviation probability; Calculate the preference loss based on the first difference between the first ratio and the second ratio; Understandably, in the actual computational process, after obtaining the intermediate game commentary generation model to be trained, the intermediate game commentary generation model to be trained can be backed up. One copy of the intermediate game commentary generation model to be trained is used for training, while the other copy of the intermediate game commentary generation model to be trained has fixed network parameters and is used as a reference for the intermediate game commentary generation model to be trained.
[0065] Understandably, the preference loss can be calculated using the following formula: In the formula, For the preference loss, This is the reference text for the second explanation. The interfering text, The input to the model is the intermediate game commentary generated during the current training, which is the game training video sample corresponding to the second commentary reference text. For the Sigmoid function, For learnable hyperparameters, This represents the intermediate game commentary generation model currently in training. This refers to the intermediate game commentary generation model to be trained.
[0066] In one implementation, the loss function may further include one or more of quality loss and generation loss; The calculation of the mass loss may include: The mass loss is calculated based on the second difference between the first ratio and the preset ratio threshold, and the third difference between the preset ratio threshold and the first ratio; The mass loss can then be calculated using the following formula: In the formula, For the mass loss, The preset ratio threshold is used.
[0067] The generation loss is the cross-entropy constructed based on the second preference probability.
[0068] Preferably, the loss function may include preference loss, quality loss and generation loss, and the loss function can be obtained by weighting the preference loss, quality loss and generation loss.
[0069] like Figure 6 As shown in the illustration, this application also provides a game commentary generation device. Optionally, the device may include: Data acquisition module 11 is used to acquire game videos; In this embodiment, the data acquisition module 11 can be used to perform... Figure 2 For a detailed description of the data acquisition module 11 shown in step A1, please refer to the description of step A1.
[0070] The commentary text acquisition module 12 is used to input the game video into a pre-trained game commentary generation model to obtain game commentary text; In this embodiment, the explanatory text acquisition module 12 can be used to perform... Figure 2 For a detailed description of the explanatory text acquisition module 12 shown in step A2, please refer to the description of step A2.
[0071] The text processing module 13 is used to split the game commentary text according to each game video frame in the game video to obtain the commentary text segment corresponding to each game video frame. In this embodiment, the text processing module 13 can be used to perform... Figure 2 For a detailed description of the text processing module 13 shown in step A3, please refer to the description of step A3.
[0072] The subtitle adding module 14 is used to add the narration text fragments as subtitles to the corresponding game video frames; In this embodiment, the subtitle adding module 14 can be used to perform... Figure 2 For a detailed description of the subtitle adding module 14 shown in step A4, please refer to the description of step A4.
[0073] In one embodiment, the device may further include a model training module 15, which is used to train the game commentary generation model. In this embodiment, the model training module 15 can be used to perform... Figure 3 For a detailed description of the model training module 15, please refer to the description of steps S1-S3 shown in the figure.
[0074] It is understood that the above-described device embodiments and method embodiments can correspond to each other, and similar descriptions of the device embodiments can be referred to the method embodiments. To avoid repetition, further details are omitted here. The game commentary generation device provided in this application can execute a game commentary generation method provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects of executing the method. The functional modules of the game commentary generation device can be implemented in hardware, in software instructions, or in a combination of hardware and software modules.
[0075] Specifically, the steps of the method embodiments of this application can be implemented by integrated logic circuits in the processor hardware and / or instructions in software form. The steps of the game commentary generation method in conjunction with the embodiments of this application can be directly implemented by a hardware encoding processor, or by a combination of hardware and software modules in the encoding processor. Optionally, the software module can be located in random access memory, and storage media such as read-only memory, programmable read-only memory, flash memory, electrically erasable programmable memory, and registers are all acceptable. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps in the above method embodiments.
[0076] This application provides an electronic device with the following structure: Figure 7 As shown. The electronic device can be as described in this embodiment. Figure 1 The server 100 or terminal 200 shown.
[0077] The electronic device includes a memory 21, a processor 22, a communication module 23, and an input / output interface 24, etc. Optionally, the memory 21, the processor 22, the communication module 23, and the input / output interface 24 can be connected and communicate with each other through a bus 25.
[0078] The memory 21 is used to store one or more computer programs and to transfer the code of the computer programs to the processor 22; when the one or more computer programs are executed by the processor 22, the game commentary generation method in this embodiment of the application is implemented.
[0079] Optionally, the electronic device can be connected to a network via communication module 23 to communicate with other devices, such as terminals or servers, to achieve data interaction. The electronic device can be various forms of digital computers, exemplarily such as desktop computers, servers, workbenches, mainframes, or other types of computers. The electronic device can also be various forms of mobile terminals, exemplarily such as smartphones, tablets, wearable devices (such as helmets, glasses, watches, etc.), and other similar mobile terminals.
[0080] Optionally, the electronic device can connect to required input / output devices, such as a keyboard or display device, via the input / output interface 24. The electronic device itself may have a display device, and other display devices can also be connected externally via the input / output interface 24. Optionally, a storage device, such as a hard disk, can also be connected via the input / output interface 24 to store data from the electronic device, read data from the storage device, or store data from the storage device in the memory 21. It is understood that the input / output interface 24 can be a wired interface or a wireless interface. Depending on the actual application scenario, the device connected to the input / output interface 24 can be a component of the electronic device or an external device connected to the electronic device when needed.
[0081] Optionally, the memory 21 may be a volatile memory and / or a non-volatile memory. The volatile memory may be a random access memory, etc., and the non-volatile memory may be a read-only memory, a programmable read-only memory, an erasable programmable read-only memory, an electrically erasable programmable read-only memory, or a flash memory, etc.
[0082] Optionally, the computer program stored in the processor 22 can be divided into one or more modules, which are stored in the memory 21 and executed by the processor 22 to perform the method provided in this embodiment. The one or more modules can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the electronic device.
[0083] Optionally, the processor 22 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 22 include, but are not limited to, a central processing unit, a graphics processing unit, a digital signal processor, various special-purpose artificial intelligence computing chips, various processors running machine learning model algorithms, and can also be any suitable controller, microcontroller, processor, etc. The processor 22 executes the various methods and processes of this embodiment, exemplarily, such as a game commentary generation method according to an embodiment of this application.
[0084] Optionally, the bus 25 may include a path for transmitting information. Depending on its function, the bus 25 may be divided into an address bus, a data bus, a control bus, etc.
[0085] In an optional implementation, this application embodiment also provides a computer storage medium storing a computer program thereon, which, when executed by a computer, enables the computer to perform the methods described in the above method embodiments. Part or all of the computer program can be loaded and / or installed on the memory 21 of an electronic device. When the computer program is executed by the processor 22, one or more steps of a game commentary generation method according to this application embodiment can be performed.
[0086] Optionally, the computer-readable storage medium may be a random access memory, a read-only memory, a programmable read-only memory, an erasable programmable read-only memory, an electrically erasable programmable read-only memory, etc.
[0087] Obviously, the above embodiments of this application are merely examples for clearly illustrating the technical solution of this application, and are not intended to limit the specific implementation of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the claims of this application should be included within the protection scope of the claims of this application.
Claims
1. A method for generating game commentary, characterized in that, The method includes: Collect game videos; The game video is input into a pre-trained game commentary generation model to obtain game commentary text; The game commentary text is split according to each game video frame in the game video to obtain the commentary text segment corresponding to each game video frame; The narration text fragments are added as subtitles to the corresponding game video frames.
2. The game commentary generation method according to claim 1, characterized in that, The pre-training of the game commentary generation model includes: Obtain game character data samples corresponding to game training video samples; The game training video samples and the game character data samples are input into a pre-trained language processing model for processing to obtain the game commentary reference text corresponding to the game training video samples. The game commentary generation model is trained using the game training video samples and the corresponding game commentary reference text to obtain the trained game commentary generation model.
3. The game commentary generation method according to claim 2, characterized in that, The step of training the game commentary generation model based on the game training video samples and the corresponding game commentary reference text to obtain the trained game commentary generation model includes: The game commentary reference texts are classified according to a preset text quality evaluation strategy to obtain a first commentary reference text and a second commentary reference text; the text quality of the second commentary reference text is higher than that of the first commentary reference text. The game commentary generation model to be trained is trained based on the first commentary reference text and the corresponding game training video samples to obtain the intermediate game commentary generation model to be trained. The intermediate game commentary generation model to be trained is trained based on the second commentary reference text and the corresponding game training video samples to obtain the trained game commentary generation model.
4. The game commentary generation method according to claim 3, characterized in that, The step of classifying the game commentary reference text according to a preset text quality evaluation strategy to obtain a first commentary reference text and a second commentary reference text includes: The game commentary reference text is scored according to a preset text quality evaluation strategy, and game commentary reference texts with scores higher than a preset score threshold are selected as candidate texts. The second explanatory reference text is selected from the candidate texts according to the preset event tags, and the game explanatory reference text other than the second explanatory reference text is used as the first explanatory reference text.
5. The game commentary generation method according to claim 3, characterized in that, The step of training the intermediate game commentary generation model based on the second commentary reference text and the corresponding game training video samples includes: Obtain the interfering text of the second explanatory reference text; Using the game training video sample corresponding to the second commentary reference text as input, the second commentary reference text as the output target, and the interference text as the output counterexample, the intermediate game commentary generation model to be trained is trained, and the training loss is calculated according to the preset loss function. The parameters of the intermediate game commentary generation model to be trained are updated according to the training loss.
6. The game commentary generation method according to claim 5, characterized in that, The loss function includes preference loss; The calculation of the preference loss includes: Obtain the first preference probability of the intermediate game commentary generation model to be trained to generate the second commentary reference text, and the first deviation probability of generating the interference text; Obtain the second preference probability of the intermediate game commentary generation model in the current training to generate the second commentary reference text, and the second deviation probability of generating the interference text; Calculate the first ratio between the first preference probability and the second preference probability, and calculate the second ratio between the first deviation probability and the second deviation probability; The preference loss is calculated based on the first difference between the first ratio and the second ratio.
7. The game commentary generation method according to claim 6, characterized in that, The loss function also includes one or more of the quality loss and generation loss; The calculation of the mass loss includes: The mass loss is calculated based on the second difference between the first ratio and the preset ratio threshold, and the third difference between the preset ratio threshold and the first ratio; The generation loss is the cross-entropy constructed based on the second preference probability.
8. The game commentary generation method according to any one of claims 2-6, characterized in that, Each of the game training video samples includes a preset number of consecutive game video frames; The acquisition of game character data samples corresponding to game training video samples includes: Image recognition is performed on each game video frame in the game video to extract game character attribute information and game character position information from each game video frame; Construct a game character information sequence by combining the game character attribute information and game character position information of all the game video frames according to the time order of the game video frames; The sequence of game character information is used as the game character data sample corresponding to the game training video sample.
9. The game commentary generation method according to claim 8, characterized in that, The process of obtaining game character data samples corresponding to game training video samples also includes: Image recognition is performed on each game video frame in the game video. When the game video frame includes game event prompt information, the game event prompt information is added to the game character information sequence according to the time of the game video frame.
10. A game commentary generation device, characterized in that, The device includes: The data acquisition module is used to capture game videos; The commentary text acquisition module is used to input the game video into a pre-trained game commentary generation model to obtain the game commentary text; The text processing module is used to split the game commentary text according to each game video frame in the game video to obtain the commentary text segment corresponding to each game video frame; The subtitle adding module is used to add the narration text fragments as subtitles to the corresponding game video frames.