Game highlight video generation method and device, computer equipment, storage medium and computer program product

Abstract

The invention provides a game highlight video generation method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: during the duration of a game process, acquiring each key picture frame of a game client in real time, and performing highlight event detection on each key picture frame to obtain highlight event detection information of each key picture frame; according to the highlight event detection information corresponding to each key picture frame, identifying the starting and ending time of each highlight event in real time; based on the starting and ending time of each highlight event, intercepting a highlight video of each highlight event from the continuously generated game video stream in real time, and understanding the video content of the highlight video of each highlight event to obtain a video content understanding result of each highlight event; and when the game process is finished, generating a game highlight video according to the video content understanding result of each highlight event, and sending the game highlight video to the game client. By adopting the method, the high-quality game highlight video can be generated.

Description

Technical Field

[0001] This application relates to the field of game video generation technology, and in particular to a method, apparatus, computer device, storage medium and computer program product for generating game highlight videos. Background Technology

[0002] In the field of game content creation, players have a strong demand for the instant sharing of "highlight moments".

[0003] Currently, the generation of game highlight videos often involves reading game backend logs, extracting fixed-length video segments from the full game video, and then analyzing these segments to generate the highlight video. However, because this method extracts fixed-length video segments, the generated video content is rather rigid, failing to convey the deeper semantics of game strategies and lacking the ability to provide a narrative explanation, resulting in poor quality game highlight videos.

[0004] Therefore, traditional technologies have the problem of being unable to generate high-quality game highlight videos. Summary of the Invention

[0005] Based on this, the purpose of this application is to at least solve one of the above-mentioned technical defects, in particular the technical defect that the prior art cannot generate high-quality game highlight videos. This application provides a game highlight video generation method, apparatus, computer equipment, computer-readable storage medium and computer program product that can generate high-quality game highlight videos.

[0006] Firstly, this application provides a method for generating game highlight videos, applied to a cloud service platform, the method comprising:

[0007] During the game process, key frames of the game client are acquired in real time, and highlight events are detected on each key frame to obtain highlight event detection information for each key frame.

[0008] Based on the highlight event detection information corresponding to each key frame, the start and end times of each highlight event are identified in real time.

[0009] Based on the start and end times of each highlight event, highlight videos of each highlight event are extracted from the continuously generated game video stream in real time, and video content understanding is performed on the highlight videos of each highlight event to obtain the video content understanding results of each highlight event.

[0010] At the end of the game, based on the understanding of the video content of each highlight event, a game highlight video is generated and sent to the game client.

[0011] In one embodiment, video content understanding is performed on the highlight videos of each highlight event to obtain the video content understanding results for each highlight event, including:

[0012] Preprocess the highlight videos of each highlight event to obtain the processed highlight videos;

[0013] Each processed highlight video is input into a multimodal large model, which analyzes the processed highlight videos to generate video content understanding results for each highlight event.

[0014] In one embodiment, the video content understanding result for each highlight event includes the start and end times of each highlight segment in the highlight event, segment description text, and segment description keywords. Based on the video content understanding results for each highlight event, a game highlight video is generated, including:

[0015] Generate a game commentary script based on the start and end times of each highlight segment, the segment description text, and the segment description keywords;

[0016] Generate audio commentary based on the game commentary script;

[0017] Based on the start and end times of each highlight clip, the highlight clips are time-aligned with the game commentary audio to generate a game highlight video.

[0018] In one embodiment, a game commentary script is generated based on the start and end times of each highlight segment, the segment description text, and the segment description keywords, including:

[0019] Determine the preset narration style, and based on the preset narration style and the start and end times of each highlight segment, segment description text and segment description keywords, generate a script to generate prompt words;

[0020] The script-generated prompts are input into a large language model. Guided by the prompts, the large language model generates a game commentary script with a preset commentary style based on the start and end times of each highlight segment, the segment description text, and the segment description keywords.

[0021] In one embodiment, specular event detection is performed on each key frame to obtain specular event detection information for each key frame, including:

[0022] Determine the highlight event detection strategy for the game process, and determine the highlight event detection model based on the highlight event detection strategy;

[0023] Each key frame is input into the highlight event detection model to obtain highlight event detection information for each key frame.

[0024] In one embodiment, the specular event detection model is a visual language model. Each key frame is input into the specular event detection model to obtain specular event detection information for each key frame, including:

[0025] Obtain highlight event detection prompts for any key frame;

[0026] Input any keyframe and any highlight event detection cue words for any keyframe into the visual language model. Guided by the highlight event detection cue words, the visual language model analyzes any keyframe and outputs the highlight event detection information for any keyframe.

[0027] Secondly, this application provides a game highlight video generation device, applied to a cloud service platform, the device comprising:

[0028] The acquisition module is used to acquire each key frame of the game client in real time during the game process, and to perform highlight event detection on each key frame to obtain the highlight event detection information of each key frame.

[0029] The recognition module is used to identify the start and end times of each highlight event in real time based on the highlight event detection information corresponding to each key frame.

[0030] The capture module is used to extract the highlight video of each highlight event from the continuously generated game video stream in real time based on the start and end times of each highlight event, and to perform video content understanding on the highlight video of each highlight event to obtain the video content understanding results of each highlight event.

[0031] The generation module is used to generate game highlight videos and send them to the game client based on the video content understanding results of each highlight event when the game process ends.

[0032] Thirdly, this application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described method.

[0033] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0034] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method.

[0035] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:

[0036] The game highlight video generation method, apparatus, computer equipment, storage medium, and computer program product provided in this application acquire key frames of the game client in real time during the continuous game process, and perform highlight event detection on each key frame to obtain highlight event detection information for each key frame; based on the highlight event detection information corresponding to each key frame, identify the start and end times of each highlight event in real time; based on the start and end times of each highlight event, extract highlight videos of each highlight event from the continuously generated game video stream in real time, and perform video content understanding on the highlight videos of each highlight event to obtain video content understanding results for each highlight event; at the end of the game process, generate game highlight videos based on the video content understanding results of each highlight event and send them to the game client; thus, because The cloud service platform only performs highlight event detection on key frames, eliminating the need for full analysis of the entire game video. This reduces computational power consumption. Furthermore, the cloud service platform simultaneously performs highlight event detection on each key frame, accurately locates the start and end times of highlight videos, efficiently extracts highlight videos, and accurately understands highlight videos throughout the game. Immediately after the game ends, based on the video content understanding results of each highlight event, it efficiently generates high-quality game highlight videos with a compelling storyline and sends them to the game client. This not only ensures the integrity of highlight event content and avoids the omission or inaccurate extraction of exciting events, but also improves the generation efficiency and timeliness of game highlight videos. Overall, it achieves fully automated, intelligent, and highly efficient generation from raw game video streams to high-quality game highlight videos. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is an application environment diagram of a game highlight video generation method in one embodiment;

[0039] Figure 2 This is a flowchart illustrating a method for generating game highlight videos in one embodiment;

[0040] Figure 3 This is a schematic diagram illustrating data interaction between an agent and a client in one embodiment.

[0041] Figure 4 This is a flowchart illustrating a method for generating game highlight videos in another embodiment;

[0042] Figure 5 This is a structural block diagram of a game highlight video generation device in one embodiment;

[0043] Figure 6 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0044] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0045] The game highlight video generation method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, the game client 102 communicates with the cloud service platform server 104 via a network. A data storage system can store the data that the server 104 needs to process. The data storage system can be integrated onto the server 104 or placed on the cloud or other network servers. During the game process, the cloud service platform server 104 acquires each key frame of the game client in real time and performs highlight event detection on each key frame to obtain highlight event detection information. Based on the highlight event detection information corresponding to each key frame, the cloud service platform server 104 identifies the start and end times of each highlight event in real time. Based on the start and end times of each highlight event, the cloud service platform server 104 extracts the highlight video of each highlight event from the continuously generated game video stream in real time, and performs video content understanding on the highlight video of each highlight event to obtain the video content understanding result of each highlight event. When the game process ends, the cloud service platform server 104 generates a game highlight video based on the video content understanding result of each highlight event and sends it to the game client 102. The game client 102 can be, but is not limited to, various personal computers, laptops, smartphones, and tablets. The cloud service platform server 104 can be implemented using a standalone server or a server cluster consisting of multiple servers.

[0046] In one exemplary embodiment, such as Figure 2 As shown, a method for generating game highlight videos is provided, which can be applied to... Figure 1 Taking server 104 of the cloud service platform as an example, the explanation includes the following steps S202 to S210. Wherein:

[0047] Step S202: During the continuous game process, acquire each key frame of the game client in real time, and perform highlight event detection on each key frame to obtain highlight event detection information for each key frame.

[0048] The duration of the game process refers to the entire time period from when the player starts the game and enters the operable state until the player voluntarily exits the game or the level officially ends.

[0049] Among them, the game client refers to the software program installed on the player's local device (such as a personal computer, game console, or mobile phone) to run the game, provide an interactive interface, and communicate with the game server.

[0050] Among them, each key frame is obtained by sampling the entire game screen according to the dynamic frame rate. The dynamic frame rate refers to the sampling rate that is adaptively adjusted according to the drastic changes in the game screen content (such as intense battles and rapid scene switching). When drastic changes in the UI or combat features are detected, the sampling rate is automatically increased to ensure details, while a lower frame rate is used in non-combat states to save resources.

[0051] Among them, highlight event detection information refers to the information obtained after analyzing a single frame. Highlight event detection information can refer to the metadata status of key frame images.

[0052] For example, the metadata state of a keyframe can be represented as "{Event: Kill / None / Win; HP: 85% (health information, used to determine if it was a narrow victory); Weapon: AK47 (weapon information, used for description); Location: Container (scene information)}". It can be seen that the metadata state of a keyframe includes data in four dimensions: Event, HP, Weapon, and Location. The Event dimension represents the highlight event detection result of the keyframe (indicating whether a highlight event occurred, such as a kill, a win, or no highlight event); the HP dimension represents the health displayed in the keyframe; the Weapon dimension represents the weapon used in the keyframe; and the Location dimension represents the scene in which the keyframe is located.

[0053] Optionally, the cloud service platform's server first samples the game screen in real time at a dynamic frame rate, sampling at a low frequency in non-combat states and automatically increasing the sampling rate when drastic UI changes or combat features are detected, thus obtaining a series of key screen frames; subsequently, specular event detection is performed on each key screen frame to obtain specular event detection information for each key screen frame.

[0054] Step S204: Based on the highlight event detection information corresponding to each key frame, identify the start and end times of each highlight event in real time.

[0055] The start and end times of a highlight event refer to the start and end times of the highlight event on the original game video stream timeline.

[0056] Optionally, the server analyzes the metadata status (highlight event detection results, health, weapon, scene) corresponding to each key frame to identify the start and end times of each independent highlight event.

[0057] Step S206: Based on the start and end times of each highlight event, extract the highlight video of each highlight event from the continuously generated game video stream in real time, and perform video content understanding on the highlight video of each highlight event to obtain the video content understanding result of each highlight event.

[0058] Among them, the continuously generated game video stream refers to the sequence of original screen frames (data stream) that is rendered in real time and continuously generated by the game client during the game process and arranged in chronological order.

[0059] Highlight video is a video clip extracted from a continuously generated game video stream that contains only a single highlight event.

[0060] Among them, the video content understanding result is the structured semantic information obtained after in-depth analysis of highlight videos, such as key actions, participating roles, and tactical significance in highlight events.

[0061] Optionally, the cloud service platform's server accurately extracts the highlight video corresponding to each independent highlight event from the continuously generated game video stream based on the start and end times of each individual highlight event, and calls the video understanding model in real time to perform content analysis on the highlight video of each highlight event, extracting rich semantic descriptions.

[0062] Step S208: At the end of the game process, based on the understanding results of the video content of each highlight event, generate a game highlight video and send it to the game client.

[0063] Among them, the game highlight video is the final composite video that integrates multiple highlight events and may be supplemented with narration and special effects.

[0064] Optionally, the cloud service platform's server integrates the video content understanding results of all highlight videos, and generates the final game highlight video by editing, splicing, adding transitions, subtitles or narration, etc., and sends the final game highlight video to the game client. In this way, users can obtain the game highlight video shortly after the game ends.

[0065] In practical applications of this application, users can obtain high-quality game highlight videos within three minutes of the game ending.

[0066] The aforementioned method for generating game highlight videos involves acquiring keyframes from the game client in real-time during the game's continuous execution, performing highlight event detection on each keyframe to obtain highlight event detection information, identifying the start and end times of each highlight event based on the highlight event detection information, extracting highlight videos of each highlight event from the continuously generated game video stream based on the start and end times of each highlight event, performing video content understanding on the highlight videos of each highlight event to obtain video content understanding results for each highlight event, and generating game highlight videos based on the video content understanding results of each highlight event at the end of the game process and sending them to the game client. Thus, since the cloud service platform only processes keyframes... The system performs highlight event detection without requiring full analysis of the entire game video, thus reducing computational consumption. Furthermore, the cloud service platform simultaneously performs highlight event detection on each key frame, accurately locates the start and end times of highlight videos, efficiently extracts highlight videos, and accurately understands highlight videos throughout the game. Immediately after the game ends, based on the video content understanding results of each highlight event, it efficiently generates high-quality game highlight videos with a compelling storyline and sends them to the game client. This not only ensures the integrity of highlight event content and avoids the omission or inaccurate extraction of exciting events, but also improves the generation efficiency and timeliness of game highlight videos. Overall, it achieves fully automatic, intelligent, and highly efficient generation from raw game video streams to high-quality game highlight videos.

[0067] In an exemplary embodiment, video content understanding is performed on the highlight videos of each highlight event to obtain video content understanding results for each highlight event, including: preprocessing the highlight videos of each highlight event to obtain processed highlight videos; inputting each processed highlight video into a multimodal large model, analyzing each processed highlight video through the multimodal large model, and generating video content understanding results for each highlight event.

[0068] Among them, the processed specular video refers to the video after the resolution of the specular video has been adjusted. By uniformly scaling or cropping the video to the fixed or optimal input size required by the multimodal large model, the model processing effect is ensured and the computational efficiency is improved.

[0069] Among them, multimodal large models refer to artificial intelligence models that can process and understand videos (including temporal image sequences and audio), and have powerful video-level semantic understanding, reasoning and description generation capabilities.

[0070] Optionally, after acquiring each highlight video, the cloud service platform's server first preprocesses them, adjusting all videos to a uniform resolution suitable for model input. This step may involve scaling while maintaining aspect ratio or center cropping. Subsequently, these processed videos are input one by one into a multimodal large model. The model analyzes each frame of the processed video and its temporal relationship, integrating visual and potentially included audio information to output a deep understanding of the video content. This includes generating detailed paragraph descriptions, extracting key action sequences, identifying important objects and characters and their interactions, and forming a structured understanding result.

[0071] In this embodiment, by employing an advanced multimodal large model to perform refined content understanding of highlight videos, a deep mining of video semantics is achieved. Compared with analyzing only a single frame or simple action classification, the multimodal large model can understand the complex spatiotemporal logic, causal relationships, and narrative structure in the video, ensuring that the generated video content understanding results not only accurately describe "what happened," but also capture "how it happened" and "why it was exciting," providing rich and accurate materials for subsequent generation of narration scripts and video arrangement.

[0072] In an exemplary embodiment, the video content understanding result for each highlight event includes the start and end times of each highlight segment in the highlight event, segment description text, and segment description keywords. Based on the video content understanding results for each highlight event, a game highlight video is generated, including: generating a game commentary script based on the start and end times of each highlight segment, segment description text, and segment description keywords; generating game commentary audio based on the game commentary script; and aligning each highlight segment with the game commentary audio on the timeline based on the start and end times of each highlight segment to generate a game highlight video.

[0073] The video content understanding results for each highlight event include the start and end times of each highlight segment in the highlight event, segment description text, and segment description keywords.

[0074] Among them, a highlight segment refers to a finer-grained video segment that is further divided within a highlight event and has independent semantics or highlights.

[0075] The start and end times of a segment are the time boundaries of the highlight segment within the video of the highlight event.

[0076] The fragment description text is a natural language description sentence of the visual content of the fragment.

[0077] Among them, the segment description keywords are keywords related to the plot of the highlight segment, indicating why a certain segment is identified as a "highlight segment".

[0078] For example, the video content understanding result of a highlight event can be output in JSON data format. When the video content understanding result of a highlight event includes the start and end times of the corresponding segments 1 and 2, the segment description text, and the segment description keywords, the video content understanding result of the highlight event can be represented as "{"clip_id":1,"start_time":"00:02","end_time":"00:06","description":"The player climbs onto the shipping container and uses an AK47 to shoot the enemy from the side with recoil control.","key_factors":["elevation advantage","recoil control"]}" and "{"clip_id":2,"start_time":"00:15","end_time":"00:21","description":"The player uses a smoke grenade to block the view and heals.","key_factors":["tactical items","healing status"]}". Here, clip_id is the clip number, start_time is the start time of the clip, end_time is the end time of the clip, description is the clip description text, and key_factors are the keywords for the clip description. key_factors can be understood as the reason for being identified as a highlight clip.

[0079] The game commentary script is a text of narration written and synchronized with the visuals for the entire highlight video.

[0080] Among them, game commentary audio is voice-over audio generated by text-to-speech (TTS) based on the commentary script. In practical applications, game commentary scripts can be converted into audio using multi-tone or zero-shot TTS technology.

[0081] Optionally, the cloud service platform's server first parses the video content understanding results of each highlight event, obtaining the start and end times, descriptive text, and descriptive keywords of each highlight segment within it. Then, based on the start and end times, descriptive text, and descriptive keywords of all highlight segments, and combined with the time sequence of all highlight segments, a coherent game commentary script is generated. Next, speech synthesis technology is used to convert the generated game commentary script into audio with a specific timbre, tone, and emotion. Finally, on the video editing timeline, each highlight segment is arranged according to its start and end times, and the generated audio track is precisely aligned and synchronized with the video track to ensure a perfect match between the commentary and the corresponding game screen, thereby outputting the final game commentary video.

[0082] In this embodiment, an automated narration script and narration audio generation based on video content understanding are introduced, and precise synchronization of audio and video is achieved, which improves the professionalism, watchability and immersion of the generated game highlight video, and solves the problem that existing automated editing of game highlight videos lacks narrative and results in poor quality of game highlight videos.

[0083] In an exemplary embodiment, a game commentary script is generated based on the start and end times, description text, and keywords of each highlight segment. This includes: determining a preset commentary style, and generating script generation prompts based on the preset commentary style, the start and end times, description text, and keywords of each highlight segment; inputting the script generation prompts into a large language model, and generating a game commentary script with the preset commentary style based on the start and end times, description text, and keywords of each highlight segment, guided by the script generation prompts.

[0084] Among them, the preset narration style refers to the pre-set narration tone and style, such as passionate, professional analysis, humorous, or calm and composed.

[0085] The script-generated prompts are carefully crafted text instructions designed to guide the large language model, which include task requirements, style settings, input data format, and expected output format.

[0086] Among them, large language models refer to artificial intelligence models that have massive parameters, have been pre-trained on large-scale texts, and are able to understand and generate natural language.

[0087] Optionally, the cloud service platform's server allows users to select or the system to configure a desired commentary style. The desired style, the start and end times of each highlight segment, descriptive text, and descriptive keywords are integrated according to a specific template to construct a detailed prompt, such as, "Please write a commentary script for the following highlight segments arranged in chronological order, in a [passionate] style. Segment 1 (00:00-00:05): [Descriptive Text]...". This prompt is then input into a large language model. Based on the style constraints and highlight segment information in the prompt, the large language model generates a coherent, style-matched, and time-corresponding complete game commentary script.

[0088] In this embodiment, by leveraging the powerful contextual understanding and text generation capabilities of the large language model, the semantics of highlight segments can be understood, and commentary rich in logic, emotion, and style can be generated accordingly. This makes the final game highlight video more personalized and attractive, significantly improving the narrative quality of the video and the user's viewing experience.

[0089] In an exemplary embodiment, highlight event detection is performed on each key frame to obtain highlight event detection information for each key frame, including: determining a highlight event detection strategy for the game process, and determining a highlight event detection model based on the highlight event detection strategy; inputting each key frame into the highlight event detection model to obtain highlight event detection information for each key frame.

[0090] Among them, the highlight event detection strategy refers to the solution for detecting high-energy events based on the definition of high-energy events in different games.

[0091] For example, in battle royale games, each kill or item use is accompanied by text descriptions in a fixed location (e.g., "You used an assault rifle..."), requiring no context. In this case, an on-device OCR model can be used for detection. In Honor of Kings, however, contextual understanding may be needed to detect highlight events, as the game screen lacks detailed text, requiring a miniature VLM for detection.

[0092] Among them, the highlight event detection model is the algorithm model that specifically performs the detection task. It can be a dedicated small model, such as an on-device OCR model or a lightweight CNN model, which can quickly identify clear UI events (such as "kill" or "down"), or it can be a miniature VLM with a very small number of parameters, which can make inferences about scenes that require a certain level of understanding (such as "enemy appears in the field of view").

[0093] Optionally, the cloud service platform's server first determines the highlight event detection strategy based on the game type of the current game process. This strategy determines the detection model to be used. For events that require contextual understanding, the VLM model may be selected. After selecting the model, any key frame obtained by dynamic sampling is input into the model. The model analyzes the data and outputs the highlight event detection information of any key frame, that is, extracts the metadata status of the key frame.

[0094] In this application, if the VLM model is selected for highlight event detection, the highlight event detection is performed on the server of the cloud service platform. If a lightweight model, such as the OCR model, is selected, the highlight event detection can be performed directly on the game client. Generally speaking, for events that need to be combined with UI information (such as kill prompts), the OCR model is selected for detection on the game client.

[0095] In this embodiment, by first determining the strategy and then selecting the adaptation model, the detection logic is ensured to be highly matched with the target game scene, thereby significantly improving the accuracy of highlight event recognition. This is a key step in ensuring the quality of the final generated game highlight video content from the source, and solves the problem of low video quality caused by inaccurate detection.

[0096] In an exemplary embodiment, the highlight event detection model is a visual language model. Each key frame throughout the game is input into the highlight event detection model to obtain highlight event detection information for each key frame. This includes: obtaining highlight event detection prompts for any given key frame; inputting the given key frame and its highlight event detection prompts into the visual language model; and, guided by the highlight event detection prompts, analyzing the given key frame through the visual language model to output highlight event detection information for that key frame.

[0097] Among them, the visual language model is a multimodal artificial intelligence model that can process both image and text inputs simultaneously and understand the relationship between the two.

[0098] Highlight event detection cue words are text instructions used to guide the visual language model to perform specific detection tasks, such as "Please analyze this game screenshot, determine whether an exciting team fight occurred, and explain your reasoning." It is important to note that the highlight event detection cue words used for each keyframe may be different.

[0099] Optionally, for each keyframe (or key frame), the cloud service platform's server will input the highlight event detection prompt for that keyframe along with the keyframe into the visual language model. The visual language model will understand the task requirements and then, based on its understanding of the image content, generate a structured text response as highlight event detection information, such as the metadata status corresponding to the keyframe mentioned above. In fact, the visual language model can also output status information in other dimensions besides highlight event detection results, health, weapons, and scene.

[0100] In this embodiment, by employing a visual language model combined with natural language prompts for highlight event detection, the visual language model can understand more complex semantics. Furthermore, by adjusting the prompt words, highlight event standards for different key frame frames can be flexibly defined without retraining the model for each new event type. This enhances the generalization ability and interpretability of highlight event detection, enabling more accurate identification of complex and exciting events that conform to human semantic understanding. Consequently, it provides a reliable detection foundation for generating higher-quality game highlight videos.

[0101] For the convenience of those skilled in the art, Figure 3 An exemplary diagram illustrating data interaction between an intelligent agent and a client is provided, corresponding to the data interaction process between the server of the cloud service platform and the game client in this application.

[0102] like Figure 3As shown, this data interaction process encompasses three main stages: highlight event detection, video segment understanding, and audio-visual narration synthesis. The first stage, highlight event detection (using a client-side or lightweight model), spans the entire game from start to finish. This stage includes real-time frame extraction and detection (the agent samples the game screen in real-time at a dynamic frame rate), a tiered detection strategy (using on-device OCR / lightweight CNN models to quickly identify clear UI events, and using a miniature visual model with minimal parameters to infer scenes requiring a certain level of understanding), and a memory module and state return (not only returning whether a highlight event occurred, but also simultaneously extracting and maintaining the metadata state of the current frame; this metadata state is cached to assist subsequent video clipping decisions). The second stage, video segment understanding (using asynchronous or parallel processing in the cloud), is triggered asynchronously during gameplay without blocking the game process. This stage includes intelligent clipping and triggering (when a valid highlight event is detected in the first stage, such as Event="..."). When a kill occurs, the system intelligently determines the start and end times of the video clip based on the context in the memory module, such as the time period during which health drops from full to 25%, rather than mechanically clipping a fixed duration. This involves dimensionality reduction inference and VLM analysis (the clipped short video segments are sent to the cloud or high-computing units, adjusted for resolution, and then input into the visual language model), high-energy location and objective description (the VLM model performs a refined understanding of the clips, outputting JSON data containing precise time periods and objective factual descriptions). The third stage is the synthesis of narration audio and video, executed immediately after the "game over" signal is triggered. This stage includes event aggregation and script generation (the system summarizes the text descriptions of n high-energy events asynchronously generated in the second stage, uses a large language model, and combines them with the user's preset style to transform the scattered objective descriptions into a coherent narration script; the input is the description and key of all clips). Factors outputs commentary that matches the video length, such as, "Look at this wave, the player takes advantage of the height difference of the container, and steadily takes down one person with the AK47! Then, he uses smoke and medicine, a textbook-perfect maneuver!", and multimodal synthesis (converts the commentary into audio using TTS technologies such as multi-tone or Zero-shot, and aligns and synthesizes the carefully edited highlight video clips, commentary audio, and automatic subtitles on the timeline, so that users can get the complete highlight video with commentary after the game ends (usually within 3 minutes).

[0103] like Figure 3As shown, the most time-consuming video understanding process is hidden within the parallel timeline of the game. When the game ends, the system only needs to process text synthesis, and the user's perceived waiting time is extremely short (on the order of minutes). The client side of this application only runs the OCR model or the VLM micro-model, the cloud VLM only processes short segments that have been filtered and downsampled, and the LLM only processes text. Compared with full video analysis, the computing power cost is reduced by several orders of magnitude. By using the health, weapon and other states recorded by the memory module and combining them with the visual description of the VLM, the system can generate commentary with deep game understanding, such as "a comeback with low health" or "a grenade forces the player to move", rather than simple image splicing.

[0104] In another embodiment, such as Figure 4 As shown, a method for generating game highlight videos is provided, which can be applied to... Figure 1 Taking server 104 of the cloud service platform as an example, the following steps are included:

[0105] Step S402: During the continuous game process, acquire each key screen frame of the game client in real time, determine the highlight event detection strategy for the game process, and determine the highlight event detection model based on the highlight event detection strategy.

[0106] Step S404: Input each key frame into the highlight event detection model to obtain highlight event detection information for each key frame.

[0107] Step S406: Based on the highlight event detection information corresponding to each key frame, identify the start and end times of each highlight event in real time.

[0108] Step S408: Based on the start and end times of each highlight event, extract the highlight video of each highlight event from the continuously generated game video stream in real time, and perform video content understanding on the highlight video of each highlight event to obtain the video content understanding result of each highlight event.

[0109] Step S410: At the end of the game process, based on the understanding results of the video content of each highlight event, generate a game highlight video and send it to the game client.

[0110] It should be noted that the specific limitations of the above steps can be found in the specific limitations of a method for generating game highlight videos described above.

[0111] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0112] The following describes the game highlight video generation apparatus provided in the embodiments of this application. The game highlight video generation apparatus has the same inventive concept as the game highlight video generation method described above. The solution to the problem provided by the apparatus is similar to the solution described in the above method. Therefore, the specific limitations of one or more game highlight video generation apparatus embodiments provided below can be referred to the limitations of the game highlight video generation method above. The game highlight video generation apparatus described below and the game highlight video generation method described above can be referred to each other, and will not be repeated here.

[0113] In one exemplary embodiment, Figure 5 This is a schematic diagram of the structure of a game highlight video generation device provided in an embodiment of this application, as shown below. Figure 5 As shown, the game highlight video generation device includes: an acquisition module 502, a recognition module 504, a cropping module 506, and a generation module 508, wherein:

[0114] The acquisition module 502 is used to acquire each key frame of the game client in real time during the continuous game process, and to perform highlight event detection on each key frame to obtain highlight event detection information of each key frame.

[0115] The recognition module 504 is used to identify the start and end times of each highlight event in real time based on the highlight event detection information corresponding to each key frame.

[0116] The interception module 506 is used to extract the highlight video of each highlight event from the continuously generated game video stream in real time based on the start and end time of each highlight event, and to perform video content understanding on the highlight video of each highlight event to obtain the video content understanding result of each highlight event.

[0117] The generation module 508 is used to generate game highlight videos and send them to the game client based on the video content understanding results of each highlight event when the game process ends.

[0118] In one embodiment, the interception module 506 is specifically used to preprocess the highlight videos of each highlight event to obtain each processed highlight video; input each processed highlight video into a multimodal large model, analyze each processed highlight video through the multimodal large model, and generate video content understanding results for each highlight event.

[0119] In one embodiment, the video content understanding result of each highlight event includes the start and end times of each highlight segment in the highlight event, segment description text, and segment description keywords. The generation module 508 is specifically used to generate a game commentary script based on the start and end times of each highlight segment, segment description text, and segment description keywords; generate game commentary audio based on the game commentary script; and align each highlight segment with the game commentary audio on the timeline based on the start and end times of each highlight segment to generate a game highlight video.

[0120] In one embodiment, the generation module 508 is specifically used to determine a preset commentary style, and based on the preset commentary style and the start and end times, description text, and keywords of each highlight segment, generate script generation prompts; input the script generation prompts into a large language model, and under the guidance of the script generation prompts, generate a game commentary script with the preset commentary style based on the start and end times, description text, and keywords of each highlight segment.

[0121] In one embodiment, the acquisition module 502 is specifically used to determine a highlight event detection strategy for the game process, and determine a highlight event detection model based on the highlight event detection strategy; input each key frame into the highlight event detection model to obtain highlight event detection information for each key frame.

[0122] In one embodiment, the highlight event detection model is a visual language model. The acquisition module 502 is specifically used to acquire highlight event detection prompts for any key frame; input any key frame and any key frame highlight event detection prompts into the visual language model; and analyze any key frame under the guidance of the highlight event detection prompts through the visual language model to output highlight event detection information for any key frame.

[0123] In one exemplary embodiment, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the game highlight video generation methods described above.

[0124] In one exemplary embodiment, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of any of the game highlight video generation methods described above.

[0125] In one exemplary embodiment, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the game highlight video generation methods described above.

[0126] Indicatively, such as Figure 6 As shown, Figure 6 This is a schematic diagram of the internal structure of a computer device 600 provided in an embodiment of this application. The computer device 600 can be provided as a server. (Refer to...) Figure 6 The computer device 600 includes a processing component 602, which further includes one or more processors, and memory resources represented by memory 601 for storing instructions, such as application programs, that can be executed by the processing component 602. The application programs stored in memory 601 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 602 is configured to execute instructions to perform the game highlight video generation method of any of the above embodiments.

[0127] The computer device 600 may also include a power supply component 603 configured to perform power management of the computer device 600, a wired or wireless network interface 604 configured to connect the computer device 600 to a network, and an input / output (I / O) interface 605. The computer device 600 may operate on an operating system stored in memory 601, such as Windows Server™, Mac OS X™, Unix™, Linux™, Free BSD™, or similar.

[0128] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0129] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0130] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.

[0131] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for generating a game highlight video, the method comprising: The method is applied to a cloud service platform, and the method comprises: During a game process, real-time acquisition of each key picture frame of a game client, and high-light event detection on each key picture frame to obtain high-light event detection information of each key picture frame; Real-time identification of start and end times of each high-light event according to high-light event detection information corresponding to each key picture frame; Real-time interception of high-light videos of each high-light event from a continuously generated game video stream based on the start and end times of each high-light event, and video content understanding on the high-light videos of each high-light event to obtain video content understanding results of each high-light event; At the end of the game process, generation of a game highlight video according to the video content understanding results of each high-light event and sending of the game highlight video to the game client.

2. The method of claim 1, wherein, The video content understanding on the high-light videos of each high-light event to obtain the video content understanding results of each high-light event comprises: Preprocessing of the high-light videos of each high-light event to obtain processed high-light videos; Input of each processed high-light video into a multi-modal large model, analysis of each processed high-light video by the multi-modal large model, and generation of video content understanding results of each high-light event.

3. The method according to claim 1 or 2, characterized in that, The video content understanding results of each high-light event comprise start and end times, description texts and description keywords of each high-light segment in the high-light event, and the generation of the game highlight video according to the video content understanding results of each high-light event comprises: Generation of a game commentary script according to the start and end times, the description texts and the description keywords of each high-light segment; Generation of game commentary audio according to the game commentary script; Time axis alignment of each high-light segment with the game commentary audio according to the start and end times of each high-light segment to generate the game highlight video.

4. The method of claim 3, wherein, The generation of the game commentary script according to the start and end times, the description texts and the description keywords of each high-light segment comprises: Determination of a preset commentary style, and generation of script generation prompt words based on the preset commentary style and the start and end times, the description texts and the description keywords of each high-light segment; Input of the script generation prompt words into a large language model, generation of a game commentary script with the preset commentary style based on the start and end times, the description texts and the description keywords of each high-light segment by the large language model under the guidance of the script generation prompt words.

5. The method of claim 1, wherein, The high-light event detection on each key picture frame to obtain high-light event detection information of each key picture frame comprises: Determination of a high-light event detection strategy for the game process, and determination of a high-light event detection model according to the high-light event detection strategy; Input of each key picture frame into the high-light event detection model to obtain high-light event detection information of each key picture frame.

6. The method of claim 5, wherein, The high-light event detection model is a visual language model, and the input of each key picture frame into the high-light event detection model to obtain high-light event detection information of each key picture frame comprises: obtaining a highlight event detection prompt word for any of the key picture frames; inputting any of the key picture frames and the highlight event detection prompt word of any of the key picture frames into the visual language model, analyzing any of the key picture frames under the guidance of the highlight event detection prompt word through the visual language model, and outputting highlight event detection information of any of the key picture frames.

7. A game highlight video generation apparatus characterized by comprising: The device is applied to a cloud service platform, and the device comprises: an acquisition module, configured to acquire each key picture frame of a game client in real time during a game process, and perform highlight event detection on each key picture frame to obtain highlight event detection information of each key picture frame; an identification module, configured to identify start and end times of each highlight event in real time according to the highlight event detection information corresponding to each key picture frame; a clipping module, configured to clip highlight videos of each highlight event from a continuously generated game video stream in real time based on the start and end times of each highlight event, and perform video content understanding on the highlight videos of each highlight event to obtain video content understanding results of each highlight event; a generation module, configured to generate a game highlight video according to the video content understanding results of each highlight event and send the game highlight video to the game client when the game process ends.

8. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, The processor executes the computer program to implement the steps of the method in any one of claims 1 to 6.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method in any one of claims 1 to 6.

Images