Game highlight video generation method and device, equipment and medium

Abstract

The invention relates to a game highlight video generation method and device, equipment and a medium, and the method comprises the steps: obtaining an image frame sequence used for displaying the real condition of a game, and extracting a local image corresponding to a preset region from the image frames of the image frame sequence; performing text detection based on the local image, and identifying a highlight prompt text; determining an editing distance between the highlight prompt text and each highlight text template preset in a template library, and when the minimum value of the editing distance meets a preset condition, triggering a highlight event corresponding to the highlight text template with the minimum value; a highlight video segment is generated that includes the image frames that trigger the highlight event. According to the method, accurate and efficient capture and rapid generation of the game highlight video are optimized through an automation technology, and interaction of game live broadcast and audience experience are improved.

Description

Technical Field

[0001] The present application relates to network live broadcast technology, and in particular to a method for generating game highlight videos and its device, equipment, and medium. Background Art

[0002] With the rapid development of the digital entertainment industry, e-sports and online games have become a popular form of entertainment. Especially on online live broadcast platforms, game anchors have attracted a large number of viewers by sharing their game processes in real time. In these live broadcasts, the highlights of the game, such as key kills and reversal victories, are often the most anticipated and exciting parts for the audience. However, due to the real-time and continuous nature of game live broadcasts, these highlights are often fleeting and difficult to be effectively captured and recorded.

[0003] In the prior art, capturing game highlights mainly relies on manual operation or simple automatic recording methods. For example, the host or audience needs to pay attention to the game process at all times and manually start recording or screenshots, which not only distracts the players' attention, but also makes it difficult to ensure the integrity and accuracy of the highlights. In addition, although some automatic recording systems can continuously record the entire game process, the generated video files are huge and contain a lot of non-highlight content, making it inefficient for users to find and share them later.

[0004] To solve this problem, some automated recording technologies have been proposed. These technologies can automatically record the game process to a certain extent, but their role is limited. In general, the existing highlight video generation technology still faces many challenges, including: The first challenge involves accuracy: the traditional manual recording method relies on players to judge and capture highlight moments in real time, which not only places high demands on players' skills and reaction speed, but also easily misses or mistakenly captures non-highlight content. Although the automated recording system can record the game process uninterruptedly, it often lacks accurate recognition of highlight moments, resulting in the generated video containing a lot of irrelevant information, reducing the pertinence of the content.

[0005] The second challenge involves recognition efficiency: when processing game videos, existing automated systems ignore the resource limitations of computer devices and usually need to analyze the entire video stream to identify highlights, which not only consumes a lot of computing resources but also makes it difficult to meet real-time requirements. Especially in highly dynamic and rapidly changing games, how to quickly and accurately filter out highlights from a large amount of data is a technical challenge faced by existing technologies.

[0006] There are also challenges in terms of user experience: For viewers, the existing video recording and sharing process is often cumbersome and inefficient. Users need to pay attention to and remember the highlights in real time during the live broadcast, or spend a lot of time browsing and editing videos after the live broadcast to find and share these wonderful moments. This user experience not only affects the audience's viewing experience, but also limits the dissemination and sharing of highlight content. Summary of the invention

[0007] The purpose of this application is to solve the above-mentioned problems and to provide a game highlight video generation method and its corresponding device, equipment, non-volatile readable storage medium, and computer program product.

[0008] According to one aspect of the present application, a method for generating a game highlight video is provided, comprising the following steps: Acquire an image frame sequence for displaying a live game, and extract a local image corresponding to a preset area from the image frames in the image frame sequence; Perform text detection based on the local image to identify highlighted text; Determine the edit distance between the highlight prompt text and each highlight text template preset in the template library, and when the minimum edit distance meets the preset condition, trigger the highlight event corresponding to the highlight text template with the minimum edit distance; A highlight video clip including the image frame triggering the highlight event is generated.

[0009] According to another aspect of the present application, a device for generating a game highlight video is provided, comprising: An image acquisition module is configured to acquire an image frame sequence for displaying a live game, and extract a local image corresponding to a preset area from image frames in the image frame sequence; A text detection module, configured to perform text detection based on the local image and identify highlighted text; An event triggering module is configured to determine the edit distance between the highlight prompt text and each highlight text template preset in the template library, and when the minimum edit distance meets a preset condition, trigger a highlight event corresponding to the highlight text template having the minimum edit distance; The video generation module is configured to generate a highlight video segment including the image frame that triggers the highlight event.

[0010] According to another aspect of the present application, a game highlight video generation device is provided, including a central processing unit and a memory, wherein the central processing unit is used to call and run a computer program stored in the memory to execute the steps of the game highlight video generation method described in the present application.

[0011] According to another aspect of the present application, a non-volatile readable storage medium is provided, which stores a computer program implemented according to the game highlight video generation method in the form of computer-readable instructions, and when the computer program is called and executed by a computer, the steps included in the method are executed.

[0012] According to another aspect of the present application, a computer program product is provided, including a computer program / instruction, wherein when the computer program / instruction is executed by a processor, the steps of the method described in any one of the embodiments of the present application are implemented.

[0013] Compared with the prior art, this application has the following significant technical advantages and beneficial effects: First of all, this application can more accurately identify highlight prompt text by performing targeted text detection and edit distance calculation in a preset area. Compared with traditional full-screen analysis, this method reduces the possibility of misjudgment and missed judgment, ensuring the accurate capture of highlight moments. By matching with the preset highlight text template, even if there is a slight error in the text recognition process, the highlight event can be correctly identified through the fault-tolerant mechanism, thereby improving the overall recognition accuracy.

[0014] Secondly, this application reduces unnecessary calculations by limiting the analysis scope to a preset area and using an edit distance algorithm to quickly identify highlight events based on text and template matching, and can quickly respond to and process image data to achieve highlight event recognition. This approach not only reduces the computational burden, but also shortens the time for highlight moment recognition, meeting real-time requirements. In highly dynamic and rapidly changing gaming environments, this efficiency improvement is particularly critical, ensuring that even in the most exciting gaming moments, no highlight moments are missed.

[0015] Furthermore, this application greatly simplifies the user's operation process. Users no longer need to manually start recording or screenshots, nor do they need to spend a lot of time browsing and editing videos after the live broadcast. This application can automatically capture highlights and generate video clips, and users can directly share or replay these wonderful moments, greatly improving the user experience. The automated processing also allows viewers to focus more on the game itself and enjoy a smoother and more immersive viewing experience.

[0016] In addition, through the technical solution of this application, the generated highlight video clips are more compact and exciting, which is convenient for users to share on social platforms. This easy-to-share content format can not only attract more viewers and divert traffic to the live broadcast room of the network live broadcast, but also increase the interactivity and viewing of the game live broadcast, thereby improving the activity and participation of the entire game community. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1This is an exemplary network architecture suitable for applying the method for generating game highlight videos of the present application; Figure 2 A flowchart of an embodiment of a method for generating a game highlight video of the present application; Figure 3 This is a principle block diagram of the device for generating game highlight videos of the present application; Figure 4 This is a structural diagram of a game highlight video generation device used in this application. DETAILED DESCRIPTION

[0018] To facilitate understanding of various embodiments of the method for generating a game highlight video of the present application, an exemplary network architecture is first introduced. Figure 1 As shown, the exemplary network architecture includes multiple computer devices, including multiple servers and multiple terminal devices. Through the interaction between these devices, the smooth live broadcast of online game live and the instant generation of highlight videos can be ensured. For example, the network architecture is deployed with a game server 81, a live broadcast server 83 and a media server 85. At the same time, the terminal device 90 of the anchor user and the terminal device 92 of the audience user are allowed to access the network architecture to communicate with each server.

[0019] The game server 81 is responsible for processing the logical operation and state maintenance of the game to ensure the smooth progress of the game. In addition, the game server 81 also interacts with the terminal device 90 of the anchor user to transmit the dynamics of the game to the anchor user in real time, so that the anchor user can interact with the game in real time and show his game skills.

[0020] The media server 85 is responsible for processing the media data of the live broadcast room of the network live broadcast, mainly processing the video stream. In one embodiment, the media server 85 receives the real-time video stream generated by the anchor user, which may include the game video stream corresponding to the live game pushed by the game server 81 and the real-time video stream generated by the camera device. The media server 85 can decode the real-time video stream on demand, and implement various technical processing in the decoding space, including mixing, editing, identifying highlight events to generate highlight videos, etc. Finally, the media server 85 corresponds to each terminal device 90, 92 in the live broadcast room, and encodes and transmits the processed video stream to each audience user for playback. In another embodiment, the media server 85 can only receive the real-time video stream from the anchor user's terminal device 90, and receive the game video stream generated by the anchor user playing the game from the game server 81 that interacts with the anchor user, and then implement decoding, editing, identifying highlight events to generate highlight videos, and encoding video streams to output to the terminal device 92 of the audience user in the live broadcast room for playback and other related processing based on these two video streams.

[0021] The media server 85 can also be responsible for storing and managing video content, including game highlight video clips. It provides video upload, storage, transcoding and distribution services to ensure efficient management and fast distribution of video content. In this way, both anchor users and audience users can easily access and share these wonderful game moments.

[0022] The live broadcast server 83 is a bridge that maintains the operation of the live broadcast room to connect the anchor user and the audience user. To this end, the live broadcast server 83 is not only responsible for providing the users in the live broadcast room with the address for pulling the video stream from the media server 85, but also handles interactive functions related to the live broadcast, such as barrage, gifts and comments, thereby enhancing the interactivity of the live broadcast and the audience's sense of participation.

[0023] In such a network architecture, a computer program product that implements the game highlight video generation method of the present application can be flexibly deployed in different network nodes. For anchor users and audience users, the program product can be integrated into a network live broadcast program and deployed in their terminal devices 90, 92, such as smart phones, tablet computers, personal computers or game consoles. On the terminal devices 90, 92, the program can analyze game image frames in real time. These game image frames are usually used to present image frames corresponding to the live game, which can be obtained from the memory of the terminal devices 90, 92, and the highlight moments are identified accordingly, and highlight video clips are automatically generated, providing users with instant viewing experience and sharing opportunities.

[0024] In addition, the computer program product can also be directly deployed in the media server 85 for centralized processing and generation of highlight videos. On the media server 85, the program can process a large number of video streams from different anchors, identify and generate highlight video clips, and then store them on the server for users to access and share. This deployment method not only improves the efficiency of video processing, but also provides powerful content management and distribution capabilities for the live broadcast platform.

[0025] In one embodiment, the host user may also run a stand-alone game in his terminal device 90. Similarly, the computer program product of the present application may also identify highlight videos based on image frames corresponding to the live game of the stand-alone game and generate corresponding highlight video clips.

[0026] From the above exemplary description of the network architecture, it can be seen that whether on the terminal device 90, 92 or on the media server 85, the viewing experience and interactivity of the live game broadcast can be effectively improved, while providing users with a richer and more convenient way to share content.

[0027] See also Figure 2According to a method for generating a game highlight video provided by the present application, the method can be implemented as a computer program product, which is installed and run in a terminal device or a server to provide a game highlight video generation system. In some embodiments thereof, the method includes the following steps: Step S3100: obtaining an image frame sequence for displaying a live game, and extracting a local image corresponding to a preset area from the image frames of the image frame sequence; Game live broadcast is the visual and auditory performance of a series of activities and events that occur in real time when players, such as anchor users, participate in the game. These activities and events include player operations, changes in the game interface, in-game sound effects, and the development of the game plot.

[0028] On the live broadcast platform, the live game is usually presented to the audience in the form of a video stream. The anchor user plays the game through the terminal device, and transmits the game screen and sound to the live broadcast platform in real time. The audience users can watch these live game video streams through the terminal device. The video stream of the live game may include the game screen, the anchor's camera screen, and the anchor's commentary sound. The game screen in the video stream of the live game will be processed in each network node, for example, in the game server, in the terminal device of the anchor user as a player, in the terminal device of the audience user in the live broadcast room of the anchor user, and in the media server of the live broadcast room. The video stream will be processed. Therefore, any network node can obtain a video stream used to display the live game. In the corresponding network node computer device, the video stream will be decoded to obtain an image frame sequence, which contains each image frame. In each network node, the computer device belonging to the terminal device can also render and display each image frame in sequence, so that the user can watch the corresponding live game. It is not difficult to understand that even a terminal device running a stand-alone game can obtain each image frame corresponding to the live game from the video memory and obtain a corresponding image frame sequence.

[0029] The image frame sequence corresponding to the live game is the basic unit of video continuity, and each image frame is a snapshot of the real-time state of the game. In order to extract the local images corresponding to the preset areas from these image frames, these preset areas need to be defined first. The preset area refers to the area pre-defined in the game interface where highlight prompt text can pop up, such as the scoreboard, kill notification, game status information, etc. These preset areas can be determined in advance based on the characteristics of the game and the experience of the players, and these preset areas are represented as coordinate data in advance for direct call. In addition, users can adjust these coordinate data to achieve customized adjustment of the preset areas.

[0030] When extracting the local image corresponding to the preset area, the image frames in the image frame sequence need to be analyzed so as to subsequently identify and locate the text information in the preset area according to the corresponding local image. Once the location of the preset area in the image frame is determined, the local images of these areas can be cropped from the image frame.

[0031] In some embodiments, the preset area can be set singly, for example, only the scoreboard in the game interface is focused on and the preset area where the scoreboard is located is set. In this case, the system will specifically extract the image frame portion corresponding to the scoreboard according to the preset area. In other embodiments, considering that different highlight events may pop up their corresponding highlight prompt texts at different positions of the image frame, multiple preset areas can be set corresponding to the areas where these highlight prompt texts may appear. In this case, the system will process each preset area in turn and extract partial images of all relevant areas from the image frame.

[0032] There may also be different embodiments for extracting a local image corresponding to a preset area from an image frame in an image frame sequence, for example: In the first part of the embodiments, the extraction process can be continuous or intermittent. In the embodiment of continuous extraction, the system processes each image frame in the image frame sequence to extract the local images of all preset areas. In the embodiment of intermittent extraction, the system can only process specific image frames in the image frame sequence, for example, extracting once every frame or within a specific time interval to determine the corresponding image frame, and then using this image frame to determine the local images corresponding to each preset area.

[0033] In the second part of the embodiments, a distribution extraction mode can be applied. Specifically, multiple preset areas can be distributed to each continuous image frame of the image frame sequence, so that the preset areas correspond to the image frames one by one, and each preset area extracts the corresponding local image from its corresponding image frame. In these continuous image frames, although each preset area obtains its corresponding local image, these local images come from different image frames, not the same image frame. Adapting to this part of the embodiments, it is not difficult to understand that since the image frame sequence continuously displays the image frames corresponding to the live game, it is also possible to continuously poll each preset area to continuously determine its corresponding image frame and then determine the corresponding local image.

[0034] In other embodiments, the above-mentioned multiple methods can be combined for flexible implementation. For example, even in each continuous image frame mentioned in the second part of the embodiments, although each preset area corresponds to an image frame in the continuous image frames, it can still be separated by a subsequent image frame, skipping this subsequent image frame and then determining the local image corresponding to the next preset area in the next image frame.

[0035] Since current game screens have a relatively high frame rate, for example, up to 60-120Hz, there are a large number of image frames per second, and the duration of a single image frame is extremely low, while the highlight prompt text corresponding to the highlight event often takes a relatively long time. In this case, in the above multiple embodiments, whether it is a method of extracting local images for preset areas by using interval image frames, or a method of distributing multiple preset areas to each continuous image frame and extracting local images for each preset area in different image frames, it can effectively identify local images containing highlight prompt texts, and at the same time, it can also greatly save system overhead. This has economies of scale for scenarios where the server side processes massive games, and for terminal devices with limited system resources, it can save system overhead and improve response rate.

[0036] On the terminal device, such as the host user's computer or game console, the process of extracting the local images corresponding to each preset area from the image frame sequence can be carried out in real time, thereby ensuring that the highlight moments can be captured immediately. Similarly, on the media server, the video stream can also be batch processed to identify and extract the highlight moments. Whether on the terminal device or on the server, the extracted local images will be used for subsequent text detection and highlight event recognition to generate highlight video clips.

[0037] In some embodiments, the system can determine whether to use continuous extraction or intermittent extraction of image frames based on the type of game to which the game belongs. For fast-paced games that require real-time capture of every action, such as e-sports competitions, the system can use a continuous extraction mode, that is, processing each frame in the image frame sequence to ensure that no highlight moments are missed. Although this method has a high demand for computing resources, it can provide the most complete visual data, thereby improving the recognition accuracy of highlight moments.

[0038] On the contrary, on resource-constrained terminal devices, or for game types that do not have very strict real-time requirements, the system can use interval extraction mode. In this mode, the system will extract image frames every certain number of frames or time intervals, for example, you can choose to extract every 5 frames or every 2 seconds. This strategy reduces the amount of data processed and reduces the consumption of computing resources, while still being able to effectively identify and record highlight events.

[0039] For the application of interval extraction mode, in order to achieve the best balance between resource utilization and real-time performance, the system can also dynamically adjust the extraction strategy based on the real-time monitored game activities or computing loads. For example, when the possibility of highlight events is detected to increase with the help of the pre-trained highlight event trend prediction model, the system can automatically switch to a more intensive extraction mode and increase the image frame extraction frequency, that is, correspondingly reduce the interval frame number of image frames used to extract the local image corresponding to the preset area; when the game activity is relatively stable, the extraction frequency is reduced and the interval frame number is increased accordingly. For another example, the system can also use a preset formula to evaluate the performance score of the terminal device based on its performance indicators, such as processor load and memory usage, and intelligently adjust the extraction frequency based on the performance score to ensure that the system performance is optimized without sacrificing the user experience.

[0040] Step S3200: performing text detection based on the local image to identify highlighted text; The technology of detecting text in a local image to identify highlighted text usually includes two stages. The first stage is to identify the outline of the text information in the local image to determine the outer frame of the text information. The concentrated text image is obtained by taking a screenshot in the local image according to the outer frame, so that the concentrated text image can remove the surrounding stray image data relative to the local image, and concentrate the image data corresponding to the text information, which helps the second stage to more efficiently and accurately identify the text information in the concentrated text image as highlighted text, and in computer devices such as terminal devices with relatively limited system resources, it can better reflect the efficiency advantage. The second stage can directly use the concentrated text image as the basis, and use various feasible text recognition technologies to realize the conversion of text information into highlighted text.

[0041] The first stage can be implemented in a variety of embodiments as needed to obtain a text mask corresponding to the indicated text information. According to the text mask, the outer frame of the text information can be accurately determined and a partial image can be captured to obtain a text concentrated image. For example: In the first embodiment, a text edge detection model based on deep learning is used, which can identify text areas in an image and generate corresponding text masks. The text mask is a binary version of a local image, in which the text area is marked as white (or a pixel with a value of 1), and the non-text area is marked as black (or a pixel with a value of 0). This mask allows the subsequent text recognition process to focus only on the area containing the text, thereby improving the accuracy and efficiency of recognition. This method can strike a balance between accurately determining the location of text information and making rational use of system resources, and can be implemented in various computer devices.

[0042] In the second embodiment, when the local image contains only text information and does not contain other objects such as images of game characters, weapons, etc., traditional image processing techniques, such as edge detection and connected domain analysis, can be used to identify text areas. By identifying edge features and connected text components in the local image, the system can construct a text mask. Although this method is not as accurate and end-to-end as deep learning-based methods, it is more efficient on resource-constrained devices.

[0043] The text mask is data used to indicate the location of text information. For example, if the local image contains the text of the game score "100 PTS", the text mask will only include the area of ​​this string of numbers and letters, and ignore the rest of the image. Even if the same highlight event may generate text of different lengths in different game live broadcasts, such as "First Blood" for the first kill or "Mega Kill" for consecutive kills, although they have different numbers of characters, resulting in differences in the actual area occupied by the corresponding text information, the text mask can accurately indicate the actual area where the actual text information is located in the image frame.

[0044] The second stage focuses on identifying the text information contained in the images in the text set as highlighted text. The implementation scheme of this stage can adopt different technical paths to achieve efficient and accurate text recognition.

[0045] In one embodiment, a deep learning model is used to process images in a text set. These models, such as convolutional neural networks (CNNs) combined with recurrent neural networks (RNNs), and models using connection temporal classification (CTC), are trained to recognize text in images and convert them into machine-readable text formats, which can be used as highlight prompt text. The advantage of deep learning models is that they can handle complex image conditions, such as different fonts, sizes and arrangements, as well as background noise. During the processing, the model first extracts features from the image with the help of CNN to identify the shape features of text lines and characters. Then, deep semantic features are extracted through RNN. Finally, classification prediction is performed through CTC to convert these features into text sequences to achieve conversion from image to text. This method is generally superior to traditional OCR technology in recognition rate and processing speed, and is particularly suitable for text recognition in complex or low-quality images. It can significantly improve the accuracy of identifying highlight prompt text and can be implemented in various computer devices.

[0046] In another embodiment, a more lightweight text recognition method can be used on a resource-constrained terminal device. In this case, the system uses a lightweight OCR engine, such as an algorithm based on template matching or feature matching, to recognize the text in the images in the text set based on the images in the text set. Although this method is slightly inferior to the deep learning method in terms of accuracy, it has a lower computational cost and is suitable for implementing real-time text recognition on performance-constrained devices.

[0047] During the game, as the image frames in the image frame sequence are streamed, a large number of local images will be generated through the detection mechanism of step S3100, and it is necessary to identify text and highlight events for a large number of local images until the corresponding highlight video clips are finally generated. Therefore, in one embodiment, a method based on an event response mechanism is used to continuously generate local images according to the image frame sequence. Specifically, every time a local image is extracted from a preset area, the system automatically creates a corresponding working thread. This working thread is specifically responsible for performing text detection, recognition and determination of highlight events on the local image, that is, the process from step S3200 to step S3300, and even this working thread can further include the execution of step S3400. By assigning an independent working thread to each local image, the system can process multiple local images in parallel, thereby improving the overall processing efficiency and response speed. When this mechanism is combined with the interval extraction mode and / or distribution extraction mode of step S3100, the technical advantages in processing efficiency and response speed can be more prominent, and the effect of quickly and smoothly identifying highlight events in real time to generate corresponding highlight video clips can be achieved in both the server and the terminal device.

[0048] It should be pointed out that when there is no text information in the local image, the subsequent processing of the corresponding local image can be terminated and it can be directly deleted from the memory to further save system overhead.

[0049] Step S3300: determining the edit distance between the highlight prompt text and each highlight text template preset in the template library, and when the minimum edit distance meets a preset condition, triggering a highlight event corresponding to the highlight text template having the minimum edit distance; Once the highlighted text is identified from the partial image, it can be matched with the preset highlighted text template. This process involves calculating the edit distance between the text and the template to determine the similarity between them. The edit distance, also known as the Levenshtein distance, is measured by counting the minimum number of single-character edits (insertions, deletions, or substitutions) required to transform one string into another.

[0050] In order to perform this step, a template library is first prepared, which contains multiple preset highlight text templates. Each highlight text template corresponds to a highlight event. When specifically expressed, the highlight event identifier and the highlight text template can be mapped and stored in the template library. These templates represent the text representation of various highlight events that may appear in the game, such as "First Blood", "Multi-Kill" or "Victory". Each template is pre-defined based on the characteristics of the game and the experience of the players.

[0051] When applying the edit distance algorithm to calculate the edit distance between the recognized highlight hint text and each highlight text template in the template library, it can be achieved through dynamic programming, in which a matrix is ​​constructed to record and update the minimum cost of different editing operations. Ultimately, a specific value in the matrix will represent the minimum number of edits required to convert the recognized text into a certain template, that is, the final edit distance.

[0052] Taking the identified highlight prompt text "Wiotory" and the highlight text template "Victory" in the template library as an example, the process of calculating the edit distance between them is as follows: First, we can use a matrix to record the transformation steps between the two strings "Wiotory" and "Victory". The size of this matrix is ​​(m+1) x (n+1), where m is the length of the recognized text "Wiotory" (7 characters) and n is the length of the template text "Victory" (7 characters). The rows of the matrix represent each character in "Wiotory" and the columns represent each character in "Victory", filling from left to right and from top to bottom.

[0053] The first row and column of the matrix are initialized to the cumulative number of steps required to transform an empty string into one of these strings. For example, the values ​​of the first row of the matrix from left to right are 0, 1, 2, ..., 7, representing the number of insertions required to gradually add each character in "Wiotory". Similarly, the values ​​of the first column from top to bottom are 0, 1,2, ..., 7, representing the number of insertions required to gradually add each character in "Victory".

[0054] Next, compare the characters in the two strings one by one and fill in the rest of the matrix. If the two characters are the same, the value of the corresponding cell in the matrix is ​​the value of the upper left cell; if they are different, then you need to consider insertion, deletion, or replacement operations. After each comparison, the system will select the minimum value of the three operations and add 1 to that value as the value of the current cell.

[0055] In the matrix corresponding to the edit distances calculated for "Wiotory" and "Victory", the value of each cell represents the minimum number of edit steps required to transform one string into another. For example, the value of the last cell in the matrix is ​​2, which is the edit distance between the two words, indicating that it takes 2 edit operations to transform "Wiotory" into "Victory": replacing 'W' with 'V' and replacing 'o' with 'c'.

[0056] After calculating the edit distances of all highlight text templates corresponding to the highlight prompt text, determine the smallest edit distance among the highlight text templates as the minimum value, and check whether it meets the preset conditions. These preset conditions can be preset thresholds corresponding to the metric edit distance. For example, if the edit distance of two strings is less than or equal to the preset threshold of 3, they are considered similar enough to trigger the corresponding highlight event. If the minimum value also fails to meet this preset condition, it means that the corresponding highlight prompt text cannot be matched to a suitable highlight event, and the subsequent steps will no longer be processed. Once a highlight text template that meets the conditions is found, its corresponding highlight event can be marked and used to generate a highlight video clip.

[0057] According to the above description and examples, in the scenario of the present application, the application of the edit distance algorithm to achieve the matching of the highlight text template and the highlight prompt text can significantly improve the accuracy and efficiency of the recognition of the highlight moments of the game. By calculating the edit distance between the highlight prompt text and the preset template, the system can quantify the similarity between the two, and accurately identify the highlight event even if there is a slight error in the text recognition process or the text is slightly different. This method not only improves the reliability of highlight moment capture, but also allows the system to flexibly handle various game text prompts, including texts of different lengths and formats. In addition, the edit distance algorithm takes up little system resources and is efficient in operation. Its application enables the system to quickly filter out the most matching template from a large number of highlight text templates, thereby quickly generating highlight video clips in real-time live broadcasts or game recordings, enhancing the audience's viewing experience and the interactivity of the content. In this way, the technical solution of the present application can effectively process and generate video content related to game highlight moments without sacrificing accuracy, providing users with a richer and more personalized entertainment experience.

[0058] Step S3400: Generate a highlight video clip including the image frame that triggers the highlight event.

[0059] Once it is determined that the edit distance between the highlight prompt text and a highlight text template in the template library meets the preset conditions, the system triggers the corresponding highlight event. The triggering of the highlight event is based on the accuracy of the text match and the user-defined threshold, ensuring that only moments that meet specific criteria are identified as highlight moments.

[0060] The triggered highlight event can carry key information, such as the highlight event identifier, the timestamp of the image frame corresponding to the triggering highlight event, etc. In addition, it can also contain other contextual information related to the generation of highlight video clips as needed. This key information is used to accurately locate and extract image frames related to highlight events from the image frame sequence of the live game. In response to these highlight events, the system records the specific moment when the highlight event is triggered, and traces back to the image frame sequence based on these timestamps. The image frame sequence containing the highlight moment is obtained according to a certain duration, such as 15 seconds. For example, according to the timestamp of the image frame corresponding to the triggering highlight event, the image frames of the first 5 seconds and the last 10 seconds are obtained, and the corresponding highlight video clip can be generated.

[0061] In some embodiments, if multiple highlight events occur in almost the same time period, for example, the highlight moments of multiple highlight events all fall within a predetermined time period, the system may treat these highlight events as concurrent highlight events, and obtain a sequence of image frames corresponding to a single highlight video clip based on the time period corresponding to the concurrent highlight events, thereby generating a single highlight video clip. This merging process allows the system to generate a single highlight video clip that contains image frames corresponding to multiple highlight moments, thereby providing a richer and more coherent viewing experience. The merging strategy may be implemented based on temporal proximity, event relevance, or other user-defined criteria.

[0062] The generated highlight video clips can be applied in a variety of scenarios. For example, they can be directly shown to the audience in the live broadcast, providing instant highlight playback. In addition, these highlight video clips can be stored on the media server for users to watch later or share on social media platforms to increase the dissemination and interaction of game content. Highlight video clips can also be used by content creators in their editing work as materials for making highlights or review videos.

[0063] In addition, the generation and application of highlight video clips can further enhance the activity of the game community, encourage players to share and discuss their wonderful moments, thereby improving the overall appeal of the game and the participation of players. Through automated highlight video generation, the technical solution of this application provides users with a novel, efficient and user-friendly way to capture and share wonderful moments in the game.

[0064] According to the above embodiments, it can be seen that the present application has the following significant technical advantages and beneficial effects: First of all, this application can more accurately identify highlight prompt text by performing targeted text detection and edit distance calculation in a preset area. Compared with traditional full-screen analysis, this method reduces the possibility of misjudgment and missed judgment, ensuring the accurate capture of highlight moments. By matching with the preset highlight text template, even if there is a slight error in the text recognition process, the highlight event can be correctly identified through the fault-tolerant mechanism, thereby improving the overall recognition accuracy.

[0065] Secondly, this application reduces unnecessary calculations by limiting the analysis scope to a preset area and using an edit distance algorithm to quickly identify highlight events based on text and template matching, and can quickly respond to and process image data to achieve highlight event recognition. This approach not only reduces the computational burden, but also shortens the time for highlight moment recognition, meeting real-time requirements. In highly dynamic and rapidly changing gaming environments, this efficiency improvement is particularly critical, ensuring that even in the most exciting gaming moments, no highlight moments are missed.

[0066] Furthermore, this application greatly simplifies the user's operation process. Users no longer need to manually start recording or screenshots, nor do they need to spend a lot of time browsing and editing videos after the live broadcast. This application can automatically capture highlights and generate video clips, and users can directly share or replay these wonderful moments, greatly improving the user experience. The automated processing also allows viewers to focus more on the game itself and enjoy a smoother and more immersive viewing experience.

[0067] In addition, through the technical solution of this application, the generated highlight video clips are more compact and exciting, which is convenient for users to share on social platforms. This easy-to-share content format can not only attract more viewers and divert traffic to the live broadcast room of the network live broadcast, but also increase the interactivity and viewing of the game live broadcast, thereby improving the activity and participation of the entire game community.

[0068] On the basis of any embodiment of the method of the present application, extracting a local image corresponding to a preset area from the image frames of the image frame sequence includes: Step S3110, obtaining a plurality of preset areas corresponding to the game settings to which the game live belongs; In order to identify the highlight prompt text, it is necessary to determine the specific areas in the game's image frame that may display highlight moment information. These areas usually include scoreboards, kill notifications, game status information, etc., which are the places where text information is most likely to appear when players achieve key achievements in the game.

[0069] To achieve this step, the relevant users can preset these areas and express them as corresponding coordinate data. First, the relevant users need to have a deep understanding of the game interface layout, including the general position of the text information of the highlight event on the screen. Next, based on the analysis of the game interface, define the areas that may contain text information corresponding to the highlight moment. When defining these areas, the information of each preset area, such as coordinates, size, etc., will be stored in the corresponding data structure as coordinate data.

[0070] In addition, considering that the game may be updated and the display method of highlight events may change accordingly, the definition of preset areas can be regularly updated and maintained. In order to improve the adaptability and user satisfaction of the method, players can also be allowed to customize these preset areas according to their own game experience and preferences.

[0071] Step S3120, polling each preset area in turn, and corresponding to each preset area, extracting a local image corresponding to the selected target image frame from the image frame sequence according to the image frame arrangement order; This embodiment uses a distributed extraction mode to extract the local image corresponding to each preset area. Through this mode, it ensures that in the image frame sequence of the game live, each preset area can be extracted one by one in the corresponding image frame. In this process, the image frame arrangement order in the image frame sequence is processed, and the local image of each preset area is extracted from the continuous image frames in sequence.

[0072] Specifically, the system first initializes the polling mechanism and prepares to traverse all preset areas. Then, the system selects the first preset area as the current preset area, and uses its coordinate data such as coordinates and size information to extract a local image from the current image frame. This current image frame is the target image frame corresponding to the first preset area at this moment. This local image contains all the visual information of the current preset area in the current game state.

[0073] After completing the partial image extraction of the current preset area, the system will move to the next image frame in the image frame sequence as the target image frame corresponding to the second preset area, and extract the corresponding partial image in the new target image frame with the second preset area as the current preset area. This process is the same as the operation of the first preset area, but for different preset areas. The system continues to extract the corresponding partial image in each subsequent image frame in the list order of the preset areas until all preset areas have been processed. If the game is still in progress, the system will loop back to the first preset area and restart polling to ensure that continuous game live broadcasts are continuously monitored.

[0074] Step S3130: extracting from the target image frame a local image corresponding only to the preset area currently polled.

[0075] In step S3120, the system has determined the corresponding target image frame of each preset area in the image frame sequence one by one according to the list order of the preset areas. In step S3130, the system will use the coordinates and size information of the preset area to accurately crop a partial image for each such target image frame. This partial image only contains the visual content of the currently polled preset area, thereby providing accurate image data for subsequent text detection and recognition.

[0076] In order to efficiently perform this step, the system can adopt a method based on an event response mechanism. Whenever a new target image frame and the current preset area are determined in step S3120, the system will automatically trigger an event and create a dedicated thread to handle this event. This thread will be responsible for extracting the local image corresponding to the current preset area from the target image frame. By assigning independent working threads to the extraction tasks of each preset area, the system can process multiple extraction tasks in parallel, thereby significantly improving the processing speed and real-time performance.

[0077] In addition, this event response mechanism-based approach also allows the system to flexibly manage computing resources. For example, if the computing load of the game is high at a certain moment, the system can dynamically adjust the number or priority of worker threads to ensure that critical extraction tasks can be completed in time while avoiding excessive impact on system performance.

[0078] In the embodiment defined by applying steps S3110 to S3130, the introduction of the polling mechanism brings significant improvements, especially in terms of efficiency and accuracy when processing real-time game image data. This mechanism ensures that each preset area can be checked and processed at the appropriate time by associating the preset areas with consecutive image frames one by one. The implementation of the polling mechanism enables the system to extract local images from each image frame in an orderly and non-overlapping manner, which not only optimizes the processing flow, but also improves the accuracy of capturing highlight moments.

[0079] Through this frame-by-frame polling method, step S3120 ensures that each preset area can be accurately extracted in its corresponding image frame. This method not only improves the processing efficiency, but also reduces the instantaneous peak demand for computing resources by distributing the processing requirements to multiple image frames, thereby helping to maintain the smooth operation of the entire system. In addition, this time-sharing extraction strategy also helps to improve the real-time performance of the system, because it allows the system to prepare for the extraction of the preset area of ​​the next image frame while processing the current image frame, thereby achieving rapid response and continuous monitoring. This strategy is particularly suitable for rapidly changing gaming environments, ensuring that highlight moments are not missed.

[0080] The flexibility of the polling mechanism is also reflected in its ability to adapt to different games and different game updates. Since the manifestation of the game's highlight events may vary from game to game, or even change with updates in the same game, the polling mechanism allows the preset areas and extraction strategies to adjust according to these changes without significantly modifying the system. This adaptability ensures the stability and reliability of the system in different scenarios.

[0081] Finally, the polling mechanism further improves the system's parallel processing capabilities through event-based thread processing. Each extraction task can be executed in parallel independently of other tasks, allowing the system to use computing resources more efficiently while maintaining the ability to capture highlights in real time. This event-based response mechanism enables the system to flexibly manage computing loads and optimize performance, especially in high-load or resource-constrained environments.

[0082] Based on any embodiment of the method of the present application, performing text detection based on the local image to identify the highlighted text includes: Step S3210, inputting the local image into a text edge detection model to obtain a text mask; In order to accurately determine the edge contour information of the text information in the local image, this embodiment is implemented by using a text edge detection model that has been trained to convergence in advance. The text edge detection model can adopt various feasible model structures, such as the DBNet structure, which is a deep learning model designed specifically for text detection and has the advantages of high efficiency and accuracy. The local image is input into the text edge detection model, and the ability acquired during the model training is used to directly obtain the text mask representing the pixel position where the text information in the local image is located.

[0083] The network architecture of the DBNet model includes a feature extraction network, a probability prediction branch, and a threshold prediction branch. The feature extraction network uses a convolutional neural network (CNN) to extract features from the input local image and generate deep semantic feature maps. These feature maps capture the key visual information in the image and provide a basis for subsequent text region recognition. Next, the probability prediction branch predicts the probability of each pixel belonging to text based on these feature maps and generates a probability map. At the same time, the threshold prediction branch predicts the decision threshold for each pixel and generates a threshold map. The outputs of these two branches are then fused, and a fusion network is used to determine whether each pixel in the local image belongs to text, and finally a text mask is generated.

[0084] Step S3220, extracting a text-focused image from the local image according to the text mask; In this embodiment, the text mask is a binary version of the local image, in which the area where the text information is located is marked, providing accurate area positioning for subsequent text extraction. Using this text mask, the system can extract the text-focused image, that is, the image area containing text information, from the local image. This step is accomplished by applying the text mask to the original local image and intercepting the area marked by the mask, thereby removing the other parts of the image that do not contain text, preparing clearer and more concentrated data for text recognition.

[0085] Step S3230: input the text set image into a text recognition conversion model to obtain a highlighted prompt text.

[0086] The images in the text set are input into a text recognition conversion model, which adopts various feasible model structures, such as the CRNN structure, which is a deep learning model that combines convolutional neural network (CNN), recurrent neural network (RNN) and connection temporal classification (CTC) loss function. CNN is used to extract the features of text images, and RNN, especially bidirectional long short-term memory network (BLSTM), is used to process these feature sequences and predict text labels. The CTC loss function allows the model to learn in text label sequences without clear segmentation, which improves the flexibility and accuracy of recognition.

[0087] Through the processing of the CRNN model, the text information in the images in the text set is converted into a machine-readable text format, namely the highlight prompt text. This text information can then be used for further analysis and processing, such as matching with a preset highlight text template to determine whether a highlight event is triggered.

[0088] By combining the text edge detection model and the text recognition conversion model described in step S3210 to step S3230, efficient recognition of highlight prompt text in the local image is achieved. The text edge detection model adopts the DBNet structure, and uses its feature extraction network and dual-branch prediction mechanism to accurately generate text masks and effectively identify the text area in the image. This process not only improves the speed and accuracy of text detection, but also optimizes the overall processing efficiency by reducing the amount of data to be analyzed in subsequent processing steps. Subsequently, the text concentrated image extracted from the local image using the text mask provides clear and concentrated text image data for the text recognition model, thereby further improving the recognition accuracy. The text recognition model of the CRNN structure can process image feature sequences and accurately predict text labels through the combination of its convolutional and recurrent neural networks, while the application of the CTC loss function enables the model to flexibly process text sequences of indefinite length and improves its adaptability to text changes. The combined use of these two models not only improves the accuracy of text detection and recognition, but also improves the processing speed and overall performance of the system through parallel processing and optimized algorithm design, making automated highlight moment capture and recognition more efficient and reliable. This technical advantage enables the system to better serve the real-time gaming environment and provide users with accurate and instant game highlight video generation services.

[0089] Based on any embodiment of the method of the present application, taking the DBNet network as an example, the local image is input into a text edge detection model to obtain a text mask, including: Step S3211, extracting deep semantic information from the local image using the feature representation network of the text edge detection model to obtain a semantic feature map; The feature representation network is responsible for extracting deep semantic information from local images. This network is usually composed of multiple convolutional layers, which can deeply understand the image content, identify and understand the visual patterns and structures in the image, and generate semantic feature maps rich in text information.

[0090] Step S3212: predicting a probability map based on the semantic feature map by the first branch network of the text edge detection model, wherein the probability map represents the probability that each pixel in the local image belongs to the text; The first branch network uses these semantic feature maps to predict a probability map. This probability map is a single-channel image in which each pixel value represents the probability that the pixel at the corresponding position belongs to text. This step is achieved by applying a series of convolutional layers and activation functions, and the final output is a continuous value map reflecting the possibility of the existence of text.

[0091] Step S3213: predicting a threshold map based on the semantic feature map by the second branch network of the text edge model, wherein the threshold map represents a decision threshold corresponding to the probability that each pixel in the local image belongs to the text; The second branch network also works based on the semantic feature map, but its purpose is to predict the threshold map. Unlike the probability map, the threshold map represents the decision threshold of the probability that each pixel belongs to the text. This threshold is used as the criterion for determining whether the pixel belongs to the text in the subsequent steps. This process also involves a deep network structure, which can predict the appropriate threshold based on the text features learned from the training data.

[0092] Step S3214: The fusion network of the text edge detection model determines whether each pixel in the local image belongs to text according to the threshold map and the probability map, and represents the corresponding pixel as a text mask.

[0093] The fusion network combines the probability map and the threshold map to generate the final text mask. This mask is a binary image where each pixel is either 0 (indicating that the pixel does not belong to text) or 1 (indicating that the pixel belongs to text). The fusion network determines the final classification of each pixel by comparing the probability map and the threshold map. This step is the key to achieving accurate text positioning because it directly affects the accuracy and effect of subsequent text recognition.

[0094] It is not difficult to understand that the text edge detection model built by ourselves according to the above architecture can also achieve the same effect as DBNet.

[0095] Through this series of steps, the text edge detection model can accurately separate the text area from the complex image background and generate a clear text mask. This mask not only identifies the exact location of the text, but also provides a high-quality input image for subsequent text recognition, thereby improving the efficiency and accuracy of the entire text detection and recognition process. This precise text detection mechanism helps to automatically capture game highlights because it ensures that text information can be accurately identified and used to trigger highlight events, thereby generating fascinating game highlight videos.

[0096] Based on any embodiment of the method of the present application, the text set image is input into a text recognition conversion model to obtain a highlight prompt text, including: Step S3231, determining the outer frame of the local image where the text is located according to the text mask, and extracting the text set image corresponding to the outer frame from the local image; In order to achieve text recognition, it is necessary to first determine the location of the text in the local image based on the text mask. The text mask is a binary image that identifies the exact area where the text exists. The system uses this mask to determine the outer frame of the text, that is, the minimum rectangular boundary of the text. Through this outer frame, the system can cut out the image area containing the text from the local image, that is, the text concentrated image.

[0097] Step S3232: extracting image feature information from the images in the text set by a feature extraction network in the text recognition conversion model; Next, the feature extraction network of the text recognition conversion model processes the images in the text set and extracts image feature information. This network is usually composed of a multi-layer convolutional neural network, such as the CNN in the CRNN model structure in the previous example, which can identify local features in the image, such as edges, corners, and textures, and integrate these features into higher-level feature representations. These high-level features can better represent the visual pattern of the text. The output of the feature extraction network is a feature map, which encodes the key information of the text image and provides a rich data foundation for text recognition.

[0098] Step S3233: The sequence prediction network in the text recognition conversion model predicts the image feature information to determine the highlighted prompt text.

[0099] The sequence prediction network predicts the extracted image feature information to determine the highlight prompt text. The sequence prediction network can use a recurrent neural network (RNN), especially a bidirectional long short-term memory network (BLSTM), to process feature sequences to predict text labels. Since the highlight prompt text in the game may be expressed in different languages ​​and formats, the sequence prediction network can learn the sequence pattern of the text and map it to the corresponding characters or words. In addition, by using the connection temporal classification (CTC) loss function, the model is able to handle the recognition problem of indefinite length text sequences and make accurate predictions even if the characters in the text are not clearly segmented. In this way, the system can identify the text content from the image features and convert it into a machine-readable text format, namely the highlight prompt text.

[0100] For example, if a highlight moment such as "First Blood" appears in the game, the text edge detection model will first identify the image area containing these words and generate a corresponding text mask. Then, based on this mask, the system will extract the image area containing the "First Blood" text from the live game image. This text set image is then input into the text recognition conversion model. The model identifies the text features in the image through its feature extraction network and converts these features into text information such as "First Blood" through the sequence prediction network. Ultimately, this information can be used to trigger highlight events and generate corresponding highlight video clips.

[0101] The embodiment defined by applying steps S3231 to S3233 has achieved significant progress at the technical level, especially in terms of the accuracy and efficiency of text recognition. By accurately determining the text frame based on the text mask and intercepting the image in the text set, this embodiment ensures that the text recognition conversion model only processes the image area containing the text, thereby reducing the interference of background noise and improving the accuracy of recognition. The feature extraction network uses deep learning technology to extract key visual features from the images in the text set, providing a rich information basis for subsequent text recognition. The sequence prediction network combines a bidirectional long short-term memory network and a connection time series classification loss function to effectively process feature sequences and accurately predict text labels, and can accurately recognize even when there is no clear segmentation between characters in the text. The combined use of these two models not only improves the flexibility and accuracy of text recognition, but also optimizes the processing speed and the overall performance of the system, making the automated highlight moment capture and recognition more efficient and reliable. This technical advantage enables the system to better serve the real-time gaming environment, provide users with accurate and instant game highlight video generation services, and at the same time improve the viewing and interactivity of game content.

[0102] Based on any embodiment of the method of the present application, before the step of performing text detection based on the partial image and identifying the highlighted prompt text, the method further includes: Step S4100, creating a corresponding working thread in response to a determination event of each partial image, each working thread is responsible for executing text detection for each partial image to obtain a highlight prompt text, and determining a highlight event based on the highlight prompt text; In this embodiment, in order to improve the efficiency of text detection and highlight event recognition, the system uses a worker thread to process multiple local images in parallel. Specifically, before performing text detection to recognize highlight prompt text, a corresponding worker thread is created for each local image. This process can be achieved by monitoring the determination event of the local image. Whenever a new local image is extracted and ready for processing, the corresponding determination event is triggered, and the system responds to this event and generates a new worker thread.

[0103] The creation of worker threads can be implemented in a variety of programming languages ​​and frameworks. For example, in Java, the Thread class or ExecutorService interface can be used to create and manage threads, and in C#, the Thread class or Task class can be used to implement similar functions. When creating a worker thread, the local image needs to be passed as a parameter to the execution function of the corresponding worker thread so that the worker thread can access and process the corresponding local image.

[0104] Each worker thread runs independently and is responsible for performing two main tasks: first, it uses the text edge detection model to identify the text area in the local image, generates a text mask, and then extracts the text set image based on the text mask; second, it inputs the text set image into the text recognition conversion model to obtain the highlight prompt text. After obtaining the text, the worker thread determines whether the highlight event is triggered based on the preset highlight text template and fuzzy matching algorithm.

[0105] For example, if a partial image comes from the scoreboard area in a game, the worker thread will detect text information in the image, such as "score" or "kill", and match it with the preset highlight event template to determine whether a highlight video clip needs to be generated.

[0106] By creating an independent working thread for each partial image, the system can process multiple partial images at the same time, significantly improving the processing speed and real-time performance. This method is particularly suitable for fast-changing game environments and can ensure rapid response and processing when highlight events occur.

[0107] The worker thread approach also brings good scalability. As the system load changes, the number of worker threads can be dynamically increased or decreased to adapt to different processing requirements. In computer devices with limited resources, the maximum number of threads can be limited to avoid excessive consumption of system resources. On multi-core processors, by reasonably allocating worker threads, the multi-core parallel processing capabilities can be fully utilized to further improve processing efficiency.

[0108] It is not difficult to understand that step S3200 and step S3300 are the method steps for completing these two main tasks. Therefore, in one embodiment, each working thread is responsible for executing the processing of step S3200 and step S3300.

[0109] Adapting to the mechanism of processing local images with a working thread, step S3400 of the present application, i.e., the step of generating a highlight video clip containing the image frame that triggers the highlight event, further includes: Step S4200: According to the highlight events determined by each working thread, all highlight events whose highlight moments are in the same time period are merged into highlight events of the same period; It is common for multiple highlight events to occur simultaneously during game play. Therefore, multiple different highlight events may be identified in the same period. If a corresponding highlight video clip is generated for each highlight event in this period, a large number of redundant clips will be generated. Therefore, further optimization can be performed. By merging all highlight events that occurred in the same period, it is ensured that all related wonderful moments are included when generating highlight video clips, thereby providing a richer and more coherent viewing experience.

[0110] Specifically, when the worker thread completes the text detection and recognition tasks, it determines whether there is a highlight event and records the specific highlight moment of the event. These moments can correspond to key game achievements, such as "first kill", "continuous scoring", etc. In order to effectively organize these events, the system implements a merging mechanism to merge all highlight events that occur in the same time window, that is, the same period, into a "contemporaneous highlight event". For example, if in an e-sports game, a player gets multiple kills in a short period of time, these kill events can be merged into a contemporaneous highlight event.

[0111] The key to the merging mechanism is to define an appropriate time window. This time window can be fixed, such as all highlight events within a few seconds are considered as the same period; or it can be dynamic and adjusted according to the specific situation of the game and user settings. For example, for fast-paced games, the time window can be set relatively short to capture fast-occurring events; while for strategy games, a relatively longer time window can be used to cover important tactical developments.

[0112] Step 4300: Control the generation of a corresponding highlight video segment according to the highlight event in the same period, so that the highlight video segment includes image frames corresponding to all highlight events in the same time period.

[0113] After determining the highlight events of the same period, a corresponding time window is actually determined, which can be used to control the generation of highlight video clips to ensure that the highlight video clips can contain all relevant image frames. Specifically, according to the time window corresponding to the highlight events of the same period, the image frames within the time window are extracted from the image frame sequence of the original game live, and they are combined in sequence into a coherent video, that is, the highlight video clip is obtained.

[0114] In addition, to improve the viewing and professionalism of the video, the system can also further process the extracted image frames, such as adjusting brightness, contrast or applying filter effects. These post-processing steps can be automatically performed based on user preferences or preset editing templates, ensuring that each generated highlight video clip has a consistent style and quality.

[0115] The above embodiments significantly improve the efficiency of capturing game highlight moments and the coherence of video content by introducing a multi-threading mechanism and a highlight event merging strategy.

[0116] First, by using worker threads to process local images in parallel, the system can quickly respond and identify highlight prompt text in each image, and then determine highlight events. This method is particularly suitable for fast-changing game environments because it reduces processing latency and ensures that even when highlight moments occur frequently, the system can capture and process every important moment in time.

[0117] Secondly, the video generation process is further optimized by merging all highlight events that occurred in the same period into one highlight event of the same period. This merging strategy avoids the generation of a large number of repeated or redundant highlight video clips, ensuring that the final generated video clips are more compact and exciting in content. At the same time, this also improves the viewing value of the video, because the audience can watch more highlight moments in a single video instead of switching between multiple scattered clips.

[0118] Furthermore, on the technical level, the multi-threading mechanism enables the system to fully utilize the multi-core characteristics of the processor to achieve true parallel processing, thereby significantly improving processing speed and real-time performance. The highlight event merging strategy reflects the in-depth understanding and intelligent processing of game live content. It provides a smoother and more immersive viewing experience by intelligently identifying and organizing related events.

[0119] In addition, this method based on multithreading and event merging has good scalability and adaptability. With the diversification of game types and player needs, the system can flexibly adapt to different game scenarios and user preferences by adjusting the size of the time window and the merging strategy. At the same time, this method also provides greater flexibility for later video editing and special effects addition, making the generated highlight video clips not only rich in content, but also more professional and attractive in form.

[0120] In summary, this embodiment not only improves the efficiency and accuracy of capturing game highlight moments by combining the advanced technologies of multi-threaded processing and highlight event merging, but also enhances the coherence and viewing quality of the final video content, providing users with a new, automated highlight video generation solution.

[0121] Based on any embodiment of the method of the present application, before obtaining the image frame sequence for displaying the live game, the method includes: Step S1100, responding to the highlight event configuration instruction, displaying the highlight event definition interface; This embodiment allows the user of the terminal device to configure how to respond to various possible highlight events to decide whether to automatically generate corresponding highlight video clips. To this end, a user-friendly interface is provided in the terminal device, which allows the user to define and configure the highlight events they want to capture. Specifically, when the user needs to make a configuration, the corresponding configuration control can be touched to trigger the corresponding highlight event configuration instruction. The system first responds to the highlight event configuration instruction and triggers the display of a highlight event definition interface. This interface is interactive and provides users with the option of defining and personalizing highlight events.

[0122] Step S1200: based on the highlight event definition interface, obtaining trigger condition information and / or generation condition information corresponding to the highlight event defined by the user; The highlight event definition interface allows users to enter keywords as trigger condition information. These keywords are user-defined to describe specific game achievements or events that they want the system to recognize. For example, users can enter keywords such as "pentakill", "super god" or "reversal victory". These words usually mark significant achievements or turning points in the game and will be displayed as text information in the image frame corresponding to the live game. Therefore, each keyword entered by the user will serve as a reference standard in the system's subsequent text detection and recognition process.

[0123] In addition, users can also set a switch flag, such as "yes" or "no", for each highlight event as generation condition information. This switch flag determines whether to generate a corresponding highlight video clip based on the user-defined highlight event. For example, if the user sets the switch to "yes", when the system recognizes text related to the event, a highlight video clip will be automatically generated and saved; if it is set to "no", even if the relevant text is recognized, no video clip will be generated. This flexible configuration option enables users to customize video content according to their preferences and needs.

[0124] Step S1300: when the trigger condition information exists, create a highlight text template corresponding to the highlight event according to the trigger condition information, and store it in the template library as a constraint condition for triggering the user-defined highlight event; Once the user enters trigger condition information, such as keywords, through the highlight event definition interface, the system will create highlight text templates based on this information. These templates represent the text information that the user wants the system to pay special attention to, and they are used in the subsequent text detection and recognition process. In the process of creating highlight text templates, one or more templates can be built based on the analysis of the keywords entered by the user, and these templates can accurately match the relevant text that may appear in the game. For example, if the user enters "five kills" as a keyword, the system can create one or more corresponding highlight text templates, which are specifically used to match the text style and format representing continuous kills in the game, such as creating multiple templates of "five kills" and "continuous kills" at the same time. These templates are then stored in the template library for use in the text detection stage.

[0125] The template library is a data structure that stores all predefined highlight text templates, each of which is associated with a specific highlight event. When the system performs real-time text detection, it matches the detected highlight prompt text with each highlight text template in the template library to determine whether the user-defined highlight event is triggered. If the text successfully matches a template, the system will assume that the corresponding highlight event has been triggered and prepare for the next step.

[0126] Step S1400: When the generation condition information exists, the generation condition information is set as a constraint condition for whether to generate a corresponding highlight video segment based on a highlight event defined by a user.

[0127] Setting the user's generation condition information as a constraint condition for whether to generate a corresponding highlight video clip based on the user-defined highlight event can be a key link in realizing the user's personalized needs. It ensures that the highlight video clip generated by the system strictly meets the user's expectations and choices.

[0128] Specifically, when the user sets the switch flag for a specific highlight event in the highlight event definition interface, these flags are captured and processed by the system as generation condition information. Generation condition information is usually reflected as a simple binary choice, such as "yes" or "no", which is directly related to the decision of whether to generate a video clip. For example, if the user sets "yes" to the highlight event "Super God", when the system recognizes the "Super God" text in the game, it will automatically start the video recording and generation process, create and save a highlight video clip containing the event. On the contrary, if it is set to "no", even if the system recognizes the same text, it will not generate a video clip.

[0129] The generation condition information input by the user can be parsed and converted into system executable configurations. These configurations will be stored in the system's configuration database or setting files, such as a template library, and associated with the corresponding highlight event templates. During text detection and recognition, the system will refer to these configurations to decide whether to perform video generation operations. This decision-making process can be implemented through programming logic, where conditional statements are used to check whether the generation conditions of each highlight event are met.

[0130] The setting of generating conditional information provides a high degree of flexibility and user control. Users can selectively turn on or off the video generation function for certain highlight events according to different game scenarios and personal preferences. For example, for players who want to save evidence of all important achievements, they can set "yes" for all types of highlight events. For players who only want to record specific types of highlight moments, they can set "yes" for certain events and "no" for other events.

[0131] Through the above embodiments, the user experience has been significantly improved, mainly in the following aspects: First, the introduction of the highlight event definition interface enables users to intuitively define and configure the highlight events they are interested in. This interactive design not only simplifies the configuration process, but also enables users to easily set personalized capture conditions based on their own gaming style and preferences. Users can directly enter keywords such as "five kills" or "super god", which they think are memorable game achievements, and the system will then automatically recognize and respond to these events.

[0132] Secondly, users have control over the generation condition information for each highlight event, providing an additional level of personalization. Users can decide whether to generate video clips for specific events, which allows content generation to be fully tailored to the user's expectations and needs. For example, if the user only wants to record particularly important victory moments, they can set the generation conditions for these specific events and ignore other less important events.

[0133] Furthermore, this approach ensures that the generated video content is closely related to the user's interests, improving the viewing experience. Users can watch and share the highlights they are truly interested in, rather than browsing a large number of irrelevant video clips. This targeted content generation enables users to enjoy and showcase their achievements in the game more effectively.

[0134] In addition, the system's automated processing reduces the user's workload in post-editing and organizing videos. Users no longer need to manually screen and edit a large number of game recordings, and the system automatically generates accurate and coherent highlight video clips for them. This automated process not only saves users' time, but also increases the convenience of sharing and reviewing the highlights of the game.

[0135] See also Figure 3 According to one aspect of the present application, a game highlight video generation device is provided, including an image acquisition module 3100, a text detection module 3200, an event triggering module 3300, and a video generation module 3400, wherein the image acquisition module 3100 is configured to acquire an image frame sequence for displaying a live game, and extract a local image corresponding to a preset area from the image frames of the image frame sequence; the text detection module 3200 is configured to perform text detection based on the local image to identify a highlight prompt text; the event triggering module 3300 is configured to determine the edit distance between the highlight prompt text and each highlight text template preset in a template library, and when the minimum edit distance meets a preset condition, trigger a highlight event corresponding to the highlight text template having the minimum edit distance; the video generation module 3400 is configured to generate a highlight video clip containing the image frame that triggers the highlight event.

[0136] Based on any embodiment of the device of the present application, the image acquisition module 3100 includes: an area acquisition module, which is configured to acquire multiple preset areas corresponding to the game settings of the game live; a polling selection module, which is configured to poll each preset area in turn, and corresponding to each preset area, extract a local image from the image frame sequence according to the image frame arrangement order corresponding to the selected target image frame; an image extraction module, which is configured to extract a local image corresponding only to the currently polled preset area from the target image frame.

[0137] Based on any embodiment of the device of the present application, the text detection module 3200 includes: a mask prediction module, which is configured to input the local image into a text edge detection model to obtain a text mask; an application extraction module, which is configured to extract a text collection image from the local image according to the text mask; and a text recognition module, which is configured to input the text collection image into a text recognition conversion model to obtain a highlight prompt text.

[0138] Based on any embodiment of the device of the present application, the mask prediction module includes: a semantic extraction module, which is configured to extract deep semantic information from the local image by the feature representation network of the text edge detection model to obtain a semantic feature map; a probability prediction module, which is configured to predict a probability map based on the semantic feature map by the first branch network of the text edge detection model, and the probability map represents the probability corresponding to each pixel in the local image belonging to the text; a threshold prediction module, which is configured to predict a threshold map based on the semantic feature map by the second branch network of the text edge model, and the threshold map represents the decision threshold corresponding to the probability that each pixel in the local image belongs to the text; a mask representation module, which is configured to determine whether each pixel in the local image belongs to the text based on the threshold map and the probability map by the fusion network of the text edge detection model, and represents it as a text mask.

[0139] Based on any embodiment of the device of the present application, the text recognition module includes: an image selection module, which is configured to determine the outer frame in the local image where the text is located according to the text mask, and to cut out the text collection image corresponding to the outer frame from the local image; a feature extraction module, which is configured to extract image feature information from the text collection image by a feature extraction network in a text recognition conversion model; and a text prediction module, which is configured to predict the image feature information by a sequence prediction network in the text recognition conversion model to determine the highlighted prompt text.

[0140] On the basis of any embodiment of the device of the present application, prior to the operation of the text detection module, the device includes: a thread creation module, configured to create a corresponding working thread in response to a determined event of each local image, each working thread is responsible for executing text detection for each local image to obtain a highlight prompt text, and determining a highlight event based on the highlight prompt text; the video generation module includes: an event merging module, configured to merge all highlight events in the same time period indicated by the highlight moments determined by each working thread into a concurrent highlight event according to the highlight events determined by the working threads; an association generation module, configured to control the generation of a corresponding highlight video clip according to the concurrent highlight event, so that the highlight video clip contains image frames corresponding to all highlight events in the same time period.

[0141] A thread creation module is configured to instantiate a corresponding matching thread for each highlight text template in the corresponding template library; a thread running module is configured to start each matching thread to determine the editing distance between its corresponding highlight text template and the highlight prompt text to determine whether to trigger the highlight event; a thread maintenance module is configured to destroy other matching threads when any of the matching threads triggers the highlight event.

[0142] On the basis of any embodiment of the device of the present application, prior to the image acquisition module 3100, the device of the present application also includes: an instruction response module, configured to respond to a highlight event configuration instruction and display a highlight event definition interface; an information receiving module, configured to obtain trigger condition information and / or generation condition information corresponding to a user-defined highlight event based on the highlight event definition interface; a template configuration module, configured to create a highlight text template corresponding to the highlight event according to the trigger condition information when the trigger condition information exists, and store it in the template library as a constraint condition for triggering a user-defined highlight event; a generation configuration module, configured to set the generation condition information as a constraint condition for whether to generate a corresponding highlight video clip based on a user-defined highlight event when the generation condition information exists.

[0143] Another embodiment of the present application also provides a device for generating a game highlight video. Figure 4 As shown, a schematic diagram of the internal structure of a game highlight video generating device. The game highlight video generating device includes a processor, a computer-readable storage medium, a memory, and a network interface connected through a system bus. Among them, the computer-readable non-volatile readable storage medium of the game highlight video generating device stores an operating system, a database, and computer-readable instructions. The database may store an information sequence. When the computer-readable instructions are executed by the processor, the processor may implement a game highlight video generating method.

[0144] The processor of the game highlight video generation device is used to provide computing and control capabilities to support the operation of the entire game highlight video generation device. The memory of the game highlight video generation device may store computer-readable instructions, which, when executed by the processor, may enable the processor to execute the game highlight video generation method of the present application. The network interface of the game highlight video generation device is used to connect and communicate with a terminal.

[0145] Those skilled in the art will understand that Figure 4 The structure shown in the figure is merely a block diagram of a partial structure related to the present application scheme, and does not constitute a limitation on the game highlight video generation device to which the present application scheme is applied. The specific game highlight video generation device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0146] In this embodiment, the processor is used to execute Figure 3The memory stores the program code and various data required to execute the above modules or submodules. The network interface is used to realize data transmission between user terminals or servers. The non-volatile readable storage medium in this embodiment stores the program code and data required to execute all modules in the game highlight video generation device of this application, and the server can call the program code and data of the server to execute the functions of all modules.

[0147] The present application also provides a non-volatile readable storage medium storing computer-readable instructions. When the computer-readable instructions are executed by one or more processors, the one or more processors execute the steps of the game highlight video generation method of any embodiment of the present application.

[0148] The present application also provides a computer program product, including a computer program / instruction, which implements the steps of the method described in any embodiment of the present application when executed by one or more processors.

[0149] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments of the present application can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a computer-readable storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0150] In summary, this application significantly improves the recognition accuracy of highlight moments by performing precise text detection in a preset area and combining the edit distance algorithm with intelligent matching of highlight text templates. This method not only reduces misjudgments and missed judgments, but also ensures that highlight events can be accurately captured through a fault-tolerant mechanism even if slight errors occur during text recognition. The automated video generation and sharing process allows users to enjoy a smooth and immersive viewing experience without manual operation, and easily share exciting moments, thereby enhancing user engagement and the activity of the gaming community.

Claims

1. A method for generating a game highlight video, characterized in that: include: Acquire an image frame sequence for displaying a live game, and extract a local image corresponding to a preset area from the image frames in the image frame sequence; Perform text detection based on the local image to identify highlighted text; Determine the edit distance between the highlight prompt text and each highlight text template preset in the template library, and when the minimum edit distance meets the preset condition, trigger the highlight event corresponding to the highlight text template with the minimum edit distance; A highlight video clip including the image frame triggering the highlight event is generated.

2. The method for generating a game highlight video according to claim 1, characterized in that: Extracting a local image corresponding to a preset area from the image frames of the image frame sequence includes: Acquire a plurality of preset areas corresponding to game settings to which the game live belongs; Polling each preset area in turn, corresponding to each preset area, extracting a local image corresponding to the selected target image frame from the image frame sequence according to the image frame arrangement order; A local image corresponding only to the preset area currently polled is extracted from the target image frame.

3. The method for generating a game highlight video according to claim 1, characterized in that: Performing text detection based on the local image to identify the highlighted text includes: Inputting the local image into a text edge detection model to obtain a text mask; Extracting a text-focused image from the local image according to the text mask; The image in the text set is input into a text recognition conversion model to obtain a highlighted prompt text.

4. The method for generating a game highlight video according to claim 3, characterized in that: Inputting the local image into a text edge detection model to obtain a text mask includes: Extracting deep semantic information from the local image using a feature representation network of the text edge detection model to obtain a semantic feature map; The first branch network of the text edge detection model predicts a probability map based on the semantic feature map, wherein the probability map represents the probability that each pixel in the local image belongs to the text; The second branch network of the text edge model predicts a threshold map based on the semantic feature map, wherein the threshold map represents a decision threshold corresponding to the probability that each pixel in the local image belongs to the text; The fusion network of the text edge detection model determines whether each pixel in the local image belongs to text according to the threshold map and the probability map, and the corresponding representation is a text mask.

5. The method for generating a game highlight video according to any one of claim 3, characterized in that: Inputting the image in the text set into the text recognition conversion model to obtain the highlighted prompt text, including: Determine the outer frame of the local image where the text is located according to the text mask, and extract the text set image corresponding to the outer frame from the local image; Extracting image feature information from the images in the text set by a feature extraction network in a text recognition conversion model; The image feature information is predicted by a sequence prediction network in the text recognition conversion model to determine the highlighted prompt text.

6. The method for generating a game highlight video according to claim 1, wherein: Before the step of performing text detection based on the partial image and identifying the highlight prompt text, the method further includes: creating a corresponding working thread in response to a determination event of each partial image, wherein each working thread is responsible for performing text detection on each partial image to obtain the highlight prompt text, and determining a highlight event based on the highlight prompt text; The step of generating a highlight video clip containing the image frame that triggers the highlight event also includes: according to the highlight events determined by each working thread, merging all the highlight events indicated by the highlight moments in the same time period into a concurrent highlight event; and controlling the generation of the corresponding highlight video clip according to the concurrent highlight event, so that the highlight video clip contains the image frames corresponding to all the highlight events in the same time period.

7. The method for generating a game highlight video according to any one of claims 1 to 6, characterized in that: Before obtaining the image frame sequence for displaying the live game, include: Respond to the highlight event configuration instruction and display the highlight event definition interface; Based on the highlight event definition interface, acquiring trigger condition information and / or generation condition information corresponding to a highlight event defined by a user; When the trigger condition information exists, creating a highlight text template corresponding to the highlight event according to the trigger condition information, and storing it in the template library as a constraint condition for triggering the user-defined highlight event; When the generation condition information exists, the generation condition information is set as a constraint condition for whether to generate a corresponding highlight video segment based on a highlight event defined by a user.

8. A device for generating a game highlight video, characterized in that: include: An image acquisition module is configured to acquire an image frame sequence for displaying a live game, and extract a local image corresponding to a preset area from image frames in the image frame sequence; A text detection module, configured to perform text detection based on the local image and identify highlighted text; An event triggering module is configured to determine the edit distance between the highlight prompt text and each highlight text template preset in the template library, and when the minimum edit distance meets a preset condition, trigger a highlight event corresponding to the highlight text template having the minimum edit distance; The video generation module is configured to generate a highlight video segment including the image frame that triggers the highlight event.

9. A game highlight video generation device, comprising a central processing unit and a memory, characterized in that: The central processing unit is used to call and run the computer program stored in the memory to execute the steps of the method according to any one of claims 1 to 7.

10. A non-volatile readable storage medium, characterized in that: It stores a computer program implemented according to the method described in any one of claims 1 to 7 in the form of computer-readable instructions, and when the computer program is called and executed by a computer, the steps included in the corresponding method are executed.

Images