Methods, equipment, storage media, and computer programs for generating event commentary

By performing layered event detection on match videos, identifying atomic actions, tactical patterns, and strategic intentions, the problem of insufficient deep semantic understanding in match commentary is solved, improving the professionalism and coherence of the commentary and enhancing the user's viewing experience.

CN123123610APending Publication Date: 2026-11-17MIGU CO LTD +1
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202610976729.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-07-01
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing methods for generating match commentary cannot effectively understand the tactical patterns and strategic intentions in a match, resulting in a lack of professionalism, coherence, and interpretability in the generated commentary text, which negatively impacts the user's viewing experience.

Method used

A hierarchical event detection mechanism is adopted to extract features and detect hierarchical events in event videos, including multi-level analysis of atomic actions, tactical patterns and strategic intentions. The deep semantics in the event are identified through a three-layer detection structure.

Benefits of technology

It enhances the deep semantic understanding of event commentary, generating more professional, coherent, and interpretable commentary texts, thereby improving the user's viewing experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123123610A_ABST
    Figure CN123123610A_ABST
Patent Text Reader

Abstract

This application provides a method, device, storage medium, and computer program product for generating sports commentary. The method includes: extracting features from a sports video to obtain video features; performing hierarchical event detection on the video features to obtain results for each hierarchical event; the hierarchical event results include: atomic action events, tactical pattern events, and strategic intent events; and generating sports commentary based on the hierarchical event results. Thus, this application embodiment, by performing hierarchical event detection on sports videos, progressively analyzes the sports content from atomic actions and tactical patterns to strategic intents, solving the problem of lacking deep semantic understanding in sports commentary generation and improving the user's viewing experience.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, and in particular to a method, device, storage medium, and computer program product for generating sports commentary. Background Technology

[0002] Sports events, e-sports competitions, and other competitive events typically require live commentary to enhance the viewing experience and understanding of the game's progress. To improve the automation of commentary generation, relevant technologies often employ rule-based template matching techniques or deep learning models driven solely by event data to automatically generate commentary. Examples include generating commentary based on preset event triggers or using neural networks to generate commentary based on event statistics.

[0003] However, the above-mentioned methods for generating commentary have the following drawbacks: rule-based methods have low template coverage, cannot cope with complex tactical combinations, and lack dynamic adaptability to different stages of the game and score differences; pure data-driven methods have poor logical consistency, are prone to generating descriptions that contradict the facts, and lack a deep understanding of tactical terminology and game rules, resulting in insufficient professionalism and interpretability, which affects the user's viewing experience. Summary of the Invention

[0004] In view of this, embodiments of this application provide a method, device, storage medium, and computer program product for generating sports commentary.

[0005] The technical solution of this application embodiment is implemented as follows: In a first aspect, embodiments of this application provide a method for generating sports commentary, the method comprising: Feature extraction is performed on the event videos to obtain video features; The video features are subjected to hierarchical event detection to obtain the results of each hierarchical event; the hierarchical event results include: atomic action events, tactical pattern events, and strategic intent events; Based on the results of the hierarchical events, the event commentary is generated.

[0006] In the above scheme, the step of performing hierarchical event detection on the video features to obtain the results of each hierarchical event includes: In the first-layer event detection, region of interest features are extracted from the video features to obtain region features; The video features and the region features are used to detect the participants in the competition, and the atomic action events of each participant are obtained. The atomic action events include target detection results and motion states.

[0007] In the above scheme, the step of performing hierarchical event detection on the video features to obtain the results of each hierarchical event includes: In the second-layer event detection, a relationship graph between the participating entities in the competition is constructed based on the atomic action events. Based on the first prediction model, the relationship graph is used to predict the tactical type and subject interaction probability to obtain the tactical pattern event; The first prediction model is obtained after training based on the first loss constraint, which is determined based on the event tactical rules; the tactical pattern event includes tactical type and subject interaction probability.

[0008] In the above scheme, the step of performing hierarchical event detection on the video features to obtain the results of each hierarchical event includes: In the third-layer event detection, temporal features are extracted from the video features; Based on recurrent neural networks and global attention mechanisms, the temporal features and event metadata are processed to obtain contextual features; Based on the second prediction model, strategic intent is predicted from the context features to obtain the strategic intent event; The second prediction model is obtained after training based on the second loss constraint, which is determined based on the event strategy rules.

[0009] In the above scheme, the step of extracting features from the event video to obtain video features includes: Multi-branch feature extraction is performed on the event video to obtain spatial features, temporal features, and motion features; The spatial features, temporal features, and motion features are fused to obtain video features.

[0010] In the above scheme, the feature fusion of the spatial features, the temporal features, and the motion features to obtain video features includes: Based on the attention mechanism, the fusion weights corresponding to the spatial features, the temporal features, and the motion features are determined; Based on the fusion weights, the spatial features, the temporal features, and the motion features are fused to obtain video features.

[0011] In the above scheme, generating event commentary based on the hierarchical event results includes: The hierarchical event results are parsed to determine the matching event type; Based on the matched event type, determine the corresponding explanation template; Obtain the event content and user preference configuration, and determine the commentary embellishment content based on the user preference configuration; Add the event content and the commentary embellishments to the commentary template to generate the event commentary.

[0012] In a second aspect, embodiments of this application provide an electronic device, including: a processor and a memory for storing a computer program capable of running on the processor, wherein the processor, when running the computer program, performs the steps of the method described in the first aspect.

[0013] Thirdly, embodiments of this application provide a storage medium storing a computer program, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0014] Fourthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in the first aspect.

[0015] In the technical solution of this application, feature extraction is performed on the event video to obtain video features; hierarchical event detection is performed on the video features to obtain the results of each hierarchical event; the hierarchical event results include: atomic action events, tactical pattern events, and strategic intent events; and event commentary is generated based on the hierarchical event results. Thus, this embodiment of the application, by performing hierarchical event detection on the event video, progressively analyzes the event content from atomic actions and tactical patterns to strategic intents, solving the problem of lacking deep semantic understanding in event commentary generation and improving the user's viewing experience. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the first process of the event commentary generation method according to an embodiment of this application; Figure 2 This is a schematic diagram of the three-layer detection structure according to an embodiment of this application; Figure 3 This is a schematic diagram of the first process of the video feature generation method according to an embodiment of this application; Figure 4 This is a schematic diagram of the second process of the video feature generation method according to an embodiment of this application; Figure 5 This is a schematic diagram illustrating tactical pattern prediction in an embodiment of this application; Figure 6 This is a schematic diagram of the second process of the event commentary generation method according to an embodiment of this application; Figure 7 This is a schematic diagram of the third process of the event commentary generation method in an embodiment of this application; Figure 8 This is a schematic diagram of the structure of the event commentary generation device according to an embodiment of this application; Figure 9 This is a schematic diagram of the structure of an electronic device according to an embodiment of this application. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0018] Sports events, esports competitions, and other competitive events typically feature live commentary to enhance the viewing experience and understanding of the competition. Traditional human commentary relies on the personal experience and improvisation of professional commentators, which has limitations such as high labor costs, inconsistent commentary quality, and difficulty in scaling up coverage of multiple events. Currently, intelligent commentary technology aims to reduce reliance on human commentary and improve efficiency by automatically generating commentary text based on artificial intelligence.

[0019] In related technologies, rule-based template matching techniques or deep learning models driven solely by event data are commonly used to automatically generate event commentary. Examples include generating commentary based on preset event triggers or using neural networks to generate commentary based on event statistics. However, these methods for generating event commentary have the following drawbacks: Rule-based template matching technology can usually only handle simple preset scenarios and cannot cope with complex tactical combinations in a match. For example, in a football match, it can only generate commentary text based on typical preset scenarios such as "goal" and "foul", and cannot cope with complex local tactical combinations such as "cross from the wing + cross movement + shot". Moreover, the generated commentary text lacks dynamic adaptability to changes in the stage and situation of the match, and subsequent injection of new rules requires continuous manual maintenance and updates.

[0020] Deep learning models driven solely by match data often suffer from poor logical consistency, easily generating descriptions that contradict reality. For example, in a football match scenario, they might generate commentary stating "the home team has possession," when the away team actually has possession. Furthermore, these models lack a deep understanding of tactical terminology and match rules, requiring extensive training with labeled samples to effectively identify low-probability events in the match.

[0021] Furthermore, some related technologies combine intelligent commentary techniques with manual annotation to enhance the depth of commentary comprehension. However, relying on manual annotation consumes significant manpower and time, making it difficult to meet the analytical efficiency requirements of real-time commentary. Therefore, it is evident that the existing commentary generation methods have significant shortcomings in both the depth and breadth of semantic understanding, only able to parse the underlying actions within the event content. This results in commentary texts lacking professionalism, coherence, and interpretability, negatively impacting the user's viewing experience.

[0022] Based on this, this application provides a method for generating sports commentary, which aims to enhance the depth of understanding of the sports content by the generated commentary and improve the user's viewing experience.

[0023] The method for generating event commentary in this embodiment of the application is as follows: Figure 1 As shown, the method includes: Step 101: Extract features from the event video to obtain video features.

[0024] Step 102: Perform hierarchical event detection on the video features to obtain the results of each hierarchical event; the hierarchical event results include: atomic action events, tactical pattern events, and strategic intent events.

[0025] Step 103: Generate event commentary based on the hierarchical event results.

[0026] Here, in this embodiment of the application, "event" refers to a competitive competition or activity involving multiple participants according to set rules, including but not limited to sports events and e-sports events; correspondingly, the event commentary in this embodiment of the application is generated based on event videos, which are image information that records the process of the event, including but not limited to the original video stream, live video stream, or recorded video stream of the event.

[0027] It should be noted that, in order to facilitate those skilled in the art to fully understand the technical solution of this application, the following embodiments use a football match as an example to illustrate the match commentary generation method of this application. However, this does not constitute an application limitation of the technical solution of this application. The match commentary generation method of this application embodiment is also applicable to other sports events, e-sports events, and other competitive competitions or activities.

[0028] Here, atomic actions, also known as basic actions, are indivisible basic action units of a participating subject during a match. They occur within a short timescale and are used to characterize the instantaneous state or action of the participating subject. For example, in a football match, atomic actions include, but are not limited to, basic actions such as passing, shooting, running, tackling, and clearing the ball with a header.

[0029] Here, tactical patterns refer to the behavioral patterns formed by multiple participating entities through coordinated cooperation in order to achieve specific local objectives during a competition. For example, in a football match, multiple players use cross-positional runs to stretch the opponent's defensive line, thus achieving the tactical pattern of "crossing runs"; and through vertical passing and cutting between wingers and central players, the tactical pattern of "crossing from the wing" is executed.

[0030] Here, strategic intent refers to the overall strategy or action plan adopted by a team participating in a competition at a specific stage or score context. Multiple stakeholders within the team execute corresponding actions or tasks based on the established strategic intent. For example, in a football match, a team might adopt a "counter-attacking" strategy, where the players move back into position, quickly make long passes after winning possession to find the striker, and maintain a lead. Conversely, a team might adopt a "possession and pressure" strategy, where the players push forward, using continuous passing and control of the ball to create more attacking opportunities.

[0031] Understandably, participants within the same team during a competition (such as athletes and team members) typically do not engage in independent competitive behavior. Instead, they collaborate to improve their chances of winning. Therefore, the individual competitive behavior of each participant actually implies the tactical patterns formed by the collaboration of multiple participants, as well as the strategic intentions that the team follows in the competition. The tactical patterns and strategic intentions adopted by the participants in the competition are key information for viewers to understand the progress of the competition and enhance their viewing experience. They are also important content for the commentary.

[0032] However, the deeper semantics of tactical patterns and strategic intentions in matches, as mentioned above, are difficult to directly capture and analyze from match videos. Specifically, this is because: firstly, tactical patterns and strategic intentions are not apparent features of match videos and cannot be directly read from underlying visual signals such as the movement trajectories and body movements of the participants. For example, analyzing a football match video can determine a player's position and whether they are participating in defense, but it cannot identify whether the player is pressing forward to execute an "offside trap" tactic. Secondly, event analysis based on a single participant also fails to obtain global semantics. Isolated events such as a single player's ball control, passing, and running only reflect individual player behavior; while the successful execution of tactics and strategies usually depends on the relative positions, timing, and role divisions of multiple players. The accumulation or connection of individual behaviors alone is insufficient to infer the team's overall strategic intentions.

[0033] It should be noted that, in order to improve the depth of understanding of the generated event commentary, this application embodiment introduces a layered event detection mechanism in the event commentary generation method to solve the problem that the tactical patterns and strategic intentions adopted in the event are difficult to directly identify from the event video.

[0034] Here, hierarchical event detection is a method that integrates complex event processing and temporal data mining. Unlike conventional methods that directly parse identifiable events, it parses layer by layer from bottom to top according to semantic granularity: first, it identifies simple atomic events at the bottom level, and then combines them into more complex events at higher levels through logic, temporal, or causal relationships, thereby producing event detection results at different semantic levels.

[0035] It should be noted that hierarchical event detection can set multiple detection levels according to the semantic complexity of a specific scenario. For example, a three-layer detection structure includes a low-level detector, a middle-level detector, and a high-level detector. The low-level detector is used to detect original, indivisible atomic events; the middle-level detector is used to detect behavioral patterns formed by the combination of multiple atomic events according to specific spatiotemporal relationships; and the high-level detector is used to detect complex events, which represent the deep semantics formed by the aggregation of multiple patterns through logical, temporal, or causal relationships.

[0036] Understandably, atomic actions are the underlying visual features that can be directly identified in a match video. However, the deeper semantics, such as tactical patterns and strategic intentions in a match, are formed by the combination of atomic actions of various participants in the spatial dimension (such as the relative positions and running routes between players) and the temporal dimension (such as the sequence and duration of the actions). Therefore, based on the hierarchical event detection mechanism, starting from the detection of underlying atomic actions in the match video, and extracting the spatial and temporal combination relationships of atomic actions through layer-by-layer analysis, the deep semantic detection from atomic actions to tactical patterns and then to strategic intentions can be achieved according to the hierarchical structure of "atomic actions - tactical patterns - strategic intentions".

[0037] Accordingly, such as Figure 2 As shown, this embodiment employs a three-layer detection structure for event detection and parsing. The three-layer detection structure includes a first-layer detector, a second-layer detector, and a third-layer detector, which are sequentially connected. The first-layer detector, also known as the L1 detector or low-level detector, is used to detect atomic action events in the event video; the second-layer detector, also known as the L2 detector or mid-level detector, is used to detect tactical pattern events in the event video; and the third-layer detector, also known as the L3 detector or high-level detector, is used to detect strategic intent events in the event video.

[0038] Specifically, in this embodiment, feature extraction is performed on the event video to extract relevant video features of the participating entities in the event video; the extracted video features are then subjected to hierarchical event detection through a first-layer detector, a second-layer detector, and a third-layer detector in sequence to detect atomic action events, tactical pattern events, and strategic intent events, which are used as the results of each hierarchical event; based on the results of each hierarchical event, an event commentary containing multi-layer semantic understanding is generated.

[0039] It is understood that the embodiments of this application solve the problem of lack of deep semantic understanding in the generation of event commentary by performing layered event detection on the event video, and progressively analyzing the event content from atomic actions, tactical patterns to strategic intentions, thereby improving the user's viewing experience.

[0040] In some embodiments, feature extraction is performed on the event video to obtain video features, including: multi-branch feature extraction of the event video to obtain spatial features, temporal features and motion features; and feature fusion of the spatial features, temporal features and motion features to obtain video features.

[0041] Spatial features are static features such as the position and posture of the participants in the event video, such as the bounding box coordinates and key point positions of the participants.

[0042] Among them, the temporal characteristic is the continuous change process of the main action in the event video, such as the run-up action and leg swing amplitude of a player before shooting in a football match.

[0043] Among them, motion characteristics are the dynamic features of the participants in the event video, such as the displacement trajectory and movement speed, for example, the flight trajectory of the football, the running speed and direction of the players in a football match.

[0044] It should be noted that in related technologies, feature extraction from sports videos often encounters the problem of feature confusion. For example, a player in a stationary position in the video may be misjudged due to feature confusion, leading to inaccurate commentary. To address this issue, this application employs a multi-branch feature parallel extraction method. Combining the characteristics of the sports event, it separates and extracts easily confused static features (spatial features) and dynamic features (motion features) from the sports video. Then, by fusing the separated spatial, temporal, and motion features, the mutual interference between static and dynamic features is avoided during the feature extraction process, improving the accuracy of feature representation.

[0045] In some embodiments, feature extraction is performed on the event video to obtain video features, including: feature extraction is performed on the event video based on a spatiotemporal multi-scale feature fusion network to obtain video features.

[0046] Here, the Spatio-temporal Multi-scale Fusion Network (ST-MSFN), also known as the Multi-scale Spatio-temporal Fusion Neural Network (MS-STFNN), is a neural network architecture for video feature extraction. Its core idea is that features of different dimensions can be extracted using different time scales and sampling strategies to achieve multi-scale modeling of video content. In this embodiment, ST-MSFN employs multiple parallel feature extraction branches to perform multi-scale feature modeling of the event video from spatial, temporal, and motion dimensions, respectively, and then fuses the extracted multi-dimensional features to output video features containing rich semantic information.

[0047] Accordingly, multi-branch feature extraction is performed on the event video to obtain spatial features, including: dividing the event video into first video segments based on the first time window in the spatial branches; and extracting features from the first video segments to obtain spatial features.

[0048] Accordingly, multi-branch feature extraction is performed on the event video to obtain spatial features, including: in the temporal branch, the event video is divided based on the second time window to obtain the second video segment; features are extracted from the second video segment to obtain temporal features.

[0049] Accordingly, multi-branch feature extraction is performed on the event video to obtain motion features, including: in the motion branch, the event video is divided based on adjacent frames to obtain video frame pairs; and feature extraction is performed on the video frame pairs to obtain motion features.

[0050] The first time window is smaller than the second time window.

[0051] In some embodiments, the first time window is set to 0.5 seconds, corresponding to uniformly sampling 16 frames from the original video at 32 frames / second as the first video segment sequence; the second time window is set to 5 seconds, corresponding to continuously extracting 160 frames from the original video as the second video segment sequence.

[0052] It is understood that the embodiments of this application extract spatial features based on short time-series segments to capture the instantaneous position and posture of the participants in the event video; and extract temporal features based on long time-series segments to model the continuous change process and contextual association of the participants' actions.

[0053] In some embodiments, multi-branch feature extraction is performed on the event video to obtain spatial features, including: in the spatial branches, multi-branch feature extraction is performed on the event video based on a lightweight three-dimensional convolutional neural network to obtain spatial features.

[0054] Among them, the lightweight 3D convolutional neural network is a 3D convolutional network architecture that improves inference efficiency by reducing the number of network parameters and computational complexity, and can be used to extract static visual features.

[0055] In some embodiments, the lightweight 3D convolutional neural network includes a 3D residual network (3D ResNet) with its channel count pruned (e.g., reducing the number of channels in the first convolutional layer from 64 to 32) and standard 3D convolutions replaced with depthwise separable convolutions to achieve lightweighting. The network contains 5 residual blocks (including downsampling layers), and the output feature map resolution is progressively reduced to 7×7.

[0056] In some embodiments, after dividing the event video using a first time window, a sequence of 16 RGB images with a resolution of 224×224 per frame is obtained, which is the first video segment. Input the first video segment into a lightweight 3D ResNet to obtain spatial features. That is, a feature map sequence with 16 frames of spatial features, each frame having a feature map size of 7×7, and 512 channels.

[0057] It is understood that the embodiments of this application extract spatial features through a lightweight three-dimensional convolutional neural network, which can reduce computational overhead while maintaining feature representation capabilities and meet the real-time requirements of event commentary.

[0058] In some embodiments, multi-branch feature extraction is performed on the event video to obtain temporal features, including: in the temporal branch, feature extraction is performed on the event video based on a temporal convolutional network to obtain temporal features.

[0059] Temporal convolutional networks are a type of temporal network architecture built by stacking causal convolutions and dilated convolutions. They can be used to extract the continuous changes in the actions of the participating subject. Causal convolutions ensure that the output at the current moment depends only on the input at previous moments, while dilated convolutions increase the receptive field through a dilation factor to capture long-term temporal dependencies.

[0060] In some embodiments, the temporal convolutional network includes 4 layers of causal convolutions with a dilation factor sequence of [1, 2, 4, 8], receptive fields of 2, 4, 8 and 16 frames respectively, the number of channels is set to 256, and the Rectified Linear Activation Function (ReLU) is used.

[0061] In some embodiments, after dividing the event video using a second time window, a sequence of 160 frames of optical flow feature maps, each with a resolution of 224×224, is obtained, which is the second video segment. The second video segment is input into a temporal convolutional network to obtain temporal features. That is, the feature vector sequence of the feature sequence with 256 dimensions and 16 time steps.

[0062] It is understood that the embodiments of this application extract temporal features through temporal convolutional networks, which can model the continuous change process of the actions of the participating subjects, capture long-term temporal dependencies, and provide a temporal semantic basis for subsequent tactical pattern recognition.

[0063] In some embodiments, multi-branch feature extraction is performed on the event video to obtain motion features, including: in the motion branch, feature extraction is performed on the event video based on an optical flow pyramid network to obtain motion features.

[0064] Among them, the optical flow pyramid network is a network architecture that captures high-speed motion trajectories through optical flow estimation and pyramid decomposition, which can be used to extract the displacement trajectory and motion velocity of the participating subject. Optical flow estimation is based on the improved Horn-Schunck algorithm to calculate the dense optical flow field between adjacent frames, and outputs optical flow features containing velocity components in the x and y directions; pyramid decomposition achieves multi-scale motion vector alignment from coarse to fine by constructing multi-level resolution optical flow pyramids.

[0065] In some embodiments, the optical flow pyramid network constructs a 3-layer optical flow pyramid, with the resolution of each layer halved layer by layer, namely 2242, 1122, and 562. Deformable convolution is introduced during feature fusion at the top layer to align multi-scale motion vectors.

[0066] In some embodiments, consecutive frames of the event video are processed. Optical flow estimation is performed to obtain the optical flow field. The optical flow field is a three-dimensional tensor of H×W×2, representing the motion velocity of each pixel in the x-axis and y-axis directions; the optical flow field is input into the optical flow pyramid network to obtain motion features. That is, a sequence of optical flow feature maps with motion features of 16 frames, each with a resolution of 224×224, containing two components: horizontal velocity and vertical velocity.

[0067] It is understood that the embodiments of this application extract motion features through optical flow pyramid networks, which can capture high-speed motion information such as ball trajectory and player displacement direction in event videos, providing a motion semantic basis for subsequent tactical pattern recognition.

[0068] In some embodiments, feature fusion of spatial features, temporal features, and motion features is performed to obtain video features, including: determining the fusion weights corresponding to spatial features, temporal features, and motion features based on an attention mechanism; and performing feature fusion of spatial features, temporal features, and motion features based on the fusion weights to obtain video features.

[0069] The attention mechanism dynamically calculates the corresponding fusion weights based on the pooling information of each branch feature to adjust the contribution of each branch feature to the video feature. The pooling information includes, but is not limited to, the global average, the maximum value, and the histogram coding distribution.

[0070] It is understood that the process of generating video features in the embodiments of this application is as follows: Figure 3 As shown, by dynamically adjusting the fusion weights of each branch feature through the attention mechanism, the features of key dimensions can be adaptively strengthened according to the game scenario. For example, in high-speed motion scenarios such as shooting in a football game, the fusion weights of motion features are automatically increased based on the attention mechanism to enhance the contribution of motion information to video features, avoid mutual interference between static and dynamic features, and improve the expressive power of video features.

[0071] In some embodiments, before performing feature fusion on spatial features, temporal features, and motion features, the method further includes: performing feature alignment on spatial features, temporal features, and motion features.

[0072] It should be noted that, in order to unify the features extracted from different branches into the same representation space for subsequent fusion, cross-modal feature alignment is required for spatial, temporal, and motion features. Specifically, spatial features are reduced to 256 channels using 1×1 convolutions; temporal features are reshaped to 256×16×1×1 by expanding the spatiotemporal dimension; motion features are extracted by concatenating the velocity components in the x and y directions and then using 3D convolutions to obtain a high-dimensional representation; finally, all branch features are uniformly mapped to a shared embedding space. Where d is the feature dimension, specifically d=512, and T is the time step.

[0073] In some embodiments, the fusion weights of the attention mechanism are obtained based on the following formula:

[0074] in, The learnable weight matrix is ​​represented by Avg, which represents pooling of the time features by the global average value; Max, which represents pooling of the motion features by the maximum value; and Hist, which represents pooling of the motion features by the histogram-encoded distribution.

[0075] Furthermore, to achieve temporal consistency adjustment, a gated recurrent unit (GRU) is introduced to smoothly transition the fusion weights. That is, the fusion weights at the current time step are calculated based on the hidden state of the previous time step and the current input features. , where h t-1 Let σ be the hidden state at the previous time step, and σ be the Sigmoid function.

[0076] In some embodiments, the fusion calculation of video features is as follows:

[0077] The fusion weights can be restricted by the loss function so that the sum of the fusion weights a1, a2, and a3 corresponding to the features of each branch is equal to 1.

[0078] In some embodiments, long-term temporal dependencies can be further modeled on the fused features using a Long Short-Term Memory (LSTM) network to output the final video features.

[0079] It should be noted that the ST-MSFN model can be trained and optimized using a multi-task loss function. Specifically, the spatial branch uses cross-entropy loss for pose classification training; the temporal branch uses temporal contrast loss for action continuity modeling; the motion branch introduces optical flow reconstruction loss for motion feature learning, and the optical flow reconstruction loss can use the L2 norm; the fusion feature uses a supervised loss jointly optimized by classification and regression. Through multi-task joint training, the accuracy of feature extraction and fusion in each branch is improved. The multi-task loss function is shown in the following equation:

[0080] Among them, L spatial For spatial branching loss, L temporal For time branch loss, L motion For motor branch loss, L fusion To incorporate feature supervision loss.

[0081] In some embodiments, model distillation techniques or 8-bit integer quantization of the convolutional layer weights may be employed to reduce the computational overhead of the ST-MSFN model and improve inference speed.

[0082] In some embodiments, the ST-MSFN model is configured in the video understanding module. The ST-MSFN model includes a spatial branch module, a temporal branch module, a motion branch module, and a feature fusion layer module. The input format, output dimension features, and technical parameters of each sub-module are shown in the table below: Table 1

[0083] Accordingly, in this embodiment, the process of generating video features based on the spatial branching module, temporal branching module, motion branching module, and feature fusion layer module is as follows: Figure 4 As shown.

[0084] In some embodiments, hierarchical event detection is performed on video features to obtain the results of each hierarchical event, including: in the first-level event detection, region of interest features are extracted from the video features to obtain region features; and the event participants are detected by combining the video features and region features to obtain the atomic action events of each event participant.

[0085] It should be noted that, in this embodiment, feature extraction is performed on the event video to obtain video features fused from multiple branches. Then, a three-layer detection structure is used to perform hierarchical event detection on the video features. Specifically, the first-layer detector outputs atomic action events; the second-layer detector performs tactical pattern recognition; and the third-layer detector predicts strategic intent. The three layers of detectors work sequentially and collaboratively to achieve a deep analysis of the event content from micro-actions to macro-strategies.

[0086] Understandably, the first-layer event detection is implemented based on the first-layer detector.

[0087] Among them, region of interest feature extraction refers to extracting regional features of the participants in the event from video features; regional features are used to characterize the local fusion feature information of the participants in the event within that region.

[0088] Here, the participants in a competition are those who participate in the competition based on the competition rules, including but not limited to the participants and the equipment used in the competition; for example, in a football match, the participants include but are not limited to: players, football, and referees.

[0089] Understandably, extracting region of interest features from video features can narrow the detection scope from global video features to regional features where the main participants in the event may exist, thereby reducing the computational load of subsequent target detection, while avoiding interference from background features and improving detection accuracy and efficiency.

[0090] In some embodiments, region features are extracted from video features based on the Region of Interest Alignment (RoIAlign) model to obtain region features. The RoIAlign model avoids quantization errors and preserves the spatial accuracy of region boundaries, making it suitable for multi-target detection scenarios with small target sizes, such as those found in sporting events.

[0091] Understandably, using video features and regional features to detect participants in a competition can improve the accuracy and robustness of detection by combining global contextual information with local fine features.

[0092] Among them, atomic action events include target detection results and motion states.

[0093] Here, the target detection results are used to characterize the static attributes of the event participants, including one or more of the following: target type, location information, and key point information. Among them, the target type can be pre-configured according to the event type. For example, in a football event, the target type can be defined as player, football, referee, etc. The target type can be represented by the category with the highest classification confidence, which is used to characterize the credibility of the detection results. The location information is the regional position of the event participants in the event video frame, which can be represented in the form of bounding box coordinates. The key point information is the position coordinates of the key sampling parts of the event participants, such as the direction of a player's legs and the rotation direction of the football in a football event scene. The number of sampled key points can be pre-configured.

[0094] In some embodiments, the static attribute detection task of the event participants in the first-layer event detection includes target type detection, bounding box detection, and key point detection; correspondingly, the target detection results include classification confidence. Bounding box coordinates and key point heatmap .

[0095] In the bounding box coordinates, x, y, w, and h are the horizontal coordinate of the center point of the bounding box, the vertical coordinate of the center point of the bounding box, the width of the bounding box, and the height of the bounding box, respectively. The resolution of the key point heatmap is one-quarter of the resolution of the original image. H / 4 and W / 4 represent the height and width of the heatmap, respectively, and 17 is the number of key points configured.

[0096] Here, motion state is used to characterize the dynamic attributes of the participants in the event, including one or more of the following: horizontal velocity, vertical velocity, motion direction angle, angular velocity, resultant velocity, and acceleration. Specifically, horizontal velocity and vertical velocity represent the velocities of the participants in the horizontal and vertical directions, respectively; motion direction angle represents the angle of the motion direction; angular velocity represents the rotational angular velocity of the body or apparatus; resultant velocity represents the magnitude of the overall velocity; and acceleration represents the rate of change of velocity. Motion state can be obtained by differential calculation of position changes over multiple consecutive frames.

[0097] It is understood that the atomic action events obtained by the first-layer event detection in this application embodiment provide basic action semantic information for subsequent tactical pattern recognition and strategic intent prediction, supporting the layer-by-layer analysis from micro-actions to macro-strategies.

[0098] In some embodiments, the first-layer detector employs an improved YOLOv7 network architecture. The backbone network in this architecture is configured as an EfficientRep structure, and the number of module repetitions in cross-stage partial network modules (CSPStage) is reduced to one-third of the original value (e.g., the original depth [1, 6, 15, 10] is reduced to [1, 2, 5, 3]), with the input resolution set to 640×640. This lightweight design reduces the computational cost of first-layer event detection, improves the inference speed of atomic action events, and meets the real-time requirements of event commentary.

[0099] It is understood that, in some embodiments, the input features of the first-layer detector include fused features based on the output of the ST-MSFN model. and the regional features extracted by the RoIAlign model. The output includes the target detection results. and motion state The object detection results include four bounding box coordinates, a classification confidence score, and 17 keypoint information, as well as the v value in the motion state. x v yθ, ω, |v|, and α are respectively the horizontal velocity, vertical velocity, direction angle, angular velocity, resultant velocity, and acceleration.

[0100] In some embodiments, hierarchical event detection is performed on video features to obtain event results for each layer, including: in the second layer event detection, constructing a relationship graph between the main participants in the event based on atomic action events; and predicting the tactical type and the probability of interaction between the main participants based on the relationship graph using a first prediction model to obtain tactical pattern events.

[0101] The first prediction model is obtained after training based on the first loss constraint, which is determined based on the event tactical rules; the tactical pattern event includes tactical type and subject interaction probability.

[0102] It should be noted that the embodiments of this application adopt a three-layer detector structure designed in series. After the first-layer detector outputs atomic action events that can be directly identified from the video features, the atomic action events are used as the input of the second-layer detector. The second-layer detector captures the cooperative relationship between multiple subjects by constructing a relationship graph of players, and then identifies local tactical patterns.

[0103] The relationship graph includes nodes and edges connecting the nodes. Nodes represent some or all of the participants in the event. For example, in a football event, the nodes in the relationship graph represent players. Edges represent the collaborative relationships or potential interactive behaviors between the participants in the event.

[0104] In some embodiments, constructing a relationship graph between the participants in the competition based on atomic action events includes: obtaining node features based on atomic action events, and determining the edges connecting the nodes according to a preset edge construction strategy; and constructing a relationship graph between the participants in the competition based on the node features and the edges.

[0105] Among them, node features are the atomic action features corresponding to the nodes, which can be generated based on information such as bounding box coordinates, key points and motion states output by the first layer detector.

[0106] In some embodiments, in a football match scenario, node features may further include ball possession status. In this context, 0 represents a state without the ball, and 1 represents a state with the ball. Accordingly, in the first-level event detection, the first-level detector will also detect the player's possession status.

[0107] In some embodiments, node features are represented by feature vectors; wherein the vector dimension of the node features is 128, as shown in the following formula:

[0108] Roi represents the regional features.

[0109] Among them, the edge construction strategy is pre-configured according to the type of competition and the corresponding competition rules. For example, in football, the edge construction strategy includes spatial adjacency, motion consistency and ball possession association. Spatial adjacency is based on the Euclidean distance between players and connects players who are close to each other. Motion consistency is based on the consistency of the speed direction of players and connects players with similar speed directions according to the cosine similarity of the speed direction. Ball possession association directly connects the ball carrier with the potential receiver for predicting passing intentions.

[0110] Understandably, by constructing a player relationship graph, the individual behaviors of the participants in the event can be transformed into a graph structure of multi-subject collaboration, providing a semantic basis for tactical pattern recognition.

[0111] It should be noted that the second-layer detector sets up a first prediction model, which is used to perform semantic parsing of the relationship graph at the tactical pattern level; in some embodiments, the first prediction model includes a Spatial-Temporal Graph Attention Network (ST-GAT), which is used to update the node features and edge weights in the relationship graph.

[0112] Here, the ST-GAT model is a graph neural network that stacks multiple layers of graph attention modules and combines multi-head attention mechanism and edge feature fusion to perform message passing and feature aggregation on node features and edge weights in the relationship graph. It can effectively capture the spatiotemporal dependencies between the participants in the event, enhance the expressive power of node features, and provide richer graph structure semantic information for subsequent tactical pattern recognition.

[0113] In some embodiments, the ST-GAT model uses a stacked 3-layer graph attention module, each layer containing a multi-head attention mechanism and edge feature fusion. The hidden layer dimension is set to 64, the number of attention heads is 4, and the dropout is 0.3. Understandably, through message passing and feature aggregation, the ST-GAT model updates node features and edge weights, outputting an updated player relationship graph.

[0114] In some embodiments, the ST-GAT model is designed as a 3-layer graph attention module, each layer containing a multi-head attention mechanism module and an edge feature fusion module. The hidden layer dimension is set to 64, the number of attention heads is 4, and the random dropout is 0.3. The edge features output by the multi-head attention mechanism module and the edge feature fusion module are shown in the following formula:

[0115] in, It is a linear transformation matrix. The attention parameter is ||, which represents the concatenation operation.

[0116] In addition to the edge features output by the multi-head attention mechanism module, the edge feature fusion module also introduces the relative position vectors between the participants in the competition. and relative velocity vector The edge features are taken as input; correspondingly, the edge feature fusion module outputs the updated edge features as shown in the following equation:

[0117] Among them, MLP represents the Multilayer Perceptron, which is a feedforward artificial neural network model consisting of an input layer, one or more hidden layers and an output layer. It is used to perform nonlinear transformation and fusion on the edge features of the input and output the enhanced edge feature representation.

[0118] Understandably, the ST-GAT model processes the initially constructed relational graph to obtain updated edge features, and then updates the node features based on the updated edge features. The specific updated node features are shown in the following formula:

[0119] Understandably, after the ST-GAT model generates updated node features and edge features, it obtains an updated relationship graph.

[0120] It should be noted that the tactical pattern event in this application embodiment includes tactical type and subject interaction probability. The tactical type represents the category of local tactics executed by multiple participating subjects in the current event, and the subject interaction probability represents the probability of behavioral interactions occurring between participating subjects beforehand; for example, in a football match, the subject interaction probability can be the probability of predicting passing intentions between players.

[0121] In some embodiments, the first prediction model includes a graph isomorphic network; accordingly, based on the first prediction model, tactical type prediction of the relationship graph includes: tactical type prediction of the relationship graph based on the graph isomorphic network.

[0122] Among them, Graph Isomorphism Network (GIN) is a type of graph neural network used to encode and aggregate node features in graph structures. The GIN model aggregates neighbor node information through learnable parameters, which can effectively distinguish different graph structures.

[0123] Understandably, the GIN model uses the updated relationship graph generated by the ST-GAT model to predict tactical types.

[0124] In some embodiments, tactical type prediction of a relational graph based on a graph isomorphic network includes: extracting neighborhood subgraphs of target nodes in the relational graph based on the graph isomorphic network; performing feature aggregation on the extracted neighborhood subgraphs; and outputting the corresponding tactical type.

[0125] Here, the target node is a node in the relationship graph whose node features contain the target features; for example, in a football match scenario, the target node is the player with the ball.

[0126] It is understandable that neighborhood subgraph extraction is performed based on the target node. The extracted neighborhood subgraph contains other nodes that may have cooperative relationships or interactive behaviors with the target node, so as to focus on node information related to tactical patterns.

[0127] Here, tactical types are generated based on the event knowledge base, including tactical tags and corresponding feature descriptions. Understandably, after obtaining the subgraph features from the set, the subgraph features are matched with the feature descriptions corresponding to each tactical tag, and the tactical tag with the highest matching degree is used as the tactical type prediction result.

[0128] In some embodiments, the second-layer detector uses a GIN model to identify ball-handling players in the relationship graph and performs feature aggregation on a two-hop neighborhood subgraph centered on the ball-handling player, outputting tactical labels through global average pooling and fully connected layers.

[0129] In some embodiments, within a football match scenario, tactical types include, but are not limited to: wing play, triangular passing, cross-field runs, crossing tactics, high pressing, deep-lying support, triangular passing, and set-piece tactics. The characteristic descriptions corresponding to each tactical type are shown in the table below: Table 2

[0130] In some embodiments, the first prediction model includes a multilayer perceptron (MLP); accordingly, based on the first prediction model, subject interaction probability prediction of the relationship graph includes: subject interaction probability prediction of the relationship graph based on the multilayer perceptron.

[0131] Understandably, the MLP model uses the updated relationship graph generated by the ST-GAT model to predict the probability of subject interactions.

[0132] Specifically, in the prediction of subject interaction probability, feature parameters that are strongly correlated with the subject interaction probability are extracted from the edge features of each edge, and the enhanced edge feature representation is calculated with the extracted feature parameters. The feature parameters include, but are not limited to: relative position vector, relative velocity vector, cosine value of the angle between the motion directions, and Euclidean distance. Then, the enhanced edge feature representation is input into a multilayer perceptron, and after nonlinear transformation, the subject interaction probability is output. The value range of the subject interaction probability is 0 to 1.

[0133] Understandably, in the context of football matches, the subject interaction probability represents the probability of predicting passing intentions. By predicting passing intentions, potential passing routes between players can be identified, providing more granular interaction information for tactical pattern recognition, while also enhancing the ability of generated match commentary to predict the course of the game.

[0134] For example, such as Figure 5 As shown, based on the first prediction model, the predicted probability of passing intention between player A and player B is 0.92, and the predicted probability of passing intention between player A and player C is 0.78. Player D defends and intercepts player A, and player E and player C coordinate their runs. The tactical type is determined to be a triangular combination centered on player A.

[0135] It should be noted that, to further address the issue of poor logical consistency in the generated match commentary and its inconsistency with match facts and rules, this application's embodiments, in addition to constructing positive prediction strategies such as edge construction strategies and tactical type prediction strategies based on match tactical rules, also incorporate match tactical rules as loss constraints into multi-task joint training during the model training phase. This guides the model to learn prediction results that conform to tactical logic, avoiding logical contradictions generated by purely data-driven models. For example, in football matches, rule constraints ensure that the distance between the three players corresponding to the "triangle pass" tactic is less than a preset threshold, thereby enhancing the interpretability and tactical rationality of the model's predictions.

[0136] In some embodiments, the first loss constraint is a tactical logic consistency loss constraint, as shown in the following equation:

[0137] Among them, s t , where is the tactical confidence level, and Rule(·) is the expert rule scoring function, determined based on the event's tactical rules.

[0138] In some embodiments, during the model inference phase, the first prediction model is trained using a multi-task joint training approach. The total loss constraint of the first prediction model is shown in the following equation:

[0139] Among them, the tactical mode prediction loss constraint Ltactic The cross-entropy loss is shown in the following formula:

[0140] Where yt is the actual tactical label corresponding to type t. This represents the probability distribution corresponding to the tactical labels.

[0141] Among them, the subject interaction probability prediction loss constraint L pass The binary cross-entropy loss is shown in the following formula:

[0142] Where, p ij Let y be the probability of interaction between the subjects. ij The labels represent actual interactions; positive samples are the paths where interactions actually occurred, while negative samples are obtained based on random sampling.

[0143] Here, the weight ratios of the tactical mode prediction loss constraint, the subject interaction probability prediction loss constraint, and the first loss constraint are 1:0.5:0.3.

[0144] Understandably, during model training, gradients are calculated based on the total loss constraint, and the network parameters of the first prediction model are updated layer by layer through the backpropagation algorithm, allowing the model's prediction results to gradually approach the true labels. Through multi-task joint training, the model can simultaneously optimize three tasks: tactical type prediction, subject interaction probability prediction, and tactical logic consistency, thereby improving the model's generalization ability and prediction accuracy.

[0145] It is understood that the tactical pattern events obtained through the second-layer event detection in this application embodiment can identify local tactical patterns formed by the collaboration of multiple entities, providing tactical-level semantic information for subsequent strategic intent prediction.

[0146] In some embodiments, hierarchical event detection is performed on video features to obtain event results for each layer, including: in the third layer event detection, extracting temporal features from video features; processing the temporal features and event metadata based on a recurrent neural network and a global attention mechanism to obtain context features; and predicting strategic intent based on the context features using a second prediction model to obtain strategic intent events.

[0147] The second prediction model is obtained after training based on the second loss constraint, which is determined based on the event strategy rules.

[0148] Here, the temporal features are also called long temporal features, which are obtained by extracting video features by setting a sliding window; in some embodiments, the length of the sliding window is set to 30 frames and the step size is set to 5 frames.

[0149] Here, event metadata is used to characterize macro-level information during the event. For example, in a football event scenario, event metadata includes one or more of the following: event time, event score, and event progress.

[0150] It should be noted that this embodiment uses both temporal features and event metadata as input to the third-layer detector, enabling the fusion of continuous action change information and macro-level event situation information from the video content. Specifically, unlike capturing the instantaneous actions of event participants, temporal features capture the continuous change process of these actions, reflecting the behavioral evolution patterns of participants within a specific time period. Event metadata provides macro-level context such as scores, times, and match stages, reflecting the objective logic behind the participants' behavior. It can be understood that the combination of temporal features and event metadata provides multi-dimensional information support from action to context for strategic intent prediction.

[0151] Here, recurrent neural networks (RNNs) are neural network architectures used to process sequential data, capable of capturing long-range dependencies in temporal features. It can be understood that by processing temporal features using RNNs, it is possible to effectively model the continuous changing patterns of the actions of participating entities during a competition, capture the temporal dependencies in tactical execution and strategic transitions, and provide temporal semantic support for predicting strategic intent.

[0152] In some embodiments, time features and event metadata are processed based on a two-layer Long Short-Term Memory (LSTM) network to obtain context features.

[0153] The LSTM model is a type of recurrent neural network that controls the memory and updating of information through three gating mechanisms: the forget gate, the input gate, and the output gate. The forget gate determines which information is discarded from the historical state, the input gate determines which new information is stored in the cell state, and the output gate determines which information is output from the cell state. This gating structure allows the LSTM model to effectively alleviate the gradient vanishing and gradient exploding problems found in traditional recurrent neural networks, making it suitable for modeling long-term sequence action changes in sports videos.

[0154] It is understood that, in the embodiments of this application, the use of a two-layer LSTM model to encode time features can capture the behavioral evolution patterns of event participants over a longer time window, providing rich time feature representations for strategic intent prediction, thereby improving the accuracy of prediction.

[0155] It should be noted that when processing long-term temporal features, the two-layer LSTM model suffers from the problem that early information is easily forgotten during transmission, making it difficult for the model to capture the significant impact of early information on strategic intent. Based on this, the embodiments of this application introduce a global attention mechanism on the two-layer LSTM model. By calculating the attention weights at each time step, key frames in the match video are dynamically focused, and the hidden states output by the two-layer LSTM model are weighted and summed to generate contextual features. This enhances the model's ability to focus on key frames such as the moment of a shot, thereby improving the accuracy of strategic intent prediction.

[0156] In some embodiments, the context features are vectors of dimension 256. ; where the global attention weight α t The following formula was used to calculate:

[0157]

[0158] Among them, W a b is a learnable weight matrix. a h is a learnable bias term. t This represents the hidden state output by the two-layer LSTM model at time step t. The attention weights α at each time step are obtained by normalization using the Softmax function. t Then, a weighted sum is performed on all hidden states to obtain the context feature c.

[0159] It should be noted that in the third-layer event detection, after generating contextual features, the contextual features and event metadata are concatenated, and the concatenated features are input into the second prediction model. The second prediction model is used for strategic intent prediction and includes a multilayer perceptron (MLP). The MLP consists of multiple fully connected layers and nonlinear activation functions. In this embodiment, the MLP structure of the second prediction model is a two-layer fully connected network with hidden layer dimensions of 256 and 64, respectively. This network is used to perform nonlinear transformations on the concatenated contextual features and event metadata and output strategic intent labels.

[0160] Here, the strategic intent label represents the type of strategic intent adopted by the participants or teams in the current stage of the competition. For example, in the context of a football match, strategic intent includes, but is not limited to, counter-attacking, possession and pressure, pressing forward with all lines, and tightening the defense.

[0161] It should be noted that, in order for the second prediction model to accurately output the corresponding strategic intent labels, this application embodiment pre-constructs a strategic intent knowledge base containing judgment rules corresponding to each strategic intent label. In some embodiments, the judgment rules adopt IF-THEN logical expressions and are constructed based on expert knowledge in the event domain. For example, in a football match scenario, the following judgment rules can be configured: when the match time exceeds 80 minutes and the score difference is 1 point, it is judged as a defensive strategy (DEFEND); when the possession rate is less than 40% and the number of goals is 0, it is judged as an offensive strategy (AGGRESSIVE); when there is 1 red card and the match time exceeds 60 minutes, it is judged as a defensive strategy (DEFEND); when the score difference is 2 points and the remaining time is less than 10 minutes, it is judged as a control strategy (CONTAIN); when there are more than 3 corner kicks and the score is 0, it is judged as a pressing strategy (PRESSURE). These judgment rules are used for rule consistency loss constraints during the model training phase and as auxiliary references during the inference phase. The judgment rules and corresponding results are shown in the following table: Table 3

[0162] It should be noted that while the training method for the first prediction model is similar, to further address the issue of poor logical consistency in the generated match commentary, which may not conform to the facts and rules of the match, this embodiment of the application introduces the match strategy rules as a loss constraint into the multi-task joint training during the training phase of the second prediction model. This guides the model to learn prediction results that conform to strategic logic, avoiding logical contradictions generated by purely data-driven models. For example, in a football match, when the match time exceeds 80 minutes and the team is only leading by one goal, the strategic rules should favor a defensive strategy. If the model predicts an all-out attack, the loss of rule consistency will increase, guiding the model to correct its prediction direction, thereby enhancing the interpretability and tactical rationality of the model's predictions.

[0163] In some embodiments, the second loss constraint is a strategic logical consistency loss constraint, as shown in the following equation:

[0164] Among them, s r For strategic confidence, Rule(·) is the expert rule scoring function, determined based on the strategic rules of the match. For example, rule R1 requires that the output result 1, representing the defensive strategy, be satisfied when the match lasts more than 80 minutes and the team is only leading by one goal.

[0165] In some embodiments, during the model inference phase, the second prediction model is trained using a multi-task joint training approach. The total loss constraint of the second prediction model is shown in the following equation:

[0166] Among them, the strategic intent prediction loss constraint Ls trat The cross-entropy loss is shown in the following formula:

[0167] Among them, y s The label representing the true strategic intent corresponding to type s. This represents the probability distribution corresponding to the strategic intent.

[0168] Among them, the pseudo-label distillation loss constraint L distill This is used to transfer knowledge generated from the strategic intent knowledge base to the second prediction model. Pseudo-labels are generated from the event metadata using the strategic intent knowledge base, and the difference between the pseudo-labels and the model's prediction results is calculated as the distillation loss. The specific constraints of the pseudo-label distillation loss are as follows:

[0169] In some embodiments, a dynamic threshold adjustment strategy is adopted to adjust the threshold or weight of the judgment rule for strategic intent according to the event time and / or the event progress; for example, the triggering conditions of the defensive strategy are relaxed after 80 minutes, and the weight of the offensive rule is strengthened during injury time.

[0170] It is understood that the strategic intent events obtained through the third-layer event detection in this application embodiment can analyze the team's strategic intent from a macro perspective, providing strategic-level semantic information for generating match commentary with deep semantic understanding.

[0171] In some embodiments, generating event commentary based on hierarchical event results includes: parsing the hierarchical event results to determine the matching event type; determining the corresponding commentary template based on the matching event type; obtaining event content and user preference configuration, and determining commentary modification content according to the user preference configuration; and supplementing the event content and commentary modification content into the commentary template to generate event commentary.

[0172] It should be noted that this application embodiment performs hierarchical event detection on video features based on a three-layer detection structure. After outputting atomic action events, tactical pattern events, and strategic intent events respectively, the output hierarchical event results are mapped to interpretable semantic events through event parsing. Specifically, this application embodiment sets up an event parser. The event parser predefines multiple event types. After the three-layer detection structure outputs hierarchical event results, the hierarchical event results and event metadata are matched with the trigger conditions corresponding to the predefined event types to determine the matched event types. For example, in a football match scenario, if an atomic action event indicates a shooting action and the match score changes, the matched event type is determined to be "goal"; if a tactical pattern event indicates a cross-run tactic and an atomic action event indicates that the home team player has the ball, the matched event type is determined to be "tactical combination".

[0173] In some embodiments, after determining the matching event type, the method further includes: contextually associating the matching event type with historical events.

[0174] Understandably, after determining the event type that matches the hierarchical event results, it is also combined with the historical events of the match to generate contextual enhancement information, thereby enhancing the contextual relevance of the generated match commentary. For example, in a football match scenario, if the currently determined matching event type is "goal," then by combining historical events and based on the number of consecutive shots, the event type is further determined to be "third consecutive shot," thus generating the commentary text "It's him again! Third consecutive shot!"

[0175] Here, the commentary template is a pre-set commentary text framework containing fixed statements and fillable parameterized variables. This application embodiment pre-sets multiple commentary templates, with one or more templates corresponding to each event type. For example, for the "shooting" event type, templates for long-range shots, headers, and volleys can be pre-set; for the "goal" event type, templates for long-range goals, rebound goals, and penalty kicks can be pre-set.

[0176] In some embodiments, the narration template is shown in the table below: Table 4

[0177] In some embodiments, if an event type corresponds to multiple explanation templates, the corresponding explanation template is determined based on the matching event type, including: determining candidate explanation templates based on the matching event type; and determining the target explanation template based on event characteristics.

[0178] Among them, the event features are obtained by processing one or more of the hierarchical event results, event metadata, and contextual enhancement information.

[0179] Specifically, this embodiment uses the BERT model to semantically encode one or more of the hierarchical event results, event metadata, and contextual enhancement information to obtain vectorized event features. Then, the semantic similarity between the event features and each candidate commentary template is calculated. Simultaneously, the rule matching score is combined with the semantic similarity and the rule matching score, and a weighted sum is taken to select the commentary template with the highest overall score as the target commentary template. The calculation method for the overall score is shown in the following formula:

[0180] Among them, s i For the comprehensive score, e represents the event characteristics, and t represents the overall score. i Let be the feature vector of the i-th explanation template, RuleMatch(.) be the rule matching score function, α be the semantic similarity weight, and β be the rule matching score weight.

[0181] Semantic similarity can be determined based on the cosine similarity between vectors.

[0182] The rule matching score can be determined based on the degree of matching between the event type and the template type.

[0183] The semantic similarity weight is greater than the rule matching score weight; in one example, the semantic similarity weight can be set to 0.7 and the rule matching score weight can be set to 0.3.

[0184] Here, the event content is used to populate the parameterized variables in the commentary template, including one or more of the following: video features, hierarchical event results, event metadata, and the event knowledge base. The event knowledge base includes, but is not limited to: knowledge of the event participants, knowledge of the event teams, and knowledge of event terminology. For example, in a football event scenario, the event knowledge base could include player nicknames, historical records, classic events, and tactical naming conventions.

[0185] Understandably, once the commentary template is determined, information such as the main participants in the event, the event process, atomic actions, tactical patterns, and strategic intentions can be extracted from the event content. This extracted information can then replace the parameterized variables to be added to the commentary template, generating a commentary that fits the current event.

[0186] In some embodiments, the explanatory template further includes sentiment variables to be supplemented; accordingly, after determining the matching event type, the method further includes: calculating an event score; and determining sentiment modifiers based on the event score.

[0187] Here, sentiment modifiers are used to replace sentiment variables in the narration template.

[0188] Here, the event score characterizes the overall importance of the current event at the tactical, strategic, and temporal levels of the match. In some embodiments, the event score is determined based on tactical patterns, strategic intentions, match time, and match progress. For example, in a football match scenario, events involving key tactical plays (such as a game-winning assist), high strategic impact (such as equalizing the score), or time-sensitive events (such as stoppage time) receive higher event scores.

[0189] Here, this application embodiment pre-defines an emotion dictionary, which stores emotion modifiers corresponding to different emotion intensities and configures a mapping relationship between event scores and emotion modifiers. It can be understood that the higher the event score, the higher the corresponding emotion intensity, and emotion modifiers with higher passion should be selected; the lower the event score, the lower the corresponding emotion intensity, and emotion modifiers that are objective and plain or reflect regret should be selected. For example, in a football match scenario, during the emotion injection phase, if the event score is higher than the corresponding upper threshold, then high-passion emotion modifiers such as "exquisite," "groundbreaking," and "spectacular" are added to the commentary template to enhance the commentary's impact; if the event score is lower than the corresponding lower threshold, then low-passion emotion modifiers such as "regrettable missed opportunity," "slightly weak," and "frequent mistakes" are added to the commentary template to reflect the event's regret or blandness.

[0190] It is understood that the embodiments of this application quantify the overall importance of events through event scoring and dynamically match emotional modifiers based on event scores, so that the generated event commentary can automatically adjust the emotional intensity according to the tactical significance, strategic impact and time urgency of the event. Compared with the commentary generation method that uses a fixed emotional template, it significantly improves the expressiveness of the commentary and the viewing experience.

[0191] Here, the user preference configuration represents the user's preferred explanatory language style, including configuration items corresponding to one or more style dimensions.

[0192] In some embodiments, the style dimensions include four dimensions: professionalism, passion, humor, and simplicity. Users can adjust the language style of the commentary by configuring the corresponding settings when watching the event.

[0193] In some embodiments, in order to accurately determine the modification content, each configuration item is a normalized parameter; for example, for the style dimension S={professionalism, passion, humor, simplicity}, the corresponding user preference configuration is w =[0.8,0.6,0.3,0.5].

[0194] Here, the explanation of the modifiers includes one or more of the following: style modifiers or style sentence structures.

[0195] Understandably, after determining the commentary style based on user preference configurations, corresponding style modifiers are added to the commentary template, or the original sentence structure in the template is improved according to the style sentence structure, so that the generated commentary matches the user's preferred style. For example, when the humor dimension configuration item in the user preference configuration is greater than 0.5, the word "shoot" in the commentary can be replaced with humorous expressions such as "trying to date the goal"; when the professionalism dimension configuration item in the user preference configuration is greater than 0.7, tactical analysis can be added to the commentary, such as "a typical application of crosses from the wing".

[0196] Understandably, dynamically adjusting the narration style through user preference configuration can meet the personalized viewing needs of different users, improve the flexibility of narration generation, and enhance the user experience.

[0197] In some embodiments, the video understanding module is further configured with a three-layer detection structure for outputting hierarchical event results; the task of generating event commentary is performed by the event commentary generation module, which includes multiple modules such as an event parser, a template selection module, a content generator, and a style injection engine; correspondingly, the event commentary generation process is as follows: Figure 6 As shown.

[0198] Specifically, after the video understanding module outputs the hierarchical event results, it inputs the hierarchical event results and event metadata into the event parser. The event parser parses the events, determines the event types with interpretable semantics, and outputs event features. The event features are input into the template selection module, which performs commentary template matching and parameter mapping, and outputs the matched target commentary template and the event content to be supplemented. The content generator supplements the event content into the target commentary template and outputs the initial event commentary. The style injection engine adjusts the style of the initial event commentary according to the user's preference configuration and outputs the final event commentary.

[0199] In some embodiments, such as Figure 7 As shown, the event video, event metadata, and external knowledge base are used as input data. Based on the processing of each module in the video understanding module and the event commentary generation module, event commentary with deep semantic understanding can be obtained.

[0200] It is understood that this application embodiment maps atomic action events, tactical pattern events, and strategic intent events into interpretable semantic events through event parsing. Based on the semantic events, a corresponding commentary template is selected, and parameters such as participating entities, action methods, tactical coordination, and strategic background from the layered event results are filled into the commentary template, thereby achieving the integration of layered event results with the event commentary. Simultaneously, the commentary embellishment content is determined based on user preference configuration to achieve a personalized commentary style. Therefore, the event commentary generated by this application embodiment not only integrates multi-layered semantic information from micro-actions to macro-strategies but also meets the personalized viewing needs of users, enhancing the expressiveness of the commentary and the user experience.

[0201] In order to implement the method of the present application embodiment, the present application embodiment also provides a sports commentary generation device, which corresponds to the above-mentioned sports commentary generation method, and the steps in the above-mentioned sports commentary generation method embodiment are also fully applicable to the present device embodiment.

[0202] like Figure 8 As shown, the event commentary generation device in this embodiment includes an extraction module 801, a detection module 802, and a generation module 803. The extraction module 801 is used to extract features from the event video to obtain video features; the detection module 802 is used to perform hierarchical event detection on the video features to obtain the results of each hierarchical event; the hierarchical event results include: atomic action events, tactical pattern events, and strategic intent events; the generation module 803 is used to generate event commentary based on the hierarchical event results.

[0203] In some embodiments, the detection module 802 is specifically used to: extract region of interest features from video features in the first layer of event detection to obtain region features; detect the participants in the event by combining the video features and the region features to obtain atomic action events of each participant in the event; wherein, the atomic action events include target detection results and motion states.

[0204] In some embodiments, the detection module 802 is specifically used to: in the second-layer event detection, construct a relationship graph between the main participants in the event based on atomic action events; and predict the tactical type and the interaction probability of the main participants based on the relationship graph using a first prediction model to obtain tactical pattern events; wherein, the first prediction model is obtained after training based on a first loss constraint, and the first loss constraint is determined based on the event tactical rules; the tactical pattern events include tactical type and the interaction probability of the main participants.

[0205] In some embodiments, the detection module 802 is specifically used to: extract temporal features from video features in the third-layer event detection; process the temporal features and event metadata based on a recurrent neural network and a global attention mechanism to obtain context features; and predict strategic intent events based on the context features using a second prediction model; wherein the second prediction model is obtained after training based on a second loss constraint, and the second loss constraint is determined based on the event strategic rules.

[0206] In some embodiments, the extraction module 801 is specifically used to: extract multi-branch features from the event video to obtain spatial features, temporal features, and motion features; and fuse the spatial features, temporal features, and motion features to obtain video features.

[0207] In some embodiments, the extraction module 801 is specifically used to: determine the fusion weights corresponding to spatial features, temporal features and motion features based on an attention mechanism; and perform feature fusion on spatial features, temporal features and motion features based on the fusion weights to obtain video features.

[0208] In some embodiments, the generation module 803 is specifically used to: perform event parsing on the hierarchical event results to determine the matching event type; determine the corresponding commentary template based on the matching event type; obtain the event content and user preference configuration, and determine the commentary modification content according to the user preference configuration; supplement the event content and commentary modification content to the commentary template to generate the event commentary.

[0209] It should be noted that the above-described event commentary generation device is only illustrated by the division of the above-described program modules when generating event commentary. In practical applications, the above processing can be assigned to different program modules as needed, that is, the internal structure of the device can be divided into different program modules to complete all or part of the processing described above. In addition, the event commentary generation device and the video display method embodiment provided above belong to the same concept, and their specific implementation process can be found in the method embodiment, which will not be repeated here.

[0210] Based on the hardware implementation of the above program modules, and in order to implement the event commentary generation method of this application embodiment, this application embodiment also provides an electronic device, such as... Figure 9 As shown, electronic device 900 includes at least one processor 901, memory 902, user interface 903, and at least one network interface 904. The various components in electronic device 900 are coupled together via bus system 905. It can be understood that bus system 905 is used to implement communication between these components. In addition to a data bus, bus system 905 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in… Figure 9The general labeled all buses as Bus System 905.

[0211] The user interface 903 may include a monitor, keyboard, mouse, trackball, click wheel, buttons, touchpad, or touch screen.

[0212] The memory 902 in this embodiment is used to store various types of data to support the operation of the electronic device 900. Examples of such data include any computer program used to operate on the electronic device 900.

[0213] The event commentary generation method disclosed in this application can be applied to or implemented by the processor 901. The processor 901 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the event commentary generation method can be completed by the integrated logic circuitry in the hardware of the processor 901 or by instructions in software form. The processor 901 can be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor 901 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the embodiments of this application can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software modules can be located in a storage medium, specifically in memory 902. The processor 901 reads information from memory 902 and, in conjunction with its hardware, completes the steps of the event commentary generation method provided in the embodiments of this application.

[0214] In an exemplary embodiment, the electronic device 900 may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), FPGAs, general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned event commentary generation method.

[0215] It is understood that memory 902 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), EEPROM, ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), Sync Link Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM). The memory 902 described in this application embodiment is intended to include, but is not limited to, these and any other suitable types of memory.

[0216] In an exemplary embodiment, this application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, such as a memory 902 that stores a computer program. This computer program can be executed by the processor 901 of the electronic device 900 to complete the steps described in the event commentary generation method of this application embodiment. The computer-readable storage medium can be a ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disc, or CD-ROM, etc.

[0217] In an exemplary embodiment, this application also provides a computer program product, including a computer program that can be executed by a processor 901 of an electronic device 900 to perform the steps described in the method of this application embodiment.

[0218] It should be noted that terms such as "first" and "second" are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0219] Furthermore, the technical solutions described in the embodiments of this application can be combined arbitrarily without conflict.

[0220] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method of generating a sports commentary, characterized by, The method includes: Feature extraction is performed on the event videos to obtain video features; The video features are subjected to hierarchical event detection to obtain the results of each hierarchical event; the hierarchical event results include: atomic action events, tactical pattern events, and strategic intent events; Based on the results of the hierarchical events, the event commentary is generated.

2. The method of claim 1, wherein, The step of performing hierarchical event detection on the video features to obtain the results of each hierarchical event includes: In the first-layer event detection, region of interest features are extracted from the video features to obtain region features; The video features and the region features are used to detect the participants in the competition, and the atomic action events of each participant are obtained. The atomic action events include target detection results and motion states.

3. The method of claim 1, wherein, The step of performing hierarchical event detection on the video features to obtain the results of each hierarchical event includes: In the second-layer event detection, a relationship graph between the participating entities in the competition is constructed based on the atomic action events. Based on the first prediction model, the relationship graph is used to predict the tactical type and subject interaction probability to obtain the tactical pattern event; The first prediction model is obtained after training based on the first loss constraint, which is determined based on the event tactical rules; the tactical pattern event includes tactical type and subject interaction probability.

4. The method of claim 1, wherein, The step of performing hierarchical event detection on the video features to obtain the results of each hierarchical event includes: In the third-layer event detection, temporal features are extracted from the video features; Based on recurrent neural networks and global attention mechanisms, the temporal features and event metadata are processed to obtain contextual features; Based on the second prediction model, strategic intent is predicted from the context features to obtain the strategic intent event; The second prediction model is obtained after training based on the second loss constraint, which is determined based on the event strategy rules.

5. The method according to any one of claims 1 to 4, characterized in that, The process of extracting features from the event video to obtain video features includes: Multi-branch feature extraction is performed on the event video to obtain spatial features, temporal features, and motion features; The spatial features, temporal features, and motion features are fused to obtain video features.

6. The method of claim 5, wherein, The feature fusion of the spatial features, the temporal features, and the motion features to obtain video features includes: Based on the attention mechanism, the fusion weights corresponding to the spatial features, the temporal features, and the motion features are determined; Based on the fusion weights, the spatial features, the temporal features, and the motion features are fused to obtain video features.

7. The method according to any one of claims 1 to 4, characterized in that, The generation of event commentary based on the hierarchical event results includes: The hierarchical event results are parsed to determine the matching event type; Based on the matched event type, determine the corresponding explanation template; Obtain the event content and user preference configuration, and determine the commentary embellishment content based on the user preference configuration; Add the event content and the commentary embellishments to the commentary template to generate the event commentary.

8. An electronic device, comprising: include: A processor and a memory for storing a computer program capable of running on the processor, wherein the processor, when running the computer program, performs the steps of the method according to any one of claims 1 to 7.

9. A storage medium, characterized by The storage medium stores a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 7.

10. A computer program product comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.