Intelligent sports event decision system based on causal driving multi-modal fusion and spatio-temporal dynamic reasoning

Abstract

The application discloses an intelligent sports event decision system based on causal driving multi-modal fusion and space-time dynamic reasoning, which comprises a multi-modal causal data acquisition and preprocessing module, a causal orientation multi-modal knowledge graph construction and updating module, a space-time perception hybrid agent module, and a causal constraint dynamic decision optimization module. The causal orientation multi-modal knowledge graph construction and updating module is arranged with a causal structure learning algorithm and an online learning framework. The causal constraint dynamic decision optimization module is arranged with a deep reinforcement learning algorithm. An explainability analysis and visualization module is used for receiving a decision strategy sent by the space-time perception hybrid agent module, generating a decision explanation according to the decision strategy, and visually presenting the decision explanation. The application aims to integrate causal reasoning, multi-modal learning and reinforcement learning to construct an intelligent system capable of providing high-precision, explainable and self-adaptive decision support, so as to significantly improve the scientificity and effectiveness of sports event analysis and decision.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence, sports data analysis, multimodal learning, and causal reasoning, specifically to an intelligent sports event decision-making system based on causal-driven multimodal fusion and spatiotemporal dynamic reasoning. Background Technology

[0002] Sports event decision support systems are a key technology for the digital and intelligent transformation of the modern sports industry, widely used in core scenarios such as match tactical analysis, player performance evaluation, injury risk prediction, and training plan optimization. With the increasing complexity of competitive sports, the event process exhibits high-dimensionality, strong dynamics, and multimodal characteristics. Existing technologies mainly rely on statistical models and shallow machine learning algorithms, which face the following limitations when handling such complex systems: First, existing methods, primarily based on correlation analysis, struggle to distinguish between causal relationships and spurious correlations between variables. For instance, a system might find a positive correlation between a player's running distance and the number of goals scored, but it cannot determine whether this relationship stems directly from running leading to goals or is driven by shared underlying factors (such as high-pressure tactics). Decision recommendations based on correlation lack robustness and reliability in the face of changing playing conditions.

[0003] Secondly, the data generated by sports events has significant multimodal characteristics, including video, audio, text, and sensor data. Traditional methods typically process each modality of data independently or perform simple feature concatenation. This approach ignores the inherent correlations and complementary information between modalities, leading to an incomplete understanding of the game situation and limiting the model's representational capabilities.

[0004] Furthermore, sports competitions are typical spatiotemporal dynamic systems, where player positions, team formations, and offensive and defensive transitions all evolve continuously in time and space. Traditional models often treat entities as isolated units, failing to effectively model the dynamic topological structure and temporal dependencies between entities, thus making it difficult to accurately capture the evolution of tactical intentions and make effective predictions.

[0005] Finally, most existing decision-making systems are static models whose parameters are fixed after offline training. Such systems are ill-suited to unforeseen events during a match, such as changes in key players or sudden shifts in opponent tactics. They lack online learning and adaptive adjustment capabilities, resulting in time lags in their decision outputs and potentially missing optimal intervention opportunities.

[0006] In conclusion, developing a novel intelligent system capable of deeply exploring causal mechanisms, effectively integrating multimodal information, accurately modeling spatiotemporal dynamics, and possessing real-time adaptive decision-making capabilities is of great significance for improving the depth and accuracy of sports event analysis. Summary of the Invention

[0007] To address the aforementioned technical problems, the present invention aims to provide an intelligent sports event decision-making system based on causal-driven multimodal fusion and spatiotemporal dynamic reasoning. This invention aims to integrate causal reasoning, multimodal learning, and reinforcement learning to construct an intelligent system capable of providing high-precision, interpretable, and adaptive decision support, thereby significantly improving the scientific rigor and effectiveness of sports event analysis and decision-making.

[0008] This invention aims to solve the key technical bottlenecks of traditional sports decision-making systems in areas such as deep causal relationship mining, deep fusion of heterogeneous data, complex spatiotemporal dynamic modeling, and adaptive decision-making strategies by constructing a causal-oriented multimodal knowledge graph, designing a spatiotemporally aware hybrid intelligent agent architecture, and implementing a causal-constrained dynamic decision optimization mechanism.

[0009] An intelligent sports event decision-making system based on causal-driven multimodal fusion and spatiotemporal dynamic reasoning includes: The multimodal causal data acquisition and preprocessing module is used to collect and process sports data containing video, audio, text, and sensor data about sports events in real time, forming multi-source heterogeneous sports data; The causal-oriented multimodal knowledge graph construction and update module is equipped with a causal structure learning algorithm and an online learning framework. It is used to receive multi-source heterogeneous sports data sent by the multimodal causal data acquisition and preprocessing module, use the causal structure learning algorithm to mine causal relationships between variables, judge the credibility of causal relationships, and use the online learning framework to construct and dynamically update a knowledge graph containing spatiotemporal attributes based on the credibility of causal relationships. The spatiotemporal-aware hybrid intelligent agent module is used to process multi-source heterogeneous sports data and knowledge graphs containing spatiotemporal attributes to obtain decision-making strategies and causal knowledge; The dynamic decision optimization module with causal constraints is equipped with a deep reinforcement learning algorithm. It uses causal knowledge as the environment state of the deep reinforcement learning algorithm and a knowledge graph containing spatiotemporal attributes as the rule boundary of the deep reinforcement learning algorithm. The reward function guides the spatiotemporal perception hybrid agent module to generate an adaptive tactical strategy that maximizes the target benefit. The interpretability analysis and visualization module is used to receive the decision strategy sent by the spatiotemporal perception hybrid intelligent agent module, generate a decision explanation based on the decision strategy, and visualize the decision explanation.

[0010] In this invention, the multimodal causal data acquisition and preprocessing module is used to acquire and process multi-source heterogeneous sports data, including video, audio, text, sensor data, and historical data, in real time and synchronously. The causal-oriented multimodal knowledge graph construction and updating module, based on an improved causal discovery algorithm incorporating a time-aware mechanism, mines causal relationships between variables from the multimodal data and utilizes an online learning framework to construct and dynamically update a knowledge graph containing spatiotemporal attributes. The improved causal discovery algorithm employs a time-aware conditional independence test, where the independence threshold is dynamically calculated based on significance level, condition set size, and time decay factor. The causal-oriented multimodal knowledge graph construction module also includes a multimodal causal strength calculation method. This method comprehensively evaluates the confidence level of causal relationships by weightedly fusing the causal contributions calculated based on mutual information across different modalities.

[0011] A spatiotemporally aware hybrid agent cluster consists of at least four cooperating agents: a causal reasoning agent for intervention and counterfactual inference, a multimodal fusion agent for deep integration of multimodal features, a spatiotemporally modeling agent for modeling spatiotemporally dynamic dependencies between entities, and a decision optimization agent for generating and optimizing decision sequences. The spatiotemporally modeling agent employs a spatiotemporal graph neural network (STGNN), which couples a graph convolutional network (GCN) to capture spatial topology and a gated recurrent unit (GRU) to capture temporal dynamics.

[0012] The causal constraint-based dynamic decision optimization module uses causal relationships in the knowledge graph as constraints and employs a deep reinforcement learning algorithm to optimize decision strategies online. The reinforcement learning algorithm used in this causal constraint-based dynamic decision optimization module has a reward function that is a composite function, including at least a task reward to measure task completion, a causal consistency reward to encourage decisions to conform to the discovered causal logic, and a temporal consistency reward to ensure smooth state transitions.

[0013] The interpretability analysis and visualization module is used to generate decision explanations based on causal chains and counterfactual analysis, and to visualize the analysis results.

[0014] The causal structure learning algorithm described is a conditional independence test algorithm that introduces a time lag effect and a causal strength measurement method that integrates cross-modal fusion, based on the Peter-Clark algorithm.

[0015] The conditional independence test algorithm for the time lag effect uses the following formula: in, Indicates at time Given a set of condition variables In the case of variables and Whether it is conditionally independent; This represents the partial correlation coefficient. Representing variables At any moment The observed values ​​or state, Representing variables The observation or state at time t.

[0016] Representing a condition set Past time window Observations within; This represents a dynamic threshold function.

[0017] This represents a dynamic threshold function, and the specific formula is as follows: in, The inverse cumulative distribution function of the standard normal distribution; Indicates the significance level (Type I error rate); Indicates at time The number of valid samples used for calculation; Indicates the number of variables in the condition set; Indicates the time decay factor. It is the interval between the data point and the current time. It is the attenuation rate.

[0018] The causal strength measurement method for cross-modal fusion uses the following formula: in, This represents the posterior probability, i.e., the probability that the new graph structure will be obtained after observing all historical data. The probability of; Indicates the likelihood term; Represents structural priors.

[0019] The online learning framework described herein adopts the Bayesian online learning framework.

[0020] The spatiotemporal perception hybrid intelligent agent module includes at least four cooperative intelligent agents; The first intelligent agent is used to perform feature fusion on the multi-source heterogeneous sports data to obtain fused features; The second agent is used to collaboratively reason using fused features and a knowledge graph containing spatiotemporal attributes to obtain a decision strategy; The third agent is used to take player information in the knowledge graph containing spatiotemporal attributes as nodes of the spatiotemporal graph network and player relationships as edges of the spatiotemporal graph network to obtain the dynamic evolution information of the team. The fourth agent is used to intervene in the team's dynamic evolution information and knowledge graph containing spatiotemporal attributes, and to make counterfactual inferences to obtain causal knowledge.

[0021] A method for intelligent sports event decision-making, employing an intelligent sports event decision-making system based on causal-driven multimodal fusion and spatiotemporal dynamic reasoning, includes the following steps: S1: Data Acquisition and Preprocessing: Real-time acquisition of multimodal data streams and extraction of high-dimensional feature vectors.

[0022] S2: Dynamic Graph Construction: Applying time-aware causal discovery algorithms, we learn causal structures from feature data, construct and continuously update a causal knowledge graph.

[0023] S3: Agent Collaborative Reasoning: Activate a hybrid agent cluster, in which a multimodal fusion agent integrates current information, a spatiotemporal modeling agent predicts future situations, and a causal reasoning agent evaluates the causal effects of potential actions.

[0024] S4: Dynamic Decision Optimization: The decision optimization agent integrates the outputs of all agents and generates the optimal decision suggestions through reinforcement learning within a decision space constrained by causal relationships.

[0025] S5: Explanation and Output: Provides causal chains and counterfactual explanations for the generated decisions and presents them to users through a visual interface.

[0026] Compared with the prior art, the present invention has the following advantages: 1. Improve the robustness and accuracy of decision-making: By introducing causal reasoning, the decision-making basis of this invention shifts from superficial correlation to deep causal mechanisms, which significantly enhances the reliability and prediction accuracy of the model in complex dynamic environments.

[0027] 2. Enhance the comprehensiveness of information utilization: Through a multimodal fusion architecture, the system can integrate and utilize complementary information from different data sources, forming a more complete and three-dimensional representation of the situation on the field.

[0028] 3. Improve the accuracy of spatiotemporal dynamic modeling: By adopting a spatiotemporal graph neural network, the dynamic spatial relationships and temporal evolution patterns between entities can be accurately captured, thereby improving the accuracy of tactical intent recognition and game trend prediction.

[0029] 4. Achieving adaptive and real-time decision-making: The reinforcement learning framework based on causal constraints enables the system to learn online and adjust dynamically, allowing it to respond quickly to changes in the game and generate adaptive decisions.

[0030] 5. Enhance the interpretability and credibility of the system: Provide an explanation mechanism based on causal chains and counterfactual analysis, making the "black box" decision-making process of the model transparent, which greatly enhances users' understanding and trust in the system. Attached Figure Description

[0031] Figure 1 This is a flowchart of the intelligent sports event decision-making system of the present invention; Figure 2 This is a structural diagram of the multimodal causal data acquisition and preprocessing module; Figure 3 A flowchart for the dynamic construction and updating of causal-oriented knowledge graphs. Detailed Implementation

[0032] The technical solution of the present invention will be described in more detail below with reference to the accompanying drawings and specific embodiments. These embodiments are only for illustrating the technical concept of the present invention, and are not intended to limit its scope of protection.

[0033] To achieve the above objectives, the technical solution of the present invention is designed as a modular system, the architecture of which is as follows: Multimodal causal data acquisition and preprocessing module: As the input layer of the system, it is responsible for the synchronous acquisition, cleaning, alignment and feature extraction of multi-source heterogeneous data.

[0034] Causal-oriented multimodal knowledge graph construction and updating module: As the system's knowledge base, it is responsible for discovering and representing causal relationships between variables from data, forming a dynamic structured knowledge model.

[0035] Spatiotemporal Aware Hybrid Intelligent Agent Cluster: As the system's inference engine, it achieves a deep understanding and prediction of the current situation on the field through the collaborative work of multiple functionally specialized intelligent agents.

[0036] The dynamic decision optimization module with causal constraints serves as the core of the system's decision-making process, generating the optimal action strategy based on the analysis of the preceding modules.

[0037] Explainable Analysis and Visualization Module: As the system's human-computer interaction interface, it is responsible for presenting the analysis results and decision-making basis to the user in an understandable way.

[0038] like Figure 1As shown, the system starts with multimodal data input. After data acquisition and preprocessing, the data flows to the causal knowledge graph construction module and the spatiotemporal-aware agent cluster, respectively. The causal knowledge graph provides prior knowledge support for the agents, and the agent cluster generates candidate decision schemes through collaborative reasoning. The reinforcement learning decision optimization module combines causal constraints and agent reasoning results to generate the optimal decision strategy. The interpretability analysis module performs causal chain tracing and counterfactual analysis on the decision-making process, ultimately outputting interpretable decision recommendations. The system achieves closed-loop optimization through an effect feedback mechanism, continuously improving the quality of decision-making.

[0039] like Figure 2 As shown, the internal structure and data flow process of the multimodal causal data acquisition and preprocessing module are described in detail. The system simultaneously processes four types of input data: video data (including visual information such as athlete movements and venue conditions), audio data (auditory information such as live sounds and commentary), text data (semantic information such as real-time comments and historical records), and sensor data (environmental parameters such as temperature, humidity, and wind speed). Each data type undergoes corresponding preprocessing steps, including noise reduction, normalization, and format unification. Feature extraction is then performed, converting the raw data into high-dimensional feature vectors. The multimodal alignment module is responsible for establishing temporal and semantic correspondences between different modalities, while the causal relationship labeling module identifies and labels potential causal relationships in the data, ultimately generating a unified feature representation for use by subsequent modules.

[0040] like Figure 3 As shown, the dynamic construction and updating mechanism of the causal-oriented knowledge graph is illustrated. The system discovers causal relationships based on historical data analysis and real-time data streams, using methods such as conditional independence testing and Granger causality testing to identify causal connections between variables. The strength of causal relationships is quantified through causal strength calculation, assigning weights to graph edges. The graph node creation module adds newly discovered causal variables, while the edge weight update module adjusts the causal relationship strength based on the latest evidence. Graph structure optimization employs heuristic algorithms to prune redundant connections, maintaining the graph's simplicity and effectiveness. An online learning update mechanism enables the graph to adapt to changes in data distribution. The completed dynamic causal graph needs to be verified for its rationality and consistency by a validation module. Only graphs that pass validation are officially released for system use; otherwise, the system returns to the optimization steps for adjustment.

[0041] The specific implementation method of this system follows a complete processing flow from raw data to decision output, including the following steps: Step S1: Multimodal causal data acquisition and unified semantic mapping The goal of this step is to build a comprehensive, synchronized, and standardized foundation of event data to provide high-quality input for subsequent analysis.

[0042] First, through a multimodal data acquisition interface, heterogeneous data streams are captured in real time from inside and outside the competition venue, forming a complete dataset. It covers video, audio, text, sensor data, and historical data.

[0043] Subsequently, the raw data is preprocessed and features are extracted, and it is mapped to a unified high-dimensional feature space to achieve semantic alignment and machine readability.

[0044] Video feature extraction: To obtain spatiotemporal information from video streams, three-dimensional convolutional neural networks (such as I3D and 3DResNet) are used to extract features.

[0045] At a certain point in time Extracted video feature vectors.

[0046] : 3D convolution operations are used to process spatiotemporal data and capture motion and appearance information.

[0047] : With the current frame The video clips within the centered time window are used as input for Conv3D.

[0048] : The dimension of the output video feature vector.

[0049] Audio feature extraction: To extract acoustic information, acoustic features such as Mel frequency cepstral coefficients (MFCC) are used.

[0050] At a certain point in time Extracted audio feature vectors.

[0051] This is a function that calculates the Mel frequency cepstral coefficients, a feature widely used in speech recognition.

[0052] Time point The original audio signal.

[0053] The number of MFCC coefficients.

[0054] The sampling frequency of the audio.

[0055] Text feature extraction: To encode text semantics, pre-trained Transformer models (such as BERT) are used for feature extraction.

[0056] : No. The feature vector of a text.

[0057] : Bidirectional Encoder Representations from Transformers model.

[0058] : The input text sequence.

[0059] : This indicates that the vector corresponding to the [CLS] special tag in the BERT output is retrieved. This vector is usually used to characterize the semantics of the entire sequence.

[0060] Sensor and historical data processing: Interpolation and normalization are performed on time-series sensor data, and structured historical data is mapped into vectors through an embedding layer.

[0061] After this step, the heterogeneous raw data is transformed into a unified, aligned set of features, providing a foundation for subsequent knowledge discovery and intelligent reasoning.

[0062] Step S2: Dynamic Construction of Causal-Driven Multimodal Knowledge Graph After data preparation is complete, this step aims to extract deep patterns from the data and construct a knowledge graph representing the causal relationships between variables. This map is the core knowledge base of the system.

[0063] 2.1 Causal Structure Learning: Time-Aware Conditional Independence Test To accurately discover causality from time-series data, this invention employs an improved PC algorithm, the core of which is the introduction of a conditional independence test with time lag effects.

[0064] At that moment Given a set of condition variables In the case of variables and Whether the conditions are independent. Partial correlation coefficient, which measures the correlation coefficient under a set of control conditions. After the impact, and The degree of linear correlation between them. : condition set Past time window The observations within the time series were lag-adjusted to address causal relationships in the time series data. A dynamic threshold function used to determine whether a correlation is statistically significant.

[0065] The dynamic threshold The time decay effect was further considered, and more recent data was given higher weight: The inverse cumulative distribution function of the standard normal distribution. Significance level (Type I error rate). At that moment The number of valid samples used for calculation. The number of variables in the condition set. : Time decay factor, where It is the interval between the data point and the current time. It is the attenuation rate.

[0066] 2.2 Causal Strength Quantification: To integrate causal evidence from different modalities, this invention defines a cross-modal causal strength measure, which performs weighted fusion of information from each modality to obtain a robust estimate of the confidence level of causal relationships.

[0067] From variable arrive The overall causal strength. The total number of data modalities. : No. The weights of each modality are set based on its signal-to-noise ratio or prior importance. In the 1st Mutual information calculated from data across multiple modalities is used to measure nonlinear dependencies between variables. In the current causal graph, nodes The set of parent nodes. The core idea of ​​this formula is: if arrive If a direct causal relationship exists, then under control... After other direct causes Still able to Provides significant information.

[0068] 2.3 Dynamic Updates to the Knowledge Graph: To adapt to the evolution of tactics and rules in the sports field, the knowledge graph adopts a Bayesian online learning framework for dynamic updates, ensuring the timeliness of its knowledge.

[0069] Posterior probability, i.e., after observing all historical data, the new graph structure is... The probability of. Likelihood term: indicates the degree to which the new graph structure explains the new data. : Structural priors, representing the structure from the old diagram The probability of transitioning to a new graph structure introduces a constraint on the smooth evolution of the graph structure.

[0070] Through the above process, the system constructs a dynamic, multimodal knowledge graph containing causal logic, providing a structured knowledge foundation for subsequent reasoning and decision-making.

[0071] Step S3: Spatiotemporally Aware Hybrid Agent Collaborative Reasoning Based on the constructed knowledge graph, this step uses a hybrid agent cluster to perform in-depth analysis and reasoning on the current situation. This cluster consists of multiple modular agents that work together to form comprehensive situational awareness.

[0072] Multimodal Fusion Agent (MFA): This agent is responsible for deeply fusing the multimodal features extracted in step S1. Its core is a Transformer-based attention mechanism that effectively learns the intrinsic relationships between different modal features.

[0073] These are the query, key, and value matrices, obtained from the input features through linear transformation.

[0074] The formula calculates the similarity score between the query and all keys, normalizes it to attention weights using softmax, and then sums the values ​​using weighted averages to obtain context-aware fusion features.

[0075] Spatiotemporal Modeling Agent (STA): This agent focuses on modeling the spatiotemporal dynamics of a match. It employs a Spatiotemporal Graph Neural Network (STGNN), treating entities such as players and the ball as graph nodes and their dynamic relationships as edges, thereby capturing the evolution of tactical structures.

[0076] Spatial Dependency Modeling (GCN): This formula is a propagation rule for one layer of a graph convolutional network, which updates the feature representation of the central node by aggregating information from neighboring nodes. . To add a self-loop to the adjacency matrix, Let it be its degree matrix.

[0077] Time-dependent modeling (GRU): This formula is the update equation for a gated recursive unit (GRU) used to model time series. It is the input at the current moment (usually the output of GCN). It is the hidden state from the previous moment. It is the updated hidden state.

[0078] Causal Reasoning Agent (CRA): This agent is the key difference between this invention and traditional models. It utilizes the causal graph constructed in step S2 to perform two core reasoning tasks: Interventional inference: Using causal inference tools such as do-calculus, to quantify the causal effect of a specific intervention (such as tactical change) on outcome variables.

[0079] Counterfactual inference: Evaluates possible outcomes under hypothetical conditions different from historical facts, used for in-depth retrospective and attribution analysis.

[0080] Decision Optimization Agent (DOA): This agent is responsible for generating policies in subsequent steps.

[0081] These agents interact through a shared memory or message bus to ensure the comprehensiveness, dynamism, and causal consistency of the analysis results.

[0082] Step S4: Dynamic Decision Optimization with Causal Constraints After gaining a deep understanding of the current situation, the system enters the decision generation phase. This step employs a deep reinforcement learning (DRL) framework, where a decision optimization agent (DOA) learns the optimal decision strategy.

[0083] A key innovation lies in the fact that the exploration and optimization process of reinforcement learning is explicitly constrained by the causal graph in step S2. This guides the agent to prioritize actions that conform to known causal laws, thereby avoiding suboptimal strategies that might violate domain knowledge and be purely based on trial and error.

[0084] This constraint is achieved through a composite reward function: Mission objective rewards are directly related to the final goal of the match, such as an increase in expected goal value (xG). Causal consistency reward. A positive reward is given when the state transition direction of an action is consistent with the direction predicted by the causal graph, and a negative reward is given otherwise. Temporal smoothness rewards penalize actions that cause illogical and drastic changes in state to ensure policy stability. The weighting coefficients for each reward item, used to balance different optimization objectives.

[0085] The optimization of the entire system is guided by a unified total loss function: Standard reinforcement learning loss functions, such as policy loss and value loss in the ActorCritic algorithm. {} = {X_i X_j _c} (0, (, )) $: Causal consistency loss, which penalizes model predictions that do not conform to the direction of the causal graph in supervised learning scenarios. Multi-agent consistency loss, through metrics such as KL divergence, promotes consistency in the outputs (e.g., probability distributions) of different agents.

[0086] Through this causal-constrained reinforcement learning paradigm, the system can generate dynamic, adaptive, and logically consistent decision recommendations.

[0087] Step S5: Interpretability Analysis and Decision Presentation To enhance system transparency and user trust, the final step is to explain the decision-making process and present it in an intuitive way. This system provides two main explanation mechanisms: Causal chain tracing: For any decision suggestion proposed by the system, its reasoning path can be displayed in the form of a causal chain. For example, suggestion X is because X → Y → Z, where → represents the causal relationship extracted from the causal graph.

[0088] Counterfactual interpretation: The system can generate and present the analysis results of counterfactual scenarios, that is, "what is the expected difference in outcome if alternative action Y is taken instead of X", thus providing a benchmark for decision-making.

[0089] All analysis results and decision recommendations are ultimately presented through an interactive visualization interface. This interface integrates various views, including player heatmaps, tactical boards, spatiotemporal evolution diagrams, and causal network diagrams, providing users with a comprehensive decision support environment.

[0090] Example 1: Professional Football Match Tactical Analysis and Decision Support System Step 1: Data Acquisition and Preprocessing Hardware Deployment: Sixteen high-definition cameras (4K resolution, 120fps) are deployed around a standard football field; a 32-channel microphone array (48kHz sampling rate) is deployed around the coaching bench and the field; all 22 players on the field wear smart wearable devices with built-in GPS (10Hz) and IMU (100Hz). Data Streaming: All data streams are initially synchronized and compressed through edge computing nodes and transmitted to the cloud server in real time.

[0091] Step 2: Feature Engineering Video Features: Video features are extracted using a pre-trained SlowFast network, which includes a slow path for static scenes and a fast path for dynamic actions, effectively capturing tactical formations and player movements. The output is a 2304-dimensional feature vector. Audio Features: In addition to MFCC, key events such as "coach shouts," "referee whistles," and "audience cheers / boos" are identified through a sound event detection model and used as classification features. Sensor Features: Kinematic features such as speed, acceleration, running distance, number of sprints, and number of sudden stops and turns are calculated from GPS / IMU data, and player fatigue levels are preliminarily assessed using heart rate variability (HRV).

[0092] Step 3: Cause-effect graph construction and application Variable Definition: Over 200 variables are defined in the causal graph, including low-level features (e.g., player A's speed), mid-level tactical indicators (e.g., team formation compactness, possession rate, and attacking pressure intensity), and high-level outcomes (e.g., number of shots, number of goals conceded). Causal Discovery: Using the time-aware PC algorithm described in step S2, a basic causal graph is learned on match data covering the last three seasons. Key causal chains discovered include: increased attacking pressure intensity leads to an increased opponent passing error rate, which in turn leads to an increased high-pressing success rate, which further leads to increased counter-attacking opportunities, and ultimately to an increased shot conversion rate. Application: During a match, when the system suggests "increasing attacking pressure intensity," the above causal chains can be presented to the coach as a basis for decision-making.

[0093] Step 4: Multi-agent cooperative reasoning and decision making Model Configuration: CRA: Based on GNN, using message passing on a causal graph to infer intervention effects. MFA: Employs a 6-layer, 8-head Transformer encoder with a fusion dimension of 768. STA: 3 GCN layers, 256 hidden state dimensions in GRU, and a 30-second time window. DOA: Uses the Soft Actor Critic (SAC) algorithm, with both its policy and value networks consisting of 4-layer MLPs and hidden layer dimensions of [512, 512, 256, 256]. Training: Offline pre-training was performed using data from approximately 5000 matches across the top five European leagues during the 2019-2022 seasons, followed by online fine-tuning in the new season. Decision Output: The system outputs decision suggestions every 30 seconds, including: 1. Current Situation Assessment: Offense and defense index, key player performance rating, fatigue warning. 2. Short-Term Tactical Suggestions: such as "In the next 5 minutes, it is recommended to switch the formation from 433 to 442 to strengthen the midfield defense." 3. Substitution Recommendations: For example, "Player B's stamina has decreased by 20%, and the number of times the opponent has broken through his area has increased. It is recommended to substitute Player C." 4. Counterfactual Analysis: For example, "If we had used a high-pressing tactic in the first half, the model predicts we would have had 3.4 more shots on target."

[0094] System Deployment and Optimization Hardware platform: The cloud uses an 8-GPU NVIDIA A100 server cluster for model training and high-concurrency inference; the venue deploys NVIDIA Jetson AGX Orin as an edge computing node for data preprocessing and some lightweight model inference.

[0095] Software stack: The deep learning framework is PyTorch, the graph computing engine is DGL, the causal inference library is DoWhy and the self-developed library CausalLib, the message queue uses Apache Kafka, and the real-time data storage uses Redis.

[0096] Performance optimization: Model distillation and quantization: Large cloud models are distilled into lightweight models and deployed at the edge, and TensorRT is used for INT8 quantization, which improves the inference speed at the edge by 4.1 times.

[0097] Asynchronous pipelined processing: Data acquisition, feature extraction, multimodal fusion, decision reasoning and other steps are designed as asynchronous pipelines to maximize hardware utilization.

[0098] Causal graph caching: Caching the subgraph structures and causal effect calculation results of frequently queried queries reduces the response time of typical queries from seconds to hundreds of milliseconds.

[0099] This invention is not only applicable to team sports such as football and basketball, but can also be applied to individual sports such as tennis, track and field, and swimming by adjusting the configuration. Its potential applications include: Professional sports clubs: for tactical analysis, opponent research, player recruitment, and injury management. Sports media and broadcasters: for generating in-depth real-time data analysis content, enriching broadcast visuals, and enhancing the viewer experience. Mass sports and youth training institutions: for personalized training guidance and scientifically assessing the potential of young athletes.

[0100] Through the above embodiments, this invention demonstrates a complete, advanced, and practical intelligent sports event decision-making solution, verifying its significant technical advantages and broad application value in multiple dimensions.

Claims

1. An intelligent sports event decision-making system based on causal-driven multimodal fusion and spatiotemporal dynamic reasoning, characterized in that, include: The multimodal causal data acquisition and preprocessing module is used to collect and process sports data containing video, audio, text, and sensor data about sports events in real time, forming multi-source heterogeneous sports data; The causal-oriented multimodal knowledge graph construction and update module is equipped with a causal structure learning algorithm and an online learning framework. It is used to receive multi-source heterogeneous sports data sent by the multimodal causal data acquisition and preprocessing module, use the causal structure learning algorithm to mine causal relationships between variables, judge the credibility of causal relationships, and use the online learning framework to construct and dynamically update a knowledge graph containing spatiotemporal attributes based on the credibility of causal relationships. The spatiotemporal-aware hybrid intelligent agent module is used to process multi-source heterogeneous sports data and knowledge graphs containing spatiotemporal attributes to obtain decision-making strategies and causal knowledge; The dynamic decision optimization module with causal constraints is equipped with a deep reinforcement learning algorithm. It uses causal knowledge as the environment state of the deep reinforcement learning algorithm and a knowledge graph containing spatiotemporal attributes as the rule boundary of the deep reinforcement learning algorithm. The reward function guides the spatiotemporal perception hybrid agent module to generate an adaptive tactical strategy that maximizes the target benefit. The interpretability analysis and visualization module is used to receive the decision strategy sent by the spatiotemporal perception hybrid intelligent agent module, generate a decision explanation based on the decision strategy, and visualize the decision explanation.

2. The intelligent sports event decision-making system based on causal-driven multimodal fusion and spatiotemporal dynamic reasoning according to claim 1, characterized in that, The causal structure learning algorithm described is a conditional independence test algorithm that introduces time lag effects and a causal strength measurement method for cross-modal fusion, which are derived from the Peter-Clark algorithm.

3. The intelligent sports event decision-making system based on causal-driven multimodal fusion and spatiotemporal dynamic reasoning according to claim 2, characterized in that, The conditional independence test algorithm for the time lag effect uses the following formula: ； in, Indicates at time Given a set of condition variables In the case of variables and Whether it is conditionally independent; This represents the partial correlation coefficient; Representing a condition set Past time window Observations within; This represents a dynamic threshold function.

4. The intelligent sports event decision-making system based on causal-driven multimodal fusion and spatiotemporal dynamic reasoning according to claim 3, characterized in that, This represents a dynamic threshold function, and the specific formula is as follows: ； in, The inverse cumulative distribution function of the standard normal distribution; Indicates the significance level. Indicates at time The number of valid samples used for calculation; Indicates the number of variables in the condition set; Indicates the time decay factor. It is the interval between the data point and the current time. It is the attenuation rate.

5. The intelligent sports event decision-making system based on causal-driven multimodal fusion and spatiotemporal dynamic reasoning according to claim 2, characterized in that, The causal strength measurement method for cross-modal fusion uses the following formula: ； in, This represents the posterior probability, i.e., the probability that the new graph structure will be obtained after observing all historical data. The probability of; Indicates the likelihood term; Represents structural priors.

6. The intelligent sports event decision-making system based on causal-driven multimodal fusion and spatiotemporal dynamic reasoning according to claim 1, characterized in that, The online learning framework described herein adopts the Bayesian online learning framework.

7. The intelligent sports event decision-making system based on causal-driven multimodal fusion and spatiotemporal dynamic reasoning according to claim 1, characterized in that, The spatiotemporal perception hybrid intelligent agent module includes at least four cooperative intelligent agents; The first intelligent agent is used to perform feature fusion on the multi-source heterogeneous sports data to obtain fused features; The second agent is used to collaboratively reason using fused features and a knowledge graph containing spatiotemporal attributes to obtain a decision strategy; The third agent is used to take player information in the knowledge graph containing spatiotemporal attributes as nodes of the spatiotemporal graph network and player relationships as edges of the spatiotemporal graph network to obtain the dynamic evolution information of the team. The fourth agent is used to intervene in the team's dynamic evolution information and knowledge graph containing spatiotemporal attributes, and to make counterfactual inferences to obtain causal knowledge.

Images