Real-time scoring statistics method and system for sports events based on AI visual recognition

Abstract

This invention provides a real-time scoring and statistics method and system for sports events based on AI visual recognition, which relates to the field of intelligent scoring technology for sports events. The system uses multiple cameras to capture images, perform target detection, and obtain data on athletes, equipment, and scoring areas. It also uses a feature pyramid network to generate joint heat maps and construct a dynamic skeleton graph structure to extract motion features. It also performs periodic equipment detection and anomaly correction. Finally, it uses a dual-stream feature projection network and a multi-head attention mechanism to perform feature matching and score calculation. The system generates a score data packet containing time, athlete number, type, and video clip, and sends it to the scoring system. This invention enables intelligent, real-time scoring for sports events, improving scoring accuracy and efficiency.

Description

Technical Field

[0001] The present invention relates to the field of intelligent scoring technology for sports events, and in particular to a real-time scoring statistics method and system for sports events based on AI visual recognition. Background Art

[0002] In sports events, accurately recording and tallying scores is a crucial component of refereeing. Traditional manual scoring relies primarily on the referee's on-site judgment. During the competition, the referee must observe the athletes' movements, equipment status changes, and the effective triggering of the scoring area in real time, and then make scoring decisions based on the competition rules. With the development of artificial intelligence and computer vision technologies, intelligent scoring systems based on visual recognition are gradually being applied to various sports events. These systems use cameras to capture game footage and deep learning algorithms to assist referees in scoring decisions.

[0003] However, existing intelligent scoring systems still have some problems in practical applications. Due to the high speed and complexity of athletes' movements, single target detection and posture estimation models are difficult to accurately capture the details of the movements, and are prone to missed detections and misjudgments; the state changes of sports equipment and the effective triggering judgment of the scoring area are often affected by factors such as occlusion and lighting, which reduces the accuracy of scoring; the existing system lacks modeling of the temporal correlation between athlete movements and equipment status, and cannot effectively distinguish between valid scoring movements and invalid movements, affecting the reliability of scoring.

[0004] In summary, the present invention aims to solve the above technical problems and proposes a real-time scoring statistics method based on multimodal feature fusion. By constructing a dynamic skeleton graph structure and a timing constraint model, it can achieve accurate modeling of athletes' movements; utilize phase consistency measurement and anomaly correction mechanism to improve the robustness of equipment state recognition; adopt dual-stream feature projection and multi-head attention mechanism to establish a temporal association between action features and equipment state, thereby improving the accuracy and reliability of scoring judgment. Summary of the Invention

[0005] The embodiments of the present invention provide a method and system for real-time sports event scoring statistics based on AI visual recognition, which can solve the problems in the existing technology.

[0006] According to a first aspect of the embodiments of the present invention,

[0007] Provides a real-time sports event scoring and statistics method based on AI visual recognition, including:

[0008] Collect image data of sports venues through multiple cameras; perform target detection on the image data to obtain athlete image data, equipment image data, and scoring area image data;

[0009] A feature pyramid network is used to generate joint heat maps from athlete image data. A dynamic connection skeleton graph structure is constructed through temporal smoothing constraints. The athlete's motion feature vectors are extracted using a graph convolutional network and recursive neural ordinary differential equations.

[0010] The phase consistency measurement network is used to perform periodic detection and anomaly correction on the equipment image data to obtain the equipment status data;

[0011] A dynamic graph structure is constructed through a two-stream feature projection network. The multi-head attention mechanism and temporal consistency constraints are used to perform feature interaction matching on the athlete's action feature vectors and equipment status data to calculate the effective score data.

[0012] Generate a scoring data packet including scoring time, scoring athlete number, scoring type and scoring video clip according to the valid scoring data;

[0013] The scoring data packet is sent to the game scoring system, and the scoring video clip is stored in the video storage server.

[0014] In an optional embodiment,

[0015] The feature pyramid network is used to generate athlete joint heat maps from athlete image data. A dynamic connection skeleton graph structure is constructed through temporal smoothing constraints. The athlete action feature vectors are extracted through graph convolutional networks and recursive neural ordinary differential equations, including:

[0016] The athlete image data is downsampled and feature mapped using a feature pyramid network to obtain feature branches of different scales. The feature branches are then convolved and mapped using an activation function to determine the spatial distribution of joints and generate a heat map of the athlete's joints.

[0017] The response value of each joint point in the athlete's joint heat map is weightedly integrated to obtain the joint space coordinates, and the smooth constraint calculation is performed in combination with the coordinate information of adjacent time frames to obtain the joint smooth coordinate sequence;

[0018] A node set is established based on the joint smooth coordinate sequence. The node connection probability is calculated through the temporal displacement changes between nodes to form a dynamic connection skeleton graph structure.

[0019] The dynamic connection skeleton graph structure is input into a multi-layer stacked graph convolutional network. In each layer of the network, the node features are normalized and the weights are updated. The velocity vector difference and position vector difference between the nodes are calculated to obtain the node motion correlation features.

[0020] Decomposing and mapping the node motion correlation features to obtain multiple basic motion representation vectors, and calculating corresponding combination weight coefficients to fuse them into a global motion feature vector;

[0021] The global action feature vector is input into a recursive neural ordinary differential equation for time series modeling, multi-scale time series features are extracted through multi-layer dilated convolution, and the athlete action feature vector is output.

[0022] In an optional embodiment,

[0023] A node set is established based on the joint smooth coordinate sequence. The node connection probability is calculated through the temporal displacement changes between nodes to form a dynamic connection skeleton graph structure, which includes:

[0024] Constructing a joint smooth coordinate sequence into an initial node set, calculating the temporal displacement difference between adjacent nodes in the initial node set, and constructing a node state vector;

[0025] Establishing a Bayesian network model based on the node state vector, encoding the spatial position relationship and temporal displacement difference between nodes into a conditional probability distribution;

[0026] Constructing a Markov random field to describe the spatial constraint relationship of nodes, wherein the spatial constraint relationship includes node distance constraint, angle constraint and topological connection constraint, and generating a spatial potential energy function of the nodes;

[0027] Combining the conditional probability distribution and the spatial potential energy function, a variational expectation maximization algorithm is used to iteratively calculate the node connection probability;

[0028] Constructing a probabilistic graph network based on the node connection probability, calculating the mutual information and conditional entropy between each pair of nodes, taking the weighted sum of the mutual information and the conditional entropy as the connection uncertainty, and setting a node connection threshold according to the connection uncertainty;

[0029] The connection relationship between the nodes is determined according to the node connection threshold, and the connection relationship is updated online based on the temporal displacement difference between the nodes to form a dynamic connection skeleton graph structure.

[0030] In an optional embodiment,

[0031] Combining the conditional probability distribution and the spatial potential energy function, using the variational expectation maximization algorithm to iteratively calculate the node connection probability includes:

[0032] Multiply the conditional probability distribution with the spatial potential energy function and normalize it to construct a joint probability distribution model of node status and connection relationship;

[0033] Constructing an auxiliary probability distribution to represent the connection relationship between nodes, and using the auxiliary probability distribution as a variational distribution to approximate the posterior distribution of the joint probability distribution model;

[0034] Constructing a variational objective function based on the joint probability distribution model and the variational distribution, wherein the variational objective function includes an observation data log-likelihood term, a posterior distribution KL divergence term, and a spatial potential energy function term;

[0035] Performing expectation maximization iterative optimization on the variational objective function, fixing the model parameters to optimize the variational distribution in the expectation step of each iteration, fixing the variational distribution in the maximization step, obtaining an adaptive learning rate by calculating the first-order moment and the second-order moment of the gradient, taking the product of the adaptive learning rate and the gradient as the parameter update amount and truncating it, optimizing the model parameters, and determining the optimized variational distribution;

[0036] The initial connection probability between node pairs is calculated based on the optimized variational distribution, and the normalized connection probability is obtained by softmax normalization combined with the temperature parameter;

[0037] The difference between the variational objective function in adjacent iterations is monitored, and when the difference is less than a preset difference threshold, the iteration convergence is determined, and the final node connection probability is output.

[0038] In an optional embodiment,

[0039] The phase consistency measurement network is used to perform periodic detection and anomaly correction on the equipment image data, and the equipment status data obtained includes:

[0040] Performing a multi-directional and multi-scale Gabor transform on the equipment image data to obtain a complex response of the equipment image data, calculating a local phase value based on the complex response, and calculating a phase consistency metric based on the amplitude of the complex response and the local phase value;

[0041] Performing a two-dimensional Fourier transform on the phase consistency metric to obtain a power spectrum, calculating a radial mean of the power spectrum, and determining a periodic feature of the equipment image data based on a maximum response of the radial mean;

[0042] Constructing a reference phase consistency template, calculating a difference between the phase consistency metric and the reference phase consistency template as a local anomaly metric, and constructing an adaptive threshold based on a mean and a standard deviation of the local anomaly metric;

[0043] Comparing the local anomaly metric with the adaptive threshold to identify a sudden abnormal area, calculating the spatial gradient of the local anomaly metric and comparing it with a preset gradient threshold to identify a gradual abnormal area;

[0044] Performing temporal correction and spatial correction on the sudden abnormal region and the gradual abnormal region, and fusing a first result of the temporal correction and a second result of the spatial correction based on an adaptive weight to obtain a modified phase consistency metric;

[0045] The modified phase consistency metric is converted into a local state feature through nonlinear mapping, and the local state feature is weightedly aggregated to obtain the equipment state data.

[0046] In an optional embodiment,

[0047] A dynamic graph structure is constructed through a two-stream feature projection network. The multi-head attention mechanism and temporal consistency constraints are used to perform feature interactive matching on the athlete's action feature vectors and equipment status data. The effective score data obtained by calculation includes:

[0048] The athlete's action feature vectors and equipment status data are collected in time series based on an adaptive sampling rate. The dimensions are unified through a two-stream feature projection network constructed by a multi-layer perceptron and a recurrent neural network to obtain action features and equipment features.

[0049] Constructing the action features and the equipment features into node representations of a dynamic graph structure, calculating edge weight values ​​between the node representations through a sparse attention weight matrix based on local sensitive hashing, and constructing a weighted graph structure;

[0050] The node representation of the graph structure is generated through a multi-scale convolutional network and position encoding to generate the query matrix, key matrix and value matrix. The scaled dot product attention mechanism adjusted by feature gradient is used for multi-head parallel calculation, and the fusion feature is determined through residual connection and layer normalization.

[0051] The temporal attention weight of the fused features is calculated through a gated recurrent unit, and the attention coefficient is calculated by combining the current node hidden state and the context vector. Based on the attention coefficient, the features corresponding to the temporal neighborhood nodes are weightedly aggregated and the node hidden state is updated using skip connections.

[0052] Calculate the similarity matrix between action features and equipment features, and calculate the Mahalanobis distance of the similarity matrix at adjacent time sequence positions through a sliding window to determine the time sequence consistency constraint;

[0053] The similarity matrix and the temporal consistency constraint are transformed nonlinearly to obtain the initial score. The exponential moving average method is used to calculate the maximum similarity mean of the initial score in the temporal dimension to determine the score confidence level.

[0054] Based on the initial score and the score confidence, anomaly detection and smoothing filtering are performed through an adaptive threshold to generate valid score data.

[0055] In an optional embodiment,

[0056] The node representation of the graph structure is generated through a multi-scale convolutional network and position encoding to generate the query matrix, key matrix and value matrix. The scaled dot product attention mechanism with feature gradient adjustment is used for multi-head parallel calculation. The fusion features are determined through residual connection and layer normalization.

[0057] The node representation of the graph structure is extracted through a multi-scale convolutional network, and feature cascade is performed to obtain a multi-scale feature representation;

[0058] Introducing position coding, dynamically adjusting the position coding through an adaptive weight matrix to obtain enhanced position coding, and performing feature enhancement operation on the enhanced position coding and the multi-scale feature representation to obtain enhanced features;

[0059] Based on the enhanced features, a query matrix, a key matrix, and a value matrix are generated respectively by querying a weight matrix, a key weight matrix, and a value weight matrix, calculating a feature gradient of the enhanced features, and inputting the feature gradient into a feature adjustment network to obtain an attention adjustment factor;

[0060] Performing multi-head parallel calculations on the query matrix, key matrix, and value matrix through a scaled dot product attention mechanism to obtain multiple attention head features, dynamically adjusting the multiple attention head features through the attention adjustment factor to obtain attention output features, and performing a residual connection between the attention output features and the multi-scale feature representation to obtain a feature fusion result;

[0061] The feature fusion result is input into the normalization network layer to generate normalization coefficients and bias parameters, and the final fusion feature is obtained through processing.

[0062] According to a second aspect of the embodiments of the present invention,

[0063] Provides a real-time sports scoring and statistics system based on AI visual recognition, including:

[0064] The first unit is used to collect image data of the sports venue through multiple cameras; perform target detection on the image data to obtain athlete image data, equipment image data and scoring area image data;

[0065] The second unit is used to generate athlete joint heat maps from athlete image data using a feature pyramid network, construct a dynamic connection skeleton graph structure through temporal smoothing constraints, and extract athlete movement feature vectors using a graph convolutional network and recursive neural ordinary differential equations;

[0066] The third unit is used to use the phase consistency measurement network to perform periodic detection and anomaly correction on the equipment image data to obtain the equipment status data;

[0067] The fourth unit is used to construct a dynamic graph structure through a two-stream feature projection network, and use a multi-head attention mechanism and temporal consistency constraints to perform feature interaction matching on the athlete's action feature vectors and equipment status data to calculate the effective score data;

[0068] The fifth unit is used to generate a scoring data packet including a scoring time, a scoring athlete number, a scoring type and a scoring video clip according to the valid scoring data;

[0069] The sixth unit is used to send the score data packet to the game scoring system and store the score video clip to the video storage server.

[0070] According to a third aspect of the embodiments of the present invention,

[0071] An electronic device is provided, comprising:

[0072] processor;

[0073] a memory for storing processor-executable instructions;

[0074] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0075] According to a fourth aspect of the embodiments of the present invention,

[0076] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method described above is implemented.

[0077] In an embodiment of the present invention, through multi-camera acquisition and advanced AI visual recognition technology, automatic statistics of real-time scores of sports events are realized, which greatly improves the accuracy and efficiency of scoring, can accurately capture the complex movement characteristics of athletes, and effectively reduces human judgment bias; the phase consistency measurement network and dual-stream feature projection network are introduced, which can accurately identify the equipment status and match it with the athlete's movement, significantly improve the accuracy of scoring judgment, enhance the system's ability to understand complex game scenes, and effectively reduce the misjudgment rate; automatically generate a score data packet containing detailed information and transmit it to the game scoring system in real time, and save relevant video clips at the same time, providing comprehensive support for event management, replay review and data analysis, which not only improves the fairness and viewing experience of the game, but also provides strong support for the digital development of sports events. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 This is a flow chart of a method for real-time sports event scoring statistics based on AI visual recognition according to an embodiment of the present invention;

[0079] Figure 2This is a comparison chart of the simulation effects of dynamic skeleton connection structure under complex high-speed movements;

[0080] Figure 3 is the parameter sensitivity heat map of the spatial potential energy function;

[0081] Figure 4 Flowchart of feature enhancement and dynamic attention mechanism. DETAILED DESCRIPTION

[0082] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0083] The following specific embodiments are used to describe the technical solution of the present invention in detail. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0084] Figure 1 FIG. 1 is a flow chart of a method for real-time sports scoring statistics based on AI visual recognition according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0085] Collect image data of sports venues through multiple cameras; perform target detection on the image data to obtain athlete image data, equipment image data, and scoring area image data;

[0086] A feature pyramid network is used to generate joint heat maps from athlete image data. A dynamic connection skeleton graph structure is constructed through temporal smoothing constraints. The athlete's motion feature vectors are extracted using a graph convolutional network and recursive neural ordinary differential equations.

[0087] The phase consistency measurement network is used to perform periodic detection and anomaly correction on the equipment image data to obtain the equipment status data;

[0088] A dynamic graph structure is constructed through a two-stream feature projection network. The multi-head attention mechanism and temporal consistency constraints are used to perform feature interaction matching on the athlete's action feature vectors and equipment status data to calculate the effective score data.

[0089] Generate a scoring data packet including scoring time, scoring athlete number, scoring type and scoring video clip according to the valid scoring data;

[0090] The scoring data packet is sent to the game scoring system, and the scoring video clip is stored in the video storage server.

[0091] In an optional embodiment, a feature pyramid network is used to generate a heat map of athlete joints from athlete image data, a dynamic connection skeleton graph structure is constructed through temporal smoothing constraints, and a graph convolutional network and recursive neural ordinary differential equations are used to extract the athlete's motion feature vectors, including:

[0092] The athlete image data is downsampled and feature mapped using a feature pyramid network to obtain feature branches of different scales. The feature branches are then convolved and mapped using an activation function to determine the spatial distribution of joints and generate a heat map of the athlete's joints.

[0093] The response value of each joint point in the athlete's joint heat map is weightedly integrated to obtain the joint space coordinates, and the smooth constraint calculation is performed in combination with the coordinate information of adjacent time frames to obtain the joint smooth coordinate sequence;

[0094] A node set is established based on the joint smooth coordinate sequence. The node connection probability is calculated through the temporal displacement changes between nodes to form a dynamic connection skeleton graph structure.

[0095] The dynamic connection skeleton graph structure is input into a multi-layer stacked graph convolutional network. In each layer of the network, the node features are normalized and the weights are updated. The velocity vector difference and position vector difference between the nodes are calculated to obtain the node motion correlation features.

[0096] Decomposing and mapping the node motion correlation features to obtain multiple basic motion representation vectors, and calculating corresponding combination weight coefficients to fuse them into a global motion feature vector;

[0097] The global action feature vector is input into a recursive neural ordinary differential equation for time series modeling, multi-scale time series features are extracted through multi-layer dilated convolution, and the athlete action feature vector is output.

[0098] In one specific embodiment, the collected athlete image data is input into a feature pyramid network consisting of five descending layers. The sampling rates of each layer are 1 / 1, 1 / 2, 1 / 4, 1 / 8, and 1 / 16 of the original size, respectively. During each downsampling process, a 3×3 convolution operation with a stride of 2 is used to halve the image size. The number of feature channels output by each scaling layer is set to 256. In this way, the original 640×480 resolution input image can generate five feature branches of different scales: 640×480×256, 320×240×256, 160×120×256, 80×60×256, and 40×30×256 feature maps.

[0099] The feature branch is mapped into a joint heatmap through a 1×1 convolution operation, with each joint corresponding to a channel. For example, for 17 human joints, the number of output channels is 17. The convolved feature map is processed through a ReLU activation function to generate a non-negative heatmap, where the value of each pixel represents the probability of the presence of a specific joint at that location. For the gymnast image data, the generated heatmap shows high response areas at key joints such as the head, shoulders, and knees, with values ​​ranging from 0 to 1.

[0100] A weighted integral is calculated for each joint's response value in the heatmap. This is done by multiplying the coordinate value of each pixel in the heatmap by the corresponding response value, summing the values ​​across the entire heatmap, and finally dividing by the sum of all response values ​​in the heatmap to obtain the floating-point coordinates of the joint. For example, for a shoulder joint heatmap with the highest response concentrated near (320, 150), the precise coordinates calculated might be (319.78, 151.23).

[0101] To reduce jitter, a temporal smoothing constraint is introduced. For each joint, the coordinates of the current frame and the previous three frames are weighted averaged, with the weight of the current frame being 0.5, the previous frame being 0.3, the previous two frames being 0.15, and the previous three frames being 0.05. For example, if the coordinates of a joint in four frames are (319.78, 151.23), (318.92, 150.87), (320.15, 152.01), and (319.56, 151.45), the smoothed coordinates are approximately (319.53, 151.24).

[0102] Based on the acquired joint smooth coordinate sequence, each joint point is regarded as a node in the graph, and the connection probability is calculated according to the displacement change of the node in adjacent frames. If the displacement change synergy between the two nodes is high, the connection probability increases. Specifically, the cosine similarity between the node pairs in two adjacent frames is calculated, and a connection relationship is established when the cosine similarity is greater than 0.8. For the action sequence of a diving athlete, during the falling process, the connection probability of the arm and torso nodes is about 0.95, while the connection probability of the arm and leg nodes is about 0.3, thereby forming a dynamic skeleton diagram that conforms to the biomechanical characteristics of the human body.

[0103] The constructed dynamic connection skeleton graph is input into a three-layer stacked graph convolutional network for feature extraction. In each layer of graph convolution, node features are normalized by subtracting the center node value from the average of its neighboring nodes and dividing by the standard deviation. For the first layer of graph convolution, the input is the original coordinates and displacement information of the joint points, and the output feature dimension is 64. For example, for a shoulder joint, the input features are the three-dimensional coordinates (319.53, 151.24, 0) and the displacement vector (0.75, 0.37, 0), which are converted into a 64-dimensional feature vector after graph convolution.

[0104] During graph convolution, motion correlation features are computed between nodes. For pairs of connected nodes, the velocity and position vector differences are calculated. For example, the velocity difference between the shoulder and elbow joints is (-1.2, 0.8, 0.5), and the position difference is (35.6, 12.3, 2.1). These differences reflect the relative motion patterns between the joints. For gymnasts performing horizontal bar movements, the arm and torso nodes exhibit highly coordinated velocity differences during rotation, with the difference typically ranging from (-2, 2).

[0105] An attention mechanism is used to decompose node features into eight basis motion representation vectors, each with a dimension of 32. Each basis vector represents a fundamental motion pattern, such as flexion and extension, rotation, and translation. A weight coefficient is determined for each basis vector by calculating an attention score. The weight values ​​range from 0 to 1, and the sum is 1. For example, for a diver's flip, the weight coefficient for the rotation basis vector is approximately 0.6, while the weight coefficient for the translation basis vector is approximately 0.1. The basis vectors are then weighted and fused according to the weight coefficients to form a 256-dimensional global motion feature vector.

[0106] The global motion feature vector is input into a recursive neural ordinary differential equation network for temporal modeling. The recursive neural ordinary differential equation network consists of three recursive layers, each of which uses dilated convolutions with dilation rates of 1, 2, 4, and 8 to extract multi-scale temporal features. For a 90-frame gymnast's motion sequence, temporal modeling generates a 512-dimensional motion feature vector that contains both temporal and spatial information about the complete motion.

[0107] In this embodiment, feature branches of different scales are extracted through a feature pyramid network and descending sampling, providing multi-level information with both fine-grained and global context for subsequent joint positioning; based on a smooth coordinate sequence, a node set is constructed and the temporal displacement and connection probability between nodes are calculated, thereby forming a skeleton graph structure that can reflect the dynamic posture changes of athletes, providing rich structural information for action recognition; through a multi-layer stacked graph convolutional network, the node features are normalized and the weights are updated, and with the help of the calculation of the difference between the velocity and position vectors, the motion correlation between the joints is effectively captured, and the expressive ability of the local motion features is enhanced; the node motion correlation features are decomposed and mapped into multiple basic motion representation vectors, and combined with the corresponding combination weight coefficients to form a global motion feature vector, thereby achieving a comprehensive description of the entire action process and improving the robustness of action recognition.

[0108] In an optional embodiment, establishing a node set based on a joint smooth coordinate sequence, calculating the node connection probability through the temporal displacement changes between the nodes, and forming a dynamic connection skeleton graph structure includes:

[0109] Constructing a joint smooth coordinate sequence into an initial node set, calculating the temporal displacement difference between adjacent nodes in the initial node set, and constructing a node state vector;

[0110] Establishing a Bayesian network model based on the node state vector, encoding the spatial position relationship and temporal displacement difference between nodes into a conditional probability distribution;

[0111] Constructing a Markov random field to describe the spatial constraint relationship of nodes, wherein the spatial constraint relationship includes node distance constraint, angle constraint and topological connection constraint, and generating a spatial potential energy function of the nodes;

[0112] Combining the conditional probability distribution and the spatial potential energy function, a variational expectation maximization algorithm is used to iteratively calculate the node connection probability;

[0113] Constructing a probabilistic graph network based on the node connection probability, calculating the mutual information and conditional entropy between each pair of nodes, taking the weighted sum of the mutual information and the conditional entropy as the connection uncertainty, and setting a node connection threshold according to the connection uncertainty;

[0114] The connection relationship between the nodes is determined according to the node connection threshold, and the connection relationship is updated online based on the temporal displacement difference between the nodes to form a dynamic connection skeleton graph structure.

[0115] In a specific embodiment, the three-dimensional coordinate data of the human skeleton joints are obtained by a depth camera or a multi-view camera system to form a time series coordinate sequence {P_t}, where P_t represents the coordinate set of all joint points at time t. In order to eliminate noise interference, a bilateral filter is used to smooth the original coordinate sequence, and the time domain window size is set to 5 frames, the spatial domain distance parameter is set to 0.05 meters, and the value domain similarity parameter is set to 0.1. Taking the human skeleton as an example, a total of 18 joints are identified, including the head, neck, shoulder, elbow, wrist, hip, knee, ankle, etc. These joints are used as the initial node set N={n_1, n_2, ..., n_18}.

[0116] Calculate the temporal displacement differences between adjacent nodes in the initial node set to construct a node state vector. For each node n_i, record its displacement change over 10 consecutive frames and calculate its relative displacement difference D_ij with its adjacent node n_j. The relative displacement difference is calculated as the difference between the displacement vectors of two nodes. For example, the average relative displacement difference between head node n_1 and neck node n_2 over 10 consecutive frames is 0.015 meters. Combine the displacement differences of each node with all its potential adjacent nodes to form a state vector S_i = {D_i1, D_i2, ..., D_iN}.

[0117] A Bayesian network model is established based on the node state vectors, encoding the spatial positional relationships and temporal displacement differences between nodes into conditional probability distributions. A directed probability graph G = (V, E) is constructed, where V is the set of nodes and E is the set of edges. For each pair of nodes (n_i, n_j), the conditional probability P(n_j | n_i) is calculated based on their state vectors S_i and S_j, representing the state distribution of n_j given the known state of n_i. The conditional probabilities are estimated using a Gaussian mixture model, and the model parameters are determined using the expectation-maximization algorithm. In practice, the number of Gaussian mixture components is set to 3, the maximum number of iterations is set to 100, and the convergence threshold is set to 0.001. For example, the conditional probability P(n_6 | n_5) between elbow node n_5 and wrist node n_6 is 0.87, indicating that the elbow state has strong predictive power for the wrist state.

[0118] A Markov random field is constructed to describe the spatial constraints between nodes. An undirected graph H = (V, E') is defined, where V is the same as the set of nodes in a Bayesian network, and E' represents the set of undirected edges between nodes. Spatial constraints include node distance constraints, angle constraints, and topological connectivity constraints. Distance constraints define the spatial distance between two nodes, such as the normal distance from the shoulder to the elbow, which ranges from 0.25 to 0.35 meters. Angle constraints specify the range of joint motion, such as the normal elbow flexion angle, which ranges from 0 to 150 degrees. Topological connectivity constraints ensure the overall structural rationality of the skeleton, such as requiring that a node be connected to at most four other nodes. Based on these constraints, a spatial potential energy function U(n_i, n_j) is generated, representing the compatibility between node pairs (n_i, n_j). Lower potential energy function values ​​indicate a higher probability of connection between two nodes. For example, the potential energy function value for adjacent joints, such as the shoulder and elbow, is 0.12, while the potential energy function value for non-adjacent joints, such as the shoulder and ankle, is 0.89.

[0119] Combining the conditional probability distribution and the spatial potential energy function, a variational expectation-maximization algorithm is used to iteratively calculate node connection probabilities. The objective function is defined as F = α·P(n_j|n_i)-β·U(n_i, n_j), where α and β are balancing parameters. Cross-validation determined α=0.6 and β=0.4. The variational expectation-maximization algorithm iterates by alternating between expectation and maximization steps. In the expectation step, the current model parameters are fixed and the posterior distribution of the latent variables is calculated. In the maximization step, the updated posterior distribution is used to optimize the model parameters. The number of iterations is set to 20, and the convergence threshold is set to 0.01. After the iterations are complete, the connection probability C_ij between the node pair (n_i, n_j) is obtained. For example, the connection probability between the shoulder node and the elbow node is 0.92, indicating that the two nodes are likely directly connected.

[0120] A probabilistic graph network is constructed based on node connection probabilities, and the mutual information and conditional entropy between each pair of nodes are calculated. For a node pair (n_i, n_j), the mutual information I(n_i; n_j) represents the amount of shared information, and the conditional entropy H(n_i|n_j) represents the uncertainty about n_i given n_j. The weighted sum of the mutual information and conditional entropy, λ·I(n_i; n_j)+(1-λ)·H(n_i|n_j), is used as the connection uncertainty U_ij, where λ is experimentally determined to be 0.7. For example, the mutual information for the shoulder and elbow node pair is 0.85, and the conditional entropy is 0.25, resulting in a calculated connection uncertainty of 0.67. A node connection threshold τ is set based on the connection uncertainty. In practice, τ = 0.75 is determined through receiver operating characteristic curve analysis.

[0121] The connection relationship between nodes is determined based on the node connection threshold. When C_ij > τ, a connection is established between nodes n_i and n_j. The connection relationship is updated online based on the temporal displacement differences between the nodes, forming a dynamic connection skeleton graph structure. During the online update process, a sliding window method is used with a window size of 30 frames. Within each window, the temporal displacement differences of the nodes are recalculated, the conditional probability distribution and spatial potential energy function are updated, and then the node connection probability is updated. When the connection probability change exceeds the threshold δ = 0.15, the connection relationship is dynamically adjusted. For example, during normal walking, the connection between the hip and knee nodes remains stable. However, during difficult movements such as aerial rotations, the connection relationship between certain joints may be temporarily adjusted to accommodate the special posture.

[0122] Existing skeleton graph construction techniques primarily rely on template matching or direct connection of predefined joints, which are difficult to adapt to movement variations and individual differences. For example, traditional methods use a fixed skeleton template and map the detected joints onto it, which is incapable of handling unconventional movements or unusual body shapes. Another approach uses deep learning to directly predict joint connections, but this relies on large amounts of annotated data and lacks physical constraints.

[0123] This embodiment introduces a probabilistic graphical model combined with temporal dynamic analysis to adaptively generate a skeleton structure; captures the correlation of joint movements through temporal displacement differences to avoid relying solely on static spatial positions; combines Bayesian networks and Markov random fields to simultaneously consider directed dependencies and undirected constraints; introduces information theory indicators to evaluate connection uncertainty to achieve dynamic adjustment of the skeleton structure.

[0124] Experimental results show that when processing complex action sequences, the method of this embodiment improves the skeleton recognition accuracy by 15.3% and the adaptability to unconventional actions by 23.7%. At the same time, it improves the real-time performance of skeleton tracking from 25 frames per second to 40 frames per second, greatly improving the skeleton structure recognition effect in dynamic environments.

[0125] like Figure 2 As shown in the figure, a comparison of the simulation effects of three different skeleton graph construction technologies when processing high-speed rotation and jumping movements (angular velocity of up to 235° / s) is demonstrated. The figure shows the skeleton connection structures generated by this technical solution, template matching method and deep learning prediction method respectively, and the corresponding connection probability value is marked on each connection line. It can be clearly seen from the figure that in the same high-speed action scene, the skeleton structure generated by this technical solution is the most accurate and complete, with a connection accuracy rate of 87.5%, a processing time of only 18 milliseconds, and an adaptability score of 84.2 points (out of 100 points). The connection relationship between all 15 joint points was correctly identified, and the connection probability was generally high, ranging from 0.98 from the head to the neck to 0.89 for the ankle joint, indicating that this method can still maintain highly reliable skeleton structure recognition even in high-speed motion conditions.

[0126] In contrast, the template matching method performed poorly under the same conditions, achieving a connection accuracy of only 57.8%, a processing time of 32 milliseconds, and an adaptability score of only 53.6. As can be seen in the figure, the template matching method has several obvious errors: the left wrist (n_7) is mistakenly connected directly to the shoulder (n_3) instead of the elbow (n_5); the connection to the right wrist (n_8) is completely missing; the left ankle (n_14) is mistakenly connected to the right knee (n_13); and the connection to the right ankle (n_15) is also completely missing. These errors are primarily due to the template matching method's inability to effectively adapt to the rapid changes in joint position during high-speed rotational movements, resulting in incorrect matches or matching failures.

[0127] The deep learning prediction method performed somewhere in between, with a connection accuracy of 69.4%, a processing time of 27 milliseconds, and an adaptability score of 67.8. While this method was able to identify all joint connections, the connection probabilities were generally low, particularly for the extremities. For example, the connection probabilities for the left and right wrists were only 0.68 and 0.67, and for the left and right ankles were 0.63 and 0.62, respectively. This suggests that the method lacks confidence in determining joint connections in high-speed action scenarios. This uncertainty can lead to instability in connection relationships during continuous action sequences, affecting the consistency of skeleton tracking.

[0128] This simulation comparison demonstrates the significant advantages of this technical solution in handling high-speed, complex movements. In particular, its method of capturing joint motion correlations by combining temporal displacement differences enables the skeleton structure to better adapt to rapidly changing postures while maintaining high processing speed and connection accuracy. These results fully demonstrate the technical innovation and practical value of this solution in dynamic skeleton structure recognition.

[0129] In this embodiment, by combining Bayesian networks and Markov random fields, the temporal dynamic relationship and spatial structural constraints of nodes are comprehensively considered, effectively enhancing the stability and expressiveness of the skeleton graph structure under different motion states; conditional probability is used to model the node state dependency, and the spatial potential energy function is used to evaluate the structural compatibility between nodes, so that the final node connection not only has statistical support, but also conforms to the physiological and motion structure logic, significantly improving the credibility of the connection relationship; the introduction of sliding windows and online update mechanisms enables the skeleton graph to adjust the connection relationship in real time according to the change of movement, which is particularly suitable for processing complex or sudden posture changes, such as jumping, rotation and other high-dynamic movements; by introducing mutual information and conditional entropy to measure the uncertainty of the connection, and setting the connection threshold for screening, the redundant connection and misconnection phenomenon are significantly reduced, and the accuracy and discriminability of the skeleton graph structure are improved; the high-dimensional three-dimensional skeleton data provided by the depth camera or multi-view system is used, combined with bilateral filtering and displacement difference modeling, to enhance the robustness of the model to multi-source data noise and ensure the stability of the node state estimation.

[0130] In an optional embodiment, combining the conditional probability distribution and the spatial potential energy function, using a variational expectation maximization algorithm to iteratively calculate the node connection probability includes:

[0131] Multiply the conditional probability distribution with the spatial potential energy function and normalize it to construct a joint probability distribution model of node status and connection relationship;

[0132] Constructing an auxiliary probability distribution to represent the connection relationship between nodes, and using the auxiliary probability distribution as a variational distribution to approximate the posterior distribution of the joint probability distribution model;

[0133] Constructing a variational objective function based on the joint probability distribution model and the variational distribution, wherein the variational objective function includes an observation data log-likelihood term, a posterior distribution KL divergence term, and a spatial potential energy function term;

[0134] Performing expectation maximization iterative optimization on the variational objective function, fixing the model parameters to optimize the variational distribution in the expectation step of each iteration, fixing the variational distribution in the maximization step, obtaining an adaptive learning rate by calculating the first-order moment and the second-order moment of the gradient, taking the product of the adaptive learning rate and the gradient as the parameter update amount and truncating it, optimizing the model parameters, and determining the optimized variational distribution;

[0135] The initial connection probability between node pairs is calculated based on the optimized variational distribution, and the normalized connection probability is obtained by softmax normalization combined with the temperature parameter;

[0136] The difference between the variational objective function in adjacent iterations is monitored, and when the difference is less than a preset difference threshold, the iteration convergence is determined, and the final node connection probability is output.

[0137] In a specific embodiment, a joint probability distribution model of node status and connection relationship is constructed, and the joint probability distribution model is obtained by multiplying the conditional probability distribution with the spatial potential energy function and normalizing it. The conditional probability distribution describes the dependence of the node status on the connection relationship, while the spatial potential energy function characterizes the relative position relationship of the nodes in space. For example, for a network containing 10 nodes, a 10×10 matrix can be used to represent the conditional probability distribution, and each element in the matrix represents the conditional probability between a pair of nodes. The spatial potential energy function can be defined based on the Euclidean distance between nodes, and the closer the distance, the lower the potential energy. By multiplying these two parts and normalizing them, a joint probability distribution model that describes the entire network structure is obtained.

[0138] Next, we introduce an auxiliary probability distribution as a variational distribution to approximate the posterior distribution of the joint probability distribution model. This auxiliary probability distribution can also be represented by a 10×10 matrix, where each element represents the probability of a connection between the corresponding node pair. Initially, all elements can be set to 0.5, indicating equal connection probability.

[0139] Next, a variational objective function is constructed. This function consists of three main components: the log-likelihood of the observed data, the KL divergence of the posterior distribution, and the spatial potential function. The log-likelihood reflects how well the model fits the observed data, the KL divergence measures the difference between the variational distribution and the true posterior distribution, and the spatial potential function introduces constraints on spatial location information. The weighted sum of these three terms constitutes the final variational objective function.

[0140] Based on the constructed variational objective function, an expectation maximization iterative optimization process is performed. In the expectation step of each iteration, the model parameters are fixed and the variational distribution is optimized. Specifically, the variational distribution can be updated by calculating the gradient of the variational objective function with respect to the variational distribution and using the gradient ascent method. In the maximization step, the variational distribution is fixed and the model parameters are optimized. The adaptive learning rate method is adopted here to determine the learning rate by calculating the first-order moment and second-order moment of the gradient. For example, the exponential moving average can be used to estimate the first-order moment and second-order moment of the gradient, and then the learning rate is dynamically adjusted based on these two statistics. The product of the adaptive learning rate and the gradient is used as the parameter update amount, and truncation is performed to prevent excessive update step size. Assuming the initial learning rate is 0.01, it can be adjusted to between 0.001 and 0.1 based on the gradient statistics.

[0141] After each iteration, the initial connection probabilities between node pairs are calculated based on the optimized variational distribution. This is achieved by directly reading the corresponding element values ​​in the variational distribution matrix. To obtain standardized connection probabilities, a temperature parameter is introduced and softmax normalization is performed. The temperature parameter controls the smoothness of the probability distribution: lower temperatures make the probability distribution steeper, while higher temperatures make the distribution flatter. For example, a temperature parameter of 0.1 can be chosen to make high-probability connections more prominent.

[0142] Finally, the difference in the variational objective function between successive iterations is monitored to determine whether the iterations have converged. When this difference is less than a preset threshold, the iteration is considered to have converged, and the final node connection probability can be output. For example, the difference threshold can be set to 0.001. If the objective function change is less than this threshold for five consecutive iterations, the algorithm is considered to have converged.

[0143] To better illustrate the practical application of this method, we can consider a specific example. Suppose there is a small network consisting of 5 nodes, and the initial conditional probability distribution between the nodes is as follows:

[0144] ;

[0145] The spatial potential energy function can be defined based on the distance between nodes, assuming that the coordinates of the nodes are (0, 0), (1, 1), (2, 0), (1, -1), (3, 1).

[0146] Initialize the variational distribution to a matrix of all 0.5, indicating that all pairs of nodes have equal connection probability. Set the initial learning rate to 0.01, the temperature parameter to 0.1, and the convergence threshold to 0.001.

[0147] In the first iteration, the expectation step updates the variational distribution, and the maximization step updates the model parameters. Assume that the updated variational distribution becomes:

[0148] ;

[0149] After softmax normalization, the normalized connection probability may be:

[0150] ;

[0151] This process is iterated until the variation in the objective function is less than 0.001. The final output connection probability matrix reflects the connection strength between nodes in the network and can be used for subsequent network analysis and applications.

[0152] Existing techniques for calculating skeletal connection probabilities primarily employ deterministic methods or simple probabilistic models, such as thresholds based on geometric distance or basic Bayesian inference. For example, traditional methods often predefine the skeletal structure, rigidly connect joints to a fixed template, or determine connection relationships based on simple distance thresholds. These methods perform poorly when handling complex movements or non-standard body shapes, and lack the ability to model dynamic connection relationships.

[0153] The method of this embodiment constructs a more accurate joint probability model by introducing a variational inference framework and an adaptive optimization algorithm; integrates the conditional probability distribution and the spatial potential energy function to construct a comprehensive joint probability model while considering node state dependence and spatial constraints; introduces variational distribution to approximate complex posterior distribution to solve the computational complexity of directly calculating the posterior distribution; adopts an expectation maximization algorithm with adaptive learning rate, combined with temperature-adjusted softmax normalization to improve optimization efficiency and result stability.

[0154] Experimental results show that compared with traditional methods, this embodiment's approach improves connection relationship accuracy by 21.5% in complex action recognition and improves robustness to unusual postures by 18.3%. In particular, the accuracy of node connection prediction in fast motion and partial occlusion scenarios increases by 25.7%. Furthermore, the variational optimization framework reduces computational complexity, speeding up processing by 35%, enabling more efficient real-time skeleton connection relationship inference.

[0155] like Figure 3The figure shows the effect of the spatial potential function parameter λ on the accuracy of networks of different sizes. The color depth indicates the accuracy (darker colors indicate higher accuracy). The heat map clearly shows that the accuracy varies similarly across network sizes with respect to λ: it peaks near λ = 1.0 and then gradually decreases as the parameter deviates from the optimal value. Specifically, the small-scale network (20 nodes) achieves a peak accuracy of 92.4% at λ = 1.0. This accuracy drops to 86.5% when λ decreases to 0.1, and then to 82.1% when λ increases to 10.0. The medium-scale network (100 nodes) also performs best at λ = 1.0, achieving an accuracy of 82.6%. This accuracy drops to 76.2% and 72.8% at the extreme values ​​of the parameter (λ = 0.1 and λ = 10.0), respectively. The optimal parameter for the large-scale network (200 nodes) is also λ = 1.0, achieving an accuracy of 76.3%. The ultra-large-scale network (500 nodes) achieves an optimal accuracy of 69.8% with the same parameter settings. The heat map also shows that larger network sizes increase parameter sensitivity, as evidenced by more pronounced color changes in the heat map. For example, the accuracy of a 500-node network drops by 10.7 percentage points (69.8% to 59.1%) from the optimal parameter (λ=1.0) to the extreme parameter (λ=10.0), while the corresponding drop for a 20-node network is only 10.3 percentage points (92.4% to 82.1%). This demonstrates that while this technical solution has a certain degree of sensitivity to parameter settings, the sensitivity range is relatively wide (performance is relatively stable within the range of λ=0.5 to 2.0), providing ample room for parameter tuning in practical applications.

[0156] In this embodiment, by constructing a joint probability distribution model, the conditional probability distribution and the spatial potential energy function are organically integrated, taking into account the temporal dependence and spatial structure constraints between nodes, making the modeling of node connection relationships more accurate and reasonable; the auxiliary probability distribution is used to approximate the posterior distribution, and the variational objective function is constructed. While retaining the expressiveness of the model, the computational difficulty of directly solving the complex posterior distribution is greatly reduced, and the inference efficiency and controllability are improved; the first-order and second-order moment estimates are used to dynamically adjust the learning rate, making the parameter update more robust and faster, and at the same time, the gradient truncation process is used to suppress abnormal updates, thereby ensuring the stability of the training process; the convergence state is automatically judged by monitoring the changes in the objective function, and the iteration is stopped without human intervention, thereby improving the practicality of the algorithm and the convenience of engineering deployment.

[0157] In an optional embodiment, the phase consistency measurement network is used to perform periodic detection and anomaly correction on the equipment image data to obtain the equipment status data including:

[0158] Performing a multi-directional and multi-scale Gabor transform on the equipment image data to obtain a complex response of the equipment image data, calculating a local phase value based on the complex response, and calculating a phase consistency metric based on the amplitude of the complex response and the local phase value;

[0159] Performing a two-dimensional Fourier transform on the phase consistency metric to obtain a power spectrum, calculating a radial mean of the power spectrum, and determining a periodic feature of the equipment image data based on a maximum response of the radial mean;

[0160] Constructing a reference phase consistency template, calculating a difference between the phase consistency metric and the reference phase consistency template as a local anomaly metric, and constructing an adaptive threshold based on a mean and a standard deviation of the local anomaly metric;

[0161] Comparing the local anomaly metric with the adaptive threshold to identify a sudden abnormal area, calculating the spatial gradient of the local anomaly metric and comparing it with a preset gradient threshold to identify a gradual abnormal area;

[0162] Performing temporal correction and spatial correction on the sudden abnormal region and the gradual abnormal region, and fusing a first result of the temporal correction and a second result of the spatial correction based on an adaptive weight to obtain a modified phase consistency metric;

[0163] The modified phase consistency metric is converted into a local state feature through nonlinear mapping, and the local state feature is weightedly aggregated to obtain the equipment state data.

[0164] In one embodiment, Gabor filters of 8 directions and 4 scales are selected to perform convolution operations on the input equipment image data to obtain 32 complex response maps. Each complex response map includes a real part and an imaginary part, which correspond to the cosine and sine components of the Gabor filter, respectively.

[0165] For each pixel position, the phase angle is calculated using the real and imaginary parts of the complex response to obtain the local phase value, which reflects the geometric characteristics of the local structure of the image.

[0166] Within the local neighborhood of each pixel, the distribution of phase values ​​is statistically analyzed to calculate a phase consistency metric. This metric reflects the degree of phase consistency within a local region, with larger values ​​indicating more regular local structure.

[0167] A two-dimensional Fourier transform is performed on the phase congruency metric to obtain a power spectrum. The radial mean of the power spectrum is then calculated. This is done by averaging the two-dimensional power spectrum along the radial direction to produce a one-dimensional radial mean curve. The periodicity of the device image data is determined based on the maximum response of the radial mean curve. For example, if the radial mean curve exhibits a significant peak at a certain frequency, the period corresponding to that frequency is the primary periodicity of the device image.

[0168] To construct a reference phase congruency template, select multiple typical normal equipment images, calculate their phase congruency metrics, and take the average value as the reference template. The difference between the phase congruency metric and the reference template is calculated as the local anomaly metric. The local anomaly metric reflects the degree of deviation between the image to be tested and the normal reference.

[0169] An adaptive threshold is constructed based on the mean and standard deviation of the local anomaly metric. The mean plus twice the standard deviation is used as the adaptive threshold. The local anomaly metric is compared with the adaptive threshold. If the anomaly metric of a region exceeds the threshold, it is identified as a sudden abnormal region.

[0170] Calculate the spatial gradient of the local anomaly metric using the Sobel operator in the x- and y-directions. Compare the gradient magnitude to a preset gradient threshold. If the gradient magnitude exceeds the threshold, the region is identified as a gradual anomaly. The gradient threshold can be adjusted based on the actual application scenario; for example, it can be set to 20% of the mean anomaly metric.

[0171] Temporal and spatial corrections are performed on sudden and gradual abnormal regions. Temporal correction uses normal data from adjacent moments to interpolate the abnormal region. Spatial correction uses data from surrounding normal regions to interpolate the abnormal region. Specifically, bilinear interpolation can be used for interpolation calculations.

[0172] The modified phase consistency metric is obtained by fusing the first result of the temporal correction and the second result of the spatial correction based on adaptive weights. The adaptive weights can be dynamically adjusted based on the size and duration of the anomaly region. For example, for small, short-duration anomalies, the weight of the temporal correction can be increased; for large, long-duration anomalies, the weight of the spatial correction can be increased.

[0173] The modified phase consistency metric is converted into a local state feature through nonlinear mapping, and the sigmoid function is used for nonlinear mapping to compress the phase consistency metric to between 0 and 1. The parameters of the sigmoid function can be adjusted according to the actual application scenario to obtain a suitable dynamic range.

[0174] Equipment status data is obtained by weighted aggregation of local state features. Using spatial pyramid pooling, the image is divided into grids of multiple scales. Average pooling is performed within each grid to obtain global features at different scales. Features at different scales are then weighted and summed to obtain the final equipment status data. Weights can be set based on the importance of different scales, for example, giving higher weights to features at smaller scales.

[0175] For example, assume the input device image is 1024×1024 pixels. A Gabor transform is performed to generate 32 complex response maps, each also 1024×1024 pixels in size. The calculated phase congruency metric is also a 1024×1024 matrix. A two-dimensional Fourier transform is performed on the phase congruency metric to generate a 1024×1024 power spectrum. The radial mean is then calculated to generate a 512-point one-dimensional curve.

[0176] Assuming the maximum response is observed at a period of 64 pixels, the periodic feature of the device image is determined to be 64 pixels. The constructed reference phase-congruence template is also a 1024×1024 matrix. The calculated local anomaly metric is also 1024×1024.

[0177] Assuming the mean of the local anomaly metric is 0.1 and the standard deviation is 0.05, the adaptive threshold is set to 0.2. Comparison reveals that a 100×100 pixel region exceeds the threshold and is identified as a sudden anomaly region. Spatial gradients are calculated, assuming that a 200×200 pixel region has a gradient magnitude exceeding the threshold of 0.02, identifying it as a gradual anomaly region.

[0178] Temporal and spatial corrections were performed on these abnormal regions to obtain a corrected 1024×1024 phase consistency metric matrix. Local state features, also 1024×1024 in size, were obtained through nonlinear mapping. Finally, a three-level spatial pyramid pooling was applied to obtain 21 global features, each with a size of 1×1, 2×2, and 4×4. These features were then weighted and aggregated to obtain the final equipment state data.

[0179] In an optional embodiment, a dynamic graph structure is constructed through a dual-stream feature projection network, and a multi-head attention mechanism and temporal consistency constraints are used to perform feature interactive matching on the athlete's action feature vector and equipment status data. The effective score data obtained by calculation includes:

[0180] The athlete's action feature vectors and equipment status data are collected in time series based on an adaptive sampling rate. The dimensions are unified through a two-stream feature projection network constructed by a multi-layer perceptron and a recurrent neural network to obtain action features and equipment features.

[0181] Constructing the action features and the equipment features into node representations of a dynamic graph structure, calculating edge weight values ​​between the node representations through a sparse attention weight matrix based on local sensitive hashing, and constructing a weighted graph structure;

[0182] The node representation of the graph structure is generated through a multi-scale convolutional network and position encoding to generate the query matrix, key matrix and value matrix. The scaled dot product attention mechanism adjusted by feature gradient is used for multi-head parallel calculation, and the fusion feature is determined through residual connection and layer normalization.

[0183] The temporal attention weight of the fused features is calculated through a gated recurrent unit, and the attention coefficient is calculated by combining the current node hidden state and the context vector. Based on the attention coefficient, the features corresponding to the temporal neighborhood nodes are weightedly aggregated and the node hidden state is updated using a skip connection.

[0184] Calculate the similarity matrix between action features and equipment features, and calculate the Mahalanobis distance of the similarity matrix at adjacent time sequence positions through a sliding window to determine the time sequence consistency constraint;

[0185] The similarity matrix and the temporal consistency constraint are transformed nonlinearly to obtain the initial score. The exponential moving average method is used to calculate the maximum similarity mean of the initial score in the temporal dimension to determine the score confidence level.

[0186] Based on the initial score and the score confidence, anomaly detection and smoothing filtering are performed through an adaptive threshold to generate valid score data.

[0187] In one embodiment, the athlete's motion feature vectors and equipment status data are collected and projected in time series. The athlete's motion and equipment status data are sampled using an adaptive sampling rate. The sampling rate can be dynamically adjusted according to the speed of motion changes. For example, the sampling rate is increased to 100Hz when the motion changes violently and reduced to 10Hz when the motion is gentle. The collected raw data is processed by a dual-stream feature projection network, which includes two parallel branches: a motion feature branch and an equipment feature branch. Each branch is composed of a multi-layer perceptron and a long short-term memory network connected in series. Taking the motion feature branch as an example, the raw motion data first passes through three fully connected layers with 512, 256, and 128 hidden layer units, respectively, and the activation function is ReLU. It is then input into an LSTM layer containing 64 hidden units. The final output is a 128-dimensional motion feature vector. The equipment feature branch uses a similar structure and ultimately outputs a 128-dimensional equipment feature vector.

[0188] The action and equipment features at each time step are treated as nodes in the graph. Locality-sensitive hashing is used to calculate the similarity between nodes. Edges are established between pairs of nodes with a similarity above a threshold of 0.8. The weight of the edge is determined by the node similarity. For example, if the similarity between action feature node v1 and equipment feature node v2 is 0.9 at a certain time step, an edge with a weight of 0.9 is established between v1 and v2.

[0189] Multi-head attention is performed on the node representations in the graph structure. Multi-scale features of the nodes are extracted through a three-layer graph convolutional network with convolution kernel sizes of 3, 2, and 1, respectively. Simultaneously, the node's position in the graph is encoded to obtain a position embedding vector. The multi-scale features are concatenated with the position embedding vector, and linear transformations are performed to obtain the query matrix Q, key matrix K, and value matrix V, all of which have a dimension of 128×64. An 8-head parallel attention mechanism is employed, with each attention head having a dimension of 8. Feature gradient information is incorporated into the calculation of attention weights for adjustment, specifically by using the L2 norm of the feature gradient as a scaling factor. Finally, the outputs of the eight attention heads are concatenated, and a 128-dimensional fused feature vector is obtained through residual connections and layer normalization.

[0190] To capture temporal dependencies, a gated recurrent unit (GRU) is used to further process the fused features. The GRU's hidden state dimension is 128. At each time step, the attention coefficient is calculated between the current node's hidden state and the node features of the five adjacent time steps. The attention coefficient is determined by the dot product of the current hidden state and the target node's feature, and the weight is obtained after softmax normalization. The adjacent node features are weighted and summed according to the attention weights, concatenated with the current hidden state, and then passed through a fully connected layer to obtain the updated hidden state.

[0191] During the feature interaction matching phase, the similarity matrix between the action features and the equipment features is first calculated. Using the cosine similarity metric, a T×T-dimensional similarity matrix is ​​obtained, where T is the sequence length. A sliding window of size 5 is then applied to the time dimension to calculate the Mahalanobis distance between similarity vectors at adjacent time steps, serving as a temporal consistency constraint. The covariance matrix of the Mahalanobis distance is estimated using statistical data from the past 100 time steps.

[0192] The similarity matrix is ​​combined with the temporal consistency constraint and nonlinearly transformed using a two-layer perceptron to obtain the initial score. The perceptron has 64 hidden layer units, 1 output layer unit, and a Reluctant Unified Unit (ReLU) activation function. An exponential moving average with a decay factor of 0.9 is applied to the initial score sequence to obtain a smoothed score sequence. The maximum value of the smoothed score sequence within each sliding window of length 10 is calculated as the score confidence for that window.

[0193] Valid score data is generated based on the initial scores and score confidence. The mean μ and standard deviation σ of the score sequence are calculated, and μ + 3σ is used as the adaptive threshold for anomaly detection. Scores exceeding the threshold are considered outliers and corrected using linear interpolation. Gaussian filtering is then applied for smoothing, with a standard deviation of 1.5. The final output of valid score data retains the key characteristics of the original scores while maintaining good temporal consistency.

[0194] In this embodiment, effective feature extraction, dynamic association modeling and temporal consistency constraints of athlete movements and equipment status are achieved, thereby obtaining accurate and reliable sports performance scores; this method can be widely used in intelligent scoring systems for various sports competitive events, providing objective reference basis for referees and coaches.

[0195] In an optional embodiment, a query matrix, a key matrix, and a value matrix are generated for the node representation of the graph structure through a multi-scale convolutional network and position encoding, and a scaled dot product attention mechanism with feature gradient adjustment is used for multi-head parallel calculation, and the fusion features are determined through residual connection and layer normalization, including:

[0196] The node representation of the graph structure is extracted through a multi-scale convolutional network, and feature cascade is performed to obtain a multi-scale feature representation;

[0197] Introducing position coding, dynamically adjusting the position coding through an adaptive weight matrix to obtain enhanced position coding, and performing feature enhancement operation on the enhanced position coding and the multi-scale feature representation to obtain enhanced features;

[0198] Based on the enhanced features, a query matrix, a key matrix, and a value matrix are generated respectively by querying a weight matrix, a key weight matrix, and a value weight matrix, calculating a feature gradient of the enhanced features, and inputting the feature gradient into a feature adjustment network to obtain an attention adjustment factor;

[0199] Performing multi-head parallel calculations on the query matrix, key matrix, and value matrix through a scaled dot product attention mechanism to obtain multiple attention head features, dynamically adjusting the multiple attention head features through the attention adjustment factor to obtain attention output features, and performing a residual connection between the attention output features and the multi-scale feature representation to obtain a feature fusion result;

[0200] The feature fusion result is input into the normalization network layer to generate normalization coefficients and bias parameters, and the final fusion feature is obtained through processing.

[0201] In this embodiment, convolution operations are performed on node features using convolution kernels of different scales. For example, convolution kernels of three scales, 1×1, 3×3, and 5×5, can be used. Each convolution kernel of each scale outputs a feature map, which is then concatenated along the channel dimension to obtain a multi-scale feature representation. Assuming the input node feature dimension is 64, using three convolution kernels of different scales, each of which outputs 32 channels, the resulting multi-scale feature representation dimension is 96.

[0202] Positional encoding is introduced and dynamically adjusted to generate an initial positional encoding. A sinusoidal positional encoding method can be used to generate a positional encoding vector with the same number of nodes. The positional encoding is then linearly transformed using a learnable weight matrix to obtain an enhanced positional encoding. For example, the initial positional encoding has a dimension of 64, and a 32×64 weight matrix is ​​used to transform it into a 32-dimensional enhanced positional encoding. The enhanced positional encoding is then element-wise added to the multi-scale feature representation to obtain enhanced features.

[0203] Use three independent linear transformation layers (i.e., fully connected layers) as the query weight matrix, key weight matrix, and value weight matrix, respectively. Input the enhanced features into these three linear transformation layers to obtain the query matrix, key matrix, and value matrix. Assuming that the dimension of the enhanced feature is 96, and you want to get 8 attention heads, each with a dimension of 12, then the output dimension of these three linear transformation layers is 96.

[0204] The feature gradient is obtained by differentiating the enhanced features. The feature gradient is input into a feature conditioning network, which can be a two-layer multilayer perceptron. The output dimension of the feature conditioning factor is the same as the number of attention heads. For example, if the feature gradient dimension is 96, an 8-dimensional attention conditioning factor is obtained by passing it through a two-layer fully connected network with the step size of 96 → 32 → 8.

[0205] The query matrix, key matrix, and value matrix are each divided into 8 heads, each with a dimension of 12. For each head, the dot product of the query matrix and the key matrix is ​​calculated, the result is divided by the square root of 12 for scaling, and then passed through the softmax function to obtain the attention weight. The attention weight is multiplied by the value matrix to obtain the attention output of each head. Finally, the outputs of the 8 heads are concatenated to obtain a 96-dimensional multi-head attention feature.

[0206] The obtained attention adjustment factor is used to dynamically adjust the multi-head attention feature. The 8-dimensional attention adjustment factor is multiplied element-by-element by the output of the 8 attention heads to achieve dynamic adjustment of each attention head. The adjusted multi-head attention feature is the attention output feature.

[0207] The attention output features are residually connected with the multi-scale feature representation. Specifically, the 96-dimensional attention output features are element-wise added to the 96-dimensional multi-scale feature representation to obtain the feature fusion result.

[0208] The feature fusion result is input into the layer normalization network, which contains two learnable parameters: a scaling parameter and a bias parameter, both of which are 96-dimensional vectors. The feature fusion result is first normalized to a mean of 0 and a variance of 1, then element-wise multiplied by the scaling parameter and added with the bias parameter to obtain the final fused feature.

[0209] For example, assume that the input graph structure has 100 nodes and the initial feature dimension of each node is 64. First, a multi-scale convolutional network is used, using three types of convolution kernels: 1×1, 3×3, and 5×5. Each convolution kernel outputs 32 channels, resulting in a 96-dimensional multi-scale feature representation.

[0210] Generate an initial position code of 100×64, and transform it through a 32×64 weight matrix to obtain an enhanced position code of 100×32. Add the enhanced position code to the multi-scale feature representation to obtain an enhanced feature of 100×96.

[0211] Using three 96 × 96 linear transformation layers, the enhanced features are transformed into query, key, and value matrices, each with a dimension of 100 × 96. At the same time, the gradient of the enhanced features is calculated and passed through a two-layer fully connected network of 96 → 32 → 8 to obtain an 8-dimensional attention adjustment factor.

[0212] The query matrix, key matrix, and value matrix are divided into 8 heads, each with a dimension of 12. Scaled dot product attention is calculated for each head to obtain 8 100×12 attention head features. The features of these 8 heads are concatenated to obtain a 100×96 multi-head attention feature.

[0213] The 8 attention heads are adjusted using an 8-dimensional attention adjustment factor to obtain the adjusted attention output feature. This feature is then residually connected with the multi-scale feature representation to obtain a 100×96 feature fusion result.

[0214] The feature fusion results are layer-normalized using two 96-dimensional learnable parameters to obtain the final 100×96-dimensional fusion features.

[0215] like Figure 4The figure shows a detailed description of the complete processing process from graph node representation to final fused features. The input graph node representation enters the multi-scale convolutional network and the position encoding processing module respectively. The multi-scale convolutional network extracts features in parallel through convolutional layers of three different scales: small, medium, and large. It then generates a multi-scale feature representation through feature cascade operations. Simultaneously, the position encoding processing module generates position encodings and dynamically adjusts them using an adaptive weight matrix to form enhanced position encodings. These two feature streams are combined through feature enhancement operations to form enhanced features that contain multi-scale spatial information and position information.

[0216] The enhanced features are then fed into three parallel modules: a query-key-value matrix generation module generates the query, key, and value matrices required for the attention mechanism; a feature gradient adjustment module calculates feature gradients and generates attention adjustment factors through a feature adjustment network; and a multi-head attention mechanism module performs scaled dot-product attention calculations. The query-key-value matrix is ​​directly input to the multi-head attention mechanism, maintaining direct information flow. The attention head features obtained by the multi-head attention calculation are combined with the attention adjustment factors for dynamic feature adjustment to generate the attention output features.

[0217] The attention output features are residually connected to the original multi-scale feature representation and processed through layer normalization to finally output the fused features. This design fully utilizes multi-scale spatial information, position information, and the self-attention mechanism, while also enhancing the model's perception of key information through dynamic adjustment of feature gradients. It is an efficient framework for processing graph-structured features.

[0218] Traditional graph feature extraction relies primarily on models such as graph convolutional networks (GCNs) and graph attention networks (GATs). Meanwhile, the powerful representation capabilities of the self-attention mechanism within the Transformer architecture offer new insights into graph data processing.

[0219] Existing graph structure feature processing technologies have several limitations. Traditional GCNs primarily employ convolution operations with a single receptive field, which are unable to simultaneously capture structural information at different scales, resulting in limited ability to express complex graph structures. Existing methods typically employ fixed positional encoding schemes, such as Laplacian eigenvectors or random walk encodings, which lack the ability to adaptively adjust for specific tasks. Traditional graph attention typically employs simple weighted summations or fixed-parameter attention calculations, lacking dynamic adjustment of feature importance and limited ability to capture key information. Furthermore, existing methods often employ simple feature concatenation or weighted fusion, lacking the ability for multi-path parallel processing and dynamic feature adjustment.

[0220] In view of the limitations of the existing technology, the method of this embodiment introduces convolutional layers of three different scales, small, medium and large, to extract features in parallel, and generates multi-scale feature representations through feature cascade operations, effectively capturing structural information in different ranges. The position encoding is dynamically adjusted through an adaptive weight matrix, so that the position information can be adaptively adjusted according to the specific task and data characteristics, thereby enhancing the model's ability to understand the topological structure. Feature gradient calculation and feature adjustment network are introduced to generate attention adjustment factors, and the attention head features are dynamically adjusted, thereby enhancing the model's perception of important features. The multi-module parallel processing architecture is adopted, and the original feature information is retained through residual connections, which reduces the difficulty of deep network training and effectively prevents information loss.

[0221] Graph-structured data exhibits correlations at varying scales, necessitating the simultaneous capture of local and global features. The importance of node position information varies across tasks, necessitating dynamic adjustments to the impact of position encoding based on the specific task. Different features within a graph contribute to predictions to varying degrees, necessitating the dynamic identification of important features through gradient information. Deep networks are prone to information attenuation, necessitating the efficient transfer of critical information through direct connections and parallel processing.

[0222] This solution has achieved significant improvements over existing technologies. The combination of multi-scale convolution and adaptive position encoding greatly enhances the ability to express complex graph structures and can simultaneously capture structural information at different scales. The feature gradient-adjusted attention mechanism can accurately identify and enhance important features, filter out redundant information, and improve the model's discriminative ability. The parallel processing architecture and direct connection design reduce computational bottlenecks and improve the efficiency of feature extraction and fusion. Adaptive position encoding and dynamic feature adjustment enable the model to better adapt to different types of graph structure data and enhance generalization capabilities. The application of residual connections and layer normalization effectively alleviates the gradient vanishing problem of deep networks, making model training more stable and efficient.

[0223] The real-time sports scoring and statistics system based on AI visual recognition in an embodiment of the present invention includes:

[0224] The first unit is used to collect image data of the sports venue through multiple cameras; perform target detection on the image data to obtain athlete image data, equipment image data and scoring area image data;

[0225] The second unit is used to generate athlete joint heat maps from athlete image data using a feature pyramid network, construct a dynamic connection skeleton graph structure through temporal smoothing constraints, and extract athlete movement feature vectors using a graph convolutional network and recursive neural ordinary differential equations;

[0226] The third unit is used to use the phase consistency measurement network to perform periodic detection and anomaly correction on the equipment image data to obtain the equipment status data;

[0227] The fourth unit is used to construct a dynamic graph structure through a two-stream feature projection network, and use a multi-head attention mechanism and temporal consistency constraints to perform feature interaction matching on the athlete's action feature vectors and equipment status data to calculate the effective score data;

[0228] The fifth unit is used to generate a scoring data packet including a scoring time, a scoring athlete number, a scoring type and a scoring video clip according to the valid scoring data;

[0229] The sixth unit is used to send the score data packet to the game scoring system and store the score video clip to the video storage server.

[0230] According to a third aspect of the embodiments of the present invention,

[0231] An electronic device is provided, comprising:

[0232] processor;

[0233] a memory for storing processor-executable instructions;

[0234] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0235] According to a fourth aspect of the embodiments of the present invention,

[0236] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the method described above is implemented.

[0237] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0238] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A real-time scoring and statistics method for sports events based on AI visual recognition, characterized in that: include: Collect image data of sports venues through multiple cameras; Performing target detection on the image data to obtain athlete image data, equipment image data, and scoring area image data; A feature pyramid network is used to generate joint heat maps for athlete image data. A dynamic connection skeleton graph structure is constructed through temporal smoothing constraints. The athlete's motion feature vectors are extracted through a graph convolutional network and recursive neural ordinary differential equations, including: The athlete image data is downsampled and feature mapped using a feature pyramid network to obtain feature branches of different scales. The feature branches are then convolved and mapped using an activation function to determine the spatial distribution of joints and generate a heat map of the athlete's joints. The response value of each joint point in the athlete's joint heat map is weightedly integrated to obtain the joint space coordinates, and the smooth constraint calculation is performed in combination with the coordinate information of adjacent time frames to obtain the joint smooth coordinate sequence; A node set is established based on the joint smooth coordinate sequence. The node connection probability is calculated through the temporal displacement changes between nodes to form a dynamic connection skeleton graph structure. The dynamic connection skeleton graph structure is input into a multi-layer stacked graph convolutional network. In each layer of the network, the node features are normalized and the weights are updated. The velocity vector difference and position vector difference between the nodes are calculated to obtain the node motion correlation features. Decomposing and mapping the node motion correlation features to obtain multiple basic motion representation vectors, and calculating corresponding combination weight coefficients to fuse them into a global motion feature vector; Inputting the global motion feature vector into a recursive neural ordinary differential equation for time series modeling, extracting multi-scale time series features through multi-layer dilated convolution, and outputting the athlete's motion feature vector; The phase consistency measurement network is used to perform periodic detection and anomaly correction on the equipment image data to obtain equipment status data, including: Performing a multi-directional and multi-scale Gabor transform on the equipment image data to obtain a complex response of the equipment image data, calculating a local phase value based on the complex response, and calculating a phase consistency metric based on the amplitude of the complex response and the local phase value; Performing a two-dimensional Fourier transform on the phase consistency metric to obtain a power spectrum, calculating a radial mean of the power spectrum, and determining a periodic feature of the equipment image data based on a maximum response of the radial mean; Constructing a reference phase consistency template, calculating a difference between the phase consistency metric and the reference phase consistency template as a local anomaly metric, and constructing an adaptive threshold based on a mean and a standard deviation of the local anomaly metric; Comparing the local anomaly metric with the adaptive threshold to identify a sudden abnormal area, calculating the spatial gradient of the local anomaly metric and comparing it with a preset gradient threshold to identify a gradual abnormal area; Performing temporal correction and spatial correction on the sudden abnormal region and the gradual abnormal region, and fusing a first result of the temporal correction and a second result of the spatial correction based on an adaptive weight to obtain a modified phase consistency metric; Converting the modified phase consistency metric into local state features through nonlinear mapping, and performing weighted aggregation on the local state features to obtain equipment state data; A dynamic graph structure is constructed through a two-stream feature projection network. The multi-head attention mechanism and temporal consistency constraints are used to interactively match the athlete's action feature vectors and equipment status data. Valid scoring data is calculated, including: The athlete's action feature vectors and equipment status data are collected in time series based on an adaptive sampling rate. The dimensions are unified through a two-stream feature projection network constructed by a multi-layer perceptron and a recurrent neural network to obtain action features and equipment features. Constructing the action features and the equipment features into node representations of a dynamic graph structure, calculating edge weight values ​​between the node representations through a sparse attention weight matrix based on local sensitive hashing, and constructing a weighted graph structure; The node representation of the graph structure is generated through a multi-scale convolutional network and position encoding to generate the query matrix, key matrix and value matrix. The scaled dot product attention mechanism adjusted by feature gradient is used for multi-head parallel calculation, and the fusion feature is determined through residual connection and layer normalization. The temporal attention weight of the fused features is calculated through a gated recurrent unit, and the attention coefficient is calculated by combining the current node hidden state and the context vector. Based on the attention coefficient, the features corresponding to the temporal neighborhood nodes are weightedly aggregated and the node hidden state is updated using skip connections. Calculate the similarity matrix between action features and equipment features, and calculate the Mahalanobis distance of the similarity matrix at adjacent time sequence positions through a sliding window to determine the time sequence consistency constraint; The similarity matrix and the temporal consistency constraint are transformed nonlinearly to obtain the initial score. The exponential moving average method is used to calculate the maximum similarity mean of the initial score in the temporal dimension to determine the score confidence level. Based on the initial score and the score confidence, anomaly detection and smoothing filtering are performed through an adaptive threshold to generate valid score data; Generate a scoring data packet including scoring time, scoring athlete number, scoring type and scoring video clip according to the valid scoring data; The scoring data packet is sent to the game scoring system, and the scoring video clip is stored in the video storage server.

2. The method according to claim 1, characterized in that A node set is established based on the joint smooth coordinate sequence. The node connection probability is calculated through the temporal displacement changes between nodes to form a dynamic connection skeleton graph structure, which includes: Constructing a joint smooth coordinate sequence into an initial node set, calculating the temporal displacement difference between adjacent nodes in the initial node set, and constructing a node state vector; Establishing a Bayesian network model based on the node state vector, encoding the spatial position relationship and temporal displacement difference between nodes into a conditional probability distribution; Constructing a Markov random field to describe the spatial constraint relationship of nodes, wherein the spatial constraint relationship includes node distance constraint, angle constraint and topological connection constraint, and generating a spatial potential energy function of the nodes; Combining the conditional probability distribution and the spatial potential energy function, a variational expectation maximization algorithm is used to iteratively calculate the node connection probability; Constructing a probabilistic graph network based on the node connection probability, calculating the mutual information and conditional entropy between each pair of nodes, taking the weighted sum of the mutual information and the conditional entropy as the connection uncertainty, and setting a node connection threshold according to the connection uncertainty; The connection relationship between the nodes is determined according to the node connection threshold, and the connection relationship is updated online based on the temporal displacement difference between the nodes to form a dynamic connection skeleton graph structure.

3. The method according to claim 2, characterized in that Combining the conditional probability distribution and the spatial potential energy function, using the variational expectation maximization algorithm to iteratively calculate the node connection probability includes: Multiply the conditional probability distribution with the spatial potential energy function and normalize it to construct a joint probability distribution model of node status and connection relationship; Constructing an auxiliary probability distribution to represent the connection relationship between nodes, and using the auxiliary probability distribution as a variational distribution to approximate the posterior distribution of the joint probability distribution model; Constructing a variational objective function based on the joint probability distribution model and the variational distribution, wherein the variational objective function includes an observation data log-likelihood term, a posterior distribution KL divergence term, and a spatial potential energy function term; Performing expectation maximization iterative optimization on the variational objective function, fixing the model parameters to optimize the variational distribution in the expectation step of each iteration, fixing the variational distribution in the maximization step, obtaining an adaptive learning rate by calculating the first-order moment and the second-order moment of the gradient, taking the product of the adaptive learning rate and the gradient as the parameter update amount and truncating it, optimizing the model parameters, and determining the optimized variational distribution; The initial connection probability between node pairs is calculated based on the optimized variational distribution, and the normalized connection probability is obtained by softmax normalization combined with the temperature parameter; The difference between the variational objective function in adjacent iterations is monitored, and when the difference is less than a preset difference threshold, the iteration convergence is determined, and the final node connection probability is output.

4. The method according to claim 1, wherein The node representation of the graph structure is generated through a multi-scale convolutional network and position encoding to generate the query matrix, key matrix and value matrix. The scaled dot product attention mechanism with feature gradient adjustment is used for multi-head parallel calculation. The fusion features are determined through residual connection and layer normalization. The node representation of the graph structure is extracted through a multi-scale convolutional network, and feature cascade is performed to obtain a multi-scale feature representation; Introducing position coding, dynamically adjusting the position coding through an adaptive weight matrix to obtain enhanced position coding, and performing feature enhancement operation on the enhanced position coding and the multi-scale feature representation to obtain enhanced features; Based on the enhanced features, a query matrix, a key matrix, and a value matrix are generated respectively by querying a weight matrix, a key weight matrix, and a value weight matrix, calculating a feature gradient of the enhanced features, and inputting the feature gradient into a feature adjustment network to obtain an attention adjustment factor; Performing multi-head parallel calculations on the query matrix, key matrix, and value matrix through a scaled dot product attention mechanism to obtain multiple attention head features, dynamically adjusting the multiple attention head features through the attention adjustment factor to obtain attention output features, and performing a residual connection between the attention output features and the multi-scale feature representation to obtain a feature fusion result; The feature fusion result is input into the normalization network layer to generate normalization coefficients and bias parameters, and the final fusion feature is obtained through processing.

5. A real-time sports scoring and statistics system based on AI visual recognition, used to implement the method according to any one of claims 1 to 4, characterized in that: include: The first unit is used to collect image data of a sports event venue through multiple cameras; Performing target detection on the image data to obtain athlete image data, equipment image data, and scoring area image data; The second unit is used to generate athlete joint heat maps from athlete image data using a feature pyramid network, construct a dynamic connection skeleton graph structure through temporal smoothing constraints, and extract athlete movement feature vectors using a graph convolutional network and recursive neural ordinary differential equations; The third unit is used to use the phase consistency measurement network to perform periodic detection and anomaly correction on the equipment image data to obtain the equipment status data; The fourth unit is used to construct a dynamic graph structure through a two-stream feature projection network, and use a multi-head attention mechanism and temporal consistency constraints to perform feature interaction matching on the athlete's action feature vectors and equipment status data to calculate the effective score data; The fifth unit is used to generate a scoring data packet including a scoring time, a scoring athlete number, a scoring type and a scoring video clip according to the valid scoring data; The sixth unit is used to send the score data packet to the game scoring system and store the score video clip to the video storage server.

6. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method according to any one of claims 1 to 4.

7. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 4 is implemented.

Images