Competitive sports-oriented skeleton trajectory deep learning compensation method and system

Abstract

The invention relates to the technical field of action recognition, in particular to a competitive sports-oriented skeleton trajectory deep learning compensation method and system, and the system comprises a data preprocessing module, a data integrity evaluation module, a core compensation network, a biomechanical rationality optimization module, a personalized habit encoder and an advanced compensation path. Compared with the prior art that a single trajectory compensation model is generally adopted, challenges of shielding scenes of different degrees are difficult to effectively deal with, and the problem of insufficient short-time shielding compensation precision or trajectory distortion under long-time shielding often occurs, an intelligent routing mechanism based on data integrity evaluation is adopted, and the method has the advantages that the complexity is reduced, and the reliability is improved. Through dynamic switching of a basic compensation path and an advanced compensation path fused with personalized prediction, adaptive processing of different shielding scenes is realized, reconstruction precision under short-time shielding is ensured, track rationality and continuity under a long-time shielding scene are remarkably improved through motion trend prediction, and the reconstruction precision is improved. And the practicability and the reliability of the system in an actual competitive environment are enhanced.

Description

Technical Field

[0001] This invention relates to the field of motion recognition technology, and in particular to a deep learning compensation method and system for skeletal trajectory in competitive sports. Background Technology

[0002] In the field of competitive sports training and analysis, computer vision-based skeletal trajectory capture technology has become an important tool for athlete technical diagnosis and performance evaluation. However, in practical applications, due to factors such as mutual occlusion between athletes, equipment obstruction, and limitations in shooting angles, skeletal trajectories acquired from video data often suffer from missing data or quality degradation at some joint points. How to accurately compensate and reconstruct incomplete or noisy skeletal trajectory data is a key technical challenge for improving the reliability of sports data analysis.

[0003] Currently, most existing skeletal trajectory compensation methods employ a uniform processing model to handle various occlusion scenarios. While these single-model methods can maintain a certain level of accuracy when facing short-term, localized occlusion, they perform poorly when dealing with occlusion of varying degrees and durations. Specifically, when encountering short-term occlusion, the model often fails to fully utilize the rich information from neighboring frames, resulting in insufficient compensation accuracy. Furthermore, when facing long-term continuous occlusion, the lack of an effective motion trend prediction mechanism easily leads to trajectory distortion and results that violate motion laws. This lack of adaptability severely limits the practical value of existing technologies in real-world competitive scenarios.

[0004] To address the aforementioned issues, this invention proposes a deep learning compensation method and system for skeletal trajectory in competitive sports. Summary of the Invention

[0005] To overcome the problems mentioned in the background art, the present invention proposes a deep learning compensation method and system for skeletal trajectory in competitive sports.

[0006] The technical solution of this invention is: a deep learning compensation system for skeletal trajectory in competitive sports, comprising: The data preprocessing module is used to perform data preprocessing operations on the original skeletal trajectory sequence data and represent the skeletal data in a standardized spatiotemporal graph structure, where joints are used as graph nodes and bones are used as graph edges. The data integrity assessment module is used to calculate the proportion of visible key points in the current input sequence and compare it with a preset integrity threshold. Based on the comparison result, the data is routed to different processing paths. The core compensation network adopts a spatiotemporally separated graph convolutional network architecture, which includes a spatial graph convolutional layer for modeling the spatial relationships between joints and a temporal convolutional layer for modeling the temporal dynamics of the joints themselves; it is used to receive the routed data and output the trajectory sequence of the preliminary compensation. The biomechanical rationality optimization module is used to apply biomechanical constraints to the preliminary compensated trajectory sequence, including joint angle limit constraints, constant bone length constraints, and motion smoothness constraints, and outputs the final high-quality bone trajectory sequence. A personalized habit encoder is used to extract low-dimensional habit embedding vectors from the historical unoccluded motion data of a specific athlete, and the habit embedding vectors are input into the core compensation network as conditional information.

[0007] As a preferred approach, the biomechanical rationality optimization module optimizes the network parameters by minimizing the composite loss function, which is expressed as follows: ; in, This is the total loss value. , , and These are the weighting coefficients for each loss term. The mean squared loss error, For joint angle constraint loss, For motion smoothness constraint loss, This is due to the loss of bone length stability.

[0008] The formula for calculating the mean squared error loss is as follows: ; in, The total number of frames in the sequence. The total number of key points. Let J be the predicted coordinates of the j-th joint in frame t. Let be the true coordinates of the j-th joint in frame t.

[0009] The joint angle constraint loss includes angle limit penalty and angle continuity penalty, and the principle formula is as follows: ; in, As an angle limit penalty, This is a penalty for angular continuity.

[0010] The motion smoothness constraint loss includes an acceleration penalty term and a jerk penalty term.

[0011] Preferably, the data preprocessing module includes the following components during operation: S11: Receive skeletal trajectory sequence data from the original data source, and perform data cleaning on the received data, detecting and removing outliers; S12: Perform spatial normalization on the cleaned data to obtain spatially normalized data in a standardized coordinate system; S13: Perform time alignment processing on the spatially normalized data; S14: Treat the key points of each time frame as nodes of the spatial graph, treat the bones as edges of the spatial graph, and connect the same key points of adjacent time frames to form temporal edges, thus constructing a spatiotemporal graph containing spatial and temporal dimensions.

[0012] The spatial normalization process for the cleaned data includes: S121: For each frame of data, select a stable body center point as the new origin of the coordinate system, and subtract the coordinates of the center point from the coordinates of all joint points. S122: Rotate the coordinate system according to the athlete's orientation so that the athlete's frontal orientation is aligned with a specific axis of the new coordinate system. The orientation is determined by the direction vector of the line connecting the body's center point to key points such as the left shoulder, right shoulder, left hip, and right hip. S123: Divide all node coordinates by the body proportion factor to eliminate the influence of differences in body shape among athletes. The body proportion factor is preferably the average length of all bones, pelvic width, or trunk length. The underlying formula is: ; in, This represents the original coordinates of the i-th joint in frame t. This represents the coordinates of the selected body center point in frame t. For body proportion factor, This represents the coordinates of the i-th joint in frame t after spatial normalization.

[0013] Preferably, the data integrity assessment module includes the following components during operation: S21: Receive standardized skeletal trajectory sequence data from the data preprocessing module, including the three-dimensional coordinates of each joint and its visibility confidence. S22: Based on visibility confidence, calculate the global data integrity score of the current input sequence and compare the global data integrity score with a preset integrity threshold; S23: Based on the comparison results, generate a routing control signal to direct the data flow to different processing paths; when the global data integrity score is greater than or equal to the preset threshold, route to the advanced compensation path containing the personalized habit encoder and short-term trajectory predictor; when the global data integrity score is less than the preset threshold, route to the basic compensation path.

[0014] Specifically, when calculating the global data integrity score of the current input sequence based on visibility confidence, the method used is as follows: For a sequence of length T frames containing J keypoints, first calculate the frame-level integrity of each frame, then take the average of the frame-level integrity of all frames to obtain the global data integrity score. The formula for calculating the frame-level integrity is as follows: ; in, This represents the frame-level integrity of frame t. The total number of key points. This represents the visibility confidence of the j-th keypoint in the t-th frame. The visibility confidence threshold. This is an indicator function that returns 1 immediately if the condition is true, and 0 otherwise. is the weight coefficient of the j-th joint.

[0015] The weighting coefficients are assigned based on the biomechanical importance of the joints in a specific motion: For lower limb-dominant movements, the hip, knee, and ankle joints are given higher weight. For upper limb-dominant movements, the shoulder, elbow, and wrist joints are given higher weight; For full-body coordination exercises, the pelvis and spinal core joints are given higher weight.

[0016] The formula for calculating the global data integrity score is as follows: ; in, The overall data integrity score is given. The total number of frames in the sequence. Let t be the frame-level integrity of the t-th frame.

[0017] Preferably, the data integrity assessment module also includes a temporal continuity assessment during operation, specifically including: Before calculating the global data integrity score, the number of consecutively occluded frames is detected. When the number of consecutively occluded frames exceeds a preset temporal threshold, the calculation of the global data integrity score is stopped, and the data flow is directly directed to an advanced compensation path that includes a personalized habit encoder and a short-term trajectory predictor.

[0018] As a preferred option, advanced compensation pathways specifically include: A11: Short-term trajectory predictor, used to predict skeletal trajectories for several future time frames based on current and past complete trajectory data and habitual embedding vectors; A12: Fusion module, connected to the short-term trajectory predictor and the core compensation network, used to perform weighted fusion of the preliminary compensated trajectory output by the core compensation network and the predicted trajectory output by the short-term trajectory predictor. Among them, the fusion weight changes over time. For a frame that has just been occluded, the weight of the compensated trajectory is greater than the weight of the predicted trajectory. As the prediction time point is delayed, the weight of the predicted trajectory gradually increases.

[0019] As a preferred option, the core compensation network specifically includes: A21: Spatial feature extraction unit, used to apply graph convolution operation to each frame of the input skeletal spatiotemporal map to extract the spatial structural features between joints; A22: Temporal feature extraction unit, used to apply a one-dimensional temporal convolution operation to the temporal trajectory of each joint point to extract the temporal dynamic features of joint motion; A23: Feature fusion unit, connected to the spatial feature extraction unit and the temporal feature extraction unit, is used to fuse spatial features and temporal features to generate compensated features containing rich spatiotemporal context information; A24: Trajectory Reconstruction Unit, used to reconstruct a complete skeletal trajectory sequence based on compensation features.

[0020] The spatial feature extraction unit employs a dual-flow graph convolutional architecture, including: Joint node flow uses joints as graph nodes and natural skeletal connections between joints as edges, and learns the structural relationships between joints through a graph convolutional neural network. Skeletal edge flow uses bone vectors as graph nodes and adjacent bone vectors as edges to learn the motion patterns of limb segments through a graph convolutional neural network. The output features of the two streams are concatenated or weighted and fused in subsequent network layers.

[0021] The graph convolution operation used in the spatial feature extraction unit is based on the following formula: ; in, This represents the feature matrix of the nodes in the (l+1)th layer. This represents the feature matrix of the nodes in the l-th layer. For adjacency matrices with added self-connections, for The degree matrix, These are trainable spatial convolution weight parameters. It is a non-linear activation function.

[0022] The temporal feature extraction unit employs dilated causal convolution, and its calculation formula is as follows: ; in,, This represents the feature of the (l+1)th layer at time t. The kernel size is [size]. As the expansion factor, These are the trainable temporal convolution weight parameters.

[0023] The core compensation network adopts an encoder-decoder architecture: The encoder part contains multiple spatiotemporal decoupling graph convolutional blocks, each containing spatial graph convolutional layers and temporal convolutional layers, for progressively extracting and compressing high-level features of the input sequence; The decoder section contains multiple deconvolutional layers and upsampling layers for reconstructing the complete skeletal trajectory sequence from high-level features; In this method, a skip connection is used between the encoder and the decoder to stitch together the high-resolution spatial features of the corresponding layer in the encoder with the features in the decoder in order to preserve detailed information.

[0024] As a preferred embodiment, the personalized habit encoder, when in operation, specifically includes: S31: Collect and preprocess historical skeletal trajectory data of athletes in an unobstructed state to construct a personalized training dataset; S32: Construct a habit feature extraction network to learn athletes' personalized movement patterns from historical data; S33: Encode the learned personalized movement patterns into low-dimensional habit embedding vectors; S34: The habitual embedding vector is used as conditional information input to the core compensation network to guide it in performing personalized compensation for occluded trajectories.

[0025] The construction of personalized training datasets includes: Collect standardized movement sequences performed by athletes in multiple training sessions, including the athlete's typical technical movements; Each action sequence undergoes rigorous quality control screening, retaining only high-quality sequences with data completeness exceeding a preset threshold; The filtered sequences are time-normalized to give them a uniform time length.

[0026] The habitual feature extraction network employs an encoder-decoder architecture, including: A temporal encoder, based on the Transformer architecture, is used to extract temporal features from skeletal trajectory sequences; The feature aggregation layer is used to aggregate temporal features into a global feature vector of fixed dimensions; The decoder, based on a deconvolutional network, is used to reconstruct the original skeletal trajectory from the global feature vector.

[0027] Specifically, when using habitual embedding vectors as conditional information input into the core compensation network, the methods include: Vector concatenation method: Concatenate the custom embedding vector with the skeletal feature vector of each frame; Feature modulation method: Adaptive instance normalization modulation of feature maps in the core compensation network is performed using conventional embedding vectors; Attention-guided approach: Use habit embedding vectors as query vectors and retrieve the most relevant motion patterns from historical templates through an attention mechanism.

[0028] Preferably, the core compensation network, when in operation, includes the following steps: S41: Receive preprocessed spatiotemporal skeletal graph data, wherein the spatiotemporal skeletal graph data contains the spatial connection relationship of joints and temporal continuous frame information; S42: A spatiotemporally separated graph convolutional network is used to extract features from the input data to obtain spatiotemporal features. Among them, the spatial graph convolutional layer aggregates the spatial features between joints along the topology of the human skeleton, and the temporal convolutional layer extracts the motion dynamic features of each joint along the time dimension. S43: Integrate the habit embedding vector from the personalized habit encoder into the spatiotemporal features to obtain the fused high-level features; S44: The fused high-level features are mapped back to the skeletal trajectory space through the decoder network to generate a preliminary compensated trajectory sequence. The decoder uses deconvolution layers and upsampling operations to gradually restore trajectory details. S45: Output the preliminary compensated trajectory sequence to the biomechanical rationality optimization module for subsequent optimization processing.

[0029] A deep learning-based method for skeletal trajectory compensation in competitive sports includes the following steps: S51: The data preprocessing module performs data preprocessing operations on the original skeletal trajectory sequence data, representing the skeletal data as a standardized spatiotemporal graph structure, where joints are graph nodes and bones are graph edges. S52: The data integrity assessment module calculates the proportion of visible key points in the current input sequence and compares it with the preset integrity threshold. Based on the comparison result, the data is routed to different processing paths. S53: Processes the routed data through the core compensation network and outputs a preliminary compensated trajectory sequence; S54: Apply biomechanical constraints to the initially compensated trajectory sequence through the biomechanical rationality optimization module, and output the final high-quality skeletal trajectory sequence; S55: Extracts low-dimensional habit embedding vectors from the historical unoccluded motion data of a specific athlete through a personalized habit encoder, and inputs the habit embedding vectors as conditional information into the core compensation network to guide it to perform personalized compensation for the trajectory of the occluded parts. In step S52, different processing paths are selected based on the data integrity assessment results. When the data integrity is higher than the preset threshold, an advanced compensation process that includes short-term trajectory prediction and fusion processing is activated; when the data integrity is lower than the preset threshold, a basic compensation process is adopted.

[0030] The beneficial effects of this invention are: 1. Compared with existing technologies that typically use a single trajectory compensation model, which is difficult to effectively cope with the challenges of different degrees of occlusion and often suffers from insufficient compensation accuracy for short-term occlusion or trajectory distortion under long-term occlusion, this invention adopts an intelligent routing mechanism based on data integrity assessment. By dynamically switching between the basic compensation path and the advanced compensation path that integrates personalized prediction, it achieves adaptive processing of different occlusion scenarios. This not only ensures the reconstruction accuracy under short-term occlusion, but also significantly improves the rationality and continuity of the trajectory under long-term occlusion scenarios through motion trend prediction, greatly enhancing the practicality and reliability of the system in actual competitive environments. 2. Compared with existing technologies that focus on the accuracy of trajectory coordinates while ignoring the physiological rationality of movement, which may lead to the generation of abnormal postures that violate human kinematics, this invention adopts a multi-objective biomechanical optimization mechanism that includes joint angle constraints, bone length stability and movement smoothness. It explicitly incorporates human physiological limitations into the loss function, ensuring that the output compensation trajectory strictly conforms to biomechanical laws, effectively avoiding the generation of unreasonable movements, and providing safer and more reliable data support for athlete technical analysis and injury prevention. 3. Compared with existing technologies that use a general model to process all athlete data, which cannot capture individual differences in movement characteristics and result in compensation results that do not match the actual technical characteristics of athletes, this invention constructs a personalized habit encoder to extract habit embedding vectors that represent the unique movement patterns of athletes from historical data and deeply integrates them as conditional information into the compensation process. This makes the generated trajectory not only accurately restore the movement process, but also reflect the athlete's unique technical style and habit patterns, significantly improving the accuracy of individualized analysis and the pertinence of training guidance. 4. Compared with existing technologies that do not pay enough attention to data preprocessing and often use raw coordinates directly, which makes the model susceptible to individual differences and noise interference, this solution designs a complete spatial normalization and temporal alignment process. Through coordinate system translation and rotation, body size scaling and temporal interpolation, it eliminates the non-critical influence of the athlete's absolute position, body size and movement speed, and provides the model with a standardized spatiotemporal graph input, which greatly improves the convergence speed, generalization ability and adaptability to athletes of different body types of deep learning models. 5. Compared with existing technologies that mostly use isolated modules in a pipeline approach, where inconsistent optimization objectives among modules may lead to error accumulation, this solution integrates preprocessing, feature extraction, personalized compensation, and biomechanical optimization into a unified system through an end-to-end joint training framework. This enables each module to learn collaboratively under the guidance of a common objective function, achieving seamless connection from raw data to the final optimized trajectory. This effectively reduces information loss between processing stages and improves the overall system's compensation efficiency and trajectory quality consistency. Attached Figure Description

[0031] Figure 1 The diagram shown is a structural schematic of the deep learning compensation system for skeletal trajectory in competitive sports according to the present invention. Figure 2 The diagram shown is a flowchart of the deep learning compensation method for skeletal trajectory in competitive sports according to the present invention. Detailed Implementation

[0032] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0033] Please see Figure 1 - Figure 2 This invention provides an embodiment: a deep learning compensation system for skeletal trajectory in competitive sports, comprising: The data preprocessing module is used to perform data preprocessing operations on the original skeletal trajectory sequence data and represent the skeletal data in a standardized spatiotemporal graph structure, where joints are used as graph nodes and bones are used as graph edges. The data integrity assessment module is used to calculate the proportion of visible key points in the current input sequence and compare it with a preset integrity threshold. Based on the comparison result, the data is routed to different processing paths. The core compensation network adopts a spatiotemporally separated graph convolutional network architecture, which includes a spatial graph convolutional layer for modeling the spatial relationships between joints and a temporal convolutional layer for modeling the temporal dynamics of the joints themselves; it is used to receive the routed data and output the trajectory sequence of the preliminary compensation. The biomechanical rationality optimization module is used to apply biomechanical constraints to the preliminary compensated trajectory sequence, including joint angle limit constraints, constant bone length constraints, and motion smoothness constraints, and outputs the final high-quality bone trajectory sequence. A personalized habit encoder is used to extract low-dimensional habit embedding vectors from the historical unoccluded motion data of a specific athlete, and the habit embedding vectors are input into the core compensation network as conditional information.

[0034] In this embodiment, the invention first standardizes the original skeletal trajectory into a spatiotemporal graph structure through a data preprocessing module, and then intelligently selects either a basic or advanced compensation pathway based on the proportion of visible joints using a data integrity assessment module. Subsequently, the core compensation network of a spatiotemporal separable graph convolutional network, guided by athlete-specific features provided by a personalized habit encoder, performs spatiotemporal feature extraction and preliminary trajectory reconstruction. Finally, a biomechanical rationality optimization module applies constraints on joint angles, bone lengths, and motion smoothness to the output trajectory. Thus, through multi-module collaboration, high-precision compensation of skeletal trajectories under occlusion conditions is achieved. This not only enhances the model's ability to capture motion features using spatiotemporal separable convolution, but also ensures that the compensation results conform to both the athlete's individualized movement patterns and the laws of human kinematics through personalized habit embedding and biomechanical constraints, significantly improving the accuracy and practicality of competitive sports data analysis.

[0035] As a preferred approach, the biomechanical rationality optimization module optimizes the network parameters by minimizing the composite loss function, which is expressed as follows: ; in, This is the total loss value. , , and These are the weighting coefficients for each loss term. The mean squared loss error, For joint angle constraint loss, For motion smoothness constraint loss, This is due to the loss of bone length stability.

[0036] The formula for calculating the mean squared error loss is as follows: ; in, The total number of frames in the sequence. The total number of key points. Let J be the predicted coordinates of the j-th joint in frame t. Let be the true coordinates of the j-th joint in frame t.

[0037] The joint angle constraint loss includes angle limit penalty and angle continuity penalty, and the principle formula is as follows: ; in, As an angle limit penalty, This is a penalty for angular continuity.

[0038] The motion smoothness constraint loss includes an acceleration penalty term and a jerk penalty term.

[0039] In this embodiment, the biomechanical rationality optimization module of the present invention optimizes the network parameters by minimizing a composite loss function, which consists of four weighted components: mean square error loss, joint angle constraint loss, motion smoothness constraint loss, and bone length stability loss. The mean square error loss ensures basic accuracy of trajectory compensation by calculating the difference between predicted and actual coordinates; the joint angle constraint loss ensures joint movement conforms to physiological limits and transitions naturally through angle limit penalties and continuity penalties; and the motion smoothness constraint loss suppresses unreasonable motion abrupt changes through acceleration and jerk penalties. Through this multi-objective optimization strategy, it improves trajectory prediction accuracy while forcing the output to meet biomechanical constraints, thereby generating a smoother and more stable skeletal trajectory that better conforms to human movement patterns. This effectively avoids physiologically unreasonable or potentially damaging motion outputs, significantly enhancing the reliability and practicality of the system in competitive sports analysis.

[0040] Preferably, the data preprocessing module includes the following components during operation: S11: Receive skeletal trajectory sequence data from the original data source, and perform data cleaning on the received data, detecting and removing outliers; S12: Perform spatial normalization on the cleaned data to obtain spatially normalized data in a standardized coordinate system; S13: Perform time alignment processing on the spatially normalized data; S14: Treat the key points of each time frame as nodes of the spatial graph, treat the bones as edges of the spatial graph, and connect the same key points of adjacent time frames to form temporal edges, thus constructing a spatiotemporal graph containing spatial and temporal dimensions.

[0041] The spatial normalization process for the cleaned data includes: S121: For each frame of data, select a stable body center point as the new origin of the coordinate system, and subtract the coordinates of the center point from the coordinates of all joint points. S122: Rotate the coordinate system according to the athlete's orientation so that the athlete's frontal orientation is aligned with a specific axis of the new coordinate system. The orientation is determined by the direction vector of the line connecting the body's center point to key points such as the left shoulder, right shoulder, left hip, and right hip. S123: Divide all node coordinates by the body proportion factor to eliminate the influence of differences in body shape among athletes. The body proportion factor is preferably the average length of all bones, pelvic width, or trunk length. The underlying formula is: ; in, This represents the original coordinates of the i-th joint in frame t. This represents the coordinates of the selected body center point in frame t. For body proportion factor, This represents the coordinates of the i-th joint in frame t after spatial normalization.

[0042] In this embodiment, the data preprocessing module of the present invention standardizes skeletal trajectory data through a multi-step processing flow. The module first cleans the raw data to remove outliers, then performs spatial normalization, eliminating differences in individual position, orientation, and body shape through coordinate system translation, rotation, and scaling. Finally, the module constructs a spatiotemporal graph structure from the processed data, using joints as nodes and bones as edges, and establishing temporal connections. Through systematic data standardization and structured representation, interference from non-critical factors is effectively eliminated, providing high-quality, well-organized input for subsequent deep learning models. This significantly improves the stability of model training and the accuracy of trajectory compensation, while the spatiotemporal graph construction method better preserves the spatiotemporal correlation features of the motion.

[0043] Preferably, the data integrity assessment module includes the following components during operation: S21: Receive standardized skeletal trajectory sequence data from the data preprocessing module, including the three-dimensional coordinates of each joint and its visibility confidence. S22: Based on visibility confidence, calculate the global data integrity score of the current input sequence and compare the global data integrity score with a preset integrity threshold; S23: Based on the comparison results, generate a routing control signal to direct the data flow to different processing paths; when the global data integrity score is greater than or equal to the preset threshold, route to the advanced compensation path containing the personalized habit encoder and short-term trajectory predictor; when the global data integrity score is less than the preset threshold, route to the basic compensation path.

[0044] Specifically, when calculating the global data integrity score of the current input sequence based on visibility confidence, the method used is as follows: For a sequence of length T frames containing J keypoints, first calculate the frame-level integrity of each frame, then take the average of the frame-level integrity of all frames to obtain the global data integrity score. The formula for calculating the frame-level integrity is as follows: ; in, This represents the frame-level integrity of frame t. The total number of key points. This represents the visibility confidence of the j-th keypoint in the t-th frame. The visibility confidence threshold. This is an indicator function that returns 1 immediately if the condition is true, and 0 otherwise. is the weight coefficient of the j-th joint.

[0045] The weighting coefficients are assigned based on the biomechanical importance of the joints in a specific motion: For lower limb-dominant movements, the hip, knee, and ankle joints are given higher weight. For upper limb-dominant movements, the shoulder, elbow, and wrist joints are given higher weight; For full-body coordination exercises, the pelvis and spinal core joints are given higher weight.

[0046] The formula for calculating the global data integrity score is as follows: ; in, The overall data integrity score is given. The total number of frames in the sequence. Let t be the frame-level integrity of the t-th frame.

[0047] In this embodiment, the data integrity assessment module of the present invention achieves adaptive selection of processing paths through an intelligent assessment mechanism. Based on the visibility confidence of key points, this module uses a weighted average algorithm to calculate the global data integrity score. The frame-level integrity calculation formula incorporates key point weight coefficients, assigning higher weights to biomechanical key joints according to different motion types. Finally, through threshold comparison, the data stream is dynamically routed to either the basic compensation path or an advanced compensation path containing personalized prediction functions. This scheme, through a multi-level assessment system combining quantitative assessment and motion characteristics, achieves precise matching of processing strategies. It fully utilizes personalized prediction to improve compensation accuracy when data is complete, while ensuring the stability of basic compensation when data is severely missing. This significantly enhances the system's adaptability to different occlusion scenarios and the reliability of the overall compensation effect.

[0048] Preferably, the data integrity assessment module also includes a temporal continuity assessment during operation, specifically including: Before calculating the global data integrity score, the number of consecutively occluded frames is detected. When the number of consecutively occluded frames exceeds a preset temporal threshold, the calculation of the global data integrity score is stopped, and the data flow is directly directed to an advanced compensation path that includes a personalized habit encoder and a short-term trajectory predictor.

[0049] In this embodiment, the present invention prioritizes detecting the number of consecutively occluded frames before calculating the global integrity score. When the duration of consecutive occlusion exceeds a preset threshold, the system directly activates the advanced compensation path. By identifying this special scenario of long-term continuous occlusion, the system proactively triggers compensation strategies based on personalized habits and motion prediction, effectively avoiding the problem of decreased compensation accuracy caused by the continuous deterioration of data quality in traditional methods under continuous occlusion. This significantly enhances the system's adaptability to complex occlusion scenarios and ensures the timeliness and reliability of trajectory compensation.

[0050] As a preferred option, advanced compensation pathways specifically include: A11: Short-term trajectory predictor, used to predict skeletal trajectories for several future time frames based on current and past complete trajectory data and habitual embedding vectors; A12: Fusion module, connected to the short-term trajectory predictor and the core compensation network, used to perform weighted fusion of the preliminary compensated trajectory output by the core compensation network and the predicted trajectory output by the short-term trajectory predictor. Among them, the fusion weight changes over time. For a frame that has just been occluded, the weight of the compensated trajectory is greater than the weight of the predicted trajectory. As the prediction time point is delayed, the weight of the predicted trajectory gradually increases.

[0051] In this embodiment, the advanced compensation path design of the present invention includes a short-term trajectory predictor and an adaptive fusion module. It predicts future motion trajectories based on historical trajectories and personalized habit embedding vectors, and performs weighted fusion with the output of the core compensation network. Innovatively, it employs a fusion weight that dynamically adjusts over time—giving higher weight to the compensation result in the initial stage of occlusion to maintain accuracy, and gradually increasing the weight of the predicted trajectory as the prediction time increases to cope with long-term occlusion. This effectively combines the advantages of data-driven compensation and personalized motion pattern prediction, ensuring reconstruction accuracy under short-term occlusion while significantly improving the trajectory rationality and smoothness in long-term occlusion scenarios through motion trend prediction, thus enhancing the system's adaptability and reliability in complex competitive environments.

[0052] As a preferred option, the core compensation network specifically includes: A21: Spatial feature extraction unit, used to apply graph convolution operation to each frame of the input skeletal spatiotemporal map to extract the spatial structural features between joints; A22: Temporal feature extraction unit, used to apply a one-dimensional temporal convolution operation to the temporal trajectory of each joint point to extract the temporal dynamic features of joint motion; A23: Feature fusion unit, connected to the spatial feature extraction unit and the temporal feature extraction unit, is used to fuse spatial features and temporal features to generate compensated features containing rich spatiotemporal context information; A24: Trajectory Reconstruction Unit, used to reconstruct a complete skeletal trajectory sequence based on compensation features.

[0053] The spatial feature extraction unit employs a dual-flow graph convolutional architecture, including: Joint node flow uses joints as graph nodes and natural skeletal connections between joints as edges, and learns the structural relationships between joints through a graph convolutional neural network. Skeletal edge flow uses bone vectors as graph nodes and adjacent bone vectors as edges to learn the motion patterns of limb segments through a graph convolutional neural network. The output features of the two streams are concatenated or weighted and fused in subsequent network layers.

[0054] The graph convolution operation used in the spatial feature extraction unit is based on the following formula: ; in, This represents the feature matrix of the nodes in the (l+1)th layer. This represents the feature matrix of the nodes in the l-th layer. For adjacency matrices with added self-connections, for The degree matrix, These are trainable spatial convolution weight parameters. It is a non-linear activation function.

[0055] The temporal feature extraction unit employs dilated causal convolution, and its calculation formula is as follows: ; in, This represents the feature of the (l+1)th layer at time t. The kernel size is [size]. As the expansion factor, These are the trainable temporal convolution weight parameters.

[0056] The core compensation network adopts an encoder-decoder architecture: The encoder part contains multiple spatiotemporal decoupling graph convolutional blocks, each containing spatial graph convolutional layers and temporal convolutional layers, for progressively extracting and compressing high-level features of the input sequence; The decoder section contains multiple deconvolutional layers and upsampling layers for reconstructing the complete skeletal trajectory sequence from high-level features; In this method, a skip connection is used between the encoder and the decoder to stitch together the high-resolution spatial features of the corresponding layer in the encoder with the features in the decoder in order to preserve detailed information.

[0057] In this embodiment, the core compensation network of this invention adopts an advanced spatiotemporal separation processing architecture. Its spatial feature extraction unit innovatively uses a dual-flow graph convolution mechanism to learn the human body structural relationships and limb movement patterns through joint node flow and skeletal edge flow, respectively, and achieves feature propagation based on the normalized graph convolution formula. The temporal feature extraction unit uses dilated causal convolution to effectively capture long-term temporal dependencies. The network as a whole adopts an encoder-decoder structure, progressively extracting high-level features through multi-layer spatiotemporal blocks and using skip connections to preserve detailed information, ultimately achieving accurate trajectory reconstruction. This scheme significantly improves the model's ability to understand the spatiotemporal characteristics of human movement through the separation and deep fusion of spatiotemporal features. While ensuring biomechanical rationality, it effectively copes with complex occlusion scenarios, providing a high-precision and highly adaptable skeletal trajectory compensation solution for competitive sports analysis.

[0058] As a preferred embodiment, the personalized habit encoder, when in operation, specifically includes: S31: Collect and preprocess historical skeletal trajectory data of athletes in an unobstructed state to construct a personalized training dataset; S32: Construct a habit feature extraction network to learn athletes' personalized movement patterns from historical data; S33: Encode the learned personalized movement patterns into low-dimensional habit embedding vectors; S34: The habitual embedding vector is used as conditional information input to the core compensation network to guide it in performing personalized compensation for occluded trajectories.

[0059] The construction of personalized training datasets includes: Collect standardized movement sequences performed by athletes in multiple training sessions, including the athlete's typical technical movements; Each action sequence undergoes rigorous quality control screening, retaining only high-quality sequences with data completeness exceeding a preset threshold; The filtered sequences are time-normalized to give them a uniform time length.

[0060] The habitual feature extraction network employs an encoder-decoder architecture, including: A temporal encoder, based on the Transformer architecture, is used to extract temporal features from skeletal trajectory sequences; The feature aggregation layer is used to aggregate temporal features into a global feature vector of fixed dimensions; The decoder, based on a deconvolutional network, is used to reconstruct the original skeletal trajectory from the global feature vector.

[0061] Specifically, when using habitual embedding vectors as conditional information input into the core compensation network, the methods include: Vector concatenation method: Concatenate the custom embedding vector with the skeletal feature vector of each frame; Feature modulation method: Adaptive instance normalization modulation of feature maps in the core compensation network is performed using conventional embedding vectors; Attention-guided approach: Use habit embedding vectors as query vectors and retrieve the most relevant motion patterns from historical templates through an attention mechanism.

[0062] In this embodiment, the personalized habit encoder of the present invention captures and applies athletes' personalized movement patterns through a systematic data processing and feature learning mechanism. This module first constructs a high-quality personalized training dataset and then uses a network architecture based on a Transformer encoder and a deconvolutional decoder to learn the athlete's unique movement habits, encoding them into low-dimensional embedding vectors. These vectors are then integrated into the core compensation network through various methods such as vector concatenation, feature modulation, or attention guidance. This approach significantly improves the individual consistency of trajectory reconstruction under occlusion conditions by establishing a deep correlation between the athlete's specific movement patterns and the trajectory compensation process. It effectively solves the problem of insufficient adaptability to individual differences in traditional methods, making the compensation results more closely match the athlete's true technical characteristics and providing a reliable data foundation for personalized technical analysis and training guidance.

[0063] Preferably, the core compensation network, when in operation, includes the following steps: S41: Receive preprocessed spatiotemporal skeletal graph data, wherein the spatiotemporal skeletal graph data contains the spatial connection relationship of joints and temporal continuous frame information; S42: A spatiotemporally separated graph convolutional network is used to extract features from the input data to obtain spatiotemporal features. Among them, the spatial graph convolutional layer aggregates the spatial features between joints along the topology of the human skeleton, and the temporal convolutional layer extracts the motion dynamic features of each joint along the time dimension. S43: Integrate the habit embedding vector from the personalized habit encoder into the spatiotemporal features to obtain the fused high-level features; S44: The fused high-level features are mapped back to the skeletal trajectory space through the decoder network to generate a preliminary compensated trajectory sequence. The decoder uses deconvolution layers and upsampling operations to gradually restore trajectory details. S45: Output the preliminary compensated trajectory sequence to the biomechanical rationality optimization module for subsequent optimization processing.

[0064] In this embodiment, the present invention first receives standardized spatiotemporal graph data, extracts structural features between key points using spatial graph convolutional layers, and captures motion temporal patterns using temporal convolutional layers. Then, it deeply fuses personalized habit embedding vectors with spatiotemporal features, and finally reconstructs a preliminary compensated trajectory using a decoder. This scheme significantly improves the model's ability to understand the multidimensional characteristics of human motion through the separation and extraction of spatiotemporal features and the adaptive fusion of personalized information. It can effectively cope with complex occlusion scenarios and ensure that the reconstructed trajectory conforms to the athlete's unique technical movement patterns, providing a high-precision, personalized trajectory compensation solution for competitive sports analysis.

[0065] A deep learning-based method for skeletal trajectory compensation in competitive sports includes the following steps: S51: The data preprocessing module performs data preprocessing operations on the original skeletal trajectory sequence data, representing the skeletal data as a standardized spatiotemporal graph structure, where joints are graph nodes and bones are graph edges. S52: The data integrity assessment module calculates the proportion of visible key points in the current input sequence and compares it with the preset integrity threshold. Based on the comparison result, the data is routed to different processing paths. S53: Processes the routed data through the core compensation network and outputs a preliminary compensated trajectory sequence; S54: Apply biomechanical constraints to the initially compensated trajectory sequence through the biomechanical rationality optimization module, and output the final high-quality skeletal trajectory sequence; S55: Extracts low-dimensional habit embedding vectors from the historical unoccluded motion data of a specific athlete through a personalized habit encoder, and inputs the habit embedding vectors as conditional information into the core compensation network to guide it to perform personalized compensation for the trajectory of the occluded parts. In step S52, different processing paths are selected based on the data integrity assessment results. When the data integrity is higher than the preset threshold, an advanced compensation process that includes short-term trajectory prediction and fusion processing is activated; when the data integrity is lower than the preset threshold, a basic compensation process is adopted.

[0066] In this embodiment, the present invention achieves high-precision reconstruction of motion trajectories under occlusion environments through a multi-stage processing flow: First, the original skeletal data is standardized and preprocessed to construct a spatiotemporal graph structure; then, based on the data integrity assessment results, a basic or advanced compensation pathway is intelligently selected; subsequently, feature extraction and trajectory compensation are performed through a spatiotemporal decoupling graph convolutional network combined with personalized habit embedding vectors; finally, a high-quality trajectory is output after biomechanical constraint optimization. This scheme effectively solves the problem of insufficient adaptability of traditional methods in complex occlusion scenarios by establishing a synergistic mechanism of data-driven compensation and personalized motion pattern prediction, significantly improving the accuracy of trajectory compensation and individual matching degree, and providing reliable technical support for technical analysis, training optimization, and injury prevention in competitive sports.

[0067] Example 1: Team Tactical Trajectory Analysis in Basketball Games In professional basketball games, a club needs to analyze the movement trajectory of its defensive formation during opponent fast breaks. Due to frequent physical contact and mutual obstruction among players, traditional vision systems capture a significant amount of missing skeletal trajectories. This system first extracts skeletons from video streams simultaneously captured by 10 high-definition cameras using a data preprocessing module. The obtained raw 3D joint coordinates are then cleaned, removing abnormal jump points caused by recognition errors. Subsequently, a standardized coordinate system is established with the pelvic center as the origin. By determining the direction of the hip joint connection, the player's facing is standardized, and body shape is normalized using the average torso length. Finally, a spatiotemporal graph containing spatial connections (joint-skeleton relationships) and temporal connections (the same joint in adjacent frames) is constructed.

[0068] The data integrity assessment module calculates the trajectory integrity of each player on the court in real time. When it detects that the visibility of the upper body joints of the point guard drops sharply to 40% (below the threshold of 60%) due to being obstructed by the defender during a breakthrough, the basic compensation pathway is immediately activated. At the same time, the system detects that although the center player has 70% visibility (above the threshold), the duration of his obstruction for 8 consecutive frames exceeds the critical value, so the advanced compensation pathway is activated simultaneously. The core compensation network adopts a two-stream spatiotemporal separation convolutional architecture. The spatial stream learns the player's limb structure relationships through graph convolution, while the temporal stream uses dilated convolution to analyze the movement rhythm. The personalized habit encoder retrieves habit embedding vectors generated from the center's historical 500 pick-and-roll actions to guide the network to compensate for the obstructed parts according to his habitual positioning posture.

[0069] The short-term trajectory predictor predicts the movement trend of the next 5 frames based on the center forward's complete trajectory and habitual vectors from the previous 15 frames. The fusion module employs a time-decay weighting strategy, assigning 80% weight to the compensation network result for the first 1-2 frames after occlusion, gradually increasing the weight of the predicted trajectory to 60% as the number of predicted frames increases. The biomechanical optimization module imposes constraints on the compensation result: ensuring that the knee flexion does not exceed the 165-degree limit, the spinal tilt angle remains continuously changing, the bone length fluctuation is controlled within 3%, and unreasonable body shaking is eliminated through acceleration smoothing. Finally, the system outputs a complete trajectory that conforms to biomechanical laws, successfully reconstructing the occluded positioning details during tactical execution.

[0070] Example 2: Optimization Analysis of Table Tennis Players' Technical Movements To optimize their athletes' forehand attack technique, a team needed to analyze the missing torso trajectory caused by racket obstruction during high-speed video recording. The system captured 100 consecutive shots by the athletes using high-speed cameras positioned on both sides of the court at 200 frames per second. After coordinate unification and outlier removal, the preprocessing module specifically labeled the data to address the periodic missing torso trajectory caused by racket obstruction. Spatial normalization was performed with the center of the chest cavity as the origin, correcting the body orientation based on the shoulder line direction, and standardizing the data using the athlete's arm span as a body shape factor.

[0071] When analyzing a backhand topspin stroke, the data integrity assessment module detected that the right arm was completely occluded for 10 consecutive frames due to rapid body rotation (30% integrity), but the data integrity of the first 30 frames reached 85%. Based on this, the system activated the advanced compensation pathway, simultaneously calling upon the personalized habit vector obtained from 2000 standard movement training sessions of the athlete. The spatial graph convolution of the core compensation network focuses on processing upper limb joint relationships, while the temporal convolution captures the temporal features of "push-rotation-backswing-hit"; although the social dynamics module was not activated, the system used the relative position of the opponent player to assist in judging the timing of the hit.

[0072] The short-term predictor predicts the subsequent swing path features based on the complete trajectory of the backswing phase. The fusion module, specifically designed for the explosive nature of table tennis movements, sets a steeper weight decay curve, increasing the prediction weight from 20% to 50% within three frames. In the biomechanical optimization phase, the system strictly constrains the wrist dorsiflexion angle to not exceed the physiological limit of 75 degrees, ensuring smooth changes in angular acceleration along the swing trajectory. It also eliminates arm length distortion that may be caused by occlusion compensation through bone length constraints. The final generated trajectory clearly shows the athlete's technical defect of excessively high elbow elevation, providing accurate data support for subsequent targeted training.

[0073] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Within the scope of knowledge possessed by those skilled in the art, various changes can be made without departing from the spirit of the present invention.

Claims

1. A skeletal trajectory deep learning compensation system for competitive sports, characterized by: The application comprises: a data preprocessing module for performing data preprocessing operations on original skeletal trajectory sequence data and representing the skeletal data in a standardized spatio-temporal graph structure, wherein the joints are nodes of the graph and the bones are edges of the graph; a data completeness evaluation module for calculating the proportion of visible joints in the current input sequence and comparing it with a preset completeness threshold, and routing the data to different processing channels according to the comparison result; a core compensation network adopting a spatio-temporal separation graph convolution network architecture, including a spatial graph convolution layer for modeling the spatial relationship between joints and a temporal convolution layer for modeling the temporal dynamics of the joints; a preliminary compensation trajectory sequence output module for receiving the routed data and outputting a preliminary compensation trajectory sequence; a biomechanical rationality optimization module for applying biomechanical constraints to the preliminary compensation trajectory sequence, including joint angle limit constraints, skeletal length constant constraints and motion smoothness constraints, and outputting a final high-quality skeletal trajectory sequence; a personalized habit encoder for extracting a low-dimensional habit embedding vector from historical unoccluded motion data of a specific athlete, and inputting the habit embedding vector as conditional information into the core compensation network.

2. The athletic sports oriented skeletal trajectory deep learning compensation system of claim 1, wherein: The biomechanical rationality optimization module optimizes the network parameters by minimizing a composite loss function, and the composite loss function is represented as: ； wherein, is a total loss value, , , and is a weight coefficient of each loss term, is a mean square loss error, is a joint angle constraint loss, is a motion smoothness constraint loss, is a bone length stability loss.

3. The athletic sports oriented skeletal trajectory deep learning compensation system of claim 2, wherein: The data preprocessing module in operation specifically comprises: S11: receiving skeletal trajectory sequence data from an original data source, and performing data cleaning on the received data to detect and eliminate outliers; S12: performing spatial normalization processing on the cleaned data to obtain spatial normalized data in a standardized coordinate system; S13: performing time alignment processing on the spatial normalized data; S14: regarding each joint in a time frame as a spatial graph node, regarding a bone as a spatial graph edge, and connecting the same joint in adjacent time frames to form a time edge, thereby constructing a spatio-temporal graph containing spatial and temporal dimensions.

4. The athletic sports oriented skeletal trajectory deep learning compensation system of claim 3, wherein: The data completeness evaluation module in operation comprises: S21: receiving standardized skeletal trajectory sequence data from the data preprocessing module, including three-dimensional coordinates of each joint and its visibility confidence; S22: calculating a global data completeness score of the current input sequence based on the visibility confidence, and comparing the global data completeness score with a preset completeness threshold; S23: generating a routing control signal to guide the data flow to different processing channels according to the comparison result; when the global data completeness score is greater than or equal to the preset threshold, routing to an advanced compensation channel containing a personalized habit encoder and a short-term trajectory predictor; when the global data completeness score is less than the preset threshold, routing to a basic compensation channel.

5. The athletic sports oriented skeletal trajectory deep learning compensation system of claim 4, wherein: The data completeness evaluation module in operation further comprises a time sequence continuity evaluation, specifically comprising: Before calculating the global data completeness score, detecting the number of consecutive occluded frames, and when the number of consecutive occluded frames exceeds a preset time sequence threshold, stopping the calculation of the global data completeness score and directly guiding the data flow to the advanced compensation channel containing the personalized habit encoder and the short-term trajectory predictor.

6. The athletic sports oriented skeletal trajectory deep learning compensation system of claim 5, wherein: The advanced compensation channel specifically comprises: A11: a short-term trajectory predictor configured to predict a skeletal trajectory in a plurality of time frames in the future based on current and past complete trajectory data and a habit embedding vector; A12: a fusion module connected to the short-term trajectory predictor and the core compensation network, configured to perform weighted fusion of a preliminary compensation trajectory output by the core compensation network and a predicted trajectory output by the short-term trajectory predictor; wherein the fusion weight varies over time, and for a frame just being occluded, the weight of the compensation trajectory is greater than the weight of the predicted trajectory, and as the prediction time point is delayed, the weight of the predicted trajectory gradually increases.

7. The athletic sports oriented skeletal trajectory deep learning compensation system of claim 6, wherein: The core compensation network specifically comprises: A21: a spatial feature extraction unit configured to separately apply a graph convolution operation to each frame in the input skeletal spatio-temporal graph to extract spatial structure features between joints; A22: a temporal feature extraction unit configured to separately apply a one-dimensional temporal convolution operation to a time sequence trajectory of each joint to extract time dynamic features of joint motion; A23: a feature fusion unit connected to the spatial feature extraction unit and the temporal feature extraction unit, configured to fuse the spatial features and the temporal features to generate compensation features containing rich spatio-temporal context information; A24: a trajectory reconstruction unit configured to reconstruct a complete skeletal trajectory sequence based on the compensation features.

8. The athletic sports oriented skeletal trajectory deep learning compensation system of claim 7, wherein: The personalized habit encoder, when in operation, specifically comprises: S31: collecting and pre-processing historical skeletal trajectory data of the athlete in an unoccluded state to construct a personalized training data set; S32: constructing a habit feature extraction network to learn the personalized motion pattern of the athlete from the historical data; S33: encoding the learned personalized motion pattern into a low-dimensional habit embedding vector; S34: inputting the habit embedding vector as conditional information into the core compensation network to guide the personalized compensation of the occluded trajectory.

9. The athletic sports oriented skeletal trajectory deep learning compensation system of claim 8, wherein: The core compensation network, when in operation, comprises the following steps: S41: receiving pre-processed skeletal spatio-temporal graph data, wherein the skeletal spatio-temporal graph data contains spatial connection relationships of the joints and continuous frame information in the time sequence; S42: using a spatio-temporal separation graph convolution network to extract features from the input data, wherein a spatial graph convolution layer aggregates spatial features between joints along the human skeletal topology structure, and a temporal convolution layer extracts motion dynamic features of each joint along the time dimension; S43: fusing the habit embedding vector from the personalized habit encoder into the spatio-temporal features to obtain fused high-level features; S44: mapping the fused high-level features back to the skeletal trajectory space through a decoder network to generate a preliminary compensation trajectory sequence, wherein the decoder uses an inverse convolution layer and an upsampling operation to gradually restore the trajectory details; S45: outputting the preliminary compensation trajectory sequence to a biomechanics rationality optimization module for subsequent optimization processing.

10. A skeletal trajectory deep learning compensation method for competitive sports, characterized in that: comprises the following steps: S51: performing data preprocessing operations on the original skeletal trajectory sequence data through a data preprocessing module to represent the skeletal data as a standardized spatio-temporal graph structure, wherein the joints are used as graph nodes and the skeleton is used as the edges of the graph; S52: Calculate the proportion of visible joints in the current input sequence through the data integrity evaluation module, and compare it with the preset integrity threshold, and route the data to different processing channels according to the comparison result; S53: Process the routed data through the core compensation network to output the preliminary compensated trajectory sequence; S54: Apply biomechanical constraints to the preliminary compensated trajectory sequence through the biomechanics rationality optimization module to output the final high-quality skeletal trajectory sequence; S55: Extract low-dimensional habit embedding vectors from the historical unoccluded motion data of a specific athlete through the personalized habit encoder, and input the habit embedding vectors as conditional information into the core compensation network to guide the personalized compensation of the trajectory of the occluded part; Among them, step S52 selects different processing channels according to the data integrity evaluation result, when the data integrity is higher than the preset threshold, the advanced compensation process containing short-term trajectory prediction and fusion processing is activated; when the data integrity is lower than the preset threshold, the basic compensation process is adopted.

Images