Athlete real-time posture analysis and correction method and system based on computer vision
By fusing deep learning data from multi-view video and multi-modal sensor data, a personalized standard action model is generated, which solves the shortcomings of existing systems in terms of accuracy, real-time performance, and personalization, and realizes high-precision, real-time football training analysis and correction.
Patent Information
- Application Number
- CN202411073574.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-06
- Publication Date
- 2024-11-12
AI Technical Summary
Existing football training analysis systems struggle to achieve high-precision, real-time, and personalized athlete posture estimation and correction, especially in multi-player training scenarios where issues such as occlusion and poor angles exist. Furthermore, they lack effective fusion of multimodal data and immediate feedback.
By combining multi-view motion video data and multi-modal sensor data, and using deep learning algorithms such as adaptive multi-scale pose estimation algorithm, dynamic temporal graph convolutional network, and hierarchical attention fusion network, a personalized standard motion model is generated and real-time correction suggestions are provided.
It achieves millimeter-level precision in capturing motion details, real-time feedback within 50 milliseconds, comprehensive evaluation and personalized analysis of multimodal data, significantly improving training efficiency and reducing the risk of injury.
Smart Images

Figure CN118942158A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence and computer vision technology, and in particular to a computer vision-based athlete real-time posture analysis and correction method and system. Background Art
[0002] As one of the most popular sports in the world, football has been constantly innovating its technical training methods. In recent years, with the development of science and technology, computer vision and artificial intelligence technologies have begun to be introduced into the field of football training, bringing new possibilities to traditional training methods. However, despite the continuous advancement of technology, the existing football training analysis system still has many limitations and cannot meet the needs of modern football for precise and personalized training.
[0003] Traditional football training mainly relies on the coach's experience observation and the players' subjective feelings. Although this method is simple and direct, it is difficult to capture the tiny details in high-speed movements. For example, a shooting action may be completed in less than 0.3 seconds, and it is difficult for the human eye to accurately capture every key link in the entire action process. Studies have shown that even experienced coaches have an accuracy rate of only about 70% when observing fast movements. This leads to the loss of many technical details and affects the accuracy of training.
[0004] With the development of technology, monocular video analysis systems have begun to be used in football training. Such systems can record players' movements and perform simple movement analysis. However, the limitations of a single perspective make it difficult for the system to accurately capture complex three-dimensional movements, especially in actual combat training with multiple people, which is prone to occlusion and poor angles. According to statistics, the accuracy of posture estimation of monocular systems in complex scenes is often less than 80%, which obviously cannot meet the needs of high-level football training.
[0005] In recent years, multi-view video analysis systems have gradually become mainstream. Such systems collect data from different angles through multiple cameras, greatly improving the accuracy of motion capture. However, relying solely on visual data still has limitations. For example, it is impossible to accurately measure key parameters such as the player's physiological state and the strength of the foot's contact with the ball. A study of top European leagues showed that the accuracy of analysis systems based solely on visual data in evaluating the quality of passes was only about 85%.
[0006] In order to make up for the shortcomings of pure visual systems, some advanced training systems have begun to integrate wearable devices, such as GPS trackers and heart rate monitors. These systems can provide more comprehensive data, including information such as the player's position, speed, heart rate, etc. However, these systems often have the problem of data silos, that is, visual data and sensor data are processed separately and lack an effective fusion mechanism. This leads to one-sided analysis results and makes it difficult to fully evaluate the player's technical movements.
[0007] In addition, existing systems also have obvious deficiencies in real-time and personalization. Most systems take a long time to process data and have difficulty providing instant feedback. According to a survey of professional football teams, the average feedback delay of existing systems is around 2-3 seconds, which is obviously not enough for football training that requires rapid adjustments. At the same time, these systems often use a unified standard action model and lack consideration of individual differences among players. A study showed that the results of action evaluation using a general model differed by up to 25% from those of professional coaches.
[0008] In the face of these problems, there is an urgent need for a football training auxiliary system that can achieve real-time fusion of multi-modal data, high-precision posture estimation, and personalized motion analysis. The present invention is an innovative solution proposed to meet this demand. Summary of the invention
[0009] The purpose of the present invention is to provide a method and system for real-time athlete posture analysis and correction based on computer vision, aiming to solve the problems of difficulty in accurately capturing and analyzing athlete movements and lack of timely and personalized correction in traditional football training.
[0010] The present invention provides a real-time athlete posture analysis and correction method based on computer vision, comprising the following steps:
[0011] Obtain multi-view action video data and multi-modal sensor data of athletes, and establish a standard action model library in advance;
[0012] Based on the multi-view action video data, obtaining a posture estimation result through an adaptive multi-scale posture estimation algorithm;
[0013] According to the posture estimation result, using a dynamic temporal graph convolutional network to realize action recognition;
[0014] Fusing the posture estimation result with the multimodal sensor data to obtain a fusion feature;
[0015] Based on the fusion features and the standard action model library, generating a personalized standard action model;
[0016] Determining the difference between the athlete's actual movement and the personalized standard movement model;
[0017] Based on the said discrepancies, corrective suggestions are generated;
[0018] Determining the best feedback method and providing the corrective suggestions to the athlete.
[0019] Specifically, the adaptive multi-scale posture estimation algorithm specifically includes:
[0020] Obtain an input image I;
[0021] Through multiple convolutional neural networks (CNN) of different scales s Process the input image I to obtain multi-scale features F s, Where s represents the scale level;
[0022] Use the soft attention mechanism SoftAttention to process the multi-scale feature F s , get the attention weight A of each scale feature s ;
[0023] Based on the attention weight A s , for the multi-scale feature F s Perform weighted summation to obtain a fusion feature F; input the fusion feature F into the regression network RegressionNet to obtain the final posture estimation result P.
[0024] Specifically, the dynamic temporal graph convolutional network specifically includes:
[0025] Get the posture feature X at time t t and the posture feature X at time t-1 t-1 ;
[0026] Based on the X t and X t-1 , through the dynamic adjacency matrix generation function f adj , get the dynamic adjacency matrix A t ; Using the dynamic adjacency matrix A t , degree matrix D and weight matrix W, perform graph convolution operation, and obtain the updated node feature H t ;
[0027] The node feature H t Input the long short-term memory network LSTM and apply the temporal attention mechanism to obtain the temporal features;
[0028] Based on the temporal features, the final action classification result Y is output through the temporal attention weighted Softmax function SoftmaxTE.
[0029] Specifically, the fusion of the posture estimation result and the multimodal sensor data specifically includes:
[0030] Obtain visual data V, sensor data S and environmental data E;
[0031] Apply the self-attention mechanism SelfAttention to the V, S and E respectively to obtain the attention feature A of each modality m ;
[0032] The attention features A of each modality m The cross-attention mechanism CrossAttention is input to learn the correlation between modalities and obtain the cross-attention feature B. The cross-attention feature B is processed by the fusion network FusionNet to obtain the final multimodal fusion feature Z.
[0033] Specifically, the generation of the personalized standard action model specifically includes:
[0034] Obtain the athlete's personal characteristics P, on-field position information F, and action type T;
[0035] Process the P, F and T through an encoder to obtain an encoding feature E;
[0036] Input the encoding feature E into the generator network Generator to create an initial standard action model M;
[0037] Based on the historical training data H, the initial standard action model M is processed by the fine-tuning network Refiner to obtain the final personalized standard action model M'.
[0038] Specifically, the determination of the difference includes:
[0039] Obtain actual action A and standard action model M;
[0040] Perform local difference calculation LocalDiff on each joint point i of A and M to obtain the local difference L i ; Perform a global difference calculation GlobalDiff on A and M to obtain a global difference G;
[0041] The local difference L is processed by the difference fusion network FusionNet i And the global difference G, the final comprehensive difference evaluation result D is obtained.
[0042] Specifically, the generation of the correction suggestion includes:
[0043] Obtain difference evaluation result D and context information C;
[0044] Processing the difference evaluation result D through a difference encoder Encoder to obtain an encoding feature E;
[0045] Input the encoded feature E and the context information C into the context fusion network ContextFusion to obtain the fusion feature F;
[0046] The LSTM decoder DecoderLSTM is used to process the fused feature F to generate the final correction suggestion text R.
[0047] Specifically, the determination of the optimal feedback mode includes:
[0048] Get the current action importance I t 、Feedback History F t and training phase information T t ;
[0049] Based on the I t 、F t and T t , the construction state represents S t ;
[0050] Process the state representation S according to the current policy π t , select feedback action A t ;
[0051] Execute the feedback action A t , observe the feedback effect E t and interference level D t ;
[0052] Based on the E t and D t , calculate the reward R t ;
[0053] Using the S t , A t , R t and the next state S t+1 , update the strategy π to maximize the long-term cumulative reward.
[0054] Specifically, it also includes long-term progress analysis and injury risk prediction, including:
[0055] Obtain historical training data sequence X;
[0056] Use convolutional neural network CNN to process the data X at each time point t in X t , and get the time series feature F t ;
[0057] The F is processed by the temporal attention mechanism TemporalAttention t , get the attention weight A t Based on the A t , for F t Perform weighted summation to obtain weighted fusion feature G;
[0058] Input G into the residual block ResidualBlock, learn the residual representation of the feature, and obtain the residual feature H;
[0059] The prediction head network PredictionHead is used to process the residual feature H to obtain the improvement trend prediction P and the injury risk prediction R.
[0060] The computer vision-based real-time athlete posture analysis and correction system for implementing the method comprises: a multi-view action video acquisition module, used to obtain the multi-view action video data of the athlete; a multi-modal sensor data acquisition module, used to obtain the multi-modal sensor data; a standard action model library, used to store pre-established standard action models; a posture estimation module, used to obtain a posture estimation result based on the multi-view action video data through an adaptive multi-scale posture estimation algorithm; an action recognition module, used to realize action recognition based on the posture estimation result using a dynamic temporal graph convolutional network; a data fusion module, used to fuse the posture estimation result with the multi-modal sensor data to obtain a fusion feature; a personalized standard action generation module, used to generate a personalized standard action model based on the fusion feature and the standard action model library; a difference analysis module, used to determine the difference between the actual action of the athlete and the personalized standard action model; a correction suggestion generation module, used to generate a correction suggestion based on the difference; and a feedback scheduling module, used to determine the best feedback method and provide the correction suggestion to the athlete.
[0061] The beneficial effects of the present invention are mainly reflected in the following aspects:
[0062] 1. High-precision posture estimation: Through multi-view data acquisition and adaptive multi-scale posture estimation algorithms, this system can capture the details of a player's movements with millimeter-level accuracy, far surpassing existing systems. This enables coaches and players to accurately analyze and improve tiny details in technical movements, such as ankle flexion and extension angles, hip rotation range, etc.
[0063] 2. Real-time performance: The end-to-end delay of this system is controlled within 50 milliseconds, achieving true real-time feedback. This means that players can get feedback the moment the action is completed, which is conducive to rapid adjustment and the formation of muscle memory.
[0064] 3. Multimodal data fusion: By integrating visual data and sensor data, the system can comprehensively evaluate the technical movements of players. For example, when analyzing shooting movements, it not only considers the correctness of the posture, but also combines the rotation and speed data of the football to give a more comprehensive evaluation.
[0065] 4. Personalized analysis: The system can generate a personalized standard action model based on the player’s physical characteristics, technical style and position on the field. This ensures that the training suggestions are targeted and avoids the problem of one-size-fits-all.
[0066] 5. Intelligent feedback: The system can automatically select the best feedback method according to the training stage and player status, such as voice commands, tactile feedback or visual prompts to ensure the effectiveness and rationality of the feedback.
[0067] 6. Long-term progress tracking: Through long-term data accumulation and analysis, the system can provide coaches and players with detailed progress reports to help formulate long-term training plans.
[0068] In general, the invention provides a comprehensive, accurate, real-time and highly personalized solution for modern football training, which is expected to significantly improve training efficiency, accelerate the technical progress of players, and reduce the risk of injuries caused by incorrect movements. The application of this system has the potential to revolutionize the way football training is conducted and provide unprecedented technical support for players and coaches. BRIEF DESCRIPTION OF THE DRAWINGS
[0069] Figure 1 It is the overall logic block diagram of the system of the present invention.
[0070] Figure 2 It is a logic block diagram of the adaptive multi-scale posture estimation algorithm of the present invention.
[0071] Figure 3 This is a logical block diagram of the dynamic temporal graph convolutional network of the present invention.
[0072] Figure 4 It is a logical block diagram of the hierarchical attention fusion network of the present invention.
[0073] Figure 5 This is a logic block diagram of the personalized standard action generation module of the present invention.
[0074] Figure 6 It is a logic block diagram of the difference analysis module of the present invention.
[0075] Figure 7 The logical block diagram of the correction suggestion generation module of the present invention.
[0076] Figure 8 It is a logic block diagram of the feedback scheduling module of the present invention.
[0077] Fig. 9 The dense point heat map is a comparison of the pose estimation accuracy of the embodiment of the present invention and comparative examples 1-3.
[0078] Fig.10 3D scatter plot performance distribution of the embodiments of the present invention and comparative examples 1-3.
[0079] Fig.11 It is the thermodynamic trajectory diagram of the embodiment of the present invention and comparative examples 1-3. DETAILED DESCRIPTION
[0080] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation methods, structures, features and effects are described in detail below in conjunction with the accompanying drawings and preferred embodiments. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.
[0081] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
[0082] The application of the present invention in football training and competition will be further described below in conjunction with the accompanying drawings and embodiments.
[0083] like Figure 1-9 As shown, the real-time posture analysis and correction method of athletes based on computer vision of the present invention comprises the following steps:
[0084] Step S1: Data acquisition.
[0085] Preferably, in one embodiment of the present invention, the data collection step is specially optimized for football training and game scenarios.
[0086] For multi-view action video data collection, this method uses 6 high-speed cameras (model: Phantomv2640) at a frame rate of 120fps to collect the movements of football players from different angles. These cameras are arranged in a regular hexagonal array around the football field, with a distance of 40 meters between every two adjacent cameras to ensure coverage of the entire standard 11-person football field (105 meters x 68 meters). The field of view of each camera is set to 94° and the focal length is 24mm to capture a wide range of actions including long shots and long passes.
[0087] Preferably, during night training or competition, the ISO value of the camera can be automatically adjusted between 100-6400, and an infrared fill light system is also equipped to ensure that the player's movements can be clearly captured in low-light environments. In addition, each camera is equipped with a rain cover and a defog system to cope with different weather conditions.
[0088] For multimodal sensor data acquisition, this method uses three subsystems designed specifically for football:
[0089] 1) Smart jersey system: The smart jerseys worn by players have built-in ultra-thin flexible sensors. Accelerometers are placed on the shoulders, waist and ankles to capture acceleration changes in passing, shooting and other actions; gyroscopes are placed on the chest and thighs to measure the player's turning and changing direction; heart rate sensors are placed on the chest to monitor the player's heart rate changes in real time. The total weight of all sensors does not exceed 50 grams to ensure that they do not affect the normal performance of the players.
[0090] 2) Smart football system: The football has built-in pressure sensors and 9-axis IMU sensors. The pressure sensors are distributed at 12 points on the sphere, which can accurately measure the pressure distribution when the foot contacts the ball, which helps analyze the kicking technique; the 9-axis IMU sensor can capture the spatial position, rotation speed and acceleration of the ball, which is used to analyze the accuracy and strength of passing and shooting. The weight and size of the smart football fully meet FIFA standards.
[0091] 3) Field environment perception system: Environmental sensors are placed at the four corners and both sides of the center line of the court, including light sensors, temperature sensors, humidity sensors and wind speed sensors. These data are used to analyze the impact of environmental factors on player performance, such as the impact of strong wind on long passes and the consumption of players' physical energy due to high temperatures.
[0092] Preferably, all sensor data is transmitted in real time to a data concentrator on the sidelines via low-power Bluetooth 5.0 technology, with a transmission delay of less than 5ms and a sampling frequency of 1000Hz to ensure that fast football action is captured. The data concentrator transmits the data to the central processing unit via the 5G network, with an end-to-end delay of no more than 20ms.
[0093] Step S2: Adaptive multi-scale pose estimation.
[0094] In a preferred embodiment of the present invention, an adaptive multi-scale pose estimation algorithm (AMSPE) is optimized specifically for soccer movements.
[0095] a) First, an input image I is obtained, which is a single frame image extracted from multi-view video data, with a size of 1920x1080 pixels and a color space of RGB.
[0096] b) Then, through three convolutional neural networks (CNN) of different scales s Process the input image I to obtain multi-scale features F s , where s∈{1,2,3} represents the scale level. Specifically: CNN 1 Focus on capturing overall posture, such as running posture, jumping action, etc. CNN 2 Focus on the separation of the upper and lower body movements, such as the twisting of the upper body and the support of the lower body when passing the ball; CNN 3Focus on local joint movements, such as ankle flexion and extension, knee bending, etc.
[0097] c) Then, the multi-scale feature F is processed using the soft attention mechanism SoftAttention s , get the attention weight A of each scale feature s The design of the attention mechanism takes into account the characteristics of different football actions. For example, in shooting actions, a higher attention weight is given to the lower limbs.
[0098] d) Based on the attention weight A s , for the multi-scale feature F s Perform weighted summation to obtain the fusion feature F.
[0099] e) Finally, the fusion feature F is input into the regression network RegressionNet to obtain the final posture estimation result P. The output of the regression network includes the 3D coordinates of 17 key points, which include important joints in football actions, such as ankles, knees, hips, shoulders, etc.
[0100] f) Using the prediction head network PredictionHead to process the residual feature H, the improvement trend prediction P and the injury risk prediction R are obtained. The improvement trend prediction P outputs the expected change rate of five key technical indicators, including passing accuracy, shooting accuracy, dribbling success rate, steal success rate and physical fitness level. The injury risk prediction R outputs the probability of common football injuries (such as ankle sprains, muscle strains, knee injuries, etc.) occurring in the next 30 days.
[0101] Preferably, the AMSPE algorithm pays special attention to capturing fast movements when processing football movements. For example, when processing a shooting movement, the algorithm can update the position of the key points of the foot at a frequency of 500Hz to accurately capture the posture of the kicking moment.
[0102] Step S3: Dynamic temporal action recognition.
[0103] In another preferred embodiment of the present invention, a dynamic temporal graph convolutional network (DT-GCN) is specially designed for football action sequences.
[0104] a) First, obtain the posture feature X at time t t and the posture feature X at time t-1 t-1 , each feature is a 51-dimensional vector (3D coordinates of 17 key points).
[0105] b) Then, based on the X t and X t-1 , through the dynamic adjacency matrix generation function f adj , get the dynamic adjacency matrix At In the football scenario, f adj The function pays special attention to certain key joint connections, such as the hip-to-knee and knee-to-ankle connections, which play a key role in passing and shooting movements.
[0106] c) Then, using the dynamic adjacency matrix A t , degree matrix D and weight matrix W, perform graph convolution operation, and obtain the updated node feature H t In football action recognition, graph convolution operations can effectively capture the coordinated motion patterns between different joints.
[0107] d) The node feature H t The long short-term memory network LSTM is input and the temporal attention mechanism is applied to obtain the temporal features. In football action recognition, the sequence length of LSTM is set to 30 frames (i.e. 0.25 seconds). This time span is sufficient to capture the complete process of most football technical actions, such as passing, shooting, and stealing. The temporal attention mechanism is specially designed to identify the key frames of the action. For example, in the shooting action, the kicking moment will get a higher attention weight.
[0108] e) Finally, based on the temporal features, the final action classification result Y is output through the Softmax function SoftmaxTE weighted by temporal attention. The output dimension of the classification layer is 20, corresponding to 20 predefined football action types, including: short pass, long pass, high ball, shot (including instep shot, inside shot, outside instep shot), header, steal, interception, dribbling, feint, turn, etc.
[0109] DT-GCN performs well in recognizing complex football action combinations. For example, it can accurately recognize a continuous action sequence such as "receiving the ball-feinting-breaking-shooting" with an accuracy of 93%. For some easily confused action pairs (such as short passes and shots), the model distinguishes them by analyzing the strength, angle, and subsequent impact of the action, keeping the error rate below 5%.
[0110] Step S4: Multimodal data fusion.
[0111] In another preferred embodiment of the present invention, the Hierarchical Attention Fusion Network (HAFN) is optimized specifically for football scenes:
[0112] a) First, obtain visual data V (i.e., posture estimation results, with a dimension of 51), sensor data S (from smart jerseys and smart balls, with a dimension of 30), and environmental data E (with a dimension of 5). The sensor data S includes the player's acceleration, angular velocity, heart rate, and the pressure distribution and motion parameters of the ball; the environmental data E includes temperature, humidity, wind speed, light intensity, and field condition score.
[0113] b) Apply the self-attention mechanism SelfAttention to the V, S and E respectively to obtain the attention feature A of each modality m In the football scenario, the self-attention mechanism can highlight important information, such as emphasizing the acceleration peak at the moment of kicking the ball in sensor data and paying attention to extreme weather conditions that may affect the performance of players in environmental data.
[0114] c) The attention features A of each modality m Input the cross-attention mechanism CrossAttention, learn the correlation between modalities and obtain the cross-attention feature B. This step can discover the association between different modal data, such as the relationship between posture data and ball speed, or the relationship between ambient temperature and player heart rate.
[0115] d) Processing the cross-attention feature B through the fusion network FusionNet to obtain the final multimodal fusion feature Z.
[0116] Preferably, HAFN can adaptively adjust the weights of different modal data when processing football training and game data. For example, during indoor training, the weight of environmental data will be relatively reduced; while in outdoor high-intensity games, the weight of sensor data (especially physiological data) will be increased accordingly. This dynamic adjustment ensures that the fused feature Z can reflect the key information in the current situation to the greatest extent.
[0117] Step S5: generating a personalized standard action model;
[0118] In another preferred embodiment of the present invention, the Dynamic Standard Action Generation Network (DSAGN) is specially optimized for different types of football players:
[0119] a) First, obtain the athlete's personal characteristics P (such as height, weight, muscle strength, dominant foot, etc., with a dimension of 10), on-field position information F (one-hot encoding, with a dimension of 11, corresponding to positions such as goalkeeper, central defender, full-back, defensive midfielder, attacking midfielder, winger, center forward, etc.) and action type T (one-hot encoding, with a dimension of 20, corresponding to the 20 predefined actions mentioned above);
[0120] b) The encoder processes P, F and T to obtain the encoded feature E. The characteristics of players in different positions are taken into consideration when designing the encoder. For example, for a forward, more attention will be paid to encoding features related to shooting and running.
[0121] c) Input the encoded feature E into the generator network Generator to create an initial standard action model M. Generator is based on the graph convolutional network structure and is particularly suitable for generating football actions that conform to the kinematic characteristics of the human body. For example, when generating a shooting action, multiple aspects such as the position of the supporting foot, the swing trajectory of the kicking foot, and the balance of the upper body are considered.
[0122] d) Based on the historical training data H, the initial standard action model M is processed by the refinement network Refiner to obtain the final personalized standard action model M'. The Refiner network can fine-tune the standard action according to the player's personal style and characteristics, such as adjusting the Ronaldo-style free kick run-up pace or Messi-style low-center-of-gravity dribbling.
[0123] Preferably, DSAGN can generate suitable standard movement models for players of different levels. For example, for young players, the generated movements will focus more on the standardization of basic techniques; while for professional players, the generated movements will include more advanced skills and personal characteristics. In addition, the system can dynamically adjust the intensity and amplitude of standard movements according to the physical condition of the players.
[0124] Step S6: difference analysis.
[0125] In another preferred embodiment of the present invention, a multi-level difference evaluation network (MDEN) is designed specifically for the characteristics of football actions:
[0126] a) Obtain the actual action A and the standard action model M', both of which are 30-frame (0.25-second) action sequences, each containing 3D coordinates of 17 key points;
[0127] b) Performing local difference calculation LocalDiff on each joint point i of A and M' to obtain the local difference L i In soccer actions, different joints have different importance. For example, in a shooting action, the difference between the ankle and the knee will be given a higher weight.
[0128] c) Performing a global difference calculation GlobalDiff on A and M' to obtain a global difference G. The global difference not only considers the geometric difference of the posture, but also the timing difference of the action. For example, when evaluating the passing action, the accuracy of the timing of the touch will be analyzed.
[0129] d) Processing the local difference L through the difference fusion network FusionNet i and global difference G, and obtain the final comprehensive difference evaluation result D. FusionNet uses an attention mechanism to dynamically adjust the importance of local and global differences according to different action types. For example, when evaluating dribbling actions, more attention will be paid to the smoothness between consecutive frames (global difference), while when evaluating shooting actions, more attention will be paid to the accuracy of key frames (local difference).
[0130] Preferably, MDEN considers not only the accuracy of the action but also the effect of the action when evaluating football actions. For example, when evaluating passing actions, in addition to the difference in posture, the data of smart football will be combined to analyze the accuracy of ball speed, rotation and landing point. For shooting actions, the position information of the goal frame will be combined to evaluate whether the angle and strength of the shot are optimal. This multi-dimensional difference analysis can provide players with more comprehensive and targeted improvement suggestions.
[0131] Step S7: Correction suggestion generation.
[0132] In yet another preferred embodiment of the present invention, the context-aware suggestion generator (CASG) is specially optimized for football training scenarios:
[0133] a) Obtain the difference evaluation result D (128-dimensional vector) and context information C (including training stage, player fatigue, game / training importance, etc., encoded as a 32-dimensional vector);
[0134] b) Processing the difference evaluation result D through a difference encoder to obtain an encoding feature E. The design of the encoder takes into account the characteristics of different football actions. For example, for shooting actions, special attention will be paid to key factors such as the position of the supporting foot and the swing trajectory of the kicking foot;
[0135] c) Input the encoded feature E and the context information C into the context fusion network ContextFusion to obtain the fusion feature F. When processing football training data, ContextFusion will dynamically adjust the level of detail and emphasis of the suggestions according to the training stage and the player's status. For example, during pre-match warm-up, concise suggestions focusing on psychological adjustment will be generated; while in technical training, more detailed technical correction suggestions will be generated;
[0136] d) Using the LSTM decoder DecoderLSTM to process the fused feature F, and generate the final correction suggestion text R. The DecoderLSTM is specially trained to generate professional suggestions that conform to football terminology.
[0137] Preferably, CASG can generate personalized suggestions based on the position and technical characteristics of the players. For example, for a midfielder who is good at long-range shooting, when evaluating his shooting action, more attention will be paid to the transmission of power and the choice of the hitting point; while for a winger, when evaluating his crossing action, more attention will be paid to the swing amplitude and the accuracy of the foot touching the ball. In addition, the system can also adjust the content of the suggestions according to the team's tactical style to ensure that the improvement of individual skills is coordinated with the requirements of team tactics.
[0138] Step S8: Determine the feedback method.
[0139] In another preferred embodiment of the present invention, the reinforcement learning-based adaptive feedback scheduler (RL-AFS) is optimized specifically for the characteristics of football training:
[0140] a) Get the current action importance I t (scalar, range 0-1, e.g., shooting is more important than simple passing), feedback history F t (the type and time of the last five feedbacks, encoded as a 25-dimensional vector) and training phase information T t (one-hot encoding, representing the four stages of warm-up, technical training, tactical training, and actual combat drills, a 4-dimensional vector);
[0141] b) Based on the I t 、F t and T t , the construction state represents S t The status representation has been specially designed to reflect the special needs of football training, such as taking into account the continuity of movements, changes in training intensity, etc.
[0142] c) Process the state representation S according to the current strategy π t , select feedback action A t Feedback actions include: voice commands (through player headphones), tactile feedback (through the vibration device of the smart jersey), visual prompts (through the big screen on the sidelines or AR glasses) and no feedback;
[0143] d) Execute the feedback action A t , observe the feedback effect E t (Motion improvement degree, range 0-1) and interference degree D t (impact on training rhythm, range 0-1);
[0144] e) Based on the E t and D t , calculate the reward R tThe design of the reward function takes into account the characteristics of football training. For example, in high-intensity training, more emphasis is placed on minimizing interference, while in technical detail training, more emphasis is placed on maximizing improvement.
[0145] f) Using the S t , A t , R t and the next state S t+1 , update the strategy π. The strategy update uses the PPO algorithm, which is particularly suitable for dealing with continuous decision-making problems in football training.
[0146] Preferably, RL-AFS can adaptively select the best feedback method according to different training scenarios. For example, in individual technical training, more immediate voice instructions may be chosen; while in team tactical training, visual prompts may be preferred to avoid interrupting the fluency of training. The system also takes into account the personal preferences and learning styles of the players. Some players may be more receptive to tactile feedback, while others may respond better to visual prompts.
[0147] Step S9: Long-term progress analysis and injury risk prediction.
[0148] In yet another preferred embodiment of the present invention, the Temporal Attention Residual Network (TARN) is optimized specifically for the long-term development of soccer players:
[0149] a) Obtain historical training data sequence X, which contains daily training data for the past 90 days. The daily data includes 20 features such as training duration, intensity, technical index score, competition performance, and recovery status.
[0150] b) Using a one-dimensional convolutional neural network CNN to process the data X at each time point t in X t , and get the time series feature F t The design of CNN takes into account the periodic characteristics of football training and can effectively capture weekly and monthly training patterns;
[0151] c) Processing the F through the temporal attention mechanism TemporalAttention t , get the attention weight A t The attention mechanism pays special attention to some key time points, such as training status before and after important games, recovery status during the off-season, etc.
[0152] d) Based on the A t , for F t Perform weighted summation to obtain weighted fusion feature G;
[0153] e) Inputting the G into the residual block ResidualBlock, learning the residual representation of the feature, and obtaining the residual feature H. Residual learning is particularly suitable for capturing small improvements or regressions in player abilities, which is crucial in long-term development analysis;
[0154] f) Using the prediction head network PredictionHead to process the residual feature H, the improvement trend prediction P and the injury risk prediction R are obtained. The improvement trend prediction P outputs the expected change rate of 5 key technical indicators, including passing accuracy, shooting accuracy, dribbling success rate, steal success rate and physical fitness level. The injury risk prediction R outputs the probability of common football injuries (such as ankle sprains, muscle strains, knee injuries, etc.) occurring in the next 30 days.
[0155] Preferably, TARN takes into account the following factors in particular when analyzing the long-term development of football players:
[0156] 1) Season cycle: The model can identify the different characteristics of the beginning, middle and end of the season, and adjust the improvement expectations and injury risk assessment accordingly.
[0157] 2) Position specificity: Different evaluation criteria and risk factors are used for players in different positions (such as forwards, midfielders, defenders).
[0158] 3) Age factor: Consider the impact of a player’s age on their room for improvement and injury risk, and provide appropriate development expectations for players of different age groups.
[0159] 4) Training load: Analyze the balance between training intensity, frequency and recovery time, and predict the risk of overtraining.
[0160] 5) Competition performance: Use the performance in actual competitions as an important reference to evaluate training effects and predict future performance.
[0161] 6) Environmental factors: Consider the impact of external factors such as weather changes and venue conditions on training results and injury risks.
[0162] Through this comprehensive analysis, TARN is able to provide coaches and players with valuable long-term development advice on how to adjust training plans to maximize improvement potential while minimizing injury risk. For example, the system may recommend that a young striker increase the intensity of shooting training in the middle of the season, but at the same time warn him of the risk of overuse of his knee joints and provide corresponding preventive training suggestions.
[0163] These steps are closely combined to form a complete football player real-time posture analysis and correction system. Through multimodal data acquisition, deep learning algorithms and personalized model generation, the system achieves accurate analysis, timely correction and long-term prediction of football players' movements, providing strong technical support for modern football training.
[0164] like Figure 2 As shown, the computer vision-based real-time football player posture analysis and correction system of the present invention includes the following modules, which work closely together to achieve all-round analysis and guidance of football players.
[0165] First, the multi-view action video acquisition module 1 serves as the "eyes" of the system, responsible for capturing every action of the players on the field. Preferably, in one embodiment of the present invention, the module consists of 6 high-speed cameras arranged in a regular hexagonal array around the football field. This layout ensures full coverage of a standard 11-person football field (105 meters x 68 meters) without visual blind spots. Each camera is specially optimized to capture fast football actions, such as sudden stops, turns, or shooting moments, at a high frame rate of 120fps. In order to adapt to various competition and training environments, these cameras are also equipped with advanced automatic exposure adjustment functions, which can maintain clear image quality under various lighting conditions from night training to direct sunlight at noon.
[0166] At the same time, the multimodal sensor data acquisition module 2 serves as the "touch" of the system, collecting rich auxiliary data. In the football scene, this module mainly includes three subsystems: smart jerseys, smart footballs, and field environment perception systems. The smart jerseys are equipped with multiple lightweight sensors that can accurately capture the player's body movements and physiological states. For example, during a shooting action, sensors on the shoulders and hips can record the body's rotation angle, while sensors at the ankles can capture changes in foot acceleration. The sensors built into the smart football can measure parameters such as ball speed, rotation, and impact force, which are crucial for analyzing passing and shooting techniques. The field environment perception system monitors various environmental factors that may affect the player's performance, such as temperature, humidity, and wind speed.
[0167] These raw data are then transmitted to the central data processing unit, which is the "brain" of the system. Preferably, in another embodiment of the present invention, the unit uses a high-performance computing cluster that can process massive data from multiple players in real time. The standard action model library is used to store pre-established standard action models; the posture estimation module 3 is used to obtain the posture estimation result based on the multi-view action video data through an adaptive multi-scale posture estimation algorithm; here, the adaptive multi-scale posture estimation algorithm (AMSPE) first processes the video data to accurately locate the key joints of the players. The action recognition module 4 is used to realize action recognition based on the posture estimation results using a dynamic temporal graph convolutional network; then, the dynamic temporal graph convolutional network (DT-GCN) analyzes the motion sequence of these joints to identify specific football actions, such as short passes, long passes or shots.
[0168] The data fusion module 5 plays a key role in this stage. It intelligently fuses the posture data from video analysis with the motion and physiological data collected by sensors. For example, when analyzing a pass, the system not only considers the player's body posture, but also combines the rotation data of the ball to comprehensively evaluate the quality of the pass. This multimodal fusion greatly improves the accuracy and comprehensiveness of motion analysis.
[0169] Based on the fused data, the personalized standard action generation module 6 creates a customized standard action model for each player. This module takes into account the player's physical characteristics, field position, and technical style to ensure that the generated standard action is both in line with technical specifications and suitable for the player's personal characteristics. For example, for a midfielder who is good at long-range shooting, the system will generate a shooting action model that emphasizes upper body stability and precise control of the kicking point.
[0170] The difference analysis module 7 then compares the player's actual movements with this personalized standard model to pinpoint areas for improvement. This analysis goes beyond simple geometric comparisons and also takes into account the effectiveness and efficiency of the movements. For example, when analyzing a pass, the system not only focuses on the standardization of the posture, but also evaluates whether the accuracy and speed of the pass have achieved the expected effect.
[0171] Based on the results of the difference analysis, the Corrective Suggestion Generation Module 8 will formulate targeted improvement suggestions. These suggestions are carefully designed to take into account both the correctness of the technique and the individual characteristics of the player and the current training stage. For example, for a player who is recovering, the system will generate more safety-oriented suggestions to avoid actions that may lead to relapse.
[0172] Finally, the feedback scheduling module 9 is responsible for selecting the most appropriate way to convey these suggestions to the players. In an intense game or high-intensity training, the system may choose a short voice command or a slight tactile prompt; in a technical training class, a detailed action comparison video may be displayed on a display screen on the sidelines.
[0173] This closed-loop system continuously collects data, analyzes, provides feedback, and learns and adjusts, thereby helping players to continuously improve their technical movements. Through long-term use, the system can also analyze the players' progress trends and provide a scientific basis for coaches to formulate long-term training plans.
[0174] In general, the system of the present invention provides a comprehensive, accurate and highly personalized solution for modern football training by closely combining advanced computer vision technology with the actual needs of football training. It can not only help coaches better understand and guide players, but also provide players with a scientific and effective way to improve themselves, thereby promoting the continuous improvement of the technical level of the entire football sport.
[0175] In order to verify the superiority of the present invention, we designed an embodiment and three comparative examples, and conducted an 8-week test in a real football training environment. The test subjects were 20 players from a professional football team, aged 20-28 years old, including players in different positions on the field.
[0176] Embodiment 1: A complete system using the present invention includes all modules such as multi-view video acquisition, multi-modal sensor, adaptive multi-scale posture estimation (AMSPE), dynamic temporal graph convolutional network (DT-GCN), personalized standard action generation and real-time feedback.
[0177] Comparative Example 1: Only a single-view high-speed camera and a traditional pose estimation algorithm are used, without multimodal data fusion and personalized model generation.
[0178] Comparative Example 2: Multi-view video acquisition and a traditional deep learning posture estimation algorithm are used, but the AMSPE and DT-GCN of the present invention are not adopted, and no personalized standard action generation is performed.
[0179] Comparative Example 3: An existing commercial football training analysis system on the market is used, which uses multiple cameras and wearable devices but lacks real-time personalized feedback and adaptive learning capabilities.
[0180] We selected the following five key indicators to evaluate the performance of the system:
[0181] 1. Posture estimation accuracy: By comparing with a professional motion capture system, the average error of key joints (in centimeters) is calculated.
[0182] 2. Action recognition accuracy: the percentage of correctly recognized football actions to the total number of actions.
[0183] 3. Real-time performance: The average delay time from when an action occurs to when the system gives feedback (unit: milliseconds).
[0184] 4. Personalization: Professional coaches evaluate the system’s recommendations for each player (out of 100 points).
[0185] 5. Technical improvement: Through standardized technical tests, the improvement (in percentage) of players in core skills such as passing, shooting, dribbling, etc. is evaluated after 8 weeks.
[0186] Detection method:
[0187] 1. Pose estimation accuracy: The Vicon motion capture system is used as the gold standard to collect data and calculate the error at the same time.
[0188] 2. Action recognition accuracy: Three professional coaches watched the video and annotated the actions, and compared the results with the system’s recognition results.
[0189] 3. Real-time: Use a high-precision timer to record the time of the entire processing flow.
[0190] 4. Degree of personalization: Five senior football coaches conducted double-blind rating of the suggestions given by the system.
[0191] 5. Technical improvement: Standardized soccer skills tests were conducted before and after 8 weeks of training, including 30-meter passing accuracy, shooting accuracy, and 30-meter dribbling around the slalom timing.
[0192] The test results are shown in the following table:
[0193]
[0194]
[0195] Combined with the above test results, the analysis and discussion are as follows:
[0196] 1. Posture estimation accuracy: Example 1 of the present invention is significantly better than other comparative examples. The multi-scale characteristics of the AMSPE algorithm enable it to capture large-scale body postures and small-scale joint details at the same time, which is particularly suitable for the complexity of football actions. For example, when analyzing a shooting action, the system can accurately capture the whole body coordinated action from hip twisting to ankle flexion and extension.
[0197] 2. Action recognition accuracy: The DT-GCN algorithm of Example 1 shows excellent performance. It can not only recognize basic actions, but also distinguish complex combination actions, such as the coherent action sequence of "fake action-breakthrough-shooting". This is crucial for analyzing the technical and tactical execution of players.
[0198] 3. Real-time: The system of the present invention has a much faster response speed than other solutions. The 45ms delay means that the system can provide feedback immediately after the player completes a quick pass, while the player is still in the closing phase of the action. This instant feedback is extremely effective for action correction and the formation of muscle memory.
[0199] 4. Personalization: The high score of Example 1 reflects the advantages of the system in adapting to the needs of different players. For example, the system can pay special attention to the quality and accuracy of a winger's cross in passing training according to his characteristics, while paying more attention to the accuracy and strength control of long passes for the central defender.
[0200] 5. Technical improvement: After 8 weeks of training, the players using the system of the present invention have made significant technical progress. An improvement of 18.5% means that, for example, a player's passing accuracy has increased from 70% to nearly 83%, which is a huge leap at the level of professional football.
[0201] Example 1 is undoubtedly the best implementation. It significantly outperforms the other comparison examples in all test indicators. It is particularly noteworthy that the system maintains extremely low latency while maintaining high accuracy, which is crucial for a fast-paced sport like football. The high degree of personalization of the system has also been recognized by coaches, indicating that it can truly adapt to the needs of different players rather than providing "one-size-fits-all" advice.
[0202] Please refer to Figure 9-11, it can be clearly observed from the scatter plot that the points of the present invention (red) are concentrated at the top of the figure, and the accuracy of most points is above 0.92, showing the best performance. The point distribution of comparative example 1 (green) is the lowest, and the accuracy of most points is between 0.65 and 0.75, which is the worst performance. The point distribution of comparative example 2 (blue) and comparative example 3 (yellow) is between the present invention and comparative example 1, but significantly lower than the present invention. The point distribution range of the present invention is the narrowest, indicating that its performance is the most stable. This visualization intuitively and in detail shows the superiority of the present invention in terms of attitude estimation accuracy. The dense scatter points not only show a large number of test samples, but also highly clearly show the performance differences through the distribution of points. The point group of the present invention is obviously distributed in a higher and more concentrated position, which strongly proves its excellent performance and stability in attitude estimation accuracy. In this 3D scatter plot, the point set of the present invention will be obviously concentrated in the upper right front of the space (high accuracy, high recognition rate, high frame rate), forming a dense red cloud. The point set of comparative example 1 will be distributed in the lower left rear, with the worst performance. The point sets of Comparative Examples 2 and 3 will be distributed in the middle, but are obviously inferior to the performance of the present invention. This thermal trajectory diagram will clearly show the superiority of the present invention throughout the training process. The trajectory of the present invention will be at the top, dark red, and the narrowest width, indicating that its performance is the highest and most stable. The trajectories of the other comparative examples will be distributed at the bottom, lighter in color, and wider in width, reflecting lower performance and stability.
[0203] These results fully demonstrate the great potential of this invention in actual football training. It can not only provide accurate technical analysis, but also significantly accelerate the process of players' technical improvement through timely and personalized feedback. This training method may revolutionize the traditional football training model, making technical training more accurate, efficient and personalized.
[0204] It should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A real-time posture analysis and correction method for athletes based on computer vision, characterized in that: include: Obtain multi-view action video data and multi-modal sensor data of athletes, and establish a standard action model library in advance; Based on the multi-view action video data, obtaining a posture estimation result through an adaptive multi-scale posture estimation algorithm; According to the posture estimation result, using a dynamic temporal graph convolutional network to realize action recognition; Fusing the posture estimation result with the multimodal sensor data to obtain a fusion feature; Based on the fusion features and the standard action model library, generating a personalized standard action model; Determining the difference between the athlete's actual movement and the personalized standard movement model; Based on the discrepancies, corrective suggestions are generated; Determining the best feedback method and providing the corrective suggestions to the athlete.
2. The method according to claim 1, characterized in that The adaptive multi-scale posture estimation algorithm specifically includes: Obtain an input image I; Through multiple convolutional neural networks (CNN) of different scales s Process the input image I to obtain multi-scale features F s, Where s represents the scale level; Use the soft attention mechanism SoftAttention to process the multi-scale feature F s , get the attention weight A of each scale feature s ; Based on the attention weight A s , for the multi-scale feature F s Perform weighted summation to obtain a fusion feature F; input the fusion feature F into the regression network RegressionNet to obtain the final posture estimation result P.
3. The method according to claim 1, characterized in that The dynamic temporal graph convolutional network specifically includes: Get the posture feature X at time t t and the posture feature X at time t-1 t-1 ; Based on the X t and X t-1 , through the dynamic adjacency matrix generation function f adj , get the dynamic adjacency matrix A t ; Using the dynamic adjacency matrix A t , degree matrix D and weight matrix W, perform graph convolution operation, and obtain the updated node feature H t ; The node feature H t Input the long short-term memory network LSTM and apply the temporal attention mechanism to obtain the temporal features; Based on the temporal features, the final action classification result Y is output through the temporal attention weighted Softmax function SoftmaxTE.
4. The method according to claim 1, characterized in that: The fusion of the posture estimation result and the multimodal sensor data specifically includes: Obtain visual data V, sensor data S and environmental data E; Apply the self-attention mechanism SelfAttention to the V, S and E respectively to obtain the attention feature A of each modality m ; The attention features A of each modality m The cross-attention mechanism CrossAttention is input to learn the correlation between modalities and obtain the cross-attention feature B. The cross-attention feature B is processed by the fusion network FusionNet to obtain the final multimodal fusion feature Z.
5. The method according to claim 1, characterized in that The generation of the personalized standard action model specifically includes: Obtain the athlete's personal characteristics P, on-field position information F, and action type T; Process the P, F and T through an encoder to obtain an encoding feature E; Input the encoding feature E into the generator network Generator to create an initial standard action model M; Based on the historical training data H, the initial standard action model M is processed by the fine-tuning network Refiner to obtain the final personalized standard action model M'.
6. The method according to claim 5, characterized in that The determination of the difference specifically includes: Obtain actual action A and standard action model M; Perform local difference calculation LocalDiff on each joint point i of A and M to obtain the local difference L i ; Perform a global difference calculation GlobalDiff on A and M to obtain a global difference G; The local difference L is processed by the difference fusion network FusionNet i And the global difference G, the final comprehensive difference evaluation result D is obtained.
7. The method according to claim 1, characterized in that The generation of the correction suggestion specifically includes: Obtain difference evaluation result D and context information C; Processing the difference evaluation result D through a difference encoder Encoder to obtain an encoding feature E; Input the encoded feature E and the context information C into the context fusion network ContextFusion to obtain the fusion feature F; The LSTM decoder DecoderLSTM is used to process the fused feature F to generate the final correction suggestion text R.
8. The method according to claim 1, characterized in that The determination of the optimal feedback method specifically includes: Get the current action importance I t 、Feedback History F t and training phase information T t ; Based on the I t 、F t and T t , the construction state represents S t ; Process the state representation S according to the current policy π t , select feedback action A t ; Execute the feedback action A t , observe the feedback effect E t and interference level D t ; Based on the E t and D t , calculate the reward R t ; Using the S t , A t , R t and the next state S t+1 , update the strategy π to maximize the long-term cumulative reward.
9. The method according to claim 1, characterized in that: It also includes long-term progress analysis and injury risk prediction, including: Obtain historical training data sequence X; Use convolutional neural network CNN to process the data X at each time point t in X t , and get the time series feature F t ; The F is processed by the temporal attention mechanism TemporalAttention t , get the attention weight A t Based on the A t , for F t Perform weighted summation to obtain weighted fusion feature G; Input G into the residual block ResidualBlock, learn the residual representation of the feature, and obtain the residual feature H; The prediction head network PredictionHead is used to process the residual feature H to obtain the improvement trend prediction P and the injury risk prediction R.
10. A computer vision-based real-time athlete posture analysis and correction system implementing the method described in any one of claims 1 to 9, characterized in that: include: A multi-view action video acquisition module, used to obtain multi-view action video data of the athlete; A multimodal sensor data acquisition module, used to obtain the multimodal sensor data; A standard action model library, used to store pre-established standard action models; A posture estimation module, used to obtain a posture estimation result based on the multi-view action video data through an adaptive multi-scale posture estimation algorithm; An action recognition module, used to realize action recognition by using a dynamic temporal graph convolutional network according to the posture estimation result; A data fusion module, used for fusing the posture estimation result with the multimodal sensor data to obtain a fusion feature; A personalized standard action generation module, used to generate a personalized standard action model based on the fusion feature and the standard action model library; a difference analysis module, used to determine the difference between the actual action of the athlete and the personalized standard action model; a correction suggestion generation module, used to generate a correction suggestion according to the difference; A feedback scheduling module is used to determine the best feedback method and provide the corrective suggestions to the athlete.
Citation Information
Cited By
Carrier loader auxiliary teaching method, system and equipment based on visual identification and medium
CN119850384A
Shielding clothes thermal radiation risk early warning method and device based on attitude image and medium
CN119939346A
Method and device for early warning of heat radiation risk of shielding clothes based on posture image, and medium
CN119939346B
Dynamic sports data visualization system and method
CN120087234A
A Dynamic Sports Data Visualization System and Method
CN120087234B
