Athlete training feedback method and system based on fatigue and action analysis

Abstract

The invention provides an athlete training feedback method and system based on fatigue and action analysis, and the method comprises the steps: collecting the multi-modal training data of an athlete through a visual sensor, an inertial measurement unit, and a myoelectricity and heart rate monitoring device, and carrying out the time alignment and feature extraction, the method comprises the following steps: constructing fatigue state features and action stability features, inputting the features into a fatigue-action decoupling analysis network FDNet, dynamically weighting various features through an attention mechanism, generating a fatigue influence coefficient by adopting outer product interaction modeling, and realizing quantitative analysis of the influence of the fatigue state on the action stability. A fatigue state score, an action stability score and personalized training feedback are output, and multi-dimensional and explainable exercise training decision support is achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of sports physiological monitoring and motion analysis, and in particular relates to an athlete training feedback method and system based on fatigue and motion analysis. Background Art

[0002] In modern competitive sports training, athlete physical condition monitoring and performance assessment are key areas of concern. Scientific and effective implementation of sports training monitoring is crucial for improving athletic performance, preventing sports injuries, and protecting athlete health. Traditional sports training monitoring relies primarily on the subjective experience of coaches, assessing training effectiveness and fatigue levels by observing athletes' movements and inquiring about their subjective feelings. This subjective assessment method is easily influenced by factors such as personal experience and differences in judgment criteria, lacks quantitative indicators, and is unable to accurately and comprehensively reflect changes in athletes' physical condition and movement technique.

[0003] In recent years, with the rapid development of modern sensing technology, the Internet of Things, and artificial intelligence, it has become possible to objectively monitor athletic performance using smart wearable devices and visual sensors. Researchers have begun experimenting with using inertial sensors to capture and analyze athletes' movements, assessing muscle fatigue through electromyographic signals and monitoring fatigue levels and recovery quality using heart rate variability. However, existing research has primarily focused on analyzing single physiological parameters or movement elements, with limited consideration of the relationship between physiological state and movement performance. In fact, an athlete's fatigue state directly impacts the stability and accuracy of movement execution. Fatigue can reduce an athlete's proprioception and body control, making them more susceptible to movement deviations and errors. Furthermore, the design of a suitable multimodal sensor solution that can achieve non-invasive data collection without disrupting athletes' training and provide personalized, targeted feedback and decision-making recommendations remains a key challenge.

[0004] Therefore, we need to develop an athlete training feedback method and system based on fatigue and motion analysis, which can integrate and analyze the athlete's physiological state and movement performance, build a quantitative correlation model between the two, and realize intelligent evaluation and early warning of the athlete's movement stability under fatigue state. Summary of the Invention

[0005] The purpose of the present invention is to provide an athlete training feedback method and system based on fatigue and motion analysis to solve the problems mentioned in the above background technology, such as the single research data of the existing sports training monitoring and feedback schemes and the lack of consideration of the correlation between physiological state and movement performance.

[0006] To achieve the above objectives, the present invention provides, on the one hand, an athlete training feedback method based on fatigue and motion analysis, the method being specifically as follows:

[0007] Step S1: Based on multi-source heterogeneous sensors, including visual sensors, inertial measurement units, electromyographic sensors, and heart rate monitoring equipment, the athlete's training process is synchronously collected to obtain multimodal motion data;

[0008] Step S2: performing preprocessing operations on the multimodal motion data, including timestamp alignment, filtering and noise reduction, and feature extraction, to obtain standardized time series feature data;

[0009] Step S3: Based on the time series feature data, extract key fatigue state features, including heart rate variability parameters, electromyographic frequency domain features, and movement stability features, including trajectory smoothness, terminal velocity deviation, and movement similarity, for quantitatively describing the athlete's fatigue state and movement stability;

[0010] Step S4: Based on the fatigue state characteristics and movement stability characteristics, a deep neural network model FDNet for end-to-end coupling analysis is designed and constructed. A nonlinear mapping relationship between the influence of fatigue state and movement stability is established through the deep neural network model, and a fatigue state score, a movement stability score, and a fatigue influence coefficient are output to form a comprehensive evaluation index;

[0011] The FDNet includes a dual-tower structure and a fatigue-action interaction layer. Based on the fatigue state characteristics and the action stability characteristics, the dual-tower structure outputs a fatigue state embedding vector and a stability state embedding vector, and the fatigue-action interaction layer obtains interaction characteristics. Based on the interaction characteristics, a fatigue influence coefficient is generated to quantitatively characterize the influence of the current fatigue state on the action stability.

[0012] Step S5: Based on the comprehensive evaluation indicators and combined with sports training domain knowledge, generate personalized intelligent feedback decisions.

[0013] Based on the above scheme, the fatigue state is embedded into the vector through the attention fusion mechanism and the stability state embedding vector Concatenate on the feature dimension to form a 32-dimensional joint embedding vector ; ,in Represents the concatenation operation on the 0th dimension (feature dimension), Represents matrix transpose.

[0014] Based on the above scheme, before obtaining the interaction features through the fatigue-action interaction layer, the joint embedding vector Input into the attention weight generation network, calculate the importance weights of fatigue state features and motion stability features, and obtain the attention weight vector , the expression is:

[0015]

[0016]

[0017] in , represents the importance score of fatigue state characteristics and motion stability characteristics, represents the importance score of fatigue status features, represents the importance score of the motion stability feature, and is a fully connected layer, ReLU is the activation function, and R is the real number field; , is the normalized attention weight vector, represents the normalized weight of fatigue state features, represents the normalized weight of the motion stability feature, 、 and .

[0018] Based on the above solution, the interaction features obtained through the fatigue-action interaction layer specifically include: embedding the fatigue state vector and the stability state embedding vector Perform outer product operation to generate interaction matrix : , ,in, Represents the element in row i and column j of the interaction matrix M, i.e. Characteristics of dimensional fatigue state and the The interaction strength of the action stability feature, the interaction matrix The influence of each dimension of fatigue state on each dimension of movement stability is explicitly characterized;

[0019] Process the interaction matrix through nonlinear transformations: ,in and is a learnable parameter, Flatten is the flattening operation, and I is the fatigue-action interaction feature.

[0020] Based on the above scheme, the fatigue influence coefficient is generated based on the interaction characteristics, and the expression is:

[0021] ,β value quantitatively represents the degree of influence of the current fatigue state on the stability of the movement. The higher the β value, the more significant the negative impact of fatigue on the movement; is the weight matrix, is the bias term.

[0022] Based on the above scheme, by fusing the attention weight vector , Interaction feature I, Fatigue state embedding vector and the stability state embedding vector , generate multimodal fusion features E: , where γ is the weight coefficient of the interaction feature I;

[0023] Through the regression output layer of FDNet, the multimodal fusion feature E is mapped to the fatigue state score , movement stability score and fatigue influence coefficient β, together constitute the comprehensive evaluation index: .

[0024] Based on the above scheme, the regression output layer includes a 16-dimensional fully connected layer And 2 parallel output fully connected output layers and , respectively used to predict and , the score range is [0, 1]; fully connected layer Then the ReLU activation function is connected, and the two output fully connected layers are connected with the Sigmoid function for numerical compression. The regression output layer expression is:

[0025]

[0026] in is an intermediate feature, The weight and bias parameters are and and The weight parameters are and , the corresponding bias parameter is a scalar and ; The Sigmoid function compresses the prediction score into the range [0, 1]: .

[0027] Based on the above scheme, FDNet is optimized through end-to-end backpropagation and gradient descent algorithm. The goal of model optimization is to minimize the difference between the predicted score and the true score. The mean square error is used as the loss function. The training set sample is denoted as ,in and They are The fatigue state characteristics and movement stability characteristics of the samples, and is the corresponding fatigue state score and movement stability score, is the total number of samples;

[0028] The training loss function of FDNet is defined as fatigue score loss and stability score loss The weighted average is as follows:

[0029]

[0030] in is the training loss output of FDNet, and Respectively represent FDNet's The predicted fatigue state score and predicted movement stability score of samples are and is the weight coefficient of the two loss terms, which is used to balance the contribution of fatigue state score and movement stability score to the total loss.

[0031] Based on the above scheme, the dual-tower structure consists of a parallel fatigue feature sub-network and a stability feature sub-network. In the fatigue feature sub-network, the fatigue state feature is used as input. The input feature first passes through a fully connected layer to map the original feature to a 256-dimensional latent space, and the hidden feature is output. The hidden feature then passes through two fully connected layers in sequence, each containing 256 neurons and using the ReLU activation function. The last fully connected layer is followed by a Dropout layer to randomly discard 50% of the neurons, and finally outputs a 16-dimensional fatigue state embedding vector.

[0032] In the stability feature subnetwork, the action stability feature is taken as input, first passes through a fully connected layer, mapped to a 128-dimensional latent space, and outputs the hidden feature. The hidden feature passes through two fully connected layers in sequence, with 128 neurons in each layer and using the ReLU activation function. The last fully connected layer is followed by a Dropout layer with a dropout probability of 0.5, and outputs a 16-dimensional stability state embedding vector.

[0033] On the other hand, the present invention provides an athlete training feedback system based on fatigue and motion analysis, which is used to implement the above-mentioned athlete training feedback method based on fatigue and motion analysis. The system includes: a multimodal motion data acquisition module, a data preprocessing module, a fatigue state assessment module, a motion stability analysis module, a deep coupling analysis model, and a feedback decision generation module;

[0034] The multimodal motion data acquisition module is responsible for synchronously collecting the athlete's training process based on multi-source heterogeneous sensors, including visual sensors, inertial measurement units, electromyographic sensors, and heart rate monitoring equipment, to obtain multimodal motion data;

[0035] The data preprocessing module performs preprocessing operations on the multimodal motion data, including timestamp alignment, filtering and noise reduction, and feature extraction, to obtain standardized time series feature data;

[0036] The fatigue state assessment module and the movement stability analysis module respectively extract key fatigue state features, including heart rate variability parameters and electromyographic frequency domain features, and movement stability features, including trajectory smoothness, terminal velocity deviation, and movement similarity, based on the time series feature data, for quantitatively describing the athlete's fatigue state and movement stability;

[0037] The deep coupling analysis model establishes a nonlinear mapping relationship between the influence of fatigue state and movement stability through a deep neural network model, and outputs a fatigue state score, a movement stability score, and a fatigue influence coefficient to form a comprehensive evaluation index;

[0038] The feedback decision generation module generates personalized intelligent feedback decisions based on the comprehensive evaluation indicators and combined with sports training field knowledge.

[0039] Compared with the prior art, the present invention has at least the following advantages and effects:

[0040] (1) Through the outer product operation of fatigue state embedding and motion stability embedding, an interaction matrix is ​​generated, and the fatigue influence coefficient β is regressed and output to quantitatively describe the influence of fatigue state on the quality of motion completion. This realizes the explicit modeling and quantitative expression of the influence of fatigue state on motion stability, directly digitally outputs the interference degree of fatigue on the motion, and forms a closed-loop control indicator.

[0041] (2) The attention mechanism is introduced to calculate the importance scores of fatigue state characteristics and movement stability characteristics, obtain the attention weight vector, and dynamically assign the importance of fatigue and movement characteristics. It can adaptively identify whether the athlete's fatigue or movement stability factor is more critical under different training states, and dynamically adjust the focus of attention. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly describes the drawings required for the specific embodiments or the description of the prior art. Similar elements or parts are generally identified by similar reference numerals throughout the drawings. Elements or parts in the drawings are not necessarily drawn to scale.

[0043] Figure 1 This is a flow chart of an athlete training feedback method based on fatigue and motion analysis provided by an embodiment of the present invention;

[0044] Figure 2 This is a structural diagram of an athlete training feedback system based on fatigue and motion analysis provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0045] In order to more clearly illustrate the purpose, technical solutions and advantages of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. The example implementation methods can be implemented in various forms and should not be understood as being limited to the examples described herein. On the contrary, these implementation methods are provided to make the present invention more comprehensive and complete, and to fully convey the concepts of the example implementation methods to those skilled in the art.

[0046] In addition, the described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present invention. However, it will be appreciated by those skilled in the art that the technical solutions of the present invention can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known methods, devices, implementations or operations are not shown or described in detail to avoid blurring various aspects of the present invention.

[0047] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically separate entities. That is, these functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0048] The flowcharts shown in the accompanying drawings are for illustrative purposes only and do not necessarily include all contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, while others may be combined or partially combined. Therefore, the actual execution order may vary depending on the actual situation.

[0049] The present invention will be described in detail below with reference to specific embodiments:

[0050] As attached Figure 1 As shown, embodiment 1 of the present invention provides an athlete training feedback method based on fatigue and motion analysis, and the specific steps of the method are as follows:

[0051] Step S1: Based on multi-source heterogeneous sensors, including visual sensors, inertial measurement units, electromyographic sensors, and heart rate monitoring equipment, the athlete's training process is synchronously collected to obtain multimodal motion data;

[0052] Step S2: performing preprocessing operations on the multimodal motion data, including timestamp alignment, filtering and noise reduction, and feature extraction, to obtain standardized time series feature data;

[0053] Step S3: Based on the time series feature data, extract key fatigue state features, including heart rate variability parameters, electromyographic frequency domain features, and movement stability features, including trajectory smoothness, terminal velocity deviation, and movement similarity, for quantitatively describing the athlete's fatigue state and movement stability;

[0054] Step S4: Based on the fatigue state characteristics and movement stability characteristics, a deep neural network model for end-to-end coupling analysis is designed and constructed. A nonlinear mapping relationship between the influence of fatigue state and movement stability is established through the deep neural network model, and a fatigue state score, a movement stability score, and a fatigue influence coefficient are output to form a comprehensive evaluation index;

[0055] Step S5: Based on the comprehensive evaluation indicators and combined with sports training domain knowledge, generate personalized intelligent feedback decisions.

[0056] Preferably, in step S1, a high-speed, high-definition RGB camera (with a resolution of not less than 1080p and a frame rate of not less than 60 fps) is used to capture RGB video at a specific location on the sports field. The camera's parameters, such as aperture, shutter speed, and ISO, are optimized based on on-site environmental conditions to ensure video quality.

[0057] To facilitate subsequent video analysis and processing, the camera's stability must be maintained during the acquisition process. In this embodiment, the camera is mounted on a tripod, allowing the camera to capture the entire action. The captured video stream data is then transmitted to a data processing terminal for subsequent analysis.

[0058] Exemplarily, the system binds inertial measurement units (IMUs) to key parts of the athlete, including but not limited to the left and right shoulders, elbows, wrists, hips, knees, and ankles. Each IMU integrates a three-axis accelerometer, a three-axis gyroscope, and a three-axis magnetometer, and can collect three-dimensional acceleration, angular velocity, and posture angle information of each key part in real time at a frequency of 100Hz to obtain skeletal inertial data.

[0059] Specifically, the IMU's placement must meet ergonomic requirements and not interfere with the athlete's normal movements. At the same time, attention must be paid to the consistency of the coordinate systems of each sensor. This embodiment uses a miniaturized IMU module based on the MPU9250 chip. Its small size, light weight, and high integration meet data acquisition requirements.

[0060] For example, myoelectric signals are important physiological signals that reflect the state of muscle activity. This example focuses on three muscles closely related to throwing movements: the posterior deltoid, the supraspinatus, and the biceps brachii. During the myoelectric signal acquisition process, disposable medical tape electrodes are attached to the target muscle groups on the athlete's skin surface. The electrodes are made of Ag-AgCl and conductive gel is used as the conductive liquid. The attachment site is hair-removed and cleaned with alcohol to minimize skin impedance. Elastic bandages are also used for fixation to minimize relative displacement of the electrodes.

[0061] Specifically, a wireless myoelectric acquisition system with 16 channels, a 1000Hz sampling rate, and 24-bit resolution is used for signal acquisition. The collected myoelectric signals are transmitted to the terminal in real time and undergo digital signal processing, including common-mode interference suppression and power frequency removal.

[0062] For example, heart rate variability (HRV) is an important indicator for assessing an athlete's fatigue and autonomic nervous system function. This embodiment uses a heart rate monitor to collect heart rate data. This device is comfortable to wear, offers high measurement accuracy, and can extract RR cycles at a 5kHz sampling frequency.

[0063] For example, athletes wear a heart rate monitor throughout their training session, which records the RR interval series in real time. The monitor has a built-in storage module and can operate continuously for five hours while offline. The collected RR interval data can be transmitted to a terminal via Bluetooth for analysis.

[0064] For example, in addition, the following points should be noted during the entire acquisition process: (1) Clock synchronization of each modality sensor. Network time synchronization protocols such as PTP or NTP can be used, or a unified system clock can be set; (2) Consistency of acquisition environment conditions. Environmental factors such as wind speed, temperature and humidity should be controlled as much as possible to reduce the impact on the acquisition process; (3) Preparation of the subjects. Before the test, let the athletes familiarize themselves with the acquisition process and adjust it to the optimal state. The acquisition process should be completed as continuously as possible without major interruptions.

[0065] Preferably, after the multimodal motion data acquisition in step S1 is completed, a preprocessing operation in step S2 needs to be performed to facilitate subsequent feature extraction and fusion analysis.

[0066] Exemplarily, the timestamp information of each modality data and a unified system clock are used to time-align the RGB video, skeletal inertial data, electromyographic signals, and heart rate data to form synchronized time series feature data.

[0067] For example, the RGB video is processed by key frame extraction, skeleton key point detection, etc. The three-dimensional coordinate sequence of the skeleton joint points corresponding to each frame is output for subsequent motion stability analysis.

[0068] For example, the skeletal inertial data is subjected to coordinate system 1, Kalman filtering denoising and posture solution to obtain the three-dimensional acceleration, angular velocity and posture angle information of each key part of the athlete, which is used to extract movement stability indicators.

[0069] For example, the power frequency interference and baseline drift in the electromyographic signal are removed, a Butterworth bandpass filter (20 Hz to 500 Hz) is designed to filter out high-frequency noise and electromyographic artifacts, and then the envelope signal is extracted for fatigue feature analysis.

[0070] For example, based on the heart rate data, i.e., RR interval data, outliers are eliminated, and the RR interval data is converted into an equally spaced time series using the cubic spline interpolation method, and then the time domain and frequency domain features are extracted for fatigue status assessment.

[0071] Preferably, in step S3, the process of extracting the fatigue state feature includes:

[0072] (1) Extract the following myoelectric frequency domain features from the myoelectric signal: Root mean square value (RMS): reflects the degree of muscle contraction, ; Median frequency (MDF): reflects the degree of muscle fatigue, ;in, The signal (electromyographic signal) is an important physiological indicator used to evaluate the athlete's muscle fatigue state. represents the power spectral density function of the electromyographic signal, f represents the frequency, represents the sampling frequency, represents the median frequency, which is the frequency value that divides the power spectrum into two parts of equal area;

[0073] (2) Extract the following heart rate variability parameters from the heart rate data: time domain indicators: AVNN (average value of RR interval), SDNN (standard deviation of RR interval), RMSSD (mean square error of adjacent RR intervals); frequency domain indicators: total power TP, low frequency component LF, high frequency component HF, LF / HF ratio;

[0074] Furthermore, fatigue state characteristics are constructed for:

[0075]

[0076] Wherein, subscripts 1 to n correspond to different target muscle groups.

[0077] Preferably, in step S3, the motion stability feature is used to quantitatively describe the stability level of the athlete during the throwing motion, and the extraction process includes:

[0078] The following three core features are extracted from RGB video and skeletal inertial data:

[0079] (1) Trajectory smoothness: This feature reflects the velocity change characteristics of the athlete's skeletal joints during the movement process and characterizes the smoothness of the movement. It is defined as the average value of the acceleration norm of the velocity vector of the joint point:

[0080]

[0081] in, For the The acceleration vector of the joint point in the frame image, is the total number of frames. The smaller the value, the smoother the motion trajectory.

[0082] (2) Terminal velocity deviation, the backswing phase after the action is completed; taking the throwing action as an example, the speed change of the end parts such as the wrist has a great influence on the throwing accuracy. This feature calculates the degree to which the wrist speed in the backswing phase deviates from the average value:

[0083]

[0084] in, For the The speed value of the sampling point at the sampling moment, is the average speed, is the total number of sampling points in the backswing phase. The smaller the value, the more stable the terminal speed.

[0085] (3) Action similarity: Using the dynamic time warping algorithm, the current action sequence is aligned with the standard action template and the similarity is calculated to obtain a normalized distance value, which is used to reflect the consistency and repeatability of the action completion.

[0086] Furthermore, the motion stability feature is constructed , including the above-mentioned trajectory smoothness, terminal velocity deviation, action similarity and other indicators, as a quantitative representation of action stability. and motion stability characteristics It will serve as the input of the subsequent deep neural network model to establish the coupling relationship between fatigue state and movement stability.

[0087] Preferably, in step S4, the deep neural network model is named fatigue-motion decoupling analysis network, hereinafter referred to as FDNet.

[0088] It should be noted that the core of the embodiment of the present invention is to build an end-to-end deep neural network model to explore and characterize the intrinsic connection between the athlete's fatigue state and movement stability, and form a quantitative comprehensive evaluation index.

[0089] Specifically, FDNet comprises a dual-tower architecture consisting of a parallel fatigue feature subnetwork and a stability feature subnetwork. The fatigue feature subnetwork processes fatigue state features and maps them into a low-dimensional fatigue state embedding space; the stability feature subnetwork processes motion stability features and maps them into a low-dimensional motion stability embedding space. These two subnetworks learn deep feature representations of fatigue state and motion stability, respectively, capturing the spatiotemporal dependencies and discriminative information within their respective modalities.

[0090] Preferably, in the fatigue feature sub-network, the fatigue state feature As input, the input features first pass through a fully connected layer, denoted as , maps the original features to a 256-dimensional hidden space and outputs the hidden features. The hidden features then pass through two fully connected layers in sequence. and , each layer contains 256 neurons and uses the ReLU activation function. To alleviate the overfitting problem, a Dropout layer is added after the last fully connected layer to randomly drop 50% of the neurons. The output of the fatigue feature sub-network is a 16-dimensional fatigue state embedding vector, denoted as . This subnetwork can be expressed as:

[0091]

[0092] In the formula Indicates the Layer hidden features.

[0093] Preferably, the stability feature sub-network adopts a structure similar to the fatigue feature sub-network, with the action stability feature As input, the input features first pass through the fully connected layer , mapped to a 128-dimensional hidden space. The hidden features pass through two fully connected layers in turn, and , 128 neurons per layer, and using ReLU activation function. The last fully connected layer is followed by a Dropout layer with a dropout probability of 0.5; the stability feature subnetwork outputs a 16-dimensional stability state embedding vector , the sub-network is expressed as:

[0094]

[0095] In summary, through the fatigue feature sub-network and the stability feature sub-network, the original fatigue state features and motion stability features are compressed into two 16-dimensional embedding spaces, obtaining the fatigue state embedding vector and the stability state embedding vector respectively. The two embedding vectors fully summarize the semantic information of each feature dimension.

[0096] It should be noted that since fatigue and motion stability during human movement are interrelated and mutually influential, simply learning the feature representations within each modality is not enough; the interaction between the two modalities must also be modeled. Therefore, FDNet further introduces an attention fusion mechanism to adaptively aggregate the deep semantics of physiological fatigue and motion stability.

[0097] Preferably, the attention fusion mechanism is used to perform feature fusion, specifically including:

[0098] Embed the fatigue state into the vector and the stability state embedding vector Concatenate on the feature dimension to form a 32-dimensional joint embedding vector :

[0099]

[0100] in Represents the concatenation operation on the 0th dimension (feature dimension), Represents matrix transpose.

[0101] Furthermore, the joint embedding vector Input to the attention weight generation network to calculate the importance weights of fatigue state features and motion stability features. The attention weight generation network consists of two fully connected layers and a Softmax normalization function, where the first fully connected layer Will Mapped from 32 dimensions to 16 dimensions, and transformed nonlinearly through the ReLU function, the second fully connected layer Further map the 16-dimensional features into a 2-dimensional vector , represents the importance score of fatigue state feature and motion stability feature. The importance score vector is normalized by Softmax function to obtain the attention weight vector , the expressions are as follows:

[0102]

[0103]

[0104] in , represents the importance score of fatigue state characteristics and motion stability characteristics, represents the importance score of fatigue status features, represents the importance score of the motion stability feature, is the normalized attention weight vector, represents the normalized weight of fatigue state features, Represents the normalized weight of the action stability feature, satisfying , and ; and The weight parameters are and , the bias parameter is and .

[0105] Preferably, in order to establish a nonlinear mapping relationship between the effect of fatigue state and motion stability, a fatigue-motion interaction layer is designed for FDNet, which is achieved by the following methods:

[0106] Embedding vector for fatigue state and the stability state embedding vector Perform outer product operation to generate interaction matrix : , ,in, Represents the element in row i and column j of the interaction matrix M, i.e. Characteristics of dimensional fatigue state and the The interaction strength of the dimensional motion stability characteristics. This interaction matrix explicitly represents the degree of influence of each dimension of fatigue state on each dimension of motion stability;

[0107] Furthermore, the interaction matrix is ​​processed by nonlinear transformation: ,in and is a learnable parameter, Flatten is the flattening operation, and I is the fatigue-action interaction feature;

[0108] Based on the interaction feature I, the fatigue influence coefficient β is generated: The β value quantitatively represents the degree of influence of the current fatigue state on the stability of the movement. The higher the β value, the more significant the negative impact of fatigue on the movement. is a weight matrix / vector used to linearly transform the interaction feature I, is the bias term, added to the linear transformation result;

[0109] Furthermore, by fusing the attention weight vector α, the interaction feature I, and the fatigue state embedding vector and the stability state embedding vector , generate multimodal fusion features E: , where γ is the weight coefficient of the interaction feature I. This design enables the model to capture the complex nonlinear relationship between fatigue state and motion stability, rather than just simple feature fusion;

[0110] Through the regression output layer, the multimodal fusion feature E is mapped to the fatigue state score , movement stability score and fatigue influence coefficient β, together constitute the comprehensive evaluation index: , these three indicators respectively reflect the athlete's fatigue level, movement stability level, and the intensity of fatigue's impact on movement, providing a comprehensive basis for training adjustments;

[0111] Specifically, the regression output layer includes a 16-dimensional fully connected layer And 2 parallel output fully connected output layers and , respectively used to predict and , the score range is [0, 1]. Fully connected layer Then the ReLU activation function is connected, and the two output fully connected layers are connected with the Sigmoid function for numerical compression. The regression output layer expression is:

[0112]

[0113] in is an intermediate feature, The weight and bias parameters are and and The weight parameters are and , the corresponding bias parameter is a scalar and ; The Sigmoid function compresses the prediction score into the range [0, 1]: ;

[0114] In summary, the entire forward propagation process of FDNet is completed, and all parameters in the model, including the fully connected layer weights, biases, and attention network parameters, are learnable. Furthermore, FDNet can be optimized through end-to-end backpropagation and gradient descent algorithms. The goal of model optimization is to minimize the difference between the predicted score and the true score, using the mean square error as the loss function. The specific training process is as follows:

[0115] The training set samples are ,in and They are The fatigue state characteristics and movement stability characteristics of the samples, and is the corresponding fatigue state score and movement stability score, is the total number of samples; the training loss function of FDNet is defined as fatigue score loss and stability score loss The weighted average is as follows:

[0116]

[0117] In the formula is the training loss output of FDNet, and Respectively represent FDNet's The predicted fatigue state score and predicted movement stability score of samples are and is the weight coefficient of the two loss terms, which is used to balance the contribution of fatigue state score and action stability score to the total loss. In this embodiment, the optimal weight is determined by grid search as =0.6, .

[0118] Furthermore, the Adam optimizer is used to update all parameters of FDNet during model training. The Adam optimizer adaptively adjusts the learning rate of each parameter, which can accelerate convergence and improve training stability. The initial learning rate is set to 0.001, and the learning rate is reduced by 50% every 10 epochs, and a maximum of 100 epochs are trained. The batch size (the number of small batch samples used in each iteration) is set to 64. During training, the model performance is evaluated on the validation set after each epoch. If the model performance does not improve for 5 consecutive epochs, the early stopping mechanism is triggered to stop training to prevent overfitting. All trainable parameters are initialized using the He initialization method, that is, the parameters are set to have a mean of 0 and a variance of Gaussian distribution, where is the number of input units of the network layer.

[0119] Furthermore, in order to improve the generalization performance and robustness of FDNet, the embodiment of the present invention introduces a data enhancement strategy in model training, including:

[0120] (1) Gaussian noise perturbation: the original fatigue state characteristics and motion stability characteristics Add random Gaussian noise , where the noise intensity Adjust according to the specific task to simulate the measurement error and individual differences of the sensor, mathematically expressed as:

[0121]

[0122] (2) Linear interpolation expansion: In the original fatigue state characteristics and motion stability characteristics A large number of virtual samples are generated by linear interpolation. A large number of virtual samples can smooth the distribution of original data, prevent the model from over-relying on certain samples or features, and improve generalization ability.

[0123] Specifically, two samples are randomly selected from the training set and , generate virtual samples :

[0124]

[0125] in is a random interpolation coefficient that obeys a uniform distribution on [0, 1]. The label of the virtual sample is given by a linear combination of the original labels, as follows:

[0126]

[0127] Furthermore, to enhance the robustness of FDNet predictions, this embodiment designed a model ensemble strategy. Specifically, using a 5-fold cross-validation approach, all training samples were randomly divided into five parts. One part was selected each time as the validation set, and the remaining four parts were used as the training set. This training process was repeated five times to obtain a basic FDNet model. During the inference phase, the outputs of these five models were arithmetic averaged to obtain the final fatigue state score and motion stability score. This ensemble strategy can reduce the occasional errors of individual models and improve the robustness of predictions.

[0128] It should be noted that the FDNet model structure and training method are exemplary and can be modified and adjusted by those skilled in the art based on specific application scenarios and data characteristics. For example, the number of layers and neurons in each subnetwork can be adjusted, other activation functions and regularization methods can be selected, and different attention mechanisms and integration strategies can be tried.

[0129] Preferably, in step S5, based on the fatigue state score and movement stability score output by FDNet, sports training domain knowledge is further integrated to intelligently generate personalized feedback auxiliary information. The core content of this feedback includes: clarifying whether the athlete's fatigue state has affected the quality of training, providing targeted guidance for optimizing the training load, and providing explainable and actionable improvement suggestions for key defects in movement completion.

[0130] In order to convey information intuitively and accurately, this embodiment proposes multiple visual feedback rendering forms. First, for the fatigue status score given by FDNet and stability score The system automatically maps it into three discrete levels: normal (green), slightly abnormal (yellow), and severely abnormal (red), which intuitively represent the athlete's current fatigue level and the quality of action completion. Corresponding to different levels, the system automatically generates corresponding text reports. When the fatigue state is normal, suggestions such as "reasonable arrangement of training and attention to ensuring recovery time" are given; when the fatigue state is slightly abnormal, suggestions such as "appropriately reduce training intensity, increase interval time, and improve sleep" are given; when the fatigue state is severely abnormal, suggestions such as "pause the current high-intensity training, arrange recovery training or rest, and obtain medical intervention if necessary" are given.

[0131] To assess stability, the system provides quantitative metrics for key movement stages, including backswing velocity mean and standard deviation, and the coefficient of variation of throwing angles. It also offers targeted optimization suggestions, such as "maintaining steady wrist velocity," "focusing on trunk and shoulder coordination," and "increasing left knee flexion and extension." Using skeletal trajectory data extracted from RGB video, the system generates a slow-motion video comparing the current movement execution process with a standard movement, overlaying key evaluation metrics to visually demonstrate the strengths and weaknesses of the movement.

[0132] In more embodiments, as shown in the attached Figure 2 As shown, an athlete training feedback system 200 based on fatigue and motion analysis is provided, which is mainly composed of a multimodal motion data acquisition module 201, a data preprocessing module 202, a fatigue state assessment module 203, a motion stability analysis module 204, a deep coupling analysis model 205 and a feedback decision generation module 206.

[0133] Preferably, the multimodal motion data acquisition module 201 is responsible for synchronously collecting the athlete's training process based on multi-source heterogeneous sensors, including visual sensors (such as high-definition RGB cameras), inertial measurement units, electromyographic sensors, and heart rate monitoring equipment, to obtain multimodal motion data, including RGB video, skeletal motion information, electromyographic signals, heart rate variability, etc.

[0134] Preferably, the data preprocessing module 202 performs a series of preprocessing operations on the collected multimodal heterogeneous data, such as timestamp alignment, coordinate system alignment, data screening, and noise filtering, to obtain standardized time series feature data, laying a data foundation for subsequent analysis.

[0135] Preferably, the fatigue state assessment module 203 extracts key fatigue state features based on the time series feature data, including heart rate variability parameters and electromyographic frequency domain features; this module focuses on the athlete's physiological fatigue level, and quantitatively describes the athlete's fatigue state by analyzing indicators such as heart rate variability parameters and electromyographic frequency domain features.

[0136] Preferably, the motion stability analysis module 204 extracts motion stability features based on the time series feature data, including trajectory smoothness, terminal velocity deviation, and motion similarity, for quantitative description of the athlete's motion stability; this module focuses on analysis from the perspective of the athlete's motion performance, and evaluates stability indicators such as trajectory smoothness, terminal velocity deviation, and motion similarity during the completion of the motion based on the joint trajectory information calculated based on RGB video content and skeletal inertia data.

[0137] Preferably, after obtaining the quantitative fatigue state characteristics and movement stability characteristics, the deep coupling analysis model 205 establishes a nonlinear mapping relationship between the influence of fatigue state and movement stability through a deep neural network model, and outputs a fatigue state score, a movement stability score and a fatigue influence coefficient to form a comprehensive evaluation index; this module makes full use of the powerful nonlinear fitting ability of the deep neural network model to construct an end-to-end fatigue state-movement stability fusion analysis model, explores the intrinsic connection between physiological fatigue and the quality of movement completion, and forms a comprehensive evaluation index.

[0138] Preferably, the final feedback decision generation module 206 generates personalized intelligent feedback decisions based on the comprehensive evaluation indicators and combined with sports training domain knowledge. This module integrates the comprehensive evaluation indicators with sports training domain knowledge to design personalized intelligent feedback decisions, forming intelligent training intervention measures tailored to the athlete's current fatigue state and movement performance. Feedback forms include various intuitive presentation methods such as evaluation reports, movement comparison playback, and fatigue-movement stability radar charts.

[0139] Through the collaborative work of the above modules, the system of this embodiment realizes a full-process closed loop from data acquisition-multimodal fusion-fatigue motion coupling analysis-intelligent feedback, providing objective, accurate and comprehensive assistance for sports training.

[0140] In practice, the system collects, analyzes, and provides real-time data at the athlete training site. Coaches and athletes can view the evaluation results in real time via tablet computers and optimize and adjust their training plans based on the system's intelligent recommendations.

[0141] In this embodiment, the athlete fatigue state-action stability coupling analysis and feedback method and system based on multimodal motion data provided by the present invention can achieve objective quantitative evaluation and decoupling of fatigue state and action performance quality, reveal the impact of physiological fatigue on the stability of action completion, and intelligently generate personalized feedback based on the evaluation results, providing strong support for scientific, refined and intelligent sports training. The system is expected to be promoted and applied in scenarios such as sports teams and fitness clubs. The implementation of the present invention is not limited to the above-mentioned embodiments. Those of ordinary skill in the art can understand that all or part of the steps in the above-mentioned method can be completed by a program to instruct the relevant hardware, and the program can be stored in a computer-readable storage medium, and the storage medium includes a read-only memory (ROM), a disk or an optical disk, etc.

Claims

1. A training feedback method for athletes based on fatigue and motion analysis, characterized in that: include: Step S1: Based on multi-source heterogeneous sensors, including visual sensors, inertial measurement units, electromyographic sensors, and heart rate monitoring equipment, the athlete's training process is synchronously collected to obtain multimodal motion data; Step S2: performing preprocessing operations on the multimodal motion data, including timestamp alignment, filtering and noise reduction, and feature extraction, to obtain standardized time series feature data; Step S3: Based on the time series feature data, extract key fatigue state features, including heart rate variability parameters, electromyographic frequency domain features, and movement stability features, including trajectory smoothness, terminal velocity deviation, and movement similarity, for quantitatively describing the athlete's fatigue state and movement stability; Step S4: Based on the fatigue state characteristics and movement stability characteristics, a deep neural network model FDNet for end-to-end coupling analysis is designed and constructed. A nonlinear mapping relationship between the influence of fatigue state and movement stability is established through the deep neural network model, and a fatigue state score, a movement stability score, and a fatigue influence coefficient are output to form a comprehensive evaluation index; The FDNet includes a dual-tower structure and a fatigue-action interaction layer. Based on the fatigue state characteristics and the action stability characteristics, the dual-tower structure outputs a fatigue state embedding vector and a stability state embedding vector, and the fatigue-action interaction layer obtains interaction characteristics. Based on the interaction characteristics, a fatigue influence coefficient is generated to quantitatively characterize the influence of the current fatigue state on the action stability. Step S5: Based on the comprehensive evaluation indicators and combined with sports training domain knowledge, generate personalized intelligent feedback decisions.

2. The athlete training feedback method based on fatigue and motion analysis according to claim 1, characterized in that: Embed the fatigue state into the vector through the attention fusion mechanism and the stability state embedding vector Concatenate on the feature dimension to form a 32-dimensional joint embedding vector ; ,in Represents the concatenation operation on the 0th dimension (feature dimension), Represents matrix transpose.

3. The athlete training feedback method based on fatigue and motion analysis according to claim 2, characterized in that: Before obtaining the interaction features through the fatigue-action interaction layer, the joint embedding vector Input into the attention weight generation network, calculate the importance weights of fatigue state features and motion stability features, and obtain the attention weight vector , the expression is: , ,in , represents the importance score of fatigue state characteristics and motion stability characteristics, represents the importance score of fatigue status features, represents the importance score of the motion stability feature, and is a fully connected layer, ReLU is the activation function, and R is the real number field; , is the normalized attention weight vector, represents the normalized weight of fatigue state features, represents the normalized weight of the motion stability feature, 、 and .

4. The athlete training feedback method based on fatigue and motion analysis according to claim 3, characterized in that: The interaction features obtained through the fatigue-action interaction layer specifically include: embedding the fatigue state vector and the stability state embedding vector Perform outer product operation to generate interaction matrix : , ,in, Represents the element in row i and column j of the interaction matrix M, i.e. Characteristics of dimensional fatigue state and the The interaction strength of the action stability feature, the interaction matrix The influence of each dimension of fatigue state on each dimension of movement stability is explicitly characterized; Process the interaction matrix through nonlinear transformations: ,in and is a learnable parameter, Flatten is the flattening operation, and I is the fatigue-action interaction feature.

5. The athlete training feedback method based on fatigue and motion analysis according to claim 4, characterized in that: The fatigue influence coefficient is generated based on the interaction characteristics, and the expression is: ,β value quantitatively represents the degree of influence of the current fatigue state on the stability of the movement. The higher the β value, the more significant the negative impact of fatigue on the movement; is the weight matrix, is the bias term.

6. The athlete training feedback method based on fatigue and motion analysis according to claim 5, characterized in that: By fusing the attention weight vector , Interaction feature I, Fatigue state embedding vector and the stability state embedding vector , generate multimodal fusion features E: , where γ is the weight coefficient of the interaction feature I; Through the regression output layer of FDNet, the multimodal fusion feature E is mapped to the fatigue state score , movement stability score and fatigue influence coefficient β, together constitute the comprehensive evaluation index: .

7. The athlete training feedback method based on fatigue and motion analysis according to claim 6, characterized in that: The regression output layer includes a 16-dimensional fully connected layer And 2 parallel output fully connected output layers and , respectively used to predict and , the score range is [0, 1]; fully connected layer Then the ReLU activation function is connected, and the two output fully connected layers are connected with the Sigmoid function for numerical compression. The regression output layer expression is: ,in is an intermediate feature, The weight and bias parameters are and and The weight parameters are and , the corresponding bias parameter is a scalar and ; The Sigmoid function compresses the prediction score into the range [0, 1]: .

8. The athlete training feedback method based on fatigue and motion analysis according to claim 6, characterized in that: FDNet is optimized through end-to-end backpropagation and gradient descent algorithms. The goal of model optimization is to minimize the difference between the predicted score and the true score. The mean square error is used as the loss function. The training set samples are denoted as ,in and They are The fatigue state characteristics and movement stability characteristics of the samples, and is the corresponding fatigue state score and movement stability score, is the total number of samples; The training loss function of FDNet is defined as fatigue score loss and stability score loss The weighted average is as follows: ,in is the training loss output of FDNet, and Respectively represent FDNet's The predicted fatigue state score and predicted movement stability score of samples are and is the weight coefficient of the two loss terms, which is used to balance the contribution of fatigue state score and movement stability score to the total loss.

9. The athlete training feedback method based on fatigue and motion analysis according to claim 1, characterized in that: The dual-tower structure consists of a parallel fatigue feature sub-network and a stability feature sub-network. In the fatigue feature sub-network, the fatigue state feature is used as input. The input feature first passes through a fully connected layer to map the original feature to a 256-dimensional latent space, and the hidden feature is output. The hidden feature then passes through two fully connected layers in sequence, each containing 256 neurons and using the ReLU activation function. The last fully connected layer is followed by a Dropout layer to randomly discard 50% of the neurons, and finally outputs a 16-dimensional fatigue state embedding vector. In the stability feature subnetwork, the action stability feature is taken as input, first passes through a fully connected layer, mapped to a 128-dimensional latent space, and outputs the hidden feature. The hidden feature passes through two fully connected layers in sequence, with 128 neurons in each layer and using the ReLU activation function. The last fully connected layer is followed by a Dropout layer with a dropout probability of 0.5, and outputs a 16-dimensional stability state embedding vector.

10. An athlete training feedback system based on fatigue and motion analysis, used to execute the athlete training feedback method based on fatigue and motion analysis according to any one of claims 1 to 9, characterized in that: It includes multimodal motion data acquisition module, data preprocessing module, fatigue state assessment module, motion stability analysis module, deep coupling analysis model and feedback decision generation module; The multimodal motion data acquisition module is responsible for synchronously collecting the athlete's training process based on multi-source heterogeneous sensors, including visual sensors, inertial measurement units, electromyographic sensors, and heart rate monitoring equipment, to obtain multimodal motion data; The data preprocessing module performs preprocessing operations on the multimodal motion data, including timestamp alignment, filtering and noise reduction, and feature extraction, to obtain standardized time series feature data; The fatigue state assessment module and the movement stability analysis module respectively extract key fatigue state features, including heart rate variability parameters and electromyographic frequency domain features, and movement stability features, including trajectory smoothness, terminal velocity deviation, and movement similarity, based on the time series feature data, for quantitatively describing the athlete's fatigue state and movement stability; The deep coupling analysis model establishes a nonlinear mapping relationship between the influence of fatigue state and movement stability through a deep neural network model, and outputs a fatigue state score, a movement stability score, and a fatigue influence coefficient to form a comprehensive evaluation index; The feedback decision generation module generates personalized intelligent feedback decisions based on the comprehensive evaluation indicators and combined with sports training field knowledge.

Images