Badminton player skill assessment method based on deep learning
Through the improved MS-TCN++ model and multimodal data fusion technology, the problems of accuracy and personalized evaluation in athlete technical evaluation are solved, and efficient evaluation of badminton players' technical movements is achieved, which is suitable for real-time monitoring and training feedback.
Patent Information
- Application Number
- CN202510872271.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-26
- Publication Date
- 2025-10-10
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing technologies have deficiencies in accuracy, real-time and personalized assessment of athletes' technical skills, especially in multimodal data fusion and addressing individual differences among athletes.
The improved MS-TCN++ model is used to combine video data with inertial measurement unit data. Through the sparse hybrid gated convolution structure, gated fusion module and cross-modal residual correction mechanism, efficient fusion and personalized evaluation of multimodal data are achieved.
It improves the accuracy and comprehensiveness of athlete movement analysis, can accurately evaluate batting movements, mobility and physical performance, has strong adaptability, is suitable for real-time monitoring and training processes of athletes, and provides scientific training feedback.
Smart Images
Figure CN120766181A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the fields of sports science and artificial intelligence technology, and in particular to a badminton player skill evaluation method based on deep learning. Background Art
[0002] With the continuous advancement of modern science and technology, especially the development of artificial intelligence and deep learning technology, athlete motion analysis has been widely used in the field of sports. However, traditional athlete technical evaluation methods mainly rely on manual observation and analysis, which are highly subjective and inefficient. These methods often find it difficult to capture subtle differences in athlete movements, and cannot provide comprehensive, objective quantitative support for technical evaluation, resulting in certain errors in the evaluation results, making it difficult to meet the requirements of accuracy and efficiency.
[0003] In recent years, the combination of video analysis and inertial measurement unit (IMU) technology has become a common method for analyzing athlete motion. Video analysis can clearly demonstrate the athlete's movements, while IMU technology can record physical parameters such as acceleration and angular velocity in real time, providing more comprehensive data support. However, existing technologies still have significant shortcomings in many aspects. Video-based analysis methods often rely on traditional image processing techniques. Although they can capture athlete movements, image analysis methods lack accuracy and robustness in fast-moving or complex backgrounds. Factors such as lighting changes and shooting angles make it difficult for video data to provide reliable analysis results in some cases. Furthermore, while IMU technology can provide athletes' physiological data, existing technologies fail to effectively integrate IMU data with video data, limiting the comprehensiveness and accuracy of analysis results.
[0004] To address these shortcomings, multimodal data fusion methods have been developed in recent years, combining video and sensor data to improve the accuracy of athlete motion analysis. While these methods have mitigated the limitations of single data sources to some extent, they still face numerous challenges. Currently, most existing deep learning models are based on fixed network structures, lacking sufficient flexibility and adaptability to account for individual differences among athletes. Furthermore, while multimodal data fusion techniques attempt to combine data from different sources, existing methods lack precision in the data alignment and fusion process, resulting in suboptimal final evaluation results.
[0005] In general, the application of existing technologies in athlete technical assessment still has certain limitations, especially in terms of accuracy, real-time and personalized assessment, which still needs to be further improved. How to more effectively integrate multimodal data, cope with individual differences among athletes, and improve the accuracy of assessment in complex scenarios are still problems that need to be solved urgently by current technology.
[0006] Therefore, how to provide a badminton player technical evaluation method based on deep learning is an urgent problem that technicians in this field need to solve. Summary of the Invention
[0007] One object of the present invention is to propose a method for evaluating the technical movements of badminton players based on deep learning. The present invention makes full use of the improved MS-TCN++ model and combines the multimodal data fusion technology of video data and inertial measurement unit data to describe in detail a method for evaluating the technical movements of badminton players through a deep learning model. The method adopts a gated fusion module and a cross-modal residual correction mechanism to effectively combine information from different feature channels. It can accurately evaluate the badminton players' hitting movements, mobility and physical performance, and has the advantages of high evaluation accuracy, strong personalized analysis, and wide adaptability.
[0008] A method for evaluating badminton player skills based on deep learning according to an embodiment of the present invention includes the following steps:
[0009] S1, collect image sequences and inertial measurement unit data, extract skeleton feature sequences and inertial feature sequences, align them by timestamp, and fuse them into a time series input tensor;
[0010] S2. Feed the time series input tensor into the improved MS-TCN++, replacing the original convolution module with a sparse hybrid gated convolution structure that includes a learnable sparse mask and a gated weight adjustment mechanism.
[0011] S3. Set up a skeleton channel and an inertial channel in the improved MS-TCN++, process the skeleton feature sequence and the inertial feature sequence respectively, and generate the skeleton channel output and the inertial channel output;
[0012] S4. Perform residual modeling on the difference between the skeleton channel output and the inertial channel output, introduce a cross-modal residual correction mechanism, and generate residual correction features;
[0013] S5. Input the residual correction feature into the gated fusion module to generate a stage fusion feature sequence, input it into the boundary prediction module, and output the action unit boundary index sequence, which is divided into action units arranged in time order;
[0014] S6. Based on the annotation information containing the scoring dimension in the training data, a scoring label is generated for each action unit, and the scoring label is used for parameter update calculation during the training process;
[0015] S7. Output the technical evaluation results including the movement ability score, the hitting action score and the physical performance score.
[0016] Optionally, the image sequence is continuous frame images acquired at a fixed frame rate, and the inertial measurement unit data includes acceleration, angular velocity and displacement, so as to construct a multimodal input data set.
[0017] Optionally, the S2 specifically includes:
[0018] S21. Input the time series input tensor into the improved MS-TCN++, where the time series input tensor is a three-dimensional tensor constructed by fusing the skeleton feature sequence and the inertia feature sequence, and includes a feature dimension, a time step dimension, and a channel dimension.
[0019] S22. In the improved MS-TCN++, a sparse hybrid gated convolution structure is used to replace the original convolution module. The sparse hybrid gated convolution structure is a convolution unit that includes both sparse control and channel gating mechanisms.
[0020] S23, the sparse hybrid gated convolution structure includes a learnable sparse mask, where the learnable sparse mask is a binary vector with the same length as the one-dimensional convolution kernel, and each bit indicates whether the convolution operation is enabled at the corresponding time position;
[0021] S24. The sparse hybrid gated convolution structure also includes a gated weight adjustment mechanism, which is composed of a trainable floating-point vector consistent with the number of output channels, and is used to adjust the convolution response strength of each channel.
[0022] Optionally, the S3 specifically includes:
[0023] S31. Setting a skeleton channel and an inertial channel in the improved MS-TCN++, wherein the skeleton channel receives a skeleton feature sequence as input, and the inertial channel receives an inertial feature sequence as input;
[0024] S32, skeleton feature sequence is represented as time length T, each time step contains D s The inertial feature sequence is represented by a two-dimensional data matrix with a time length of T and each time step containing D i The two-dimensional data matrix of the feature vector is expanded into feature vector sequences according to the time step.
[0025] S33. Perform one-dimensional convolution operations on the skeleton feature sequence and the inertia feature sequence respectively, and calculate the channel output:
[0026]
[0027] Among them, m represents the channel type, which can be either skeleton channel s or inertial channel i. represents the output vector of the channel at time step t, Represents the feature vector of the input sequence at time step t+k, with a length equal to the input feature dimension, Represents the convolution kernel parameter matrix at the relative displacement k, the number of rows is the channel output feature dimension, the number of columns is the input feature dimension, b (m) Represents the bias vector, whose dimension is equal to the output feature dimension.
[0028] Optionally, the S4 specifically includes:
[0029] S41, performing residual modeling on the difference between the skeleton channel output and the inertial channel output, where the residual is represented as the time-step difference between the skeleton channel output and the inertial channel output;
[0030] S42. Input the residual into the cross-modal residual correction mechanism. The cross-modal residual correction mechanism includes a residual correction module, an adjustment module, and a feature fusion module. The residual correction module corrects the residual through nonlinear mapping. The adjustment module adjusts the corrected residual according to the adjustment coefficient obtained through training. The feature fusion module combines the corrected residual with the output of the skeleton channel and the inertial channel. The residual correction operation is expressed as:
[0031]
[0032] Among them, R t ′ is the corrected residual, R t is the original cross-modal residual, f(·;θ) is the correction function, which represents the nonlinear mapping learned through training, θ is the trainable parameter of the correction function, and λ is the regularization coefficient, which controls the influence of the regularization term on the residual correction. is the gradient of the residual, which represents the rate of change of the residual relative to the input and is used to measure the change of cross-modal features;
[0033] S43, the corrected residual R t ′ is applied to the output of the skeleton channel and the inertial channel to generate residual correction features.
[0034] Optionally, the S5 specifically includes:
[0035] S51. Input the residual correction feature into the gated fusion module. The gated fusion module controls the fusion ratio of the residual correction feature and the skeleton channel feature according to the gating coefficient of each time step:
[0036]
[0037] Among them, Y t is the fusion feature of time step t, W t is the gate coefficient, controlling the residual correction feature R t ′ and skeleton channel characteristics The fusion ratio, R ′ t is the residual correction feature at time step t, is the skeleton channel feature at time step t;
[0038] S52, generate a stage fusion feature sequence, the stage fusion feature sequence is expressed as Y = {Y1, Y2, ..., Y T}, where T is the number of time steps, contains the information of each time step, and has temporal dependence;
[0039] S53: Input the stage fusion feature sequence into the boundary prediction module. The boundary prediction module processes the fusion feature through time series modeling to generate an action unit boundary index sequence:
[0040]
[0041] Where B is the action unit boundary index sequence, indicating the boundary position of the action unit, σ(·) is the sigmoid activation function, cos(·) is the cosine similarity function, f(·; θ1) and f(·; θ2) are learning mapping functions based on fusion features, θ1 and θ2 are trainable parameters to optimize the accuracy of boundary prediction;
[0042] S54. Divide the stage fusion feature sequence into action units arranged in time order according to the action unit boundary index sequence.
[0043] Optionally, the boundary prediction module specifically includes:
[0044] Feature extraction module, used to extract time series information from the stage fusion feature sequence and generate time series feature representation;
[0045] LSTM network is used to model the temporal feature representation, capture the temporal dependency of input features, and generate temporal dependency features;
[0046] The prediction module is used to generate the action unit boundary index sequence for each time step based on the temporal dependency features output by the LSTM network and determine the location of the action unit boundary;
[0047] The verification module is used to optimize and correct the generated action unit boundary index sequence to ensure the accuracy of the boundary position.
[0048] Optionally, the S6 specifically includes:
[0049] S61. Based on the annotation information containing the scoring dimension in the training data, generate a scoring label corresponding to each action unit, wherein the scoring label includes a movement ability score, a hitting action score, and a physical performance score, and associate the scoring label with the corresponding action unit;
[0050] S62. Calculate the loss function of each action unit based on the score label. The loss function consists of L2 loss and cross entropy loss, which represents the difference between the predicted value and the true value:
[0051] L=λ1·L2(y pred ,y true )+λ2·CE(y pred ,y true );
[0052] Among them, L2 is the L2 loss function, which represents the square difference between the predicted value and the true value, CE is the cross entropy loss function, which represents the difference between the probability distributions, and y pred is the predicted value, y true is the true label, λ1 and λ2 are weight coefficients used to balance the various parts of the loss function;
[0053] S63. Calculate the gradient based on the loss function and update the model parameters through the back propagation algorithm to optimize the model and reduce the loss.
[0054] The beneficial effects of the present invention are:
[0055] This invention significantly improves the accuracy and comprehensiveness of athlete motion analysis by combining video data with inertial measurement unit data, using an improved MS-TCN++ model and multimodal data fusion technology. Prior art methods for analyzing athlete motion based on video data cannot effectively address motion changes in complex environments, and inertial measurement unit data is typically not efficiently fused with video data. By introducing a gated fusion module and a cross-modal residual correction mechanism, this invention precisely controls the fusion method of different feature channels and optimizes and corrects the data, effectively addressing the shortcomings of the prior art.
[0056] Through this method, a badminton player's technical movements, especially his hitting movements, mobility and physical performance, can be accurately evaluated. Due to the use of deep learning algorithms and dynamic adjustment mechanisms, the present invention has the ability to efficiently handle individual differences among different athletes, thereby providing more scientific data support for personalized athlete training and technical improvement. At the same time, through the improved deep learning model, the present invention can handle more complex dynamic motion situations, overcoming the problems of low accuracy and poor adaptability in the existing technology, making technical evaluation more comprehensive and accurate.
[0057] The present invention also has the advantages of strong adaptability and high real-time evaluation. It can be widely used in the real-time monitoring and training of athletes, helping coaches and athletes to better understand their sports technical level and provide a scientific basis for training and competition. These advantages make the present invention a significant improvement over the existing technology, especially in terms of accuracy, personalized evaluation and training feedback. BRIEF DESCRIPTION OF THE DRAWINGS
[0058] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:
[0059] Figure 1 This is a flowchart of a badminton player skill evaluation method based on deep learning proposed by the present invention;
[0060] Figure 2 This is a schematic diagram of the network structure of the improved MS-TCN++ model for the deep learning-based badminton player skill evaluation method proposed in the present invention;
[0061] Figure 3 This is a structural diagram of the action unit boundary prediction module of the badminton player skill evaluation method based on deep learning proposed by the present invention. DETAILED DESCRIPTION
[0062] The present invention will now be described in further detail with reference to the accompanying drawings, which are simplified schematic diagrams that illustrate the basic structure of the present invention in a schematic manner.
[0063] refer to Figure 1-3 , a badminton player skill evaluation method based on deep learning, comprising the following steps:
[0064] S1, collect image sequences and inertial measurement unit data, extract skeleton feature sequences and inertial feature sequences, align them by timestamp, and fuse them into a time series input tensor;
[0065] S2. Feed the time series input tensor into the improved MS-TCN++, replacing the original convolution module with a sparse hybrid gated convolution structure that includes a learnable sparse mask and a gated weight adjustment mechanism.
[0066] S3. Set up a skeleton channel and an inertial channel in the improved MS-TCN++, process the skeleton feature sequence and the inertial feature sequence respectively, and generate the skeleton channel output and the inertial channel output;
[0067] S4. Perform residual modeling on the difference between the skeleton channel output and the inertial channel output, introduce a cross-modal residual correction mechanism, and generate residual correction features;
[0068] S5. Input the residual correction feature into the gated fusion module to generate a stage fusion feature sequence, input it into the boundary prediction module, and output the action unit boundary index sequence, which is divided into action units arranged in time order;
[0069] S6. Based on the annotation information containing the scoring dimension in the training data, a scoring label is generated for each action unit, and the scoring label is used for parameter update calculation during the training process;
[0070] S7. Output the technical evaluation results including the movement ability score, the hitting action score and the physical performance score.
[0071] By fusing video data and inertial measurement unit data, this paper proposes a deep learning-based badminton player skill evaluation method. Through the improved MS-TCN++ model and gated fusion module, the present invention can evaluate the player's movements, including hitting movements, mobility, and physical performance, with high evaluation accuracy, and can provide coaches with accurate training feedback.
[0072] In this embodiment, the image sequence is a continuous frame image acquired at a fixed frame rate, and the inertial measurement unit data includes acceleration, angular velocity and displacement, constructing a multimodal input data set.
[0073] The present invention integrates specific image sequences with inertial measurement unit data and utilizes the sparse hybrid gated convolution structure in the deep learning model to automatically identify and process key features in different modal data, avoiding the tediousness and shortcomings of traditional manual feature extraction and improving the accuracy and efficiency of multimodal data processing.
[0074] In this embodiment, S2 specifically includes:
[0075] S21. Input the time series input tensor into the improved MS-TCN++, where the time series input tensor is a three-dimensional tensor constructed by fusing the skeleton feature sequence and the inertia feature sequence, and includes a feature dimension, a time step dimension, and a channel dimension.
[0076] S22. In the improved MS-TCN++, a sparse hybrid gated convolution structure is used to replace the original convolution module. The sparse hybrid gated convolution structure is a convolution unit that includes both sparse control and channel gating mechanisms.
[0077] S23, the sparse hybrid gated convolution structure includes a learnable sparse mask, where the learnable sparse mask is a binary vector with the same length as the one-dimensional convolution kernel, and each bit indicates whether the convolution operation is enabled at the corresponding time position;
[0078] S24. The sparse hybrid gated convolution structure also includes a gated weight adjustment mechanism, which is composed of a trainable floating-point vector consistent with the number of output channels, and is used to adjust the convolution response strength of each channel.
[0079] The present invention adopts the skeleton channel and inertial channel settings in the improved MS-TCN++ model, so that video data and inertial data can be processed separately and optimized for fusion, which can better extract the multi-dimensional motion characteristics of athletes and provide an efficient and accurate solution for badminton technical evaluation.
[0080] In this embodiment, S3 specifically includes:
[0081] S31. Setting a skeleton channel and an inertial channel in the improved MS-TCN++, wherein the skeleton channel receives a skeleton feature sequence as input, and the inertial channel receives an inertial feature sequence as input;
[0082] S32, skeleton feature sequence is represented as time length T, each time step contains D s The inertial feature sequence is represented by a two-dimensional data matrix with a time length of T and each time step containing D i The two-dimensional data matrix of the feature vector is expanded into feature vector sequences according to the time step.
[0083] S33. Perform one-dimensional convolution operations on the skeleton feature sequence and the inertia feature sequence respectively, and calculate the channel output:
[0084]
[0085] Among them, m represents the channel type, which can be either skeleton channel s or inertial channel i. represents the output vector of the channel at time step t, Represents the feature vector of the input sequence at time step t+k, with a length equal to the input feature dimension, Represents the convolution kernel parameter matrix at the relative displacement k, the number of rows is the channel output feature dimension, the number of columns is the input feature dimension, b (m) Represents the bias vector, whose dimension is equal to the output feature dimension.
[0086] Through residual modeling and cross-modal residual correction, this invention precisely adjusts the deviations and errors between different feature channels, ensuring more accurate data fusion and feature extraction. This innovative technology can effectively improve the accuracy of motion assessment and avoid the error propagation problem in existing technologies.
[0087] In this embodiment, the S4 specifically includes:
[0088] S41, performing residual modeling on the difference between the skeleton channel output and the inertial channel output, where the residual is represented as the time-step difference between the skeleton channel output and the inertial channel output;
[0089] S42. Input the residual into the cross-modal residual correction mechanism. The cross-modal residual correction mechanism includes a residual correction module, an adjustment module, and a feature fusion module. The residual correction module corrects the residual through nonlinear mapping. The adjustment module adjusts the corrected residual according to the adjustment coefficient obtained through training. The feature fusion module combines the corrected residual with the output of the skeleton channel and the inertial channel. The residual correction operation is expressed as:
[0090]
[0091] Among them, R t ′ is the corrected residual, R t is the original cross-modal residual, f(·;θ) is the correction function, which represents the nonlinear mapping learned through training, θ is the trainable parameter of the correction function, and λ is the regularization coefficient, which controls the influence of the regularization term on the residual correction. is the gradient of the residual, which represents the rate of change of the residual relative to the input and is used to measure the change of cross-modal features;
[0092] S43, the corrected residual R t ′ is applied to the output of the skeleton channel and the inertial channel to generate residual correction features.
[0093] The present invention achieves real-time and accurate evaluation of badminton players' movements by combining a gated fusion module with a cross-modal residual correction mechanism. The generated stage fusion feature sequence can effectively improve the accuracy of the action unit boundary index sequence, thereby accurately dividing the action units and providing strong data support for technical analysis.
[0094] In this embodiment, the S5 specifically includes:
[0095] S51. Input the residual correction feature into the gated fusion module. The gated fusion module controls the fusion ratio of the residual correction feature and the skeleton channel feature according to the gating coefficient of each time step:
[0096]
[0097] Among them, Y t is the fusion feature of time step t, W t is the gate coefficient, controlling the residual correction feature R t ′ and skeleton channel characteristics The fusion ratio, R ′ t is the residual correction feature at time step t, is the skeleton channel feature at time step t;
[0098] S52, generate a stage fusion feature sequence, the stage fusion feature sequence is expressed as Y = {Y1, Y2, ..., YT}, where T is the number of time steps, contains the information of each time step, and has temporal dependence;
[0099] S53: Input the stage fusion feature sequence into the boundary prediction module. The boundary prediction module processes the fusion feature through time series modeling to generate an action unit boundary index sequence:
[0100]
[0101] Where B is the action unit boundary index sequence, indicating the boundary position of the action unit, σ(·) is the sigmoid activation function, cos(·) is the cosine similarity function, f(·; θ1) and f(·; θ2) are learning mapping functions based on fusion features, θ1 and θ2 are trainable parameters to optimize the accuracy of boundary prediction;
[0102] S54. Divide the stage fusion feature sequence into action units arranged in time order according to the action unit boundary index sequence.
[0103] This paper combines scoring label generation with loss function optimization to make parameter updates during training more efficient and accurate. By combining L2 loss and cross-entropy loss, the scoring results for each action unit can be calculated more accurately, thereby improving the overall accuracy of technical evaluation.
[0104] In this embodiment, the boundary prediction module specifically includes:
[0105] Feature extraction module, used to extract time series information from the stage fusion feature sequence and generate time series feature representation;
[0106] LSTM network is used to model the temporal feature representation, capture the temporal dependency of input features, and generate temporal dependency features;
[0107] The prediction module is used to generate the action unit boundary index sequence for each time step based on the temporal dependency features output by the LSTM network and determine the location of the action unit boundary;
[0108] The verification module is used to optimize and correct the generated action unit boundary index sequence to ensure the accuracy of the boundary position.
[0109] The present invention can generate accurate scoring data for each player's batting action, mobility and physical performance, providing coaches with a scientific evaluation system to help players quickly identify and improve deficient technical aspects, thereby improving training results.
[0110] In this embodiment, S6 specifically includes:
[0111] S61. Based on the annotation information containing the scoring dimension in the training data, generate a scoring label corresponding to each action unit, wherein the scoring label includes a movement ability score, a hitting action score, and a physical performance score, and associate the scoring label with the corresponding action unit;
[0112] S62. Calculate the loss function of each action unit based on the score label. The loss function consists of L2 loss and cross entropy loss, which represents the difference between the predicted value and the true value:
[0113] L=λ1·L2(y pred ,y true )+λ2·CE(y pred ,y true );
[0114] Among them, L2 is the L2 loss function, which represents the square difference between the predicted value and the true value, CE is the cross entropy loss function, which represents the difference between the probability distributions, and y pred is the predicted value, y true is the true label, λ1 and λ2 are weight coefficients used to balance the various parts of the loss function;
[0115] S63. Calculate the gradient based on the loss function and update the model parameters through the back propagation algorithm to optimize the model and reduce the loss.
[0116] The present invention uses an accurate boundary prediction module and a back-propagation optimization method during training to precisely divide the boundary position of each action unit, further improving the accuracy of action evaluation and the adaptability of the model, making personalized analysis of athletes possible and improving the targeted nature of training.
[0117] Example 1:
[0118] In order to verify the feasibility of the present invention, the method of the present invention was applied to the training process of a badminton club. The training aims to accurately evaluate the technical movements of badminton players. By evaluating the hitting movements, mobility and physical performance, it helps coaches provide players with more targeted training programs. Three players participated in this evaluation training, namely a junior player (player A), an intermediate player (player B) and a senior player (player C). The training movements of each player include standard movements such as forehand strokes, backhand strokes and step movements. All training data are collected synchronously by video equipment and inertial measurement unit (IMU) equipment to ensure the comprehensiveness and accuracy of the data. The frame rate of the video data is 30fps, and the IMU equipment records the physiological data such as the acceleration and angular velocity of the players in real time.
[0119] After data acquisition is completed, the method of the present invention first preprocesses the video data and inertial data, aligns and fuses the data using the technology proposed in the present invention, and generates a time series input tensor. The processed input data is analyzed by the improved MS-TCN++ model, which uses a gated fusion module to effectively fuse the features of video and inertial data, and corrects the data through a cross-modal residual correction mechanism to optimize the feature fusion process of different data channels. Finally, the model accurately divides the time range of each action through the action unit boundary index sequence and extracts the key features of the action unit.
[0120] After completing the data processing, the system generated a batting action score, a movement ability score, and a physical performance score for each player. These scores are calculated based on the player's performance in movement accuracy, pace, movement fluency, and physical recovery demonstrated in training. The evaluation results were further improved by calculating the loss function and optimizing the model using the backpropagation algorithm.
[0121] During training, Player A's strokes were rated 65 / 100, his mobility was 55 / 100, and his physical performance was 60 / 100. The results showed that Player A's strokes were unstable, his pace was slow, and his physical recovery was poor. Player B's strokes were rated 82 / 100, his mobility was 75 / 100, and his physical performance was 72 / 100, indicating that Player B performed well in terms of speed and agility, but still had room for improvement. Player C's strokes were rated 95 / 100, his mobility was 90 / 100, and his physical performance was 88 / 100, indicating that Player C possessed a very high level of technical skill, with accurate strokes, agile pace, and excellent physical performance.
[0122] During the evaluation process, athlete A's inertial data showed significant fluctuations, indicating a lack of fluidity and stability in his movements. Athlete B's acceleration data showed minimal fluctuations, indicating a relatively smooth movement. Athlete C's acceleration and angular velocity data showed smooth and efficient changes, reflecting the efficiency and stability of his movements. Using this evaluation data, coaches can quickly identify each athlete's strengths and weaknesses and develop personalized training plans.
[0123] Compared with traditional manual observation and evaluation methods, the method of the present invention provides a more objective and comprehensive evaluation system, which is significantly superior to traditional methods in terms of evaluation accuracy and time efficiency. In particular, when facing the personalized needs of different players, it can provide more accurate analysis results. The method of the present invention can quickly identify the technical bottlenecks of players and provide strong data support for coaches, thereby helping athletes improve their level in the shortest time.
[0124] The above description is only a preferred specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any technician familiar with the technical field, within the technical scope disclosed by the present invention, who makes equivalent replacements or changes based on the technical solution and inventive concept of the present invention, should be covered by the scope of protection of the present invention.
Claims
1. A badminton player skill evaluation method based on deep learning, characterized in that: The steps include: S1, collect image sequences and inertial measurement unit data, extract skeleton feature sequences and inertial feature sequences, align them by timestamp, and fuse them into a time series input tensor; S2. Feed the time series input tensor into the improved MS-TCN++, replacing the original convolution module with a sparse hybrid gated convolution structure that includes a learnable sparse mask and a gated weight adjustment mechanism. S3. Set up a skeleton channel and an inertial channel in the improved MS-TCN++, process the skeleton feature sequence and the inertial feature sequence respectively, and generate the skeleton channel output and the inertial channel output; S4. Perform residual modeling on the difference between the skeleton channel output and the inertial channel output, introduce a cross-modal residual correction mechanism, and generate residual correction features; S5. Input the residual correction feature into the gated fusion module to generate a stage fusion feature sequence, input it into the boundary prediction module, and output the action unit boundary index sequence, which is divided into action units arranged in time order; S6. Based on the annotation information containing the scoring dimension in the training data, a scoring label is generated for each action unit, and the scoring label is used for parameter update calculation during the training process; S7. Output the technical evaluation results including the movement ability score, the hitting action score and the physical performance score.
2. A badminton player skill evaluation method based on deep learning according to claim 1, characterized in that: The image sequence is a continuous frame image acquired at a fixed frame rate, and the inertial measurement unit data includes acceleration, angular velocity and displacement, constructing a multimodal input data set.
3. The method for evaluating badminton player skills based on deep learning according to claim 1, wherein: The S2 specifically includes: S21. Input the time series input tensor into the improved MS-TCN++, where the time series input tensor is a three-dimensional tensor constructed by fusing the skeleton feature sequence and the inertia feature sequence, and includes a feature dimension, a time step dimension, and a channel dimension. S22. In the improved MS-TCN++, a sparse hybrid gated convolution structure is used to replace the original convolution module. The sparse hybrid gated convolution structure is a convolution unit that includes both sparse control and channel gating mechanisms. S23, the sparse hybrid gated convolution structure includes a learnable sparse mask, where the learnable sparse mask is a binary vector with the same length as the one-dimensional convolution kernel, and each bit indicates whether the convolution operation is enabled at the corresponding time position; S24. The sparse hybrid gated convolution structure also includes a gated weight adjustment mechanism, which is composed of a trainable floating-point vector consistent with the number of output channels, and is used to adjust the convolution response strength of each channel.
4. The method for evaluating badminton player skills based on deep learning according to claim 1, wherein: The S3 specifically includes: S31. Setting a skeleton channel and an inertial channel in the improved MS-TCN++, wherein the skeleton channel receives a skeleton feature sequence as input, and the inertial channel receives an inertial feature sequence as input; S32, skeleton feature sequence is represented as time length T, each time step contains D s The inertial feature sequence is represented by a two-dimensional data matrix with a time length of T and each time step containing D i The two-dimensional data matrix of the feature vector is expanded into feature vector sequences according to the time step. S33. Perform one-dimensional convolution operations on the skeleton feature sequence and the inertia feature sequence respectively, and calculate the channel output: Among them, m represents the channel type, which can be either skeleton channel s or inertial channel i. represents the output vector of the channel at time step t, Represents the feature vector of the input sequence at time step t+k, with a length equal to the input feature dimension, Represents the convolution kernel parameter matrix at the relative displacement k, the number of rows is the channel output feature dimension, the number of columns is the input feature dimension, b (m) Represents the bias vector, whose dimension is equal to the output feature dimension.
5. The method for evaluating badminton player skills based on deep learning according to claim 1, wherein: The S4 specifically includes: S41, performing residual modeling on the difference between the skeleton channel output and the inertial channel output, where the residual is represented as the time-step difference between the skeleton channel output and the inertial channel output; S42. Input the residual into the cross-modal residual correction mechanism. The cross-modal residual correction mechanism includes a residual correction module, an adjustment module, and a feature fusion module. The residual correction module corrects the residual through nonlinear mapping. The adjustment module adjusts the corrected residual according to the adjustment coefficient obtained through training. The feature fusion module combines the corrected residual with the output of the skeleton channel and the inertial channel. The residual correction operation is expressed as: Among them, R t is the corrected residual, R t is the original cross-modal residual, f(·;θ) is the correction function, which means Through training and learning of nonlinear mapping, θ is the trainable parameter of the correction function, and λ is the regularization coefficient, which controls the influence of the regularization term on the residual correction. is the gradient of the residual, which represents the rate of change of the residual relative to the input and is used to measure the change of cross-modal features; S43, the corrected residual R t Applied to the output of the skeleton channel and the inertial channel to generate residual correction features.
6. A badminton player skill evaluation method based on deep learning according to claim 1, characterized in that: The S5 specifically includes: S51. Input the residual correction feature into the gated fusion module. The gated fusion module controls the fusion ratio of the residual correction feature and the skeleton channel feature according to the gating coefficient of each time step: Among them, Y t is the fusion feature of time step t, W t is the gate coefficient, controlling the residual correction feature R t Skeleton channel features The fusion ratio, R ′ t is the residual correction feature at time step t, is the skeleton channel feature at time step t; S52, generate a stage fusion feature sequence, the stage fusion feature sequence is expressed as Y = {Y1, Y2, ..., Y T }, where T is the number of time steps, contains the information of each time step, and has temporal dependence; S53: Input the stage fusion feature sequence into the boundary prediction module. The boundary prediction module processes the fusion feature through time series modeling to generate an action unit boundary index sequence: Where B is the action unit boundary index sequence, indicating the boundary position of the action unit, σ(·) is the sigmoid activation function, cos(·) is the cosine similarity function, f(·; θ1) and f(·; θ2) are learning mapping functions based on fusion features, θ1 and θ2 are trainable parameters to optimize the accuracy of boundary prediction; S54. Divide the stage fusion feature sequence into action units arranged in time order according to the action unit boundary index sequence.
7. A badminton player skill evaluation method based on deep learning according to claim 6, characterized in that: The boundary prediction module specifically includes: Feature extraction module, used to extract time series information from the stage fusion feature sequence and generate time series feature representation; LSTM network is used to model the temporal feature representation, capture the temporal dependency of input features, and generate temporal dependency features; The prediction module is used to generate the action unit boundary index sequence for each time step based on the temporal dependency features output by the LSTM network and determine the location of the action unit boundary; The verification module is used to optimize and correct the generated action unit boundary index sequence to ensure the accuracy of the boundary position.
8. The method for evaluating badminton player skills based on deep learning according to claim 1, wherein: The S6 specifically includes: S61. Based on the annotation information containing the scoring dimension in the training data, generate a scoring label corresponding to each action unit, wherein the scoring label includes a movement ability score, a hitting action score, and a physical performance score, and associate the scoring label with the corresponding action unit; S62. Calculate the loss function of each action unit based on the score label. The loss function consists of L2 loss and cross entropy loss, which represents the difference between the predicted value and the true value: L=λ1·L2(y pred ,y true )+λ2·CE(y pred ,y true ); Among them, L2 is the L2 loss function, which represents the square difference between the predicted value and the true value, CE is the cross entropy loss function, which represents the difference between the probability distributions, and y pred is the predicted value, y true is the true label, λ1 and λ2 are weight coefficients used to balance the various parts of the loss function; S63. Calculate the gradient based on the loss function and update the model parameters through the back propagation algorithm to optimize the model and reduce the loss.