Joint torque prediction method and system based on multiple branches and time-frequency loss
By adopting multi-branch adaptive module and coding module in the joint torque prediction method, combined with a small batch training scheme of time domain and frequency domain loss functions, the problems of insufficient redundant information, high computing resources and dynamic change capture in the existing methods are solved, and more efficient and accurate joint torque prediction is achieved.
Patent Information
- Application Number
- CN202411548021.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-01
- Publication Date
- 2025-03-07
AI Technical Summary
The current joint torque prediction method is prone to generate redundant information in high batch training, has high computing resource requirements, is complex in processing, and is insufficient in capturing transient dynamic change characteristics, which affects prediction efficiency and accuracy.
The joint torque prediction method based on multi-branch adaptive module and encoding module is adopted, and small-batch training and layer normalization are carried out through the loss functions of time domain loss and frequency domain loss, which reduces the complexity of the model, and combines the single-layer encoder structure of the Transformer model to improve the processing effect of the model on long-term dependence.
It effectively improves the efficiency and accuracy of joint torque prediction, reduces computing resource consumption, improves the training stability and convergence speed of the model, and enhances the ability to capture dynamic changes.
Smart Images
Figure CN119564233A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of joint torque prediction, and in particular to a joint torque prediction method and system based on multi-branch and time-frequency loss. Background Art
[0002] With the rapid development of deep learning and neural networks, the prediction of joint torque has been widely used in fields such as prosthetic control and human motion assessment, and has also shown important value in fields such as rehabilitation medicine. However, traditional methods for obtaining joint torque mainly rely on anatomical data and invasive surgical measurements, which limits its application in clinical practice. To meet this challenge, recent inventions tend to predict biological signals by forming overlapping data windows and training at high batch sizes.
[0003] Chang developed the TranSEMG model, which integrates spatial and temporal sEMG features to accurately predict hip joint torque, surpassing the existing dual-stream architecture model; Shepherd used deep convolutional neural networks to develop and verify a machine learning estimator to dynamically control the ankle exoskeleton and enhance walking efficiency at different speeds; Kang developed and verified a gait phase estimator based on convolutional neural networks for real-time control of robotic hip exoskeleton under various gait conditions, significantly improving the accuracy of gait phase estimation; Hossain et al. developed DL-Kinetics-FM-Net, which uses a single IMU sensor to accurately predict hip, knee, and ankle joint torques under various walking conditions, showing significant improvements over traditional deep learning models; Rania Kolaghassi et al. developed a deep learning model to predict the stable gait trajectory of the exoskeleton reference trajectory for children with cerebral palsy. Although these methods have enhanced the ability to process time series data and long-term prediction to a certain extent, the following problems still exist when applied to joint torque prediction:
[0004] 1. High batch training may lead to redundant information in the model training process, especially when the input data contains significant noise, which introduces too much unnecessary information, thereby reducing the utilization efficiency of the input data and affecting the efficiency of joint torque prediction.
[0005] 2. The high demand for computing resources results in a long model reaction time, which is not suitable for joint torque pre-application that requires real-time or near real-time response. This significantly limits its practicality in fields that require rapid decision-making and accurate dynamic prediction, such as biomedicine and rehabilitation technology.
[0006] 3. When processing time series data, the use of overlapping windows enhances the memory capacity of the model, but increases the complexity of model design, especially in parameter adjustment and model structure selection, and complex models will reduce the efficiency of joint torque prediction.
[0007] 4. The existing models are still insufficient in capturing the characteristics of transient dynamic changes, especially in the short period after the sEMG signal is generated, and fail to effectively respond to the rapid changes in muscle activity; and within 200-300 milliseconds after the sEMG signal is generated, the cumulative effect of short-term rapid changes and long-term movement trends will significantly affect the joint torque, which is not fully considered by the traditional model, thereby affecting the accuracy of joint torque prediction.
[0008] Therefore, how to provide a joint torque prediction method and system based on multi-branch and time-frequency loss to improve the efficiency and accuracy of joint torque prediction has become a technical problem that needs to be solved urgently. Summary of the invention
[0009] The technical problem to be solved by the present invention is to provide a joint torque prediction method and system based on multi-branch and time-frequency loss, so as to improve the efficiency and accuracy of joint torque prediction.
[0010] In a first aspect, the present invention provides a joint torque prediction method based on multi-branch and time-frequency loss, comprising the following steps:
[0011] Step S1, obtaining time series data of surface electromyographic signals and joint angles of a large number of joints, and constructing a data set after preprocessing each of the time series data;
[0012] Step S2, creating a joint torque prediction model based on the multi-branch adaptive module and the encoding module, and setting a loss function of the joint torque prediction model based on time domain loss and frequency domain loss;
[0013] Step S3, training the joint torque prediction model using the data set until the loss value of the loss function is less than a preset loss threshold;
[0014] Step S4: using the trained joint torque prediction model to perform joint torque prediction.
[0015] Furthermore, the step S1 is specifically as follows:
[0016] Surface electromyographic signals of a large number of joints and time series data of joint angles corresponding to the surface electromyographic signals are obtained, each time series data is preprocessed by removing outliers and filling missing values, and each preprocessed time series data is labeled with joint torque to construct a data set.
[0017] Further, in step S2, the multi-branch adaptive module is provided with a plurality of adaptive units, each of which is used to extract spatial features from surface electromyographic signals, joint angles, or a combination of surface electromyographic signals and joint angles, and transmit the spatial features to the encoding module;
[0018] Each of the adaptive units is composed of a depthwise separable convolution and a first multi-layer perceptron; the depthwise separable convolution is used to extract spatial sub-features from each channel of the input time series data; the first multi-layer perceptron is used to learn the importance of each channel at different time points to generate adaptive weights of each of the spatial sub-features, and then perform weighted summation of each of the spatial sub-features based on each of the adaptive weights to obtain a spatial feature;
[0019] The encoding module is constructed based on a single-layer encoder structure of the Transformer model, and a second multi-layer perceptron is used to replace the multi-head attention mechanism of the Transformer model; the encoding module is used to capture the nonlinear relationship and long-distance dependency of the spatial features output by each adaptive unit to obtain local features, and fuse each of the local features to obtain a fused feature, and output the joint torque prediction result based on the fused feature.
[0020] Furthermore, in step S2, the formula of the loss function is:
[0021] Total loss =F loss +T loss ;
[0022]
[0023] Frequency domain =Abs(FFT(Error));
[0024] Error = Prediction - Target;
[0025] T loss =W T ×Mean(Error 2 );
[0026] Among them, Total loss Represents the loss value of the loss function; F loss represents the frequency domain loss; T loss represents the time domain loss; W F represents the frequency domain weight; W T Represents the time domain weight; Mean() represents the average function; Frequency domainRepresents frequency domain features; Abs() represents absolute value function; FFT() represents Fourier transform function; Error represents the error between the predicted joint torque result and the true value of the joint torque; Prediction represents the predicted joint torque result; Target represents the true value of the joint torque.
[0027] Furthermore, the step S3 is specifically as follows:
[0028] The data set is divided into a training set and a validation set based on a preset ratio. The joint torque prediction model is trained using the training set. During the training process, a preset batch threshold and layer normalization are combined until the loss value of the loss function is less than a preset loss threshold. The trained joint torque prediction model is then verified using the validation set. If the verification is passed, the training is terminated. If the verification fails, the training set is expanded to continue the training.
[0029] In a second aspect, the present invention provides a joint torque prediction system based on multi-branch and time-frequency loss, comprising the following modules:
[0030] A data set construction module is used to obtain the surface electromyographic signals of a large number of joints and the time series data of the joint angles, and to construct a data set after preprocessing each of the time series data;
[0031] A joint torque prediction model creation module, used to create a joint torque prediction model based on the multi-branch adaptive module and the encoding module, and set the loss function of the joint torque prediction model based on the time domain loss and the frequency domain loss;
[0032] A joint torque prediction model training module, used for training the joint torque prediction model through the data set until the loss value of the loss function is less than a preset loss threshold;
[0033] The joint torque prediction module is used to perform joint torque prediction using the trained joint torque prediction model.
[0034] Furthermore, the data set construction module is specifically used for:
[0035] Surface electromyographic signals of a large number of joints and time series data of joint angles corresponding to the surface electromyographic signals are obtained, each time series data is preprocessed by removing outliers and filling missing values, and each preprocessed time series data is labeled with joint torque to construct a data set.
[0036] Further, in the joint torque prediction model creation module, the multi-branch adaptive module is provided with a plurality of adaptive units, each of the adaptive units is used to extract spatial features from surface electromyographic signals, joint angles, or a combination of surface electromyographic signals and joint angles, and transmit the spatial features to the encoding module;
[0037] Each of the adaptive units is composed of a depthwise separable convolution and a first multi-layer perceptron; the depthwise separable convolution is used to extract spatial sub-features from each channel of the input time series data; the first multi-layer perceptron is used to learn the importance of each channel at different time points to generate adaptive weights of each of the spatial sub-features, and then perform weighted summation of each of the spatial sub-features based on each of the adaptive weights to obtain a spatial feature;
[0038] The encoding module is constructed based on a single-layer encoder structure of the Transformer model, and a second multi-layer perceptron is used to replace the multi-head attention mechanism of the Transformer model; the encoding module is used to capture the nonlinear relationship and long-distance dependency of the spatial features output by each adaptive unit to obtain local features, and fuse each of the local features to obtain a fused feature, and output the joint torque prediction result based on the fused feature.
[0039] Furthermore, in the joint torque prediction model creation module, the formula of the loss function is:
[0040] Total loss =F loss +T loss ;
[0041]
[0042] Frequency domain =Abs(FFT(Error));
[0043] Error = Prediction - Target;
[0044] T loss =W T ×Mean(Error 2 );
[0045] Among them, Total loss Represents the loss value of the loss function; F loss represents the frequency domain loss; T loss represents the time domain loss; W F represents the frequency domain weight; W T Represents the time domain weight; Mean() represents the average function; Frequency domain Represents frequency domain features; Abs() represents absolute value function; FFT() represents Fourier transform function; Error represents the error between the predicted joint torque result and the true value of the joint torque; Prediction represents the predicted joint torque result; Target represents the true value of the joint torque.
[0046] Furthermore, the joint torque prediction model training module is specifically used for:
[0047] The data set is divided into a training set and a validation set based on a preset ratio. The joint torque prediction model is trained using the training set. During the training process, a preset batch threshold and layer normalization are combined until the loss value of the loss function is less than a preset loss threshold. The trained joint torque prediction model is then verified using the validation set. If the verification is passed, the training is terminated. If the verification fails, the training set is expanded to continue the training.
[0048] The advantages of the present invention are:
[0049] By acquiring a large number of surface electromyographic signals of joints and time series data of joint angles, each time series data is preprocessed to construct a data set; then a joint torque prediction model is created based on a multi-branch adaptive module and a coding module, and the loss function of the joint torque prediction model is set based on time domain loss and frequency domain loss. The joint torque prediction model is trained through the data set until the loss value of the loss function is less than the preset loss threshold, and finally the trained joint torque prediction model is used to predict the joint torque; since the joint torque prediction model is trained in small batches in combination with a preset batch threshold during training, redundant information is avoided during training, and the utilization efficiency of the data set is effectively improved, so as to improve the training speed of the joint torque prediction model; through the training process, the joint torque prediction model is trained in small batches in combination with the preset batch threshold, so as to avoid the generation of redundant information during training, and effectively improve the utilization efficiency of the data set, so as to improve the training speed of the joint torque prediction model; In the process, batch threshold and layer normalization are combined to effectively improve training stability and accelerate convergence speed; the encoding module is constructed through the single-layer encoder structure of the Transformer model, that is, the decoder is removed from the Transformer model, which effectively reduces the complexity of the model and reduces the consumption of computing resources; the traditional mean square error only focuses on time domain information, and fails to fully consider the impact of frequency domain information on dynamic changes. The high-frequency component is related to the rapid change of joint torque, and the low-frequency component corresponds to the slower change. Therefore, the loss function of the joint torque prediction model is set by time domain loss and frequency domain loss, that is, the time domain information and frequency domain information are comprehensively considered to improve the accuracy of joint torque prediction, which ultimately greatly improves the efficiency and accuracy of joint torque prediction. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] The present invention will be further described below in conjunction with embodiments with reference to the accompanying drawings.
[0051] Figure 1 It is a flow chart of a joint torque prediction method based on multi-branch and time-frequency loss of the present invention.
[0052] Figure 2 It is a structural schematic diagram of a joint torque prediction system based on multi-branch and time-frequency loss of the present invention.
[0053] Figure 3 It is a schematic diagram of the process of joint torque prediction of the present invention.
[0054] Figure 4 Schematic diagram of the multi-branch adaptive module of the present invention.
[0055] Figure 5 It is a schematic diagram of the encoding module of the present invention. DETAILED DESCRIPTION
[0056] The technical solution in the embodiments of the present application has the following overall idea: during the training process of the joint torque prediction model, small batch training is performed in combination with a preset batch threshold to avoid redundant information during the training process; by combining batch thresholds and layer normalization during the training process, the training stability is effectively improved and the convergence speed is accelerated; the encoding module is constructed through the single-layer encoder structure of the Transformer model to effectively reduce the model complexity and reduce the consumption of computing resources; the loss function of the joint torque prediction model is set through time domain loss and frequency domain loss, and the time domain information and frequency domain information are comprehensively considered to improve the efficiency and accuracy of joint torque prediction.
[0057] Please refer to Figures 1 to 5 As shown, a preferred embodiment of the joint torque prediction method based on multi-branch and time-frequency loss of the present invention comprises the following steps:
[0058] Step S1, obtaining time series data of surface electromyographic signals and joint angles of a large number of joints, and constructing a data set after preprocessing each of the time series data;
[0059] Step S2, creating a joint torque prediction model based on a multi-branch adaptive module (MAE) and a coding module (SFT), setting a loss function (HTFLoss) of the joint torque prediction model based on time domain loss and frequency domain loss, so as to better integrate time domain information and frequency domain information, enhance the joint torque feature structure, and smoothly predict the transition state, so as to improve the prediction quality and dynamic response capability;
[0060] Step S3, training the joint torque prediction model using the data set until the loss value of the loss function is less than a preset loss threshold;
[0061] Step S4: using the trained joint torque prediction model to perform joint torque prediction.
[0062] The present invention introduces a multi-branch adaptive module of a multi-branch input strategy (several adaptive units) to dynamically allocate weights of different surface electromyographic signals (sEMG) and joint angles, thereby optimizing the processing of small sample data. At the same time, a coding module for simplified features is designed, namely, a multi-head attention mechanism of a Transformer model is replaced by a second multi-layer perceptron and the decoder is removed, thereby improving small batch training and long-term prediction capabilities. The present invention also combines a loss function of a mixed time domain loss and a frequency domain loss to significantly enhance the model's processing effect on long-term dependencies, thereby significantly improving prediction accuracy and generalization capabilities.
[0063] The step S1 is specifically as follows:
[0064] Surface electromyographic signals of a large number of joints and time series data of joint angles corresponding to the surface electromyographic signals are obtained, each time series data is preprocessed by removing outliers and filling missing values, and each preprocessed time series data is labeled with joint torque to construct a data set.
[0065] In step S2, the multi-branch adaptive module is provided with a plurality of adaptive units, each of which is used to extract spatial features from surface electromyographic signals, joint angles, or a combination of surface electromyographic signals and joint angles, and transmit the spatial features to an encoding module;
[0066] Each of the adaptive units is composed of a depthwise separable convolution (DSC) and a first multi-layer perceptron (MLP); the depthwise separable convolution is used to extract spatial sub-features from each channel of the input time series data, which can reduce the complexity of model parameters and enhance the flexibility of feature extraction; the first multi-layer perceptron is used to learn the importance of each channel at different time points to generate adaptive weights of each of the spatial sub-features, and then perform weighted summation of each of the spatial sub-features based on each of the adaptive weights to obtain spatial features;
[0067] The calculation formula of the depth-wise separable convolution is:
[0068] X out =ReLU(BN(PC(ReLU(BN(DC(X))))));
[0069] Among them, X out Represents the output of depth-wise separable convolution; X represents the input of depth-wise separable convolution; ReLU() represents the activation function; BN() represents batch normalization; PC() represents point convolution operation; DC() represents depth-wise convolution operation;
[0070] The encoding module is constructed based on the single-layer encoder structure of the Transformer model, and a second multi-layer perceptron is used to replace the multi-head attention mechanism of the Transformer model; the encoding module is used to capture the nonlinear relationship and long-distance dependency of the spatial features output by each adaptive unit to obtain local features, and fuse the local features to obtain fusion features, and output the joint torque prediction results based on the fusion features. The second multi-layer perceptron flexibly learns the relationship between input sequence features by compressing and expanding the input dimension, effectively captures the complex nonlinear relationship and long-distance dependency in the biological signal, focuses on capturing the local features of the surface electromyography signal and the joint angle at different time intervals, and enhances the modeling ability of the biological signal.
[0071] In step S2, the formula of the loss function is:
[0072] Total loss =F loss +T loss ;
[0073]
[0074] Frequency domain =Abs(FFT(Error));
[0075] Error = Prediction - Target;
[0076] T loss =W T ×Mean(Error 2 );
[0077] Among them, Total loss Represents the loss value of the loss function; F loss represents the frequency domain loss; T loss represents the time domain loss; W F represents the frequency domain weight, with a value range of [0.6, 0.9], which is used to emphasize the impact of frequency domain errors on the overall loss; W T Represents the time domain weight, the value range is [0.1,0.4]; Mean() represents the average function; Frequency domain Represents frequency domain features; Abs() represents absolute value function; FFT() represents Fourier transform function; Error represents the error between the predicted joint torque result and the true value of the joint torque; Prediction represents the predicted joint torque result; Target represents the true value of the joint torque.
[0078] The calculation process of the loss function is as follows: calculate the error between the predicted joint torque and the true value of the joint torque, decompose the error into frequency domain features using fast Fourier transform, assign weights to the losses in the time domain and frequency domain respectively, and finally add the two losses to obtain the total loss. This method can better capture the local characteristics of joint motion, improve the prediction ability of the model in dynamic motion tasks, and ensure that significant changes in joint dynamics will not be ignored.
[0079] The step S3 is specifically as follows:
[0080] The data set is divided into a training set and a validation set based on a preset ratio. The joint torque prediction model is trained with the training set. During the training process, small batch training and layer normalization (LayerNorm) are performed in combination with a preset batch threshold until the loss value of the loss function is less than the preset loss threshold. The trained joint torque prediction model is then verified with the validation set. If the verification is passed, the training is terminated. If the verification fails, the training set is expanded to continue the training.
[0081] A preferred embodiment of a joint torque prediction system based on multi-branch and time-frequency loss of the present invention includes the following modules:
[0082] A data set construction module is used to obtain the surface electromyographic signals of a large number of joints and the time series data of the joint angles, and to construct a data set after preprocessing each of the time series data;
[0083] A joint torque prediction model creation module is used to create a joint torque prediction model based on a multi-branch adaptive module (MAE) and a coding module (SFT), and to set a loss function (HTFLoss) of the joint torque prediction model based on time domain loss and frequency domain loss, so as to better integrate time domain information and frequency domain information, enhance the joint torque feature structure, and smoothly predict the transition state, so as to improve the prediction quality and dynamic response capability;
[0084] A joint torque prediction model training module, used for training the joint torque prediction model through the data set until the loss value of the loss function is less than a preset loss threshold;
[0085] The joint torque prediction module is used to perform joint torque prediction using the trained joint torque prediction model.
[0086] The present invention introduces a multi-branch adaptive module of a multi-branch input strategy (several adaptive units) to dynamically allocate weights of different surface electromyographic signals (sEMG) and joint angles, thereby optimizing the processing of small sample data. At the same time, a coding module for simplified features is designed, namely, a multi-head attention mechanism of a Transformer model is replaced by a second multi-layer perceptron and the decoder is removed, thereby improving small batch training and long-term prediction capabilities. The present invention also combines a loss function of a mixed time domain loss and a frequency domain loss to significantly enhance the model's processing effect on long-term dependencies, thereby significantly improving prediction accuracy and generalization capabilities.
[0087] The data set construction module is specifically used for:
[0088] Surface electromyographic signals of a large number of joints and time series data of joint angles corresponding to the surface electromyographic signals are obtained, each time series data is preprocessed by removing outliers and filling missing values, and each preprocessed time series data is labeled with joint torque to construct a data set.
[0089] In the joint torque prediction model creation module, the multi-branch adaptive module is provided with a plurality of adaptive units, each of which is used to extract spatial features from surface electromyographic signals, joint angles, or a combination of surface electromyographic signals and joint angles, and transmit the spatial features to the encoding module;
[0090] Each of the adaptive units is composed of a depthwise separable convolution (DSC) and a first multi-layer perceptron (MLP); the depthwise separable convolution is used to extract spatial sub-features from each channel of the input time series data, which can reduce the complexity of model parameters and enhance the flexibility of feature extraction; the first multi-layer perceptron is used to learn the importance of each channel at different time points to generate adaptive weights of each of the spatial sub-features, and then perform weighted summation of each of the spatial sub-features based on each of the adaptive weights to obtain spatial features;
[0091] The calculation formula of the depth-wise separable convolution is:
[0092] X out =ReLU(BN(PC(ReLU(BN(DC(X))))));
[0093] Among them, X out Represents the output of depth-wise separable convolution; X represents the input of depth-wise separable convolution; ReLU() represents the activation function; BN() represents batch normalization; PC() represents point convolution operation; DC() represents depth-wise convolution operation;
[0094] The encoding module is constructed based on the single-layer encoder structure of the Transformer model, and a second multi-layer perceptron is used to replace the multi-head attention mechanism of the Transformer model; the encoding module is used to capture the nonlinear relationship and long-distance dependency of the spatial features output by each adaptive unit to obtain local features, and fuse the local features to obtain fusion features, and output the joint torque prediction results based on the fusion features. The second multi-layer perceptron flexibly learns the relationship between input sequence features by compressing and expanding the input dimension, effectively captures the complex nonlinear relationship and long-distance dependency in the biological signal, focuses on capturing the local features of the surface electromyography signal and the joint angle at different time intervals, and enhances the modeling ability of the biological signal.
[0095] In the joint torque prediction model creation module, the formula of the loss function is:
[0096] Total loss =F loss +T loss ;
[0097]
[0098] Frequency domain =Abs(FFT(Error));
[0099] Error = Prediction - Target;
[0100] T loss =W T ×Mean(Error 2 );
[0101] Among them, Total loss Represents the loss value of the loss function; F loss represents the frequency domain loss; T loss represents the time domain loss; W F , the value range is [0.6,0.9], which is used to emphasize the impact of frequency domain errors on the overall loss; W T Represents the time domain weight, the value range is [0.1,0.4]; Mean() represents the average function; Frequency domain Represents frequency domain features; Abs() represents absolute value function; FFT() represents Fourier transform function; Error represents the error between the predicted joint torque result and the true value of the joint torque; Prediction represents the predicted joint torque result; Target represents the true value of the joint torque.
[0102] The calculation process of the loss function is as follows: calculate the error between the predicted joint torque and the true value of the joint torque, decompose the error into frequency domain features using fast Fourier transform, assign weights to the losses in the time domain and frequency domain respectively, and finally add the two losses to obtain the total loss. This method can better capture the local characteristics of joint motion, improve the prediction ability of the model in dynamic motion tasks, and ensure that significant changes in joint dynamics will not be ignored.
[0103] The joint torque prediction model training module is specifically used for:
[0104] The data set is divided into a training set and a validation set based on a preset ratio. The joint torque prediction model is trained with the training set. During the training process, small batch training and layer normalization (LayerNorm) are performed in combination with a preset batch threshold until the loss value of the loss function is less than the preset loss threshold. The trained joint torque prediction model is then verified with the validation set. If the verification is passed, the training is terminated. If the verification fails, the training set is expanded to continue the training.
[0105] In summary, the advantages of the present invention are:
[0106] By acquiring a large number of surface electromyographic signals of joints and time series data of joint angles, each time series data is preprocessed to construct a data set; then a joint torque prediction model is created based on a multi-branch adaptive module and a coding module, and the loss function of the joint torque prediction model is set based on time domain loss and frequency domain loss. The joint torque prediction model is trained through the data set until the loss value of the loss function is less than the preset loss threshold, and finally the trained joint torque prediction model is used to predict the joint torque; since the joint torque prediction model is trained in small batches in combination with a preset batch threshold during training, redundant information is avoided during training, and the utilization efficiency of the data set is effectively improved, so as to improve the training speed of the joint torque prediction model; through the training process, the joint torque prediction model is trained in small batches in combination with the preset batch threshold, so as to avoid the generation of redundant information during training, and effectively improve the utilization efficiency of the data set, so as to improve the training speed of the joint torque prediction model; In the process, batch threshold and layer normalization are combined to effectively improve training stability and accelerate convergence speed; the encoding module is constructed through the single-layer encoder structure of the Transformer model, that is, the decoder is removed from the Transformer model, which effectively reduces the complexity of the model and reduces the consumption of computing resources; the traditional mean square error only focuses on time domain information, and fails to fully consider the impact of frequency domain information on dynamic changes. The high-frequency component is related to the rapid change of joint torque, and the low-frequency component corresponds to the slower change. Therefore, the loss function of the joint torque prediction model is set by time domain loss and frequency domain loss, that is, the time domain information and frequency domain information are comprehensively considered to improve the accuracy of joint torque prediction, which ultimately greatly improves the efficiency and accuracy of joint torque prediction.
[0107] Although the specific implementation modes of the present invention are described above, those skilled in the art should understand that the specific implementation modes described are only illustrative and are not intended to limit the scope of the present invention. Equivalent modifications and changes made by those skilled in the art in accordance with the spirit of the present invention should be included in the scope of protection of the claims of the present invention.
Claims
1. A joint torque prediction method based on multi-branch and time-frequency loss, characterized in that: The steps include: Step S1, obtaining time series data of surface electromyographic signals and joint angles of a large number of joints, and constructing a data set after preprocessing each of the time series data; Step S2, creating a joint torque prediction model based on the multi-branch adaptive module and the encoding module, and setting a loss function of the joint torque prediction model based on time domain loss and frequency domain loss; Step S3, training the joint torque prediction model using the data set until the loss value of the loss function is less than a preset loss threshold; Step S4: Use the trained joint torque prediction model to predict joint torque.
2. The method for predicting joint torque based on multi-branch and time-frequency loss according to claim 1, characterized in that: The step S1 is specifically as follows: Surface electromyographic signals of a large number of joints and time series data of joint angles corresponding to the surface electromyographic signals are obtained, each time series data is preprocessed by removing outliers and filling missing values, and each preprocessed time series data is labeled with joint torque to construct a data set.
3. The joint torque prediction method based on multi-branch and time-frequency loss according to claim 1, characterized in that: In step S2, the multi-branch adaptive module is provided with a plurality of adaptive units, each of which is used to extract spatial features from surface electromyographic signals, joint angles, or a combination of surface electromyographic signals and joint angles, and transmit the spatial features to an encoding module; Each of the adaptive units is composed of a depthwise separable convolution and a first multi-layer perceptron; the depthwise separable convolution is used to extract spatial sub-features from each channel of the input time series data; the first multi-layer perceptron is used to learn the importance of each channel at different time points to generate adaptive weights of each of the spatial sub-features, and then perform weighted summation of each of the spatial sub-features based on each of the adaptive weights to obtain a spatial feature; The encoding module is constructed based on a single-layer encoder structure of the Transformer model, and a second multi-layer perceptron is used to replace the multi-head attention mechanism of the Transformer model; the encoding module is used to capture the nonlinear relationship and long-distance dependency of the spatial features output by each adaptive unit to obtain local features, and fuse each of the local features to obtain a fused feature, and output the joint torque prediction result based on the fused feature.
4. The method for predicting joint torque based on multi-branch and time-frequency loss according to claim 1, characterized in that: In step S2, the formula of the loss function is: Total loss =F loss +T loss ; Frequency domain =Abs(FFT(Error)); Error = Prediction - Target; T loss =W T ×Mean(Error 2 ); Among them, Total loss Represents the loss value of the loss function; F loss represents the frequency domain loss; T loss represents the time domain loss; W F represents the frequency domain weight; W T Represents the time domain weight; Mean() represents the average function; Frequency domain Represents frequency domain features; Abs() represents absolute value function; FFT() represents Fourier transform function; Error represents the error between the predicted joint torque result and the true value of the joint torque; Prediction represents the predicted joint torque result; Target represents the true value of the joint torque.
5. The method for predicting joint torque based on multi-branch and time-frequency loss according to claim 1, characterized in that: The step S3 is specifically as follows: The data set is divided into a training set and a validation set based on a preset ratio. The joint torque prediction model is trained using the training set. During the training process, a preset batch threshold and layer normalization are combined until the loss value of the loss function is less than a preset loss threshold. The trained joint torque prediction model is then verified using the validation set. If the verification is passed, the training is terminated. If the verification fails, the training set is expanded to continue the training.
6. A joint torque prediction system based on multi-branch and time-frequency loss, characterized in that: Includes the following modules: A data set construction module is used to obtain the surface electromyographic signals of a large number of joints and the time series data of the joint angles, and to construct a data set after preprocessing each of the time series data; A joint torque prediction model creation module, used to create a joint torque prediction model based on the multi-branch adaptive module and the encoding module, and set the loss function of the joint torque prediction model based on the time domain loss and the frequency domain loss; A joint torque prediction model training module, used for training the joint torque prediction model through the data set until the loss value of the loss function is less than a preset loss threshold; The joint torque prediction module is used to perform joint torque prediction using the trained joint torque prediction model.
7. A joint torque prediction system based on multi-branch and time-frequency loss as claimed in claim 6, characterized in that: The data set construction module is specifically used for: Surface electromyographic signals of a large number of joints and time series data of joint angles corresponding to the surface electromyographic signals are obtained, each time series data is preprocessed by removing outliers and filling missing values, and each preprocessed time series data is labeled with joint torque to construct a data set.
8. The joint torque prediction system based on multi-branch and time-frequency loss according to claim 6, characterized in that: In the joint torque prediction model creation module, the multi-branch adaptive module is provided with a plurality of adaptive units, each of which is used to extract spatial features from surface electromyographic signals, joint angles, or a combination of surface electromyographic signals and joint angles, and transmit the spatial features to the encoding module; Each of the adaptive units is composed of a depthwise separable convolution and a first multi-layer perceptron; the depthwise separable convolution is used to extract spatial sub-features from each channel of the input time series data; the first multi-layer perceptron is used to learn the importance of each channel at different time points to generate adaptive weights of each of the spatial sub-features, and then perform weighted summation of each of the spatial sub-features based on each of the adaptive weights to obtain a spatial feature; The encoding module is constructed based on a single-layer encoder structure of the Transformer model, and a second multi-layer perceptron is used to replace the multi-head attention mechanism of the Transformer model; the encoding module is used to capture the nonlinear relationship and long-distance dependency of the spatial features output by each adaptive unit to obtain local features, and fuse each of the local features to obtain a fused feature, and output the joint torque prediction result based on the fused feature.
9. The joint torque prediction system based on multi-branch and time-frequency loss according to claim 6, characterized in that: In the joint torque prediction model creation module, the formula of the loss function is: Total loss =F loss +T loss ; Frequency domain =Abs(FFT(Error)); Error = Prediction - Target; T loss =W T ×Mean(Error 2 ); Among them, Total loss Represents the loss value of the loss function; F loss represents the frequency domain loss; T loss represents the time domain loss; W F represents the frequency domain weight; W T Represents the time domain weight; Mean() represents the average function; Frequency domain Represents frequency domain features; Abs() represents absolute value function; FFT() represents Fourier transform function; Error represents the error between the predicted joint torque result and the true value of the joint torque; Prediction represents the predicted joint torque result; Target represents the true value of the joint torque.
10. The joint torque prediction system based on multi-branch and time-frequency loss according to claim 6, characterized in that: The joint torque prediction model training module is specifically used for: The data set is divided into a training set and a validation set based on a preset ratio. The joint torque prediction model is trained using the training set. During the training process, a preset batch threshold and layer normalization are combined until the loss value of the loss function is less than a preset loss threshold. The trained joint torque prediction model is then verified using the validation set. If the verification is passed, the training is terminated. If the verification fails, the training set is expanded to continue the training.