A gesture trajectory prediction method based on motion intention extraction

Through the gesture trajectory prediction method based on motion intention extraction, a convolutional neural network is used to establish a relationship model between electromyography and hand movements, which solves the problem that traditional prosthetic control is difficult to achieve complex movements and response delays, and improves the motion accuracy of prosthetic limb and adaptability to response speeds, enabling natural and precise hand operation.

CN118732861BActive Publication Date: 2024-11-22HEFEI UNIV OF TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411225397.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-09-03
Publication Date
2024-11-22
Estimated Expiration
2044-09-03

AI Technical Summary

Technical Problem

Traditional prosthetic control is difficult to achieve complex and fine hand movements, and the delay in response leads to a time difference between user operation and prosthetic movements, affecting the user experience and operation fluency, and it is difficult to adapt to individual different EMG signals and exercise habits.

Method used

Using gesture trajectory prediction method based on motion intention extraction, the electromyography signal is collected through electromyography sensors, preprocessing and feature extraction, and a convolutional neural network algorithm is used to establish a relationship model between electromyography and hand movements to predict users in real time. The hand movement intention is to control the prosthetic movement according to the prediction results, and the response speed correction is performed through the feedback data of multiple sensors during the actual control process.

Benefits of technology

It improves the accuracy of prosthetic movement and the accuracy of response speed, reduces the user's operating burden, adapts to individual differences, and achieves natural and precise hand movements.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118732861B_ABST
    Figure CN118732861B_ABST
Patent Text Reader

Abstract

The present invention discloses a gesture trajectory prediction method based on motion intention extraction, which relates to the field of prosthetic trajectory control technology. The method extracts relevant features from preprocessed electromyographic signals, trains the extracted relevant features through a convolutional neural network algorithm, establishes a relationship model between electromyographic signals and hand movements, collects the electromyographic signals of the user's arm in real time, and inputs them into the trained relationship model after preprocessing and extracting relevant features, predicts the user's hand movement intention, and controls the movement of the prosthesis according to the predicted hand movement intention. The prediction system preprocesses the electromyographic signals collected from the arm through machine learning, predicts the hand movement through the electromyographic signals and establishes a relationship, and subsequently controls the movement of the prosthesis through the electromyographic signals, which not only enables the prosthesis to respond in advance, but also effectively improves the accuracy of the prosthetic movement.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of prosthetic trajectory control, and in particular to a gesture trajectory prediction method based on motion intention extraction. Background Art

[0002] A prosthesis is a device used to replace part or all of the lost limb function. It is mainly used to help amputees recover part or all of their limb function. The development of prosthetic technology has evolved from simple mechanical prostheses to modern intelligent prostheses. With the advancement of science and technology, the functions and performance of prostheses are constantly improving.

[0003] The prior art has the following defects:

[0004] 1. The control of traditional prostheses usually relies on simple mechanical structures or basic electric control, which makes it difficult to accurately perform complex and delicate hand movements. In addition, the response delay of the prosthesis will cause a time difference between the user's operation and the prosthetic movement, affecting the user experience and the smoothness of the operation, making it difficult to achieve natural hand movements;

[0005] 2. Since each user’s electromyographic signals and movement habits are different, traditional prostheses are difficult to adapt to these individual differences, resulting in unsatisfactory control effects.

[0006] Based on this, the present invention proposes a gesture trajectory prediction method based on motion intention extraction. The electromyographic signals collected from the arm are preprocessed through machine learning, and the hand movements are predicted and a relationship is established through the electromyographic signals. Subsequently, the electromyographic signals are used to control the movements of the prosthesis, which not only enables the prosthesis to respond in advance, but also effectively improves the accuracy of the prosthetic movement. Summary of the invention

[0007] The purpose of the present invention is to provide a gesture trajectory prediction method based on motion intention extraction to solve the shortcomings of the background technology.

[0008] In order to achieve the above object, the present invention provides the following technical solution: a gesture trajectory prediction method based on motion intention extraction, the prediction method comprising the following steps:

[0009] Use electromyographic sensors to collect electromyographic signals from the user's arm, record muscle activity data, filter and denoise the collected electromyographic signals, and extract relevant features from the pre-processed electromyographic signals;

[0010] The extracted relevant features are trained through the convolutional neural network algorithm to establish a relationship model between electromyographic signals and hand movements. The electromyographic signals of the user's arm are collected in real time, and after preprocessing and extracting relevant features, they are input into the trained relationship model to predict the user's hand movement intention and control the movement of the prosthesis according to the predicted hand movement intention.

[0011] In the actual control process, by analyzing the feedback data from multiple types of sensors, it is determined whether the response speed of the prosthesis needs to be corrected in real time, and the corresponding control strategy is generated based on the judgment results.

[0012] In a preferred embodiment, extracting relevant features from the preprocessed electromyographic signal comprises the following steps:

[0013] A segment of electromyographic signal data is collected, and the relevant features include time domain features, frequency domain features and time-frequency domain features. The extracted time domain features include the waveform length and root mean square value of the electromyographic signal, the extracted frequency domain features include the frequency band energy and frequency center of the electromyographic signal, and the extracted time-frequency domain features include the short-time Fourier transform and wavelet transform of the electromyographic signal.

[0014] In a preferred embodiment, the extracted relevant features are trained by a convolutional neural network algorithm to establish a relationship model between electromyographic signals and hand movements, including the following steps:

[0015] The acquired root mean square value, waveform length, frequency band energy, frequency center, short-time Fourier transform, and wavelet transform are all normalized, and the normalized root mean square value, waveform length, frequency band energy, frequency center, short-time Fourier transform, and wavelet transform are annotated to form a label data set;

[0016] After the convolutional neural network model is built, the gap between the predicted value and the true value is calculated by the mean square error, and the network weight is updated through the optimization algorithm SGD. The convolutional neural network model is iteratively trained using the training data set, and the weights are continuously adjusted and the loss function value is reduced. After the convolutional neural network model training is completed, the performance of the convolutional neural network model is evaluated through the verification data set, and the hyperparameter adjustment is optimized. Finally, the test data set is used to verify that the generalization ability of the convolutional neural network model meets the standard, and the relationship model between the electromyographic signal and the hand movement is output.

[0017] In a preferred embodiment, in the actual control process, by analyzing the feedback data of multiple types of sensors, it is determined whether the response speed of the prosthesis needs to be corrected in real time, and a corresponding control strategy is generated according to the judgment result, including the following steps:

[0018] After the prosthetic action is completed, the position error, speed error and fatigue index are obtained, and the correction coefficient is obtained by comprehensive calculation of the position error, speed error and fatigue index. The expression is: , where is the correction factor, is the position error, is the speed error, is the fatigue index, , are the proportional coefficients of position error and velocity error respectively, and , All are greater than 0;

[0019] If the correction coefficient is less than or equal to the second coefficient threshold, and the correction coefficient is greater than or equal to the first coefficient threshold, it is judged that there is no need to perform real-time correction on the response speed of the prosthesis. If the correction coefficient is greater than the second coefficient threshold, it is judged that the response speed of the prosthesis needs to be reduced. If the correction coefficient is less than the second coefficient threshold, it is judged that the response speed of the prosthesis needs to be increased. The correction algorithm expression is:

[0020]

[0021] In the formula, is the response speed after correction, is the response speed before correction, is the correction factor, is the first coefficient threshold, is the second coefficient threshold.

[0022] In a preferred embodiment, the actual position of the prosthesis is monitored by a position sensor, and the position error of the prosthesis is calculated. , , where For prosthesis in time The actual location at the moment, For prosthesis in time The predicted position at the moment;

[0023] The actual speed of the prosthesis movement is obtained based on the actual position. The expression is:

[0024] , the expected speed is: , where is the actual speed, is the expected speed, For time The time derivative of the actual position of the prosthesis at time , For time The time derivative of the expected position of the prosthesis at time t, then the velocity error The expression is: ;

[0025] Calculate the fatigue waveform length, the expression is: , where is the fatigue waveform length, is the number of electromyographic signal data, For the The electromyographic signal value, For the The electromyographic signal values ​​are accumulated and calculated during the entire action period of the prosthesis to obtain the fatigue index, which is expressed as: , where is the fatigue index, During the movement of the prosthesis, The length of the fatigue waveform at the time, The time it takes for the prosthesis to complete the movement.

[0026] In a preferred embodiment, the gap between the predicted value and the true value is calculated by the mean square error, and the network weight is updated by the optimization algorithm SGD, and the convolutional neural network model is iteratively trained using the training data set to continuously adjust the weight and reduce the loss function value, including the following steps:

[0027] Get the predicted value of the convolutional neural network model output , use the mean square error loss function to calculate the gap between the predicted value and the true value, the expression is: , where is the mean square error, is the number of predicted values, For the Prediction values, For the The actual value corresponding to the predicted value is used to calculate the gradient of the loss function relative to the network weight and is used to update the weight. The expression is: , where is the gradient of the loss function corresponding to the predicted value, is the predicted value, is the true value, and the network weight is updated using the stochastic gradient descent optimization algorithm. The expression is: , where is the updated network weight, is the network weight before updating, is the learning rate, and the forward propagation, loss calculation, backpropagation, and weight update steps are repeated until the loss function value converges.

[0028] In a preferred embodiment, a segment of electromyographic signal data is collected, and the extracted time domain features include the waveform length and root mean square value of the electromyographic signal. The waveform length calculation expression is: , the calculation expression of the root mean square value is: , where is the waveform length, is the root mean square value, Indicates the number of electromyographic signal data. Indicates Data value, Indicates Data value;

[0029] Collect a segment of EMG signal data, and extract frequency domain features including the frequency band energy and frequency center of the EMG signal. The calculation expression of the frequency band energy is: , the calculation expression of the frequency center is: , where is the band energy, is the frequency center, at the frequency The power spectral density at The upper and lower bounds of the frequency range represent the frequency interval for calculating the energy of the frequency band. is the maximum frequency, indicating the upper limit of the frequency range of the power spectrum.

[0030] In a preferred embodiment, a segment of electromyographic signal data is collected, and the extracted time-frequency domain features include short-time Fourier transform and wavelet transform of the electromyographic signal. The calculation expression of the short-time Fourier transform is:

[0031] , where The collected electromyographic signal is is a time point, indicating the center position of the window, is the angular frequency, and ,in is the frequency, is the window function, using a Gaussian window and weighting at the time point, is a complex exponential function, representing the basis function of Fourier transform, Represents myoelectric signal At the point in time And the angular frequency The short-time Fourier transform is performed at the position, and the calculation expression of the wavelet transform is: , where Represents myoelectric signal In the scale parameter and translation parameters The wavelet transform result is: is the scale parameter, which represents the expansion factor of the wavelet and controls the frequency resolution. is the translation parameter, which represents the time displacement of the wavelet and controls the time resolution. is the mother wavelet function, is the complex conjugate of the mother wavelet function.

[0032] In the above technical solution, the technical effects and advantages provided by the present invention are:

[0033] 1. The present invention extracts relevant features from the preprocessed electromyographic signals, trains the extracted relevant features through the convolutional neural network algorithm, establishes a relationship model between electromyographic signals and hand movements, collects the electromyographic signals of the user's arm in real time, and after preprocessing and extracting relevant features, inputs them into the trained relationship model to predict the user's hand movement intention, and controls the movement of the prosthesis according to the predicted hand movement intention. The prediction system preprocesses the electromyographic signals collected from the arm through machine learning, predicts the hand movement through the electromyographic signals and establishes a relationship, and subsequently controls the movement of the prosthesis through the electromyographic signals, which not only enables the prosthesis to respond in advance, but also effectively improves the accuracy of the prosthesis movement.

[0034] 2. In the actual control process, the present invention determines whether it is necessary to perform real-time correction on the response speed of the prosthesis by analyzing the feedback data of multiple types of sensors, and generates a corresponding control strategy based on the judgment result, so as to accurately adjust the response speed of the prosthesis and avoid the increase in the user's prosthesis usage burden caused by uncoordinated movements due to too fast or too slow response speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0036] Figure 1 The figure is a flow chart of the method of the present invention. DETAILED DESCRIPTION

[0037] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] Example 1: Please refer to Figure 1 As shown, the gesture trajectory prediction method based on motion intention extraction described in this embodiment includes the following steps:

[0039] Use electromyographic sensors to collect electromyographic signals (EMG) from the user's arm, record muscle activity data, filter and denoise the collected electromyographic signals, remove environmental noise and interference, extract effective electromyographic signal features, and extract relevant features from the preprocessed electromyographic signals. The relevant features include time domain features, frequency domain features, and time-frequency domain features for subsequent pattern recognition and prediction. The extracted relevant features are trained through a convolutional neural network algorithm to establish a relationship model between electromyographic signals and hand movements. The electromyographic signals of the user's arm are collected in real time, and after preprocessing and extracting relevant features, they are input into the trained relationship model to predict the user's hand movement intention. The movement of the prosthesis is controlled according to the predicted hand movement intention, so that the prosthesis can respond to the user's intention in advance and achieve natural and precise hand movements. In the actual control process, by analyzing the feedback data of multiple types of sensors, it is determined whether the response speed of the prosthesis needs to be corrected in real time, and the corresponding control strategy is generated based on the judgment result.

[0040] This application extracts relevant features from the preprocessed electromyographic signals, trains the extracted relevant features through the convolutional neural network algorithm, establishes a relationship model between electromyographic signals and hand movements, collects the electromyographic signals of the user's arm in real time, and after preprocessing and extracting relevant features, inputs them into the trained relationship model to predict the user's hand movement intention, and controls the movement of the prosthesis according to the predicted hand movement intention. The prediction system preprocesses the electromyographic signals collected from the arm through machine learning, predicts hand movements through electromyographic signals and establishes relationships, and subsequently controls the movement of the prosthesis through electromyographic signals, which not only enables the prosthesis to respond in advance, but also effectively improves the accuracy of the prosthetic movement.

[0041] During the actual control process, the present application determines whether it is necessary to perform real-time correction on the response speed of the prosthesis by analyzing feedback data from multiple types of sensors, and generates a corresponding control strategy based on the judgment result, so as to accurately adjust the response speed of the prosthesis and avoid the increase in the user's burden of using the prosthesis due to uncoordinated movements caused by too fast or too slow response speed.

[0042] The prediction system includes: EMG signal feature extraction module, relationship model building module, action prediction module, and control strategy generation module;

[0043] EMG signal feature extraction module: Use EMG sensors to collect EMG signals from the user's arms, record muscle activity data, filter and denoise the collected EMG signals, remove environmental noise and interference, extract effective EMG signal features, and extract relevant features from the pre-processed EMG signals. The relevant features include time domain features, frequency domain features, and time-frequency domain features for subsequent pattern recognition and prediction. The relevant features are sent to the relationship model building module;

[0044] Relationship model building module: The extracted relevant features are trained through the convolutional neural network algorithm to build a relationship model between the electromyographic signal and the hand movement, and the relationship model is sent to the movement prediction module;

[0045] Action prediction module: collects the electromyographic signals of the user's arm in real time, and after preprocessing and extracting relevant features, inputs them into the trained relationship model to predict the user's hand movement intention. The movement of the prosthesis is controlled according to the predicted hand movement intention, so that the prosthesis can respond to the user's intention in advance and achieve natural and accurate hand movements. The prosthesis control process is sent to the control strategy generation module;

[0046] Control strategy generation module: In the actual control process, by analyzing the feedback data of multiple types of sensors, it is determined whether the response speed of the prosthesis needs to be corrected in real time, and the corresponding control strategy is generated based on the judgment results.

[0047] Embodiment 2: Using an electromyographic sensor to collect electromyographic signals (EMG) from a user's arm and record muscle activity data, including the following steps:

[0048] The EMG sensor is correctly placed on the user's arm, usually at the location of the major muscle groups, to ensure that the muscle activity signal can be accurately captured. The EMG sensor is started to collect the electrical signals generated by the user's muscle activity in real time. The collected signals are raw, unprocessed EMG data, and the EMG data is recorded.

[0049] The collected electromyographic signals are filtered and denoised to remove environmental noise and interference and extract effective electromyographic signal features, including the following steps:

[0050] Bandpass filtering: Use a bandpass filter to remove high-frequency and low-frequency noise and retain the EMG signal within the target frequency range. Usually the frequency range of 20Hz to 450Hz is selected.

[0051] Low-pass filter: removes high-frequency noise above 450Hz.

[0052] High pass filter: removes low frequency noise below 20 Hz and baseline drift.

[0053] Notch filter: Use notch filter to remove power frequency interference (50Hz or 60Hz), which is common power supply noise.

[0054] Average filtering: Perform sliding average processing on the signal to smooth out short-term noise.

[0055] Median filtering: Use a median filter to remove sharp burst noise and maintain the edge characteristics of the signal.

[0056] Amplitude normalization: Normalize the EMG signal to a standard range (such as 0 to 1 or -1 to 1) to make the signals between different samples comparable.

[0057] Mean Removal: Adjusts the mean of a signal to zero to remove DC offset.

[0058] Extracting relevant features from the preprocessed EMG signals for subsequent pattern recognition and prediction includes the following steps:

[0059] A segment of electromyographic signal data is collected, and the relevant features include time domain features, frequency domain features and time-frequency domain features. The extracted time domain features include the root mean square value and waveform length of the electromyographic signal, the extracted frequency domain features include frequency band energy and frequency center, and the extracted time-frequency domain features include short-time Fourier transform and wavelet transform.

[0060] Collect a section of EMG signal data, and extract the time domain features including the waveform length and RMS value of the EMG signal. The waveform length calculation expression is: , the calculation expression of the root mean square value is: , where is the waveform length, is the root mean square value, Indicates the number of electromyographic signal data. Indicates Data value, Indicates Data value;

[0061] Collect a segment of EMG signal data, and extract frequency domain features including the frequency band energy and frequency center of the EMG signal. The calculation expression of the frequency band energy is: , the calculation expression of the frequency center is: , where is the band energy, is the frequency center, at the frequency The power spectral density at The upper and lower bounds of the frequency range represent the frequency interval for calculating the energy of the frequency band. is the maximum frequency, indicating the upper limit of the frequency range of the power spectrum;

[0062] Collect a section of EMG signal data, and extract the time-frequency domain features including the short-time Fourier transform and wavelet transform of the EMG signal. The calculation expression of the short-time Fourier transform is:

[0063] , where The collected electromyographic signal is is a time point, indicating the center position of the window, is the angular frequency, and ,in is the frequency, is the window function, using a Gaussian window and weighting at the time point, is a complex exponential function, representing the basis function of Fourier transform, Represents myoelectric signal At the point in time And the angular frequency The short-time Fourier transform is performed at the position, and the calculation expression of the wavelet transform is: , where Represents myoelectric signal In the scale parameter and translation parameters The wavelet transform result is: is the scale parameter, which represents the expansion factor of the wavelet and controls the frequency resolution. is the translation parameter, which represents the time displacement of the wavelet and controls the time resolution. is the mother wavelet function, which is the basic wavelet shape. is the complex conjugate of the mother wavelet function.

[0064] The extracted relevant features are trained by the convolutional neural network algorithm to establish a relationship model between the electromyographic signal and the hand movement, including the following steps:

[0065] The acquired root mean square value, waveform length, frequency band energy, frequency center, short-time Fourier transform, and wavelet transform are normalized, and the normalized root mean square value, waveform length, frequency band energy, frequency center, short-time Fourier transform, and wavelet transform are labeled to form a labeled data set for input into the convolutional neural network to improve the stability and convergence speed of model training;

[0066] After the convolutional neural network model is built, the gap between the predicted value and the true value is calculated by the mean square error, and the network weight is updated through the optimization algorithm SGD. The convolutional neural network model is iteratively trained using the training data set, and the weights are continuously adjusted and the loss function value is reduced. After the convolutional neural network model training is completed, the performance of the convolutional neural network model is evaluated through the verification data set, and the hyperparameter adjustment is optimized. Finally, the test data set is used to verify that the generalization ability of the convolutional neural network model meets the standard, and the relationship model between the electromyographic signal and the hand movement is output.

[0067] In this application, the normalized root mean square value, waveform length, frequency band energy, frequency center, short-time Fourier transform, and wavelet transform are labeled, which means that the normalized root mean square value, waveform length, frequency band energy, frequency center, short-time Fourier transform, and wavelet transform are associated with the corresponding hand action label. For example, the hand action label is "fist" (the corresponding label is 1).

[0068] The convolutional neural network model construction includes the following steps:

[0069] Input layer: inputs the feature data of the label data set. Each input feature in the feature data has a corresponding output label, and the feature data includes 6 dimensions;

[0070] Convolutional layer: Use different convolution kernels to detect feature patterns and extract feature maps. The size of the feature map is 6×16 (16 feature maps, each with a size of 6);

[0071] Activation layer: Use the ReLU activation function to introduce nonlinearity to the output feature map of the convolution layer and obtain the feature map after applying ReLU;

[0072] Pooling layer: Use maximum pooling with a window size of 2 and a stride of 2 to reduce the size of the feature map after ReLU, extract the main features, and prevent overfitting;

[0073] Fully connected layer: flattens the feature map output by the pooling layer, connects it to the fully connected layer, and maps the extracted feature map to the output space;

[0074] Output layer: Use softmax activation function to predict hand movements.

[0075] The gap between the predicted value and the true value is calculated by the mean square error, and the network weight is updated by the optimization algorithm SGD. The convolutional neural network model is iteratively trained using the training data set to continuously adjust the weight and reduce the loss function value, including the following steps:

[0076] Get the predicted value of the convolutional neural network model output , the mean square error (MSE) loss function is used to calculate the gap between the predicted value and the true value, and the expression is: , where is the mean square error, is the number of predicted values, For the Prediction values, For the The actual value corresponding to the predicted value is used to calculate the gradient of the loss function relative to the network weight and is used to update the weight. The expression is: , where is the gradient of the loss function corresponding to the predicted value, is the predicted value, is the true value, and the network weights are updated using the stochastic gradient descent (SGD) optimization algorithm, expressed as: , where is the updated network weight, is the network weight before updating, is the learning rate, and the forward propagation, loss calculation, backpropagation, and weight update steps are repeated until the loss function value converges.

[0077] After the convolutional neural network model training is completed, the performance of the convolutional neural network model is evaluated through the validation data set, and the hyperparameters are adjusted and optimized. Finally, the generalization ability of the convolutional neural network model is verified by the test data set, and the relationship model between the electromyographic signal and the hand movement is output, including the following steps:

[0078] Calculate the performance indicators of the model on the validation data set, such as accuracy, precision, recall, and F1 score.

[0079] Assume that the performance of the model on the validation dataset is:

[0080] Accuracy: 90%

[0081] Precision: 88%

[0082] Recall: 89%

[0083] F1 Score: 88.5%.

[0084] According to the performance indicators on the validation set, adjust the model's hyperparameters to optimize the model performance;

[0085] Hyperparameters example:

[0086] Learning rate: adjusted from 0.01 to 0.001;

[0087] Batch-Size: adjusted from 32 to 64;

[0088] Number of Convolutional Layers: Increased from 2 to 3 layers;

[0089] Activation-Function: changed from ReLU to Leaky-ReLU;

[0090] Retrain the model, using the optimized hyperparameters, and evaluate the model's performance on the validation set.

[0091] Assume that the performance of the adjusted model is:

[0092] Accuracy: 92%

[0093] Precision: 90%

[0094] Recall: 91%

[0095] F1 Score: 90.5%

[0096] After the model is evaluated through validation data set evaluation and hyperparameter adjustment optimization, an independent test data set is used to verify the generalization ability of the model. After the model performance reaches the expected level (i.e., the accuracy is greater than or equal to the accuracy threshold, the precision is greater than or equal to the precision threshold, the recall is greater than or equal to the recall threshold, and the F1 score is greater than or equal to the equal division threshold), the relationship model between the electromyographic signal and the hand movement is output.

[0097] The electromyographic signals of the user's arm are collected in real time, and after preprocessing and extracting relevant features, they are input into the trained relationship model to predict the user's hand movement intention. The movement of the prosthesis is controlled according to the predicted hand movement intention, so that the prosthesis can respond to the user's intention in advance and achieve natural and precise hand movements, including the following steps:

[0098] Use electromyographic sensors to collect electromyographic signals (EMG) from the user's arm in real time and record the raw data of the user's muscle activity;

[0099] Example: The collected electromyographic signal: [0.45, 0.47, 0.49, …];

[0100] Extracted features: root mean square value (RMS), waveform length (WL), band energy, frequency center, short time Fourier transform (STFT), wavelet transform (CWT).

[0101] Eigenvector = [RMS, WL, Band Energy, Frequency Center, STFT, CWT];

[0102] Normalize the extracted features to meet the requirements of model input.

[0103] Normalized feature vector: [0.45, 0.47, 0.50, 0.52, 0.55, 0.58];

[0104] The normalized feature vector is input into the trained convolutional neural network (CNN) relational model to predict the user's hand movement intention. According to the prediction results output by the model, the user's hand movement intention is judged. According to the predicted hand movement intention, the movement of the prosthesis is controlled, and the electromyographic signal is converted into specific movement instructions of the prosthesis through the prosthesis control system.

[0105] In the actual control process, by analyzing the feedback data of multiple types of sensors, it is determined whether the response speed of the prosthesis needs to be corrected in real time, and a corresponding control strategy is generated based on the judgment result, including the following steps:

[0106] After the prosthetic action is completed, the position error, speed error and fatigue index are obtained, and the correction coefficient is obtained by comprehensive calculation of the position error, speed error and fatigue index. The expression is:

[0107]

[0108] , where is the correction factor, is the position error, is the speed error, is the fatigue index, , are the proportional coefficients of position error and velocity error respectively, and , All are greater than 0;

[0109] If the correction coefficient is less than or equal to the second coefficient threshold, and the correction coefficient is greater than or equal to the first coefficient threshold, it is judged that there is no need to perform real-time correction on the response speed of the prosthesis. If the correction coefficient is greater than the second coefficient threshold, it is judged that the response speed of the prosthesis needs to be reduced. If the correction coefficient is less than the second coefficient threshold, it is judged that the response speed of the prosthesis needs to be increased. The correction algorithm expression is:

[0110]

[0111] In the formula, is the response speed after correction, is the response speed before correction, is the correction factor, is the first coefficient threshold, is the second coefficient threshold.

[0112] The actual position of the prosthesis is monitored by the position sensor, and the position error of the prosthesis is calculated , , where For prosthesis in time The actual location at the moment, For prosthesis in time The predicted position at the moment. When the position error is large, it indicates that the prosthesis responds quickly. When the position error is small, it indicates that the prosthesis responds slowly.

[0113] The actual speed of the prosthesis movement is obtained based on the actual position. The expression is:

[0114] , the expected speed is: , where is the actual speed, is the expected speed, For time The time derivative of the actual position of the prosthesis at time , For time The time derivative of the expected position of the prosthesis at time t, then the velocity error The expression is: When the speed error is large, it indicates that the prosthesis responds faster. When the speed error is small, it indicates that the prosthesis responds slower.

[0115] The myoelectric sensor is used to obtain the user's electromyographic signal during the movement of the prosthesis. Muscle fatigue causes the irregularity of electrical activity to increase, and the waveform length increases. The fatigue waveform length is calculated, and the expression is: , where is the fatigue waveform length, is the number of electromyographic signal data, For the The electromyographic signal value, For the The electromyographic signal values ​​are accumulated and calculated during the entire action period of the prosthesis to obtain the fatigue index, which is expressed as: , where is the fatigue index, During the movement of the prosthesis, The length of the fatigue waveform at the time, It is the time it takes for the prosthesis to complete an action. The larger the fatigue index is, the more fatigue the user's arm will experience during the movement of the prosthesis. When the prosthesis responds too fast or too slow, it will cause fatigue in the user's arm.

[0116] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0117] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0118] The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to only specific implementation methods. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. A gesture trajectory prediction method based on motion intention extraction, characterized in that: The prediction method comprises the following steps: Use electromyographic sensors to collect electromyographic signals from the user's arm, record muscle activity data, filter and denoise the collected electromyographic signals, and extract relevant features from the pre-processed electromyographic signals; The extracted relevant features are trained through the convolutional neural network algorithm to establish a relationship model between electromyographic signals and hand movements. The electromyographic signals of the user's arm are collected in real time, and after preprocessing and extracting relevant features, they are input into the trained relationship model to predict the user's hand movement intention and control the movement of the prosthesis according to the predicted hand movement intention. In the actual control process, the feedback data from multiple types of sensors are analyzed to determine whether the response speed of the prosthesis needs to be corrected in real time, and the corresponding control strategy is generated based on the judgment results; Extracting relevant features from the preprocessed electromyographic signal includes the following steps: Collect a section of electromyographic signal data, the relevant features include time domain features, frequency domain features and time-frequency domain features, the extracted time domain features include the waveform length and root mean square value of the electromyographic signal, the extracted frequency domain features include the frequency band energy and frequency center of the electromyographic signal, and the extracted time-frequency domain features include short-time Fourier transform and wavelet transform of the electromyographic signal; The extracted relevant features are trained by the convolutional neural network algorithm to establish a relationship model between the electromyographic signal and the hand movement, including the following steps: The acquired root mean square value, waveform length, frequency band energy, frequency center, short-time Fourier transform, and wavelet transform are all normalized, and the normalized root mean square value, waveform length, frequency band energy, frequency center, short-time Fourier transform, and wavelet transform are annotated to form a label data set; After the convolutional neural network model is built, the difference between the predicted value and the true value is calculated through the mean square error, and the network weight is updated through the optimization algorithm SGD. The convolutional neural network model is iteratively trained using the training data set, and the weight is continuously adjusted and the loss function value is reduced. After the convolutional neural network model is trained, the performance of the convolutional neural network model is evaluated through the verification data set, and the hyperparameter adjustment and optimization are performed. Finally, the test data set is used to verify that the generalization ability of the convolutional neural network model meets the standard, and the relationship model between the electromyographic signal and the hand movement is output; In the actual control process, by analyzing the feedback data of multiple types of sensors, it is determined whether the response speed of the prosthesis needs to be corrected in real time, and a corresponding control strategy is generated based on the judgment result, including the following steps: After the prosthetic action is completed, the position error, speed error and fatigue index are obtained, and the correction coefficient is obtained by comprehensive calculation of the position error, speed error and fatigue index. The expression is: , where is the correction factor, is the position error, is the speed error, is the fatigue index, , are the proportional coefficients of position error and velocity error respectively, and , All are greater than 0; If the correction coefficient is less than or equal to the second coefficient threshold, and the correction coefficient is greater than or equal to the first coefficient threshold, it is judged that there is no need to perform real-time correction on the response speed of the prosthesis. If the correction coefficient is greater than the second coefficient threshold, it is judged that the response speed of the prosthesis needs to be reduced. If the correction coefficient is less than the second coefficient threshold, it is judged that the response speed of the prosthesis needs to be increased. The correction algorithm expression is: In the formula, is the response speed after correction, is the response speed before correction, is the correction factor, is the first coefficient threshold, is the second coefficient threshold.

2. The method for predicting hand gesture trajectories based on motion intention extraction according to claim 1, characterized in that: The actual position of the prosthesis is monitored by the position sensor, and the position error of the prosthesis is calculated , , where For prosthesis in time The actual location at the moment, For prosthesis in time The predicted position at the moment; The actual speed of the prosthesis movement is obtained based on the actual position. The expression is: , the expected speed is: , where is the actual speed, is the expected speed, For time The time derivative of the actual position of the prosthesis at time , For time The time derivative of the expected position of the prosthesis at time t, then the velocity error The expression is: ; Calculate the fatigue waveform length, the expression is: , where is the fatigue waveform length, is the number of electromyographic signal data, For the The electromyographic signal value, For the The electromyographic signal values ​​are accumulated and calculated during the entire action period of the prosthesis to obtain the fatigue index, which is expressed as: , where is the fatigue index, During the movement of the prosthesis, The length of the fatigue waveform at the time, The time it takes for the prosthesis to complete the movement.

3. The method for predicting hand gesture trajectories based on motion intention extraction according to claim 2, characterized in that: The gap between the predicted value and the true value is calculated by the mean square error, and the network weight is updated by the optimization algorithm SGD. The convolutional neural network model is iteratively trained using the training data set to continuously adjust the weight and reduce the loss function value, including the following steps: Get the predicted value of the convolutional neural network model output , use the mean square error loss function to calculate the gap between the predicted value and the true value, the expression is: , where is the mean square error, is the number of predicted values, For the Prediction values, For the The actual value corresponding to the predicted value is used to calculate the gradient of the loss function relative to the network weight and is used to update the weight. The expression is: , where is the gradient of the loss function corresponding to the predicted value, is the predicted value, is the true value, and the network weight is updated using the stochastic gradient descent optimization algorithm. The expression is: , where is the updated network weight, is the network weight before updating, is the learning rate, and the forward propagation, loss calculation, backpropagation, and weight update steps are repeated until the loss function value converges.

4. The method for predicting hand gesture trajectories based on motion intention extraction according to claim 3, characterized in that: Collect a section of EMG signal data, and extract the time domain features including the waveform length and RMS value of the EMG signal. The waveform length calculation expression is: , the calculation expression of the root mean square value is: , where is the waveform length, is the root mean square value, Indicates the number of electromyographic signal data. Indicates Data value, Indicates Data value; Collect a segment of EMG signal data, and extract frequency domain features including the frequency band energy and frequency center of the EMG signal. The calculation expression of the frequency band energy is: , the calculation expression of the frequency center is: , where is the band energy, is the frequency center, at the frequency The power spectral density at The upper and lower bounds of the frequency range represent the frequency interval for calculating the energy of the frequency band. is the maximum frequency, indicating the upper limit of the frequency range of the power spectrum.

5. The method for predicting hand gesture trajectories based on motion intention extraction according to claim 4, characterized in that: Collect a section of EMG signal data, and extract the time-frequency domain features including the short-time Fourier transform and wavelet transform of the EMG signal. The calculation expression of the short-time Fourier transform is: , where The collected electromyographic signal is is a time point, indicating the center position of the window, is the angular frequency, and ,in is the frequency, is the window function, using a Gaussian window and weighting at the time point, is a complex exponential function, representing the basis function of Fourier transform, Represents myoelectric signal At the point in time And the angular frequency The short-time Fourier transform is performed at the position, and the calculation expression of the wavelet transform is: , where Represents myoelectric signal In the scale parameter and translation parameters The wavelet transform result is: is the scale parameter, which represents the expansion factor of the wavelet and controls the frequency resolution. is the translation parameter, which represents the time displacement of the wavelet and controls the time resolution. is the mother wavelet function, is the complex conjugate of the mother wavelet function.