Robustness improvement method for electromyography control system based on generative adversarial network

Abstract

The application discloses a myoelectricity control system robustness improving method based on a generative adversarial network, which comprises the following steps: using a discriminator of a generative adversarial network (GAN) as a discriminator for judging whether a classification model result of a myoelectricity controller is a known action category. When the effect of the method is presented, a simple CNN model is first built as a K-class known category classifier as a simulation of an original executor classification model, and the CNN model will give a K-dimensional prediction output to test data. The discriminator needs to select the known classification result according to the prediction output and reject the unknown result. Finally, the classification result allowed to be output will be executed by the executor, and after rejecting the unknown action, the default or previous state is maintained, so that the action error rate of the executor is reduced. The discriminator is trained by using the GAN. The application has the advantages of improving the accuracy of the actual output action, improving the robustness of the action execution process, being capable of distinguishing the known and unknown actions, having a simple structure and requiring small computing power.

Description

Technical Field

[0001] This invention relates to the field of electromyography (EMG) signal pattern recognition technology, and in particular to a method for improving the robustness of an EMG control system based on generative adversarial networks. Background Technology

[0002] Electromyography (EMG), a bioelectrical signal reflecting human muscle activity, has been widely used in human motion recognition and medical rehabilitation. However, for a long time, EMG actuators, such as those used in neuroprosthetics, have been constrained by the high variability of EMG signals. Numerous studies have proposed and validated advanced deep learning algorithms and improved acquisition devices to address the stability and accuracy of EMG signal recognition. However, existing algorithms struggle to maintain good classification performance when EMG signals of unknown movements are present, making it difficult to detect movements and leading to errors in movement execution.

[0003] Current commercial electromyography (EMG) control systems (such as neuroprosthetics) typically use only EMG as the source of movement intention recognition, and can achieve effective control simply by wearing the device. However, this inevitably makes them susceptible to the high variability of EMG signals, leading to a decline in recognition performance with long-term use and without calibration. Summary of the Invention

[0004] This invention addresses the shortcomings of existing technologies by providing a robustness improvement method for electromyography (EMG) control systems based on generative adversarial networks (GANs). The method correctly classifies and executes actions known to the algorithm, while rejecting the execution of unknown, newly emerging actions. This maintains the device in its existing or default state, reducing the Active Error Rate (AER, the proportion of erroneous actions among the actual actions executed by the actuator), thereby improving the overall robustness of the system.

[0005] To achieve the above-mentioned objectives, the technical solution adopted by the present invention is as follows:

[0006] A method for improving the robustness of an electromyography (EMG) control system based on generative adversarial networks includes the following steps:

[0007] Step 1, Data Preprocessing: After the user puts on the electromyography (EMG) actuator, the surface EMG signal acquisition device collects EMG signals. These EMG signals are filtered and time-windowed to become EMG signal segments, which are then input into an EMG signal classifier (not limited to any specific machine learning or deep learning classification model; the following will use a convolutional neural network (CNN) as an example of a classifier).

[0008] The following data classification is required during the pre-training phase; it is not necessary when using the testing environment:

[0009] Known classes: Used for training CNN and Generative Adversarial Network (GAN) models, and also appearing in test scenarios;

[0010] Unknown class: Not used for model training, only appears in testing scenarios.

[0011] Step 2: The classifier calculates the feature vector.

[0012] Construct a CNN classifier that classifies known classes, containing convolutional layers and fully connected layers. Pre-train the CNN model on a known class dataset; the dimension of the output feature vector is the number of known classes it can distinguish. For each sample, the CNN outputs a prediction, which is computed up to the last fully connected layer. The output of this layer is then used as the input to the unknown class discriminator.

[0013] Step 3: Determine whether it is a known action class and output the final classification result of the accepted action;

[0014] 1) Training the unknown class discriminator

[0015] The discriminator for unknown classes is trained using only a dataset containing known classes. Known classes are treated as real samples, while samples generated by the GAN's generator are treated as fake samples, which are then fed into the GAN's discriminator to determine whether they are real or fake.

[0016] In each training epoch, the GAN's discriminator is tested on a test set containing both known and unknown class data to verify its ability to distinguish between known and unknown classes. After training, the discriminator with the best discrimination performance is used as the discriminator for the unknown class.

[0017] 2) Select the best discriminator for the unknown class

[0018] The GAN discriminator predicts the probability of an input sample, indicating its degree of authenticity. The optimization metric for training the discriminator is its performance in distinguishing between real and fake samples on the test set during training. AUC (Area Under Curve) is used as the classification metric for the GAN discriminator on the test set.

[0019] The direct definition of AUC is the area under the ROC curve. AUC values ​​range from 0 to 1; the closer to 1, the better the classification performance. During training, the discriminator with the highest AUC value is selected as the optimal discriminator for the unknown class.

[0020] The selected optimal unknown class discriminator still requires a threshold value during use because the discriminator outputs a value between 0 and 1. Values ​​above the threshold are classified as true (known class action), and values ​​below the threshold are classified as false (unknown class action). Based on the AUC value, a corresponding ROC curve can be plotted. The point on the ROC curve closest to the top left corner is the required threshold.

[0021] 3) Use an unknown class discriminator to complete the unknown determination.

[0022] When the feature vector calculated by the classification model is input into the unknown class discriminator, if the output probability is greater than or equal to the threshold, the sample is considered to belong to the known class, the sample is accepted, and the final action classification result is calculated; if the probability of the sample is lower than the threshold, the sample is rejected, and the default action or the previous action is maintained until the next accepted action is executed.

[0023] Furthermore, step 1 is detailed as follows:

[0024] Ten channels of surface electromyography (EMG) signals were acquired using a surface EMG signal acquisition device. Continuous signals were windowed for 200 ms and then directly fed into a convolutional neural network (CNN). During training, some EMG signal movements were assigned to the known class, while others were assigned to the unknown class.

[0025] Furthermore, the generator of the GAN contains a certain number of fully connected layers, accepts random noise, and generates a 1*X output; the discriminator of the GAN contains three fully connected layers, accepts a 1*X input, and outputs a predicted probability value normalized to between 0 and 1.

[0026] Furthermore, the acquisition device in step 1 is a Shimmer 3. Before attaching the electrodes to the arm, the area to be attached is cleaned with alcohol. Then, three pairs of differential electrodes are attached to the skin above the main active muscles of the forearm, and the calibration end is placed on the bony prominence of the arm. The Shimmer 3 connects to the computer in real time via Bluetooth and transmits the acquired electromyographic signals.

[0027] Type A hand gestures were collected, with each gesture repeated 10 times and each gesture lasting 10 seconds. The signal acquisition program on the computer was written in Matlab. Continuous signals were sampled in a 200ms time window with a 50ms time step, and used as input for neural network training.

[0028] Assign training / test sets, with the 2nd, 5th, and 7th repetitions of each action used as the test set. The method for assigning known and unknown classes is to take the first X classes of action A as the known class and the last Y classes as the unknown class.

[0029] Furthermore, in step 2, the CNN needs to correctly classify the known categories. Both the training and test sets of the CNN contain only data of the known categories. The trained CNN is then used to construct the dataset after feature extraction. All raw electromyography signal samples will undergo CNN feature extraction, and the output will be a vector with a dimension equal to the number of known categories X.

[0030] Furthermore, in step 3, after each training round, the GAN discriminator calculates the AUC value on the test set. Training is completed within 2000 epochs, and the optimal AUC value is recorded. Once the ideal AUC value is obtained, the corresponding GAN discriminator becomes the unknown class discriminator.

[0031] Furthermore, the discriminator with the maximum AUC value is set with a threshold, its ROC curve is plotted, and the optimal threshold is obtained based on the Youden index. When the discriminator's output is greater than or equal to this threshold, it is considered a known class; otherwise, it is considered an unknown class. Thus, a usable discriminator for the unknown class is obtained.

[0032] Compared with the prior art, the advantages of the present invention are as follows:

[0033] This invention can improve the accuracy of actual output actions in real-world environments with a large number of novel actions, while preventing the execution of unknown actions, thus enhancing the robustness of the entire actuator's action execution process. The discriminator of a GAN, after training, serves as the unknown class discriminator. It is fed into the output of the neural network classification model to evaluate the model's output, ultimately providing a rejection or acceptance result. During the training of the neural network classification model and the GAN, no samples other than those with known actions are involved. This invention can distinguish between known and unknown actions without encountering any novel actions, a significant advantage given the difficulty humans face in identifying hard-to-classify electromyographic signals. Furthermore, the invention has a simple structure (the unknown class discriminator contains only a few fully connected layers), requires minimal computational power, and is easier to deploy on edge computing devices such as electromyographic actuators. This invention addresses the practical problems of electromyographic signals while being simple and convenient enough to improve existing devices. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of the framework structure of the method for improving the robustness of the electromyography control system according to an embodiment of the present invention.

[0035] Figure 2 This is a schematic diagram of the CNN classifier structure according to an embodiment of the present invention.

[0036] Figure 3 This is a schematic diagram of the generator and discriminator structure of the GAN in this embodiment of the invention.

[0037] Figure 4 These are schematic diagrams of 10 hand gestures according to embodiments of the present invention.

[0038] Figure 5 This is a schematic diagram of the ROC curve and its optimal cutoff point in an embodiment of the present invention.

[0039] Figure 6 This is a schematic diagram illustrating the impact of changes in the number of unknown classes on the AER of this invention.

[0040] Figure 7 This is a schematic diagram illustrating the impact of changes in the number of known classes on the effectiveness of the present invention in an embodiment of the invention. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and examples.

[0042] I. Methodological Framework Based on Unknown Class Rejection

[0043] This section provides a detailed explanation of the implementation of the method for improving the stability of electromyographic signal control based on unknown class rejection. Its overall structure is as follows: Figure 1 As shown.

[0044] The complete framework of the electromyographic signal control stability improvement method based on unknown class rejection proposed in this invention includes three parts: data processing, classifier feature extraction, and output of the classification result of the accepted action.

[0045] 1.1 Data Preprocessing

[0046] Ten channels of surface electromyography (EMG) signals were acquired using a surface EMG signal acquisition device at a sampling rate of 1000 Hz. Continuous signals were windowed for 200 ms and then directly fed into a convolutional neural network (CNN) without further data processing. The EMG signals came from 10 action categories defined in this embodiment, of which 6 categories were designated as known categories and the remaining 4 as unknown categories.

[0047] Known classes: used for training CNN and Generative Adversarial Network (GAN) models, and also appearing in test scenarios (including the test set of the unknown class discriminator);

[0048] Unknown class: Not used for model training, it only appears in the test scenario (including the test set of the unknown class discriminator).

[0049] 1.2 Classifier calculates feature vectors

[0050] like Figure 2 As shown, a CNN classifier for classifying known classes is constructed, consisting of 3 convolutional layers and 2 fully connected layers. The CNN model used here has been pre-trained on a dataset of known classes, and the dimension of the output feature vector is the number of known classes it can distinguish. For each sample, the CNN outputs a prediction, but here we only compute up to the last fully connected layer, and then use the output of this layer as the input to the unknown class discriminator.

[0051] like Figure 3 As shown, the generator of the GAN contains three fully connected layers, accepts random noise of size 1*4, and generates a 1*6 output; the discriminator of the GAN contains three fully connected layers, accepts a 1*6 input (real or generated sample), and outputs a 1*1 predicted probability value normalized to between 0 and 1.

[0052] 1.3 Output the final classification result of the accepted actions.

[0053] 1) Training the unknown class discriminator

[0054] The discriminator for unknown classes is trained using only a dataset containing known classes. Known classes are treated as real samples, while samples generated by the GAN's generator are treated as fake samples, which are then fed into the GAN's discriminator to determine whether they are real or fake. This training method is consistent with conventional GAN ​​training methods.

[0055] In each training epoch, the GAN's discriminator is tested on a test set containing six known classes and four unknown classes to verify its ability to distinguish between known and unknown classes. After training, the discriminator with the best performance is selected as the discriminator for the unknown classes.

[0056] 2) Select the best discriminator for the unknown class

[0057] The GAN discriminator predicts the probability of an input sample, indicating its true or false nature. AUC is used as the classification metric for the GAN discriminator on the test set.

[0058] During training, the GAN discriminator with the highest AUC value is used as the unknown class discriminator in this method. Next, an ROC curve is plotted based on the AUC value to obtain the classification threshold.

[0059] 3) Use an unknown class discriminator to complete the classification.

[0060] When the feature vector calculated by the classification model obtains an unknown class discriminant prediction probability greater than or equal to this threshold, this embodiment considers the sample to belong to the known class, accepts this classification, and calculates the final action classification result; when the probability of the feature vector is lower than this threshold, this method will reject this classification result, and the system will maintain the default action or the previous action until the next accepted action is executed.

[0061] II. Experiment

[0062] 2.1 Dataset Selection and Construction

[0063] This embodiment uses self-collected data and publicly available datasets to test the method.

[0064] 2.1.1 Self-collected data

[0065] In this embodiment, the acquisition device consists of two Shimmer 3 electrodes, a type of wet electrode electromyography (EMG) signal acquisition device. Before attaching the electrodes to the arm, the area to be attached is cleaned with alcohol. Then, three pairs of differential electrodes are attached to the skin above the main active muscles of the forearm, with the calibration end placed on bony prominences (such as joints) on the arm. The Shimmer 3 electrodes have built-in batteries and connect to the computer in real-time via Bluetooth to transmit the acquired EMG signals. Considering both the Bluetooth signal packet loss rate and the frequency range of the EMG signals, this embodiment sets the signal sampling rate to 1000Hz. To improve signal stability, the network card on the computer uses... Wi-Fi 6AX200 with an external antenna.

[0066] This embodiment collected 10 hand gestures (see...) Figure 4 Each movement was repeated 10 times, each movement lasting 10 seconds, followed by a 10-second rest. The rest time between movements was not mandatory. During motion acquisition, the subject was required to sit with their arms resting on a table to minimize the impact of arm and torso movement on the electromyographic signals. The signal acquisition program on the computer was written in Matlab. Continuous signals were sampled in 200ms time windows with 50ms time steps for input into the neural network for training.

[0067] The method for allocating training / test sets is to use the 2nd, 5th, and 7th repetitions of each action as the test set. The method for assigning known and unknown classes is to take the first six of the 10 action classes as the known class and the last four as the unknown class.

[0068] 2.1.2 Public Datasets

[0069] The Ninapro db1 dataset was used to test the performance and compared with self-collected data. The time window length was 200ms and the step size was 50ms. A subset of actions were selected as known classes, and 20% of the samples were randomly selected as the test set. This constructed the training set and test set for the known classes, as well as the discriminator validation set containing all action categories.

[0070] 2.2 Model Training

[0071] 2.2.1 Training the CNN model

[0072] CNNs need to correctly classify known categories, so their training and test sets only contain data from known classes. The trained CNN is then used to construct a dataset after feature extraction. All raw EMG signal samples undergo CNN feature extraction, outputting fixed-dimensional vectors. The new dataset is then used to allocate the training and test sets in a 4:1 ratio, and unknown categories are removed from the training set. This step ensures that the GAN does not encounter any data from unknown categories. Meanwhile, the test set, which comprises 20% of the total data, does not require additional attention regarding the ratio of known to unknown classes, as the AUC is not affected by the proportion of binary classification data.

[0073] The training of GANs follows the common process of alternating training of the generator and discriminator. The difference is that this embodiment no longer uses the trend and value of the loss functions of both as the end-of-training criterion, although these are still important indicators for observing whether the training process is normal. After each training round, the GAN discriminator calculates the AUC value on the test set. According to the experiments in this embodiment and the report by Kongshu et al., the optimal AUC value does not appear regularly. Therefore, this embodiment typically completes training within 2000 epochs and records the optimal AUC value. If the requirements of this embodiment are not met, other parameters are adjusted and the attempt is repeated. This embodiment completed most of the work on an NVIDIA RTX 3050 graphics card. After obtaining the ideal AUC value, the corresponding GAN discriminator becomes the unknown class discriminator in this embodiment.

[0074] The discriminator that obtains the maximum AUC value can still only output predicted probability values. Only by setting a threshold for its output can it be used for binary classification to determine known and unknown classes. This embodiment then plots the discriminator's ROC curve and obtains the optimal threshold based on the Youden index. Figure 5 If the output of the discriminator is greater than or equal to this threshold, it is considered a known class; otherwise, it is considered an unknown class. Thus, this embodiment has obtained a usable unknown class discriminator.

[0075] 2.2.2 Record AUC, ROC, TPR, and FPR

[0076] A higher AUC value indicates a steeper ROC curve, and the closer the optimal threshold point is to the top left corner of the image. The closer to the optimal threshold in the top left corner, the smaller the FPR (Fixed Probability Reduction) on the horizontal axis, indicating that only a small proportion of the unknown class is identified as positive examples; the larger the TPR (Total Probability Reduction) on the vertical axis, indicating that most of the known class is correctly detected. Therefore, the ROC curve can balance the recognition performance of both classes of samples without being affected by changes in the ratio between positive examples (known class) and negative examples (unknown class).

[0077] 2.3 Effectiveness Testing of Rejection Methods

[0078] To verify the effectiveness of this method in improving the stability of electromyographic signals, this embodiment selects the Ninapro DB1 public dataset as the validation object. Ninapro, as the largest and oldest public dataset in the field, has been used for validation in many studies. To maximize the determination of the method's effectiveness, this embodiment combines known and unknown classes in different proportions to explore the impact of data proportions on the method's performance. Considering that this embodiment requires a CNN as the classifier for the known classes, different numbers of known classes will lead to changes in the corresponding classifier, thus requiring separate consideration of changes in the number of known and unknown classes. This embodiment adopts two methods of changing the proportions: one where the number of known classes remains constant while the number of unknown classes changes; and another where the number of unknown classes remains constant while the number of known classes changes. Since neither the known class classifier nor the unknown class discriminator learns unknown class data in this method, this proportion is actually used to select and test the unknown class discriminator's dataset.

[0079] 2.3.1 Keep known classes unchanged, change unknown classes

[0080] To maximize the exploration of the impact of the relative proportions of known and unknown classes, this embodiment fixes the number of known classes at 10 and sets the number of unknown classes to 5, 10, 15, 20, 30, and 42 (6 levels in total). This arrangement ensures that the ratio of the two classes is distributed around 1:1, while setting the test data with the largest possible proportion of known classes, thus more closely resembling the usage scenario of electromyographic actuators. This test selected data from 5 individuals. First, a classifier for the known class was trained on each individual. The classifier then calculated the feature values ​​of the corresponding proportion of electromyographic signal dataset for all known and unknown class data. The new dataset composed of these feature values ​​will be used to train the GAN and select the discriminator.

[0081] 2.3.2 Keep the unknown class unchanged, change the known class

[0082] When the number of known classes changes, the corresponding known class classifier also needs to be adjusted. In practical applications, this corresponds to the model pre-training stage. To achieve a more comprehensive ratio setting, this embodiment sets the unknown classes to two fixed values: 4 and 20. For each scenario with a fixed number of unknown classes, there are datasets with known class numbers of 4, 6, 8, 10, 12, 16, and 20. The ratio of known to unknown classes increases from 1:1 in the former case and decreases from 1:1 in the latter, thus covering as many scenarios as possible. To avoid the known class classifier encountering any unknown classes, actions 1-20 of the 52 actions in the entire db1 dataset are set as known classes, and actions 21-40 are set as unknown classes. When training the known class classifier, each class needs to train 7 CNNs. Then, the CNNs calculate the feature values ​​for the known classes corresponding to this ratio and all data from the two combined datasets. This will be the GAN's dataset and test set. The GAN will be trained on the known class dataset and selected from the test and validation sets containing the corresponding ratio of data to obtain the final unknown class discriminator.

[0083] III. Results

[0084] 3.1 Usability Assessment of the Robustness Improvement Method

[0085] This section presents a performance comparison of unknown class discriminators selected for validation and test sets with different class ratios, along with corresponding analysis. Changes in the number of known classes will cause the classifier to change accordingly; therefore, this embodiment first fixes the number of known classes, and then adds different numbers of unknown class actions to the test and validation sets to construct datasets with corresponding ratios, such as... Figure 6 There are six ratios in total. "Close" represents the test results of the original model on a closed set (containing only known classes), therefore its value does not change with the ratio of the test set. The other two are obtained on test sets of their respective ratios. For "Open" and "OpenGan," the most obvious phenomenon is that the AER increases with the number of unknown class actions, which is consistent with the definition of the error rate of the final action execution. After introducing unknown class discrimination and rejection, the AER decreases significantly compared to "Open," but when the ratio is greater than or equal to 1:3, the AER of "OpenGan" is still higher than 50%.

[0086] In fact, according to the records of the unknown class discriminator in this embodiment (Table 1), the AUC does not decrease with the increase of the number of unknown classes. Therefore, it is difficult to explain the slow increase of AER by the discriminator performance. Although a clear downward trend can be observed in the F1-score (Table 2), the F1-score can only be obtained after calculating the optimal threshold from the ROC curve, and cannot be used as a reference standard during the training phase. This embodiment believes that the reason why the AER of OpenGAN also increases with the increase of the proportion of unknown classes may be that the absolute number of unknown classes increases, which leads to a lower unknown class acceptance ratio, resulting in a large number of unknown classes in the received data. When the number of known classes does not fluctuate significantly, more unknown class data raises the AER. In summary, if it is determined that an accuracy of more than 50% is required for practical application, then this embodiment believes that an effect of 1:3 and below is meaningful.

[0087] Table 4.1 AUC values ​​of the unknown class discriminator on the test set

[0088]

[0089] Table 4.2 F1 scores of the unknown class discriminator on the test set

[0090]

[0091] like Figure 7 As shown, the results displayed are the final accuracy results of the original CNN model on the accepted sample data after the rejection process. Figure 7 This is the average of 5 people. Close: the original CNN result; open: the original open set accuracy; opengan,20: the final accuracy on the received data.

[0092] The previous experiment demonstrated that the effectiveness of this method is limited by the proportion of unknown classes in the test set. Based on this conclusion, we can further explore the impact of changes in the number of known classes on the performance. Since the known class classifier also needs to be modified accordingly, to control variables as much as possible, this embodiment keeps the basic CNN structure unchanged, only modifying the number of neurons in the last layer to match the number of known classes. Although this reduces the difference in model structure, it also causes a slight decrease in model accuracy when the number of known classes increases. Figure 7 (close). At the same time, as the number of known classes increases, their relative proportion on the test set also increases, which improves the model accuracy on the test set. Figure 7 The number of "open" values ​​also increases, and the accuracy is lower for scenarios with a higher number of unknown classes (unknown = 20) than for scenarios with a lower number of unknown classes (unknown = 4). This method ( Figure 7The "opengan" method performs exceptionally well in unknown scenarios (unknown = 4), with the accuracy (ACC) for known scenarios (10-20) almost identical to the original. In unknown scenarios (unknown = 20), it shows a significant improvement over open, but only approaches the original accuracy at a near 1:1 ratio. Interestingly, the accuracy of opengan for unknown = 20 never surpasses that of open or opengan for unknown = 4 in any test environment, suggesting that the improvement of this method may have an upper limit under certain ratios. Similarly, it's noticeable that the opengan,4 value is relatively stable across all scenarios; even when the open,4 accuracy increases from 50% to 70%, the opengan,4 value fluctuates between 80-90%, further supporting the hypothesis of an upper limit.

[0093] like Figure 7 As shown, the results displayed are the final accuracy results of the original CNN model on the accepted sample data after the rejection process. Figure 7 This is the average of 5 people. Close: Original CNN results; open,4 vs. opengan,4: 4. Original open set accuracy for unknown class scenarios, final accuracy with accepted data. open,20 vs. opengan,20: 20. Original open set accuracy for unknown class scenarios, final accuracy with accepted data.

[0094] Clearly, the Accuracy Correction (ACC) is significantly improved after rejecting the unknown class ("opengan" in the figure) using this method. The higher the proportion of positive examples, the better the ACC recovery, which confirms the same conclusion as the previous experiment. Furthermore, this embodiment did not find that structural changes in the classification model (i.e., CNN) significantly affected the effectiveness of this method. Further, this embodiment also found that the accuracy improvement may have an upper limit, which is highly correlated with the proportion of known unknown classes. Combining the two experiments, this embodiment suggests that when the known class is less than the unknown class, but the ratio is less than 1:3, the final accuracy can still exceed 50%. However, whether this level of accuracy has practical value depends on the application scenario. When the known class is more than the unknown class in the application scenario (i.e., more than 1:1), this method can effectively approximate the original accuracy of the model and has practical value.

[0095] Select the discriminator from the test set to achieve the best results.

[0096] 3.2 Self-collected data testing

[0097] Following the data acquisition method in section 2.1, ten hand gestures were collected and divided into known and unknown classes in a 6:4 ratio. The CNN classification model in this embodiment achieved 97.6% accuracy on a test set with six known classes. Then, a GAN was used to obtain a discriminator for the unknown class. The CNN achieved 57.6% accuracy on an open set test set containing both known and unknown classes. After rejecting some unknown classes using the discriminator, the accuracy recovered to 81.2%. This embodiment is compared with Ninapro db1 values ​​of the same proportion and number. Figure 7 The two methods (unknown=4, known=6) have similar performance, verifying the usability of the method in this embodiment. After introducing the method of this embodiment, by rejecting unknown classes and accepting known classes, the actual error rate (AER) in the output actions is significantly reduced, effectively improving the stability of the entire decision-making process.

[0098] Table 4.3 Comparison of self-collected data and Ninapro db1 results

[0099]

[0100] * Six categories are considered known actions, and four categories are considered unknown actions.

[0101] In fact, since neural network models can classify all samples as belonging to a certain known category, when this model is applied to an actuator, its model accuracy is equivalent to the activation action accuracy (1-AER).

[0102] The method of this invention fills the gap in the control process that uses electromyography (EMG) signals as the source of actuator control in open, real-world environments. The model, combined with transfer learning, achieves higher accuracy in EMG signal pattern recognition across multiple individuals and time periods, and this higher accuracy also raises the upper limit of the accuracy that this method can recover. Therefore, the method of this invention can complement transfer learning well, jointly improving the robustness of control.

[0103] Those skilled in the art will recognize that the embodiments described herein are intended to help the reader understand the implementation methods of the present invention, and should be understood that the scope of protection of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations based on the technical teachings disclosed in this invention without departing from the spirit of the invention, and these modifications and combinations are still within the scope of protection of the present invention.

Claims

1. A method for improving the robustness of an electromyography (EMG) control system based on generative adversarial networks, characterized in that, Includes the following steps: Step 1, Data Preprocessing; After the user puts on the electromyography actuator, the surface electromyography signal acquisition device will collect electromyography signals. After filtering and time windowing, the electromyographic (EMG) signal becomes an EMG signal segment, which is then used as input to the EMG signal classifier. Two datasets need to be created during the model training phase, namely: Known datasets: used for training Convolutional Neural Network (CNN) and Generative Adversarial Network (GAN) models, and also appearing in CNN testing scenarios; Unknown datasets: These are not used for model training and only appear in GAN testing scenarios. Step 2: The classifier calculates the feature vector. Construct a CNN classifier that classifies known classes, which includes convolutional layers and fully connected layers; pre-train the CNN model on the known class dataset, and the dimension of the output feature vector is the number of known classes it can distinguish; for each sample, the CNN outputs a prediction result, which is calculated up to the last fully connected layer, and then the output of this layer is used as the input of the unknown class discriminator; Step 3: Determine whether it is a known action class and output the final classification result of the accepted action; 1) Training the unknown class discriminator The discriminator for unknown classes is trained using only known class datasets; known classes are used as real samples, and samples generated by the GAN generator are used as fake samples, which are then fed into the GAN discriminator to distinguish between real and fake samples. In each training epoch, the GAN's discriminator is tested on a test set containing both known and unknown class data to verify its ability to distinguish between known and unknown classes. After training, the discriminator with the best discrimination performance is used as the discriminator for the unknown class. 2) Select the best discriminator for the unknown class The GAN discriminator predicts the probability of an input sample, indicating its degree of authenticity. The optimization metric for training the discriminator is its performance in distinguishing between real and fake samples on the test set during training. AUC (Area Under Curve) is used as the classification evaluation metric for the GAN discriminator on the test set. During training, the discriminator with the highest AUC value is selected as the best discriminator for the unknown class. The selected best unknown class discriminator still needs a threshold when used, because the discriminator outputs a value between 0 and 1; if the value is higher than the threshold, it is judged as true, that is, a known class action, and if the value is lower than the threshold, it is judged as false, that is, an unknown class action; based on the AUC value, the corresponding ROC curve can be plotted; the point on the ROC curve closest to the top left corner is the required threshold. 3) Use an unknown class discriminator to complete the unknown determination. When the feature vector calculated by the classification model is input into the unknown class discriminator, if the output probability is greater than or equal to the threshold, the sample is considered to belong to the known class, the sample is accepted, and the final action classification result is calculated; if the probability of the sample is lower than the threshold, the sample is rejected, and the default action or the previous action is maintained until the next accepted action is executed.

2. The method for improving the robustness of an electromyography control system based on generative adversarial networks according to claim 1, characterized in that: Step 1 is as follows: Ten channels of surface electromyography (EMG) signals were acquired using a surface EMG signal acquisition device; continuous signals were directly fed into a convolutional neural network (CNN) after being windowed for 200ms; during training, some EMG signal actions were set as known classes, and others as unknown classes.

3. The method for improving the robustness of an electromyography control system based on generative adversarial networks according to claim 1, characterized in that: The generator of the GAN contains a certain number of fully connected layers, accepts random noise, and generates a 1*X output; the discriminator of the GAN contains a certain number of fully connected layers, accepts a 1*X input, and outputs a predicted probability value normalized to between 0 and 1.

4. The method for improving the robustness of an electromyography control system based on generative adversarial networks according to claim 1, characterized in that: In step 1, the acquisition device is a Shimmer 3. Before attaching the electrodes to the arm, the area to be attached is cleaned with alcohol. Then, three pairs of differential electrodes are attached to the skin above the main active muscles of the forearm of the subject, and the calibration end is placed on the bony prominence of the arm. The Shimmer 3 connects to the computer in real time via Bluetooth and transmits the acquired electromyographic signals. Type A hand gestures were collected, with each gesture repeated 10 times and each gesture lasting 10 seconds. The signal acquisition program on the computer was written in Matlab. Continuous signals were sampled in a 200ms time window with a 50ms time step, and used as input for neural network training. The training set / test set is assigned, and the 2nd, 5th, and 7th repetitions of each action are used as the test set. The method for assigning known and unknown classes is to take the first X classes of action A as the known class and the last Y classes as the unknown class.

5. The method for improving the robustness of an electromyography control system based on generative adversarial networks according to claim 1, characterized in that: In step 2, the CNN needs to correctly classify the known categories. The training set and test set of the CNN only contain data of the known categories. The trained CNN will then be used to construct the dataset after feature extraction. All the original electromyography signal samples will be processed by the CNN feature extraction, and the output will be a vector with a dimension equal to the number of known categories X.

6. The method for improving the robustness of an electromyography control system based on generative adversarial networks according to claim 1, characterized in that: In step 3, after each training round, the GAN discriminator calculates the AUC value on the test set; the training is completed within 2000 epochs, and the best AUC value is recorded; after obtaining the ideal AUC value, the corresponding GAN discriminator becomes the unknown class discriminator.

7. The method for improving the robustness of an electromyography control system based on generative adversarial networks according to claim 1, characterized in that: To obtain the discriminator with the maximum AUC value, a threshold needs to be set, the ROC curve of the discriminator needs to be plotted, and the optimal threshold needs to be obtained based on the Youden index. When the output of the discriminator is greater than or equal to this threshold, it is considered to be a known class; otherwise, it is considered to be an unknown class. At this point, a usable discriminator for unknown classes is obtained.

Images