An electromyographic gesture recognition method and system based on iterative self-training domain adaptation

Abstract

The application belongs to the field of gesture recognition, and provides an electromyography gesture recognition method and system based on iterative self-training domain adaptation. The method comprises the following steps: obtaining electromyography data of a historical user and electromyography data of a new user; extracting source domain time-frequency features based on the electromyography data of the historical user, and extracting target domain time-frequency features based on the electromyography data of the new user; using a difference-based domain adaptation method to align the source domain time-frequency features and the target domain time-frequency features; training a model based on the aligned target domain time-frequency features, judging whether the model meets the iteration number requirement, if yes, predicting the target domain time-frequency features of the electromyography data of the new user to obtain a gesture recognition result; otherwise, giving pseudo-labels to unlabeled data of the target domain time-frequency features, selecting label classes with a number of pseudo-labels less than a certain value for upsampling to balance all classes, repeating the process of aligning the source domain time-frequency features and the target domain time-frequency features until the iteration number judgment process until a set iteration number is reached.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of gesture recognition, and particularly relates to an electromyography gesture recognition method and system based on iterative self-training domain adaptation. BACKGROUND

[0002] The statements in this section merely provide background information related to the present application and do not necessarily constitute prior art.

[0003] Electromyography gesture recognition is a sensor-based gesture recognition method. Compared with gesture recognition based on vision, ultrasonic wave and other sensors, electromyography gesture recognition can realize more natural human-computer interaction. Electromyography signals can directly perceive and analyze human muscle activity, and are not sensitive to light and noise. Gesture recognition based on surface electromyography signals has a wide range of applications, especially in the field of rehabilitation medicine. For example, it can assist in the development of prosthesis control systems and help stroke patients recover.

[0004] Electromyography signals are a kind of physiological electrical signals of the human body, and have strong user dependence. More specifically, the signal distribution between different users has large differences. Traditional domain adaptation models have made great progress in solving the distribution inconsistency in computer vision, natural language processing and other problems, but their performance on such time-series physiological signals is not good. Therefore, in order to realize effective cross-user surface electromyography gesture recognition, traditional domain adaptation models face important challenges. SUMMARY

[0005] In order to solve the technical problems existing in the background art, the present application provides an electromyography gesture recognition method and system based on iterative self-training domain adaptation, that is, an algorithm based on iterative self-training domain adaptation (STDA) is used. This method only uses a small amount of labeled data and a small amount of unlabeled data in the target domain to realize effective transfer from the source domain to the target domain. STDA mainly consists of discrepancy-based domain adaptation (DDA) and pseudo-label iterative update (PIU). DDA uses a distance constraint based on a Gaussian kernel to align the data of existing users and the unlabeled data of new users. PIU iteratively updates the pseudo-labels to generate more accurate labeled data of new users with class balance.

[0006] In order to achieve the above purpose, the present application adopts the following technical solutions:

[0007] The first aspect of the present application provides an electromyography gesture recognition method based on iterative self-training domain adaptation.

[0008] An electromyographic gesture recognition method based on iterative self-training domain adaptation, comprising:

[0009] Obtaining electromyographic data of a historical user and electromyographic data of a new user;

[0010] Based on the electromyographic data of the historical user, extracting source domain time-frequency features, and based on the electromyographic data of the new user, extracting target domain time-frequency features;

[0011] Aligning the source domain time-frequency features and the target domain time-frequency features using a difference-based domain adaptation method;

[0012] Based on the aligned target domain time-frequency features, training a model, determining whether the model meets the iteration number requirement, if yes, predicting the target domain time-frequency features of the electromyographic data of the new user to obtain a gesture recognition result; otherwise, giving pseudo-labels to the unlabeled data of the target domain time-frequency features, selecting label classes with a number of pseudo-labels less than a certain value for upsampling to balance all classes, repeating the process of aligning the source domain time-frequency features and the target domain time-frequency features until the iteration number judgment process until a set iteration number is reached.

[0013] Further, the process of aligning the source domain time-frequency features and the target domain time-frequency features using the difference-based domain adaptation method comprises: mapping the source domain time-frequency features and the target domain time-frequency features to the same shared space, measuring the distance between the two distributions of the source domain time-frequency features and the target domain time-frequency features; and optimizing the target domain time-frequency features according to the distance.

[0014] Still further, the shared space is a reproducing kernel Hilbert space.

[0015] Still further, the distance between the two distributions of the source domain time-frequency features and the target domain time-frequency features is measured using the following formula:

[0016]

[0017] where φ represents a function that maps the original data to H, φ(x) = k(·, x), and k generally takes a Gaussian kernel function.

[0018] Further, the process of training the model further comprises: training a model containing a small amount of labeled data; predicting the class labels of unlabeled samples using the trained model; using a threshold to filter out pseudo-labels that meet the confidence condition; and jointly training the model using labeled and pseudo-labeled data until the model converges to obtain a trained model.

[0019] Still further, the model convergence is determined according to a loss function, and the loss function is:

[0020]

[0021] wherein Y represents an actual label, represents a predicted output, C represents a total number of classifications, p i,k represents a probability that the ith sample is predicted as the kth class.

[0022] Further, the pseudo label calculation formula satisfying the confidence degree is:

[0023]

[0024] wherein, for each sample in a C-classification problem,

[0025] The second aspect of the application provides an electromyography gesture recognition system based on iterative self-training domain adaptation.

[0026] An electromyography gesture recognition system based on iterative self-training domain adaptation, comprising:

[0027] A data acquisition module configured to acquire electromyography data of a historical user and electromyography data of a new user;

[0028] A feature extraction module configured to extract source domain time-frequency features based on the electromyography data of the historical user, and extract target domain time-frequency features based on the electromyography data of the new user;

[0029] An alignment module configured to align the source domain time-frequency features and the target domain time-frequency features by using a difference-based domain adaptation method;

[0030] A training recognition module configured to train a model based on the aligned target domain time-frequency features, judge whether the model meets a number of iterations requirement, if yes, predict the target domain time-frequency features of the electromyography data of the new user to obtain a gesture recognition result, and if not, label pseudo labels to unlabeled data of the target domain time-frequency features, select a label class with a number of pseudo labels less than a certain value for up-sampling to balance all classes, and repeat the process of aligning the source domain time-frequency features and the target domain time-frequency features until the number of iterations is judged to reach a set number of iterations.

[0031] The third aspect of the application provides a computer readable storage medium.

[0032] A computer readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the steps in the electromyography gesture recognition method based on iterative self-training domain adaptation according to the first aspect.

[0033] The fourth aspect of the application provides a computer device.

[0034] A computer device comprises a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the method for electromyography gesture recognition based on iterative self-training domain adaptation according to the first aspect.

[0035] Compared with the prior art, the present application has the following beneficial effects:

[0036] The present application provides a method and system for electromyography gesture recognition based on iterative self-training domain adaptation, which adopts a self-training-based domain adaptation method STDA. The STDA mainly consists of two parts: difference-based domain adaptation (DDA) and pseudo-label iterative update (PIU). The DDA uses a distance constraint based on a Gaussian kernel to align the existing user's data and the new user's unlabeled data. The PIU iteratively and continuously updates the pseudo-labels to generate more accurate label data of the new user with class balance. The pseudo-labels generated by self-training supervise the feature decoupling process to achieve effective transfer from the source domain to the target domain, thereby improving the accuracy of electromyography gesture recognition. BRIEF DESCRIPTION OF DRAWINGS

[0037] The drawings accompanying the specification of the present application form a part thereof and serve to provide further understanding of the present application, the illustrative embodiments of which and its description serve to explain the present application. The present application should not be limited by the inappropriate interpretation of the above description of the illustrative embodiments and its description.

[0038] Figure 1 is a flowchart of the method for electromyography gesture recognition based on iterative self-training domain adaptation shown in the present application;

[0039] Figure 2 is a method framework diagram based on iterative self-training domain adaptation shown in the present application;

[0040] Figure 3 is a schematic diagram of 8 gestures in CapgMyo shown in the present application;

[0041] Figure 4 is a schematic diagram of 12 gestures in CapgMyo shown in the present application;

[0042] Figure 5(a) is a confusion matrix analysis result diagram on the DB-1 dataset shown in the present application;

[0043] Figure 5(b) is a confusion matrix analysis result diagram on the DB-5 dataset shown in the present application;

[0044] Figure 5(c) is a confusion matrix analysis result diagram on the CapgMyo DB-a dataset shown in the present application;

[0045] Figure 5(d) is a confusion matrix analysis result diagram on the CapgMyo DB-b dataset shown in the present application;

[0046] Figure 5(e) is a plot of the confusion matrix analysis result on CapgMyo DB-c dataset shown by the present disclosure;

[0047] Figure 6(a) is a plot of the parameter epoch sensitivity analysis result shown by the present disclosure;

[0048] Figure 6(b) is a plot of the parameter thres sensitivity analysis result shown by the present disclosure. DETAILED DESCRIPTION

[0049] The present disclosure will be further described with reference to the drawings and examples.

[0050] It should be noted that the following detailed description is merely exemplary in nature and is intended to provide further description of the present disclosure. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs.

[0051] It is also to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and / or "comprising," when used in this specification, specify the presence of stated features, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, steps, operations, elements, components, and / or groups thereof.

[0052] It should be noted that the flowchart and block diagrams in the drawings show the architecture, functionality, and operation of possible implementations of methods and systems according to various embodiments of the present disclosure. It should also be noted that each block in the flowchart and block diagrams and / or combinations of blocks in the flowchart and block diagrams can be implemented by a combination of hardware and software, or hardware alone, as desired. Further, it should be noted that each block in the flowchart and block diagrams and / or combinations of blocks in the flowchart and block diagrams can be implemented by special-purpose hardware-based computer systems which are specifically programmed, configured, or constructed to carry out one or more computer program instructions stored in the memory of the computer system. In this manner, the various steps and / or functions illustrated in the flowchart and block diagrams can be realized by a combination of hardware and software.

[0053] Embodiment One

[0054] The embodiment provides an electromyography gesture recognition method based on iterative self-training domain adaptation. The embodiment takes the method applied to a server as an example. It can be understood that the method can also be applied to a terminal and can also be applied to a system including a terminal and a server and is realized through interaction between the terminal and the server. The server can be a physical server, a server cluster composed of multiple physical servers or a distributed system, or a cloud server providing basic cloud computing services such as cloud service, cloud database, cloud computing, cloud function, cloud storage, network server, cloud communication, middleware service, domain name service, security service CDN, and big data and artificial intelligence platform. The terminal can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch and the like, but is not limited thereto. The terminal and the server can be directly or indirectly connected through wired or wireless communication, which is not limited in the application. In the embodiment, the method comprises the following steps:

[0055] Obtaining electromyography data of a historical user and electromyography data of a new user;

[0056] Extracting source domain time-frequency features based on the electromyography data of the historical user and target domain time-frequency features based on the electromyography data of the new user;

[0057] Aligning the source domain time-frequency features and the target domain time-frequency features by using a difference-based domain adaptation method;

[0058] Training a model based on the aligned target domain time-frequency features, judging whether the model meets the iteration number requirement, if yes, predicting the target domain time-frequency features of the electromyography data of the new user to obtain a gesture recognition result, and if not, labeling the target domain time-frequency features with pseudo labels, selecting a label class with a number of pseudo labels less than a certain value for up-sampling to balance all classes, and repeating the process of aligning the source domain time-frequency features and the target domain time-frequency features until the iteration number judgment process until a set iteration number is reached.

[0059] A flowchart of the electromyography gesture recognition method based on iterative self-training domain adaptation is shown in Figure 1 .

[0060] 1) Start

[0061] 2) Extract source domain time-frequency features and target domain time-frequency features

[0062] 3) Judge whether the pre-training round number is met, if yes, 4), and if not, 5)

[0063] 4) Align the source domain and target domain features

[0064] 5) Train the classifier

[0065] 6) Judge whether the set training round number is reached, if yes, 7), and if not, 8)

[0066] 7) Predict on target domain test data, go to 10)

[0067] 8) Pseudo-label target domain unlabeled data

[0068] 9) Up-sample minority class to balance all classes, go to 4)

[0069] 10) End.

[0070] 1. Problem definition

[0071] Data of existing users constitute source domain where N s denotes the number of samples in source domain, denotes the i-th sample in source domain, denotes the label corresponding to the i-th sample in source domain. Data of target users constitute target domain where denotes the small amount of labeled data in target domain (for a C(C∈N + ) classification problem, it has only C samples), denotes the relatively large amount of unlabeled data in target domain, denotes the test data in target domain, and satisfies where N l denotes the number of labeled samples in target domain. where N u denotes the number of unlabeled samples in target domain. where N te denotes the number of test samples in target domain. In the cross-user scenario, there is domain shift, i.e. the distributions of source domain and target domain are inconsistent The above x s ,x t are four-dimensional data vectors, y s ,y t are class labels. The goal of cross-user electromyography gesture recognition is to obtain accurate test data labels using the existing users' data a small amount of labeled data on the target domain and a relatively large amount of unlabeled data on the target domain

[0072] 2. Method based on iterative self-training domain adaptation

[0073] Iterative Self-Training based Domain Adaptation (STDA) is a domain adaptation method that generates pseudo-labels through iterative self-training to supervise the process of feature decoupling, and realizes effective transfer from source domain to target domain. The overall framework of the STDA method is shown in FIG. 1, which is composed of two parts: discrepancy-based domain adaptation (DDA) and pseudo-label iterative update (PIU), which are used to align features and generate more accurate target domain labels, respectively. Figure 2

[0074] DDA uses maximum mean discrepancy (MMD) as a distance measure to align the source and target domain features in the reproducing kernel Hilbert space (RKHS). PIU mainly consists of two parts: one is to generate more accurate pseudo-labels through iterative self-training, and the other is to make the generated pseudo-labels more balanced through data-level upsampling. The basic idea of STDA is to supervise the process of feature decoupling through more accurate target domain pseudo-labels generated by continuous iteration to realize effective transfer from source domain to target domain.

[0075] (1) Discrepancy-based domain adaptation

[0076] The source domain is represented as The target domain is represented as The input space and label space of the source domain and the target domain are the same, but the probability distributions of the two are inconsistent, i.e. S ≠P t The purpose of feature alignment is to learn a good mapping (f) that simultaneously maps the source domain and the target domain to a shared space (H) so that the distance between the two is relatively close The kernel learning method based on maximum mean discrepancy (MMD) effectively measures the difference between distributions. Maximum mean discrepancy measures the distance between two distributions in the reproducing kernel Hilbert space (RKHS). The distance between two distributions P s and P t can be defined as:

[0077]

[0078] where, ​denotes a set of functions in RKHS, E x～ denotes expectation in source domain or target domain. When the distribution of source domain and target domain is close, the distance D is close to 0. The MMD between source domain and target domain can be calculated as:

[0079]

[0080]

[0081] where φ denotes a function that maps the original data to H, φ(x)=k(·,x), and k is usually taken as Gaussian kernel function. In this high-dimensional space, the distribution difference between source domain and target domain samples can be measured by this distance. Therefore, the MMD distance is usually considered as a loss, embedded into the high layer of the network of deep learning, and then optimized.

[0082] (2) Pseudo label iterative update

[0083] The pseudo label iterative update mainly includes two parts, one is iterative self-training, and the other is class rebalancing. The iterative self-training in the embodiment mainly includes the following four steps: 1) training a model containing a small amount of labeled data; 2) using the trained model to predict the class label of the unlabeled sample; 3) using a threshold to filter out the pseudo label with confidence satisfying the condition; 4) training the model with labeled and pseudo-labeled data jointly, and repeating 1)-4) until the model converges. For multi-classification problems, cross-entropy is used, and the calculation formula is as follows:

[0084]

[0085] where Y represents the actual label, represents the predicted output, C represents the total number of classifications, p i,k represents the probability of the i-th sample being predicted as the k-th class. For each sample in a C-classification problem, The pseudo label based on the softmax confidence is calculated as follows:

[0086]

[0087] The class rebalancing in the embodiment is to prevent the misleading of the model learning by the unbalanced class samples, and a data-level method oversampling is adopted to make the minority class and the majority class samples comparable. In order to balance the classes, a balance loss is defined. Assuming that the number of a group of gesture classes is x1, x2, …, x n , the calculation formula of the balance loss is as follows:

[0088]

[0089]

[0090] wherein the model comprises mainly two parts, domain difference based adaptation (DDA) and pseudo label iterative update (PIU) as shown in Figure 2 Firstly, the time-frequency features of the existing users' electromyographic data in the source domain are extracted, and the time-frequency features of the new users in the target domain are also extracted; secondly, the source domain and the target domain are extracted features through convolutional neural network (CNN) and self-attention (Self-attention); then the maximum mean difference (MMD) distance of the source domain features and the target domain features is used as the loss of the model optimization, at the same time, the target domain model is used to give the target domain unmarked data a pseudo label, and the category rebalance is used to balance all classes on the target domain; iterate until the model converges.

[0091] 3. Experimental evaluation

[0092] (1) Dataset

[0093] Table 1 Commonly used public datasets

[0094]

[0095] NinaPro: NinaPro DB-1 and DB-5 are electromyographic data collected by sparse electrode sensing devices, which are one of the most commonly used public datasets for electromyographic gesture recognition. At present, the NinaPro dataset has developed to DB10, which contains data of healthy subjects and amputee patients. The DB-1 sub-dataset includes 27 healthy subjects, a total of 52 gestures, including 12 basic finger movements, 8 basic hand movements, 9 basic wrist movements, and 23 grip and functional movements. The surface electromyographic data of DB1 dataset is collected by Otto Bock MyoBock with 10 channels (differential electrodes), and the sampling frequency is 100 Hz. DB-5 includes 10 subjects, 52 gestures, 16 channels, and a sampling frequency of 200 Hz.

[0096] CapgMyo: CapgMyo is electromyographic data collected by high-density electrode array sensing devices, which is one of the most commonly used public datasets for electromyographic gesture recognition. CapgMyo is 128-channel high-density surface electromyographic data of 23 healthy subjects collected by the self-developed device of Professor Geng Weidong's team of Zhejiang University, with a sampling frequency of 1000 Hz, including three sub-datasets, DB-a, DB-b and DB-c. DB-a contains 8 finger gestures of 18 subjects, DB-b contains 8 gestures collected at two different time periods of 10 subjects, and DB-c contains 12 basic finger gestures of 10 subjects.

[0097] The 8 gestures and 12 gestures in CapgMyo are respectively as followsFigure 3 and Figure 4 .

[0098] (2) Comparison methods

[0099] In order to verify the effect of the STDA method, seven methods are selected as comparison methods in this embodiment, including:

[0100] • A variant of STDA that only uses the source domain to train the model (Only-Source);

[0101] • A variant of STDA that only uses the target domain to train the model (Only-Target);

[0102] • A fine-tuning-based method that first decomposes the channel and then fuses the features (Multi-Stream');

[0103] • A dual-stream supervised domain adaptation framework (MDSDA);

[0104] • A domain adaptation method based on kernel space distance (SGAS);

[0105] • An unsupervised domain adaptation method with pseudo group contrast mechanism (Self-Tuning);

[0106] • A cyclic self-training domain adaptation method (CST).

[0107] Among them, Only-Source and Only-Target are two benchmark methods, Multi-Stream' is a fine-tuning method based on deep learning, MDSDA and SGAS are two supervised domain adaptation methods, and Self-Tuning and CST are two unsupervised domain adaptation methods. The experimental environment is Linux 125GB, the development environment is Python3.8.3, and the main third-party library is pytorch 1.10.2+cu113. During the experiment, the main parameter settings are as follows: the confidence parameter is set to 0.99, the pre-training number of rounds is set to 400, and the learning rate is set to 0.001.

[0108] (3) Comparison experiment results

[0109] Table 2 Comparison experiment results

[0110]

[0111] The comparative experimental results are shown in Table 2. According to the experimental results shown in Table 2, the STDA method proposed in the embodiment is obviously superior to other methods on the public benchmark myoelectric gesture recognition dataset. Compared with the baseline method, it is improved by more than 25%; compared with the fine-tuning method, it is improved by more than 8% on the remaining datasets except the DB-1 dataset; compared with the supervised domain adaptation method, it is improved by more than 5%; compared with the unsupervised domain adaptation method, it is improved by more than 24%.

[0112] (4) Confusion matrix analysis

[0113] The confusion matrix analysis results are shown in Table 3. Figures 5(a)-5(e) From the confusion matrices on the DB-1 and DB-5 datasets, it can be seen that the 28th and 51st gestures on DB-1 have higher accuracy than other gestures; the 1st, 4th, 16th, 24th, and 27th gestures on DB-5 have higher accuracy than other gestures, indicating that the design of a complex gesture set is also crucial in the construction of a gesture recognition system. In addition, some gestures are also easily confused. For example, on the DB-1 dataset, the 50th gesture has about a 24% probability of being misjudged as the 49th gesture. Similarly, the 14th gesture is also easily misjudged as the 13th gesture. Similar phenomena also occur on the DB-5 dataset. For example, the 8th gesture is easily misjudged as the 10th gesture, and the 13th gesture is easily misjudged as the 14th gesture. The false positive rate is as high as 30%. Similar conclusions are also obtained on the three sub-datasets DB-a, DB-b, and DB-c of CapgMyo.

[0114] From the two eight-class datasets DB-a and DB-b of the CapgMyo dataset, it can be seen that in DB-a, the fifth gesture has the highest accuracy of 71.6%, and in DB-b, the fifth gesture has the second highest accuracy, which is very close to the highest accuracy of 81.7% in DB-b. At the same time, the third gesture has the lowest accuracy in DB-a and DB-b, with an accuracy of 35.2% in DB-a and an accuracy of 57.5% in DB-b. One possible reason is that the fifth gesture is relatively different from the other seven gestures, while the third gesture is very similar to several of the remaining seven gestures.

[0115] (5) Ablation experiment analysis

[0116] Table 3 Ablation experiment results

[0117]

[0118] The ablation experiment results are shown in Table 3. To explore the contribution of each component, we conducted ablation experiments. The SDTA method mainly consists of two modules: feature space alignment (MMD) and iterative update of pseudo-labels (self-training). Overall, the STDA method combining feature alignment and self-training exhibits the best performance, demonstrating its effectiveness. Furthermore, experimental results show that the self-training strategy contributes the most to the DB-5, DB-a, DB-b, and DB-c datasets. On the DB-1 dataset, the main contribution is feature space alignment.

[0119] (6) Parameter sensitivity analysis

[0120] The results of the parameter sensitivity analysis are as follows: Figures 6(a)-6(b) As shown in the figure, the SDDA method is primarily sensitive to two parameters: the number of pre-training epochs ("epoch") and the confidence threshold for iterative self-training ("thres"). To evaluate the impact of these parameters on the performance of the STDA method, we used a univariate approach, meaning we changed one variable while keeping the other constant. The range of the parameter "epoch" was set to {50, 100, 200, 400, 600, 800}; the range of the parameter "thres" was set to {0.7, 0.8, 0.9, 0.95, 0.99}. The triangle in the figure represents the optimal values. It can be seen that the STDA method performs best when "epoch" is 100, and worst when "epoch" is 600. This indicates that appropriate pre-training is beneficial for model learning, while excessive pre-training may lead to overfitting. The STDA method performs best when "thres" is 0.95, and worst when "thres" is 0.8. This indicates that too low a confidence level can lead to a large number of incorrect labels, misleading the model's learning, while too high a confidence level can also have an adverse effect on the model.

[0121] Example 2

[0122] This embodiment provides an electromyographic gesture recognition system based on iterative self-training domain adaptation.

[0123] An electromyographic gesture recognition system based on iterative self-training domain adaptation includes:

[0124] The data acquisition module is configured to acquire electromyography (EMG) data of historical users and EMG data of new users.

[0125] The feature extraction module is configured to: extract source domain time-frequency features based on historical users' electromyography (EMG) data, and extract target domain time-frequency features based on new users' EMG data;

[0126] an alignment module configured to align the source domain time-frequency features and the target domain time-frequency features by using a difference-based domain adaptation method;

[0127] a training recognition module configured to train the model based on the aligned target domain time-frequency features, determine whether the model reaches a required number of iterations, if yes, predict the target domain time-frequency features of the new user's electromyography data to obtain a gesture recognition result, and if not, label the target domain time-frequency features with pseudo labels, select a label class with a number of pseudo labels less than a certain value for up-sampling to balance all classes, and repeat the process of aligning the source domain time-frequency features and the target domain time-frequency features until the number of iterations reaches a set number of iterations.

[0128] It should be noted that the data acquisition module, the feature extraction module, the alignment module and the training recognition module are the same as the examples and application scenarios realized by the steps in Embodiment One, but are not limited to the content disclosed in Embodiment One. It should be noted that the above modules as part of the system can be executed in a computer system such as a set of computer executable instructions.

[0129] Embodiment Three

[0130] The embodiment provides a computer readable storage medium, which stores a computer program, and the program is executed by a processor to realize the steps in the electromyography gesture recognition method based on iterative self-training domain adaptation as described in Embodiment One.

[0131] Embodiment Four

[0132] The embodiment provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor realizes the steps in the electromyography gesture recognition method based on iterative self-training domain adaptation as described in Embodiment One when executing the program.

[0133] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can be in the form of a hardware embodiment, a software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can be in the form of a computer program product implemented on one or more computer usable storage media (including but not limited to magnetic disk storage and optical storage, etc.) containing computer usable program code.

[0134] The embodiments of methods, apparatuses (systems) and computer program products according to the present application can be described in the general context of method steps and processes, which can be implemented in one embodiment by a program of instructions on a computer-readable storage medium executed by a computer or other programmable apparatus. The apparatuses can be specially constructed for executing the embodiments of methods, apparatuses (systems) and computer program products according to the present application or can include a computer or other programmable apparatus. Figure 1 The flow and / or block diagram in which each flow and / or block includes the functions specified in that flow or block. Figure 1 The apparatus that carries out the functions specified in one or more flows and / or blocks.

[0135] The computer program instructions can also be loaded onto a computer or other programmable apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flow Figure 1 The flow and / or block diagram in which each flow and / or block includes the functions specified in that flow or block. Figure 1 The apparatus that carries out the functions specified in one or more flows and / or blocks.

[0136] The computer program instructions can also be loaded onto a computer or other programmable apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flow Figure 1 The flow and / or block diagram in which each flow and / or block includes the functions specified in that flow or block. Figure 1 The apparatus that carries out the functions specified in one or more flows and / or blocks.

[0137] A person of ordinary skill in the art can understand that all or part of the above-mentioned method embodiments can be implemented by computer program instructions, and the program can be stored in a computer readable storage medium. When the program is executed, the program can include the flow of the above-mentioned method embodiments. The storage medium can be a magnetic disc, an optical disc, a read-only memory (ROM) or a random access memory (RAM).

[0138] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. An electromyography gesture recognition method based on iterative self-training domain adaptation, characterized in that, The method comprises the following steps: obtaining historical user's electromyography data and new user's electromyography data; extracting source domain time-frequency features based on the historical user's electromyography data, and extracting target domain time-frequency features based on the new user's electromyography data; extracting features of the source domain and the target domain through a convolutional neural network and self-attention; aligning the source domain time-frequency features and the target domain time-frequency features by using a difference-based domain adaptation method; training a model based on the aligned target domain time-frequency features, judging whether the model meets the iteration number requirement, if yes, predicting the target domain time-frequency features of the new user's electromyography data to obtain a gesture recognition result, otherwise, labeling the unlabeled data of the target domain time-frequency features with pseudo labels, selecting a label type with a number of pseudo labels less than a certain value for upsampling to balance all categories, repeating the alignment of the source domain time-frequency features and the target domain time-frequency features until a set iteration number is reached.

2. The method of claim 1, wherein, The process of aligning the source domain time-frequency features and the target domain time-frequency features by using the difference-based domain adaptation method comprises the following steps: mapping the source domain time-frequency features and the target domain time-frequency features to the same shared space, measuring the distance between the two distributions of the source domain time-frequency features and the target domain time-frequency features; and optimizing the target domain time-frequency features according to the distance.

3. The method of claim 2, wherein, The shared space is a reproducing kernel Hilbert space.

4. The method of claim 2, wherein, The distance between the two distributions of the source domain time-frequency features and the target domain time-frequency features is measured by using the following formula: The source domain is represented as The target domain is represented as ; Indicates the number of samples in the source domain. Indicates the first in the source domain One sample, Indicates the first in the source domain The label corresponding to each sample; This indicates the number of samples in the target domain. Let j represent the j-th sample in the target domain; where, This represents a mapping of raw data to... The function, And k takes the Gaussian kernel function.

5. The method of claim 1, wherein, In the process of training the model, the method further comprises the following steps: training a model containing a small amount of labeled data; predicting the class labels of unlabeled samples by using the trained model; screening out pseudo labels with confidence satisfying a condition by using a threshold; and jointly training the model by using the data with labels and pseudo labels until the model converges, thereby obtaining a trained model.

6. The method of myoelectric gesture recognition based on iterative self-training domain adaptation according to claim 5, characterized in that, The model convergence is judged according to a loss function, and the loss function is as follows: where, denotes the actual label, denotes the predicted output, denotes the number of samples in the source domain, denotes the total number of classes, denotes the probability that the th sample is predicted to be the th class.

7. The method of claim 5, wherein, The pseudo label with the confidence satisfying the condition is calculated by using the following formula: wherein, for one each sample in the classification problem, .

8. An electromyography gesture recognition system based on iterative self-training domain adaptation, characterized in that, The method comprises the following steps: a data acquisition module configured to obtain historical user's electromyography data and new user's electromyography data; a feature extraction module configured to extract source domain time-frequency features based on the historical user's electromyography data, and extract target domain time-frequency features based on the new user's electromyography data; extracting features of the source domain and the target domain through a convolutional neural network and self-attention; an alignment module configured to align the source domain time-frequency features and the target domain time-frequency features by using a difference-based domain adaptation method; a training and recognition module configured to train a model based on the aligned target domain time-frequency features, judge whether the model meets the iteration number requirement, if yes, predict the target domain time-frequency features of the new user's electromyography data to obtain a gesture recognition result, otherwise, label the unlabeled data of the target domain time-frequency features with pseudo labels, select a label type with a number of pseudo labels less than a certain value for upsampling to balance all categories, and repeat the alignment of the source domain time-frequency features and the target domain time-frequency features until a set iteration number is reached.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the steps in the electromyography gesture recognition method based on iterative self-training domain adaptation according to any one of claims 1-7.

10. A computer device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the program to implement the steps in the electromyography gesture recognition method based on iterative self-training domain adaptation according to any one of claims 1-7.

Images