Cross-user electromyographic gesture recognition method and system based on multi-source domain integration migration

Abstract

The present invention discloses a cross-user electromyographic gesture recognition method and system based on multi-source domain integrated migration, including: constructing an electromyographic gesture recognition source domain individual model for each user; using the accuracy of the target user on the single source domain individual model and the complexity index of the single source domain individual model to measure the fitness of the target user on the source domain individual model, and calculating the fitness evaluation score; updating the single source domain individual model according to the fitness evaluation score; integrating the updated single source domain individual model through a majority voting mechanism to obtain an integrated source domain individual model; obtaining the electromyographic signal data of the target user, and obtaining the electromyographic gesture recognition result of the target user based on the integrated source domain individual model. The present invention is based on a multi-source domain integrated migration learning algorithm, uses electromyographic signal data for gesture recognition, and can obtain a high-accuracy model based on a source model construction strategy and a target model adaptation strategy.

Description

Technical Field

[0001] The present invention relates to the field of gesture recognition technology, and in particular to a cross-user electromyographic gesture recognition method and system based on multi-source domain integrated migration. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Gesture recognition is widely used in health assistance, interactive entertainment, smart home, smart car, smart supermarket, industrial quality inspection and other fields; hands, as the most flexible part of the human body, are of great significance to an individual's daily life.

[0004] Myoelectric gesture recognition is an important gesture recognition method. Gesture recognition based on electromyography can directly perceive and analyze human muscle activities, so it can achieve more natural human-computer interaction. Myoelectric gesture recognition has the advantages of recognizing fine gestures, being non-intrusive, and not being affected by light and noise. Myoelectric signals are physiological signals that are easily affected by physiological factors such as muscle distribution and skin condition, because the distribution of myoelectric signals of different users is different. The gesture recognition model built based on traditional machine learning methods has obvious user dependence problems. In the process of achieving effective myoelectric gesture recognition, there are the following challenges: the personalized adaptation of the myoelectric gesture recognition model is difficult, and obtaining a large amount of labeled data for new users will seriously increase the burden on users. Therefore, how to use a small number of samples to achieve personalized adaptation of existing models is an important challenge facing myoelectric gesture recognition. Summary of the invention

[0005] In order to solve the above problems, the present invention proposes a cross-user electromyographic gesture recognition method and system based on multi-source domain integrated migration. The transfer learning method based on multi-source domain integration is a transfer learning framework suitable for integrated learning, including a source model construction strategy and a target model adaptation strategy. The source model construction strategy constructs a model with strong generalization ability by constructing "good but different" individual models; the target model adaptation strategy updates the individual model by mining the strong and weak relationships between different users under the same gesture, so as to improve the accuracy of electromyographic gesture recognition.

[0006] In some embodiments, the following technical solutions are adopted:

[0007] A cross-user electromyographic gesture recognition method based on multi-source domain integration transfer, comprising:

[0008] Build an individual model of the source domain of electromyographic gesture recognition for each user;

[0009] The accuracy of the target user on a single source domain individual model and the complexity index of a single source domain individual model are used to measure the fitness of the target user on the source domain individual model, and the fitness evaluation score is calculated;

[0010] The single source domain individual model is updated according to the fitness evaluation score; the updated single source domain individual model is integrated through the majority voting mechanism to obtain the integrated source domain individual model;

[0011] The target user's electromyographic signal data is obtained, and the target user's electromyographic gesture recognition result is obtained based on the integrated source domain individual model.

[0012] As an optional solution, a myoelectric gesture recognition source domain individual model is constructed for each user, and the source domain individual model is constructed using a convolutional neural network; the source domain individual model is trained using source domain data.

[0013] As an optional solution, calculate the accuracy of the target user on a single source domain individual model, specifically:

[0014] For the target domain The data pairs in the dataset are labeled, and the number of data pairs whose predicted labels and true labels are equal obtained by a single source domain individual model is calculated. This number is equal to the target domain The ratio of the total number of data pairs in is the accuracy of the source domain individual model; Provide labeled data for the target domain.

[0015] As an optional solution, the complexity of the target user on a single source domain individual model is calculated as follows:

[0016] The ratio of the sum of the parameters of each layer of the i-th source domain individual model to the sum of the parameters of each layer of all source domain individual models.

[0017] As an optional solution, the fitness evaluation score of the i-th source domain individual model is:

[0018] score i =ω*acc i +(1-ω)*com i

[0019] Among them, acc i is the accuracy of the individual model of the i-th source domain, com i is the complexity of the individual model of the i-th source domain, and ω is the adaptive weight.

[0020] As an optional solution, a single source domain individual model is updated according to the fitness evaluation score, specifically:

[0021] Calculate the number of rounds that a single source domain individual model needs to be adjusted based on the evaluation score:

[0022] epoch i =c-score i *p

[0023] Among them, epoch i It indicates the number of rounds that the i-th model needs to be adjusted, and c and p are both constants.

[0024] As an optional solution, the updated individual models of the source domain are integrated through a majority voting mechanism, specifically:

[0025] Assume that there are N different source domain individual models in the source domain, and there is a classification task of K types of gestures;

[0026] Any input data is passed through each source domain individual model to obtain the corresponding gesture type;

[0027] Traverse all output values, count the number of each gesture type, and select the gesture type with the largest number of outputs as the output of the integrated source domain individual model.

[0028] In other embodiments, the following technical solutions are adopted:

[0029] A cross-user electromyographic gesture recognition system based on multi-source domain integrated migration, comprising:

[0030] An individual model building module is used to build an individual model of the source domain of electromyographic gesture recognition for each user;

[0031] The individual model evaluation module is used to measure the fitness of the target user on the source domain individual model by using the accuracy of the target user on the single source domain individual model and the complexity index of the single source domain individual model, and calculate the fitness evaluation score;

[0032] The individual model integration module is used to update the individual model of a single source domain according to the fitness evaluation score; the updated individual model of the single source domain is integrated through the majority voting mechanism to obtain the integrated source domain individual model;

[0033] The gesture recognition module is used to obtain the target user's electromyographic signal data and obtain the target user's electromyographic gesture recognition result based on the integrated source domain individual model.

[0034] In other embodiments, the following technical solutions are adopted:

[0035] A terminal device includes a processor and a memory, wherein the processor is used to implement various instructions; and the memory is used to store multiple instructions, wherein the instructions are suitable for being loaded by the processor and executing the above-mentioned cross-user electromyographic gesture recognition method based on multi-source domain integrated migration.

[0036] In other embodiments, the following technical solutions are adopted:

[0037] A computer-readable storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor of a terminal device and executing the above-mentioned cross-user electromyographic gesture recognition method based on multi-source domain integrated migration.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] (1) The present invention is based on a transfer learning algorithm based on multi-source domain integration, uses electromyographic signal data for gesture recognition, and builds a model with strong generalization ability by constructing "good but different" individual models based on the source model construction strategy; based on the target model adaptation strategy, the individual model is updated by mining the strong and weak relationships between different users under the same gesture, thereby obtaining a model with high accuracy. The present invention has conducted detailed experiments on a public dataset of electromyographic gesture recognition. The experimental results show that the present invention has a significant improvement in accuracy.

[0040] (2) The present invention uses the accuracy of the target user on a single source domain individual model and the complexity index of a single source domain individual model to measure the fitness of the target user on the source domain individual model, and uses the fitness score to fine-tune the parameters of the individual model, thereby achieving effective migration from the source domain to the target domain.

[0041] Other features and advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through the practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is a flow chart of a cross-user electromyographic gesture recognition method based on multi-source domain integrated migration in an embodiment of the present invention;

[0043] Figure 2 It is a framework diagram of a transfer learning method based on multi-source domain integration in an embodiment of the present invention;

[0044] Figure 3 Schematic diagram of 52 gestures of NinaPro DB1 data set in an embodiment of the present invention;

[0045] FIG4(a)-(b) are schematic diagrams of 8 gestures and 12 gestures in the CapgMyo dataset in an embodiment of the present invention;

[0046] Figure 5(a)-(d) are schematic diagrams of confusion matrix analysis results;

[0047] Figure 6(a)-(d) are schematic diagrams of the results of the labeled data volume analysis;

[0048] Figure 7(a)-(b) are schematic diagrams of parameter sensitivity analysis results. DETAILED DESCRIPTION

[0049] It should be noted that the following detailed descriptions are illustrative and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used in the present invention have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0050] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0051] Embodiment 1

[0052] Transfer learning is a new learning model that is different from traditional machine learning. Its goal is to apply the knowledge or patterns learned in one field or task to another different but related field or task. Specifically, transfer learning can use the model built for task A as a starting point and reuse it in the process of building the recognition model for task B. Transfer learning can make model building more efficient and faster.

[0053] Based on this, the embodiment of the present invention is based on a multi-source integration based transfer learning algorithm (Multi-Source Integration based Transfer Learning, MSITL). This method only uses a small amount of target domain data to effectively transfer source domain knowledge to the target domain. MSITL is an integrated model consisting of two parts: a source model construction strategy (Source Model Construction Strategy, SMCS) and a target model adaptation strategy (Target Model Adaptation Strategy, TMAS). Figure 2 The source model construction strategy builds a model with strong generalization ability through a hierarchical integrated model. The first layer builds a good performance model for each user; the second layer integrates the built individual models through a simple majority voting mechanism. The target model adaptation strategy (TMAS) first evaluates the source model using the target domain data and calculates the evaluation score based on the two indicators of model complexity and accuracy. Then, under the guidance of the evaluation score, the individual model is fine-tuned to obtain a high-accuracy model.

[0054] In one or more embodiments, a cross-user electromyographic gesture recognition method based on multi-source domain integration migration is disclosed, combined with Figure 1 , specifically including the following process:

[0055] (1) Build an individual model of the source domain for myoelectric gesture recognition for each user;

[0056] Specifically, cross-user surface electromyography gesture recognition is a multi-classification task. Assuming that there is already some user labeled data, the source domain can be represented as Where N represents the number of users, S i represents the i-th source domain (i.e., the i-th user). Among them, M i represents the number of samples of the i-th user. The target domain can be expressed as in It is labeled data. It is unlabeled data. Where P represents the number of labeled samples in the target domain. Where Q represents the number of unlabeled samples in the target domain. The data dimensions are the same, that is, in The label space is the same, i.e. The data distribution is inconsistent, that is Visible during training, The goal of cross-user surface electromyography gesture recognition is to use source domain data. and target domain data Predicting target domain data The label, that is

[0057] MSITL is based on a simple majority voting strategy and is integrated from multiple neural networks. Existing theories show that: represents the generalization error value of a single classifier, Represents the ambiguity of a single classifier. As can be seen from the formula, the higher the accuracy of the source domain individual model (individual classifier) ​​and the greater the diversity, the better the ensemble effect. Therefore, in order to build an ensemble with good generalization performance, a single source domain individual model needs to be "good but different". The basic idea of ​​MSITL is to mine the association between new users and existing users under the same gesture to construct a high-accuracy model, while ensuring the diversity between each model.

[0058] MSITL consists of two parts: source model construction strategy and target model adaptation strategy, which are used to build an integrated model with strong generalization ability and to fine-tune individual models respectively.

[0059] The source model building strategy mainly consists of two steps:

[0060] First, build a separate gesture recognition model for each user to increase the diversity of the entire model and retain personalized information.

[0061] Second, integrate various models to make a comprehensive judgment on gesture categories.

[0062] In this embodiment, a source domain individual model is constructed for each user, and we use convolutional neural networks and fully connected networks with different numbers of layers. The convolution layer uses a one-dimensional convolutional network to more effectively extract the features of the time series signal. Under the setting of the parameters in the present invention, the calculation formula of the one-dimensional convolution is:

[0063]

[0064] Among them, kernel_size is the kernel size of the convolution layer, and stride is the sliding step size of the convolution layer. The input data format is input=(N,C in ,L in ), the output data format is output=(N,C out ,L out ), where N represents the number of samples, C in and C out Respectively represent the number of input and output channels, L in and L out Represent the input and output sequence lengths respectively. Build a model for each user and get N models. is the training data of the i-th model, Model i is the trained model. The loss value is calculated using the cross entropy loss function, and the calculation formula is:

[0065]

[0066] Among them, Y represents the true value, P represents the predicted value, K represents the total number of labels (i.e. the number of gesture categories), and p i,k It represents the probability that the i-th sample is predicted to be the k-th label.

[0067] (2) The accuracy of the target user on a single source domain individual model and the complexity index of a single source domain individual model are used to measure the fitness of the target user on the source domain individual model, and the fitness evaluation score is calculated;

[0068] The target model adaptation strategy also mainly includes two steps:

[0069] First, evaluate the individual source model to measure the relevance of different users (different users refer to users in a certain source domain and new users in the target domain).

[0070] Second, update the source model to adapt it to the target domain. For some individual models, if the accuracy of the new user on the model is high, the model does not need to be adapted, otherwise it needs to be adjusted appropriately.

[0071] In addition, for a relatively complex model, further complication should be avoided. Based on the above two points, this embodiment uses the accuracy of the target user on a single model and the complexity of a single model to measure the fitness of the target user on the source domain. Calculate the accuracy of the individual model, specifically: The data pairs in the dataset are labeled, and the number of data pairs whose predicted labels and true labels are equal obtained by a single source domain individual model is calculated. This number is equal to the target domain The ratio of the total number of data pairs in is the accuracy of the individual model in the source domain.

[0072] The calculation formula is:

[0073]

[0074] Among them, x represents the input data, Represents the indicator function, the true value y and the predicted value If they are the same, it is 1; if they are different, it is 0. Represents: for the target domain This data set (set), take out all the data and add labels (a (x, y) represents an element in the set, a data pair), x is the data, y is the true label, calculate the predicted label Is it equal to the true label y? Count (sum) the equal ones.

[0075] The complexity of the source domain individual model is specifically: the ratio of the sum of the parameters of each layer of the i-th source domain individual model to the sum of the parameters of each layer of all source domain individual models.

[0076] The calculation formula is:

[0077]

[0078] Among them, param jk It represents the number of parameters in the kth layer of the jth individual model. The number of parameters refers to the number of parameters. For example, a model has 2 layers, the first layer has 5 parameters, a1, a2, a3, a4, a5; the second layer has 6 parameters, b1, b2, b3, b4, b5, b6. The total number of parameters of this model is 5+6=11.

[0079] The evaluation score is calculated according to the evaluation function. The calculation formula is:

[0080] scorei =ω*acc i +(1-ω)*com i (5)

[0081] Among them, ω is an adaptive parameter, score i Represents the evaluation score of the ith model.

[0082] (3) Update the individual source domain models according to the fitness evaluation scores; integrate the updated individual source domain models through the majority voting mechanism to obtain the integrated source domain individual model;

[0083] In this embodiment, the number of rounds that the individual model needs to be adjusted is calculated based on the evaluation score, and the calculation formula is:

[0084] epoch i =c-score i *1000 (6)

[0085] Among them, epoch i represents the number of rounds that the i-th model needs to be adjusted, and c is a constant related to the specific experimental dataset.

[0086] (4) Obtain the target user’s electromyographic signal data and obtain the target user’s electromyographic gesture recognition results based on the integrated source domain individual model.

[0087] In this embodiment, the idea of ​​ensemble learning is adopted to integrate the source domain models using a simple majority voting mechanism. Assume that the source domain has N different source domain individual models (individual classifiers), C i (i=1,2,...,N), there is a K classification task (i.e. K gestures). Any input data x j The output o is obtained by all individual classifiers i (i=1,2,...,N). Traverse all output values ​​and count the number of each category. k (k=1,2,...,K). Calculate the maximum value of these K categories, max{count k}(k=1,2,...,K), the obtained value is the output of the integrated model.

[0088] In this embodiment, for the target user's electromyographic signal data, the outputs of N source domain individual models are obtained, and the predicted output of the current electromyographic signal data is obtained through a simple majority voting mechanism.

[0089] This embodiment uses the data set given in Table 1 to experimentally verify the method of this embodiment.

[0090] Table 1 Commonly used public datasets

[0091]

[0092] Among them, NinaPro DB1 dataset: NinaPro DB1 is electromyographic data collected by sparse electrode sensing devices. It is one of the most commonly used public datasets for electromyographic gesture recognition. Currently, the NinaPro dataset has developed to DB10, which contains data from healthy subjects and patients with amputated limbs. The DB1 sub-dataset includes 27 healthy subjects and a total of 52 gestures, such as Figure 3 As shown, there are 12 basic finger movements, 8 basic hand movements, 9 basic wrist movements, and 23 grasping and functional movements. The surface electromyography data of the DB1 dataset was collected using a 10-channel (differential electrode) OttoBock MyoBock with a sampling frequency of 100 Hz.

[0093] CapgMyo dataset: CapgMyo is electromyographic data collected by a high-density electrode array sensing device. It is one of the most commonly used public datasets for electromyographic gesture recognition. CapgMyo is a 128-channel high-density surface electromyographic data of 23 healthy subjects collected by Professor Geng Weidong's team at Zhejiang University using self-developed equipment. The sampling frequency is 1000Hz and contains 3 sub-datasets, DB-a, DB-b and DB-c. DB-a contains 8 finger gestures of 18 subjects, DB-b contains 8 gestures collected by 10 subjects in 2 different time periods, and DB-c contains 12 basic finger gestures of 10 subjects, as shown in Figure 4 (a)-(b).

[0094] In order to verify the effect of the method of this embodiment, 7 methods are selected as comparison methods, including:

[0095] ① Direct modeling using labeled data from the target domain (DIRECT);

[0096] ② A variant of MSITL, using a unified fine-tuning strategy to replace TMAS (AVG);

[0097] ③ A variant of MSITL, using a single-source domain strategy to replace SMCS (SINGLE);

[0098] ④ A two-step recurrent neural network method (2SRNN) that alternately freezes the weights of the domain adaptation layer and the sequence classifier;

[0099] ⑤ Fine-tuning method using continuous wavelet transform for preprocessing (CWT);

[0100] ⑥ Unsupervised domain adaptation method based on adversarial learning (MADA);

[0101] ⑦ Difference-based unsupervised domain adaptation method (DRCN).

[0102] Among them, DIRECT, AVG and SINGLE are three benchmark methods, 2SRNN and CWT are two deep learning methods, MADA and DRCN are two unsupervised domain adaptation methods. The experimental environment is MECHREVO Intel(R)Core(TM)i7-10870HCPU 2.20GHz 32GB, the development environment is Python 3.9.0, and the main dependent third-party library is pytorch 1.9.0+cu111. During the experiment, the main parameters are set as follows: for fair comparison, the sliding window is set to 200ms, the batch size is set to 256, and the parameters are optimized using the Adam optimizer. To accelerate model convergence, the learning rate is set to 0.001 on the DB1 dataset, and the learning rate is set to 0.0001 on the DB-a, DB-b and DB-c datasets. According to trial and error, in different cases, the number of training rounds is selected as {20, 50, 60, 100, 120, 200} to obtain better accuracy. Each accuracy value is the average of 10 runs. Table 2 gives the comparative experimental results.

[0103] Table 2 Comparative experimental results

[0104]

[0105] As can be seen from Table 2, the data consists of accuracy and standard deviation (with %). According to the experimental results shown in Table 2, the MSITL method proposed in the present invention is significantly better than other methods on the public benchmark electromyographic gesture recognition dataset. Compared with the DIRECT method, the accuracy is improved by 10.78% to 28.05%; compared with the SINGLE method, the accuracy is improved by 8% to 55.48%; compared with the 2SRNN method, the accuracy is improved by 4% to 40.86%; compared with the CWT method, the accuracy is improved by 1.98% to 28.4%; compared with the MADA method, the accuracy is improved by 59.83% to 68.81%; compared with the DRCN method, the accuracy is improved by 60.23% to 74.33%; compared with the AVG method, the accuracy is improved by 4%, 0.3% and 0.1% on DB1, DB-a and DB-b respectively, and is only 1.11% lower than AVG on DB-c, but the MSITL method is more stable (5.45<5.93).

[0106] The confusion matrix analysis results are shown in Figure 5(a)-(d). Figure 5(a) shows the confusion matrix analysis result of the NinaPro DB1 dataset, Figure 5(b) shows the confusion matrix analysis result of the CapgMyo DB-a dataset, Figure 5(c) shows the confusion matrix analysis result of the CapgMyo DB-b dataset, and Figure 5(d) shows the confusion matrix analysis result of the CapgMyo DB-c dataset.

[0107] Among the 52 gestures on the DB1 dataset, the second gesture has the highest accuracy, which is 87%; the 32nd gesture has the lowest accuracy, which is 52%. Among the 8 gestures on the DB-a dataset, the third gesture has the highest accuracy, which is 94%; the fourth and fifth gestures have the lowest accuracy, which is 77%. Among the 8 gestures on the DB-b dataset, the first gesture has the highest accuracy, which is 95%; the second and eighth gestures have the lowest accuracy, which is 81%. Among the 12 gestures on the DB-c dataset, the eighth gesture has the highest accuracy, which is 92%; the ninth gesture has the lowest accuracy, which is 59%. Therefore, the recognition difficulty is different for different gestures. In addition to the gesture recognition algorithm, a good gesture set design also helps to improve the overall effect of the recognition system.

[0108] In addition, some gestures are easily confused with each other. In the DB-a data, the probability of misidentifying the fifth gesture as the fourth gesture is 11%; the probability of misidentifying it as the sixth gesture is 10%. In the DB-b dataset, the probability of misidentifying the eighth gesture as the fourth gesture is 10%. In the DB-c dataset, the ninth and tenth gestures are easily misidentified as the eleventh gesture, with probabilities of 13% and 11% respectively. Therefore, in the process of gesture set design, the similarity between gestures must be considered.

[0109] The results of the labeled data volume analysis for NinaPro DB1, CapgMyo DB-a, CapgMyo DB-b, and CapgMyo DB-c are shown in Figure 6(a)-(d), respectively. Figure 6(a)-(d) shows the average recognition accuracy of all users as a function of the amount of labeled data. The data includes the average accuracy and standard deviation. On the DB1, DB-a, DB-b, and DB-c datasets, the amount of labeled data in the target domain increases from 4% to 44%, and the overall accuracy on the target domain shows an upward trend. When the amount of labeled data is relatively small, the average accuracy increases with the increase in the amount of data. However, when the scale of the labeled data is large, the upward trend of the accuracy tends to stabilize.

[0110] The results of parameter sensitivity analysis of parameter ω and parameter c are shown in Figure 7(a)-(b), respectively. The MSITL method is mainly sensitive to two parameters, the adaptive parameter ω and the fine-tuning constant c. In the experiment, ω is set to {0.1, 0.2, ..., 0.9} and c is set to {50, 100, ..., 400}. By fixing one parameter and changing the other parameter, the average accuracy of MSITL is used for evaluation. The triangles are the points with the best accuracy. When ω is 0.1 and 0.5, the optimal and suboptimal accuracies are obtained, respectively. When ω∈[0.1, 0.6], the choice of parameter ω has relatively little effect on the gesture recognition accuracy. However, the effect of c on the gesture recognition accuracy is obvious, and the gesture recognition accuracy is positively correlated with the value of c. The choice of c is a trade-off process. It is feasible to improve the gesture recognition accuracy by increasing the number of fine-tuning rounds, but it also means that more time is required.

[0111] The method of this embodiment first constructs an integrated model with strong generalization ability through a two-layer architecture, then uses the target domain data to evaluate the model using its accuracy on the individual model and the complexity of the individual model itself, and finally uses the evaluation score to fine-tune the individual model, which can achieve effective migration from the source domain to the target domain.

[0112] Embodiment 2

[0113] In one or more embodiments, a cross-user electromyographic gesture recognition system based on multi-source domain integrated migration is disclosed, including:

[0114] An individual model building module is used to build an individual model of the source domain of electromyographic gesture recognition for each user;

[0115] The individual model evaluation module is used to measure the fitness of the target user on the source domain individual model by using the accuracy of the target user on the single source domain individual model and the complexity index of the single source domain individual model, and calculate the fitness evaluation score;

[0116] The individual model integration module is used to update the individual model of a single source domain according to the fitness evaluation score; the updated individual model of the single source domain is integrated through the majority voting mechanism to obtain the integrated source domain individual model;

[0117] The gesture recognition module is used to obtain the target user's electromyographic signal data and obtain the target user's electromyographic gesture recognition result based on the integrated source domain individual model.

[0118] It should be noted that the specific implementation methods of the above modules have been described in Example 1 and will not be described in detail here.

[0119] Embodiment 3

[0120] In one or more embodiments, a terminal device is disclosed, including a server, the server including a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the cross-user electromyographic gesture recognition method based on multi-source domain integrated migration in Embodiment 1 when executing the program. For the sake of brevity, it will not be described in detail here.

[0121] It should be understood that in this embodiment, the processor may be a central processing unit CPU, and the processor may also be other general-purpose processors, digital signal processors DSP, application-specific integrated circuits ASIC, off-the-shelf programmable gate arrays FPGA or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0122] The memory may include a read-only memory and a random access memory, and provide instructions and data to the processor. A portion of the memory may also include a non-volatile random access memory. For example, the memory may also store information about the device type.

[0123] In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in a processor or an instruction in the form of software.

[0124] Embodiment 4

[0125] In one or more embodiments, a computer-readable storage medium is disclosed, in which a plurality of instructions are stored, wherein the instructions are suitable for being loaded by a processor of a terminal device and executing the cross-user electromyographic gesture recognition method based on multi-source domain integrated migration described in Embodiment 1.

[0126] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A cross-user electromyographic gesture recognition method based on multi-source domain integration migration, characterized in that: include: Build an individual model of the source domain of electromyographic gesture recognition for each user; The accuracy of the target user on a single source domain individual model and the complexity index of a single source domain individual model are used to measure the fitness of the target user on the source domain individual model, and the fitness evaluation score is calculated; The single source domain individual model is updated according to the fitness evaluation score; the updated single source domain individual model is integrated through the majority voting mechanism to obtain the integrated source domain individual model; Obtain the target user's electromyographic signal data, and obtain the target user's electromyographic gesture recognition result based on the integrated source domain individual model; Among them, the fitness evaluation score of the i-th source domain individual model is: score i =ω*acc i +(1-ω)*com i ; Among them, acc i is the accuracy of the individual model of the i-th source domain, com i is the complexity of the individual model of the i-th source domain, ω is the adaptive weight; The individual model of a single source domain is updated according to the fitness evaluation score, specifically: Calculate the number of rounds that a single source domain individual model needs to be adjusted based on the evaluation score: epoch i =c-score i *p; Among them, epoch i Indicates the number of rounds that the i-th model needs to be adjusted, and c and p are both constants; The updated individual models of the single source domain are integrated through a majority voting mechanism, specifically: Assume that there are N different source domain individual models in the source domain, and there is a classification task of K types of gestures; Any input data is passed through each source domain individual model to obtain the corresponding gesture type; Traverse all output values, count the number of each gesture type, and select the gesture type with the largest number of outputs as the output of the integrated source domain individual model.

2. The cross-user electromyographic gesture recognition method based on multi-source domain integrated migration as claimed in claim 1 is characterized in that: A source domain individual model for electromyographic gesture recognition is constructed for each user, wherein the source domain individual model is constructed using a convolutional neural network; and the source domain individual model is trained using source domain data.

3. The cross-user electromyographic gesture recognition method based on multi-source domain integrated migration as claimed in claim 1 is characterized in that: Calculate the accuracy of the target user on a single source domain individual model, specifically: For the target domain The data pairs in the dataset are labeled, and the number of data pairs whose predicted labels and true labels are equal obtained by a single source domain individual model is calculated. This number is equal to the target domain The ratio of the total number of data pairs in is the accuracy of the source domain individual model; Provide labeled data for the target domain.

4. The cross-user electromyographic gesture recognition method based on multi-source domain integrated migration as claimed in claim 1, characterized in that: Calculate the complexity of the target user on a single source domain individual model, specifically: The ratio of the sum of the parameters of each layer of the i-th source domain individual model to the sum of the parameters of each layer of all source domain individual models.

5. A cross-user electromyographic gesture recognition system based on multi-source domain integration migration, characterized in that: include: An individual model building module is used to build an individual model of the source domain of electromyographic gesture recognition for each user; The individual model evaluation module is used to measure the fitness of the target user on the source domain individual model by using the accuracy of the target user on the single source domain individual model and the complexity index of the single source domain individual model, and calculate the fitness evaluation score; The individual model integration module is used to update the individual model of a single source domain according to the fitness evaluation score; the updated individual model of the single source domain is integrated through the majority voting mechanism to obtain the integrated source domain individual model; The gesture recognition module is used to obtain the target user's electromyographic signal data and obtain the target user's electromyographic gesture recognition result based on the integrated source domain individual model; Among them, the fitness evaluation score of the i-th source domain individual model is: score i =ω*acc i +(1-ω)*com i ; Among them, acc i is the accuracy of the individual model of the i-th source domain, com i is the complexity of the individual model of the i-th source domain, ω is the adaptive weight; The individual model of a single source domain is updated according to the fitness evaluation score, specifically: Calculate the number of rounds that a single source domain individual model needs to be adjusted based on the evaluation score: epoch i =c-score i *p; Among them, epoch i Indicates the number of rounds that the i-th model needs to be adjusted, and c and p are both constants; The updated individual models of the single source domain are integrated through a majority voting mechanism, specifically: Assume that there are N different source domain individual models in the source domain, and there is a classification task of K types of gestures; Any input data is passed through each source domain individual model to obtain the corresponding gesture type; Traverse all output values, count the number of each gesture type, and select the gesture type with the largest number of outputs as the output of the integrated source domain individual model.

6. A terminal device, comprising a processor and a memory, wherein the processor is used to implement each instruction; and the memory is used to store multiple instructions, characterized in that: The instructions are suitable for being loaded by a processor and executing the cross-user electromyographic gesture recognition method based on multi-source domain integrated migration according to any one of claims 1 to 4.

7. A computer-readable storage medium storing a plurality of instructions, characterized in that: The instructions are suitable for being loaded by a processor of a terminal device and executing the cross-user electromyographic gesture recognition method based on multi-source domain integrated migration according to any one of claims 1 to 4.

Images