Model obtaining system, gesture recognition method and device, equipment and storage medium

Abstract

The application relates to the field of electrophysiological signal recognition, and discloses a model acquisition system, a gesture recognition method and device, equipment and a storage medium. The model acquisition system comprises an input unit configured to acquire current electromyographic data and a gesture label of a current individual; and a processing unit configured to acquire an initial gesture recognition model in the case that the current individual does not have a personal gesture recognition model, wherein the initial gesture recognition model is generated according to a plurality of meta-learning training generated based on historical electromyographic data and gesture labels of a plurality of individuals, a support set of each meta-learning task is generated based on historical electromyographic data and gesture labels of an individual at an earlier time, and a query set is generated based on historical electromyographic data and gesture labels of an individual at a later time; and the initial gesture recognition model is fine-tuned according to the current electromyographic data and the gesture label of the current individual to obtain a current gesture recognition model. The electromyographic data and the gesture label are used to construct a meta-learning task based on time sequence, and the robustness of the current gesture recognition model is enhanced.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the field of electrophysiological signal recognition, and in particular to a model acquisition system, a gesture recognition method and device, equipment and a storage medium. BACKGROUND

[0002] Electromyography (EMG) is an electrophysiological signal obtained by recording the discharge of motor neurons of muscles through electromyography electrodes. It contains rich neural information, which can be decoded into many limb-related activity signals. EMG acquisition has the characteristics of no harm to the human body, easy acquisition and easy operation, and has good application prospects in the field of gesture recognition classification, especially in the medical, entertainment and other industries involving machine control.

[0003] Traditional electromyographic gesture recognition uses a large amount of single individual pre-acquired data to train the corresponding pattern recognition classifier. Common classifier models include traditional machine learning support vector machines, random forest models, and models constructed based on convolutional neural networks and recurrent neural networks in deep learning. At present, there are many inherent problems in the application of electromyographic signals in gesture recognition. Due to the specificity of individual muscle nerve distribution and the influence of factors such as skin impedance and muscle structure, which are highly dependent on individual characteristics, there are significant individual differences in electromyographic signals, and the classification accuracy depends on a large amount of data from a single individual. In addition, due to the influence of factors such as electrode displacement and environment, the characteristics of electromyographic signals change very quickly, resulting in a gradual decrease in the accuracy of the model trained on a large amount of individual data over time. Based on the above reasons, the model trained based on a large amount of individual data often leads to overfitting and unstable model performance due to limited data acquisition and prior knowledge waste.

[0004] In order to solve the problem of prior knowledge waste, the method of transfer learning provides a solution, but transfer learning will face the problem of catastrophic forgetting, and in order to ensure the accuracy of the model, a large amount of individual data is still needed for training. The model training and model application scene switching are still relatively complex, and the training and adjustment efficiency is low. SUMMARY

[0005] The embodiments of the present application aim to provide a model acquisition system, a gesture recognition method, a device, equipment and a storage medium, which utilize the way of meta-learning to accurately obtain a current gesture recognition model that adapts to the current individual characteristics through a small amount of individual data, improve the efficiency of gesture recognition model training and adjustment, and then complete accurate recognition of the current individual gesture.

[0006] To solve one or more of the above problems and achieve the above purposes, the embodiments of the present application provide a model obtaining system, comprising: an input unit configured to obtain current electromyography data and gesture labels of a current individual, wherein the gesture labels comprise gesture information; a processing unit configured to: in a case where there is no personal gesture recognition model for the current individual, obtain an initial gesture recognition model, wherein the initial gesture recognition model is generated by meta-learning training according to a plurality of meta-learning tasks generated based on historical electromyography data and gesture labels of a plurality of individuals, and each of the meta-learning tasks is generated based on historical electromyography data and gesture labels of the same individual; and fine-tune the initial gesture recognition model according to the current electromyography data and gesture labels of the current individual to obtain a current gesture recognition model of the current individual.

[0007] To solve one or more of the above problems and achieve the above purposes, the embodiments of the present application provide a model obtaining method, comprising: obtaining current electromyography data and gesture labels of a current individual, wherein the gesture labels comprise gesture information; in a case where there is no personal gesture recognition model for the current individual, obtaining an initial gesture recognition model, wherein the initial gesture recognition model is generated by meta-learning training according to a plurality of meta-learning tasks generated based on historical electromyography data and gesture labels of a plurality of individuals, and each of the meta-learning tasks is generated based on historical electromyography data and gesture labels of the same individual; and fine-tuning the initial gesture recognition model according to the current electromyography data and gesture labels of the current individual to obtain a current gesture recognition model of the current individual.

[0008] To solve one or more of the above problems and achieve the above purposes, the embodiments of the present application provide a gesture recognition method, comprising: obtaining a current gesture recognition model of a current individual by the model obtaining system described above; obtaining real-time electromyography data of the current individual; and obtaining a gesture of the current individual according to the real-time electromyography data by the current gesture recognition model.

[0009] To solve one or more of the above problems and achieve the above purposes, the embodiments of the present application further provide a gesture recognition device, comprising: a first obtaining module configured to obtain a current gesture recognition model of a current individual by the model obtaining system described above; a second obtaining module configured to obtain real-time electromyography data of the current individual; and a recognition module configured to obtain a gesture of the current individual according to the real-time electromyography data by the current gesture recognition model.

[0010] To solve one or more of the above problems and achieve the above purposes, the embodiments of the present application further provide an electronic device, comprising: at least one processor; and a memory in communication connection with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the gesture recognition method or execute the model obtaining method.

[0011] To solve one or more of the above problems and achieve the above purposes, the embodiments of the present application further provide a computer-readable storage medium storing a computer program, and the computer program is executed by a processor to implement the gesture recognition method or the model obtaining method.

[0012] The model obtaining system provided in the present application, in the case that the current individual does not have a personal gesture recognition model, first acquires the current electromyography data and gesture information of the current individual through an input unit, and fine-tunes the initial gesture recognition model according to the current electromyography data and gesture label of the current individual to obtain a current gesture recognition model suitable for the current individual. The initial gesture recognition model is generated by meta-learning training based on a plurality of meta-learning tasks generated based on historical electromyography data and gesture labels of multiple individuals, and each of the meta-learning tasks is generated based on the historical electromyography data and gesture labels of the same individual. The initial gesture recognition model, which is acquired by the meta-learning training and is adapted to the crowd and has good generalization ability, is formed according to the historical electromyography data and gesture labels of multiple individuals, and is fine-tuned according to the current electromyography data and gesture label of the current individual when applied to a specific current individual. In this way, the initial gesture recognition model is fine-tuned accurately and efficiently with a small amount of individual data to obtain a current gesture recognition model suitable for the current individual, thereby ensuring the accuracy of gesture recognition. BRIEF DESCRIPTION OF DRAWINGS

[0013] One or more embodiments are exemplified by the pictures in the drawings corresponding thereto, which do not constitute a limitation on the embodiments, and elements with the same reference numerals in the drawings represent similar elements. Unless otherwise specified, the drawings do not constitute a proportional limitation.

[0014] Figure 1 is a model obtaining system structure schematic diagram in the embodiments of the present application;

[0015] Figure 2 is a meta-learning task structure schematic diagram in the embodiments of the present application;

[0016] Figure 3is a meta-learning training process schematic diagram in an embodiment of the present application;

[0017] Figure 4 is a model acquisition method flowchart in another embodiment of the present application;

[0018] Figure 5 is a gesture recognition method flowchart in another embodiment of the present application;

[0019] Figure 6 is a gesture recognition device structure schematic diagram in another embodiment of the present application;

[0020] Figure 7 is an electronic device structure schematic diagram in another embodiment of the present application. DETAILED DESCRIPTION

[0021] As known from the background, the conventional model training method has the problems of overfitting and unstable model performance, and the model obtained through transfer learning has low training and adjustment efficiency and needs to rely on a large amount of individual data to ensure recognition accuracy. Therefore, how to provide a model acquisition method capable of efficiently switching among different application scenarios and having high training and adjustment efficiency to ensure accurate and efficient gesture recognition is an urgent problem to be solved.

[0022] To solve the above problems, an embodiment of the present application provides a model acquisition system, comprising: an input unit configured to obtain current electromyography data and a gesture label of a current individual, the gesture label containing gesture information; a processing unit configured to: in the case that there is no personal gesture recognition model for the current individual, obtain an initial gesture recognition model, wherein the initial gesture recognition model is generated by meta-learning training according to a plurality of meta-learning tasks generated based on historical electromyography data and gesture labels of a plurality of individuals, and each meta-learning task is generated based on historical electromyography data and a gesture label of the same individual; and fine-tune the initial gesture recognition model according to the current electromyography data and the gesture label of the current individual to obtain a current gesture recognition model for the current individual.

[0023] The model acquisition system provided in the application obtains, before obtaining a current gesture recognition model of a current individual, historical electromyographic data and gesture labels of multiple individuals, forms a meta-learning task based on the historical electromyographic data and the gesture labels of each individual, and generates a general initial gesture recognition model through meta-learning training. Then, when obtaining the current gesture recognition model of the current individual in the case that the current individual does not have a personal gesture recognition model, the current electromyographic data and the gesture labels of the current individual are obtained through the input unit, and the initial gesture recognition model is fine-tuned according to the obtained current electromyographic data and gesture labels to obtain a current gesture recognition model suitable for the current individual. The initial gesture recognition model is obtained through the meta-learning training mode based on the historical electromyographic data and the gesture labels of multiple individuals, is adapted to the crowd, and has good generalization ability. When applied to a specific current individual, the initial gesture recognition model is fine-tuned according to the current electromyographic data and the gesture labels of the current individual. In this way, the initial gesture recognition model is fine-tuned accurately and efficiently with a small amount of individual data to obtain a current gesture recognition model suitable for the current individual, thereby ensuring the accuracy of gesture recognition.

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in detail below with reference to the drawings. However, those skilled in the art can understand that, in the embodiments of the present application, many technical details are proposed to make the readers better understand the present application. However, the technical solutions claimed by the present application can be implemented even without these technical details and various changes and modifications based on the following embodiments. The division of the following embodiments is for the convenience of description, and should not constitute any limitation on the specific implementation modes of the present application. The embodiments can be combined and referenced with each other on the premise of no contradiction.

[0025] The implementation details of the model acquisition system described in the present application will be specifically described below in combination with specific embodiments. The following content only provides implementation details for the convenience of understanding, and is not essential for the present solution.

[0026] The first aspect of the embodiments of the present application provides a model acquisition system, as shown in Figure 1 The model acquisition system includes an input unit 101 and a processing unit 102.

[0027] The input unit 101 is configured to obtain current electromyographic data and gesture labels of a current individual, and the gesture labels contain gesture information.

[0028] Specifically, when obtaining the current gesture recognition model with good recognition effect for the current individual, the electromyographic data of the current individual in the current state for a certain time length and the gesture label corresponding to the gesture of the individual during the collection of the electromyographic data are collected and recorded by the input unit 101 to obtain the current electromyographic data and the gesture label of the current individual. The gesture label is a data label created when storing the gesture. Different gesture labels correspond to different gestures. For example, gesture label 1 corresponds to the individual's index finger, gesture label 2 corresponds to the individual's middle finger, and gesture label 3 corresponds to the individual's fist. In specific applications, the meanings of the labels can be set as needed, and the specific settings and meanings of the gesture labels in the present embodiment are not limited.

[0029] For example, when collecting the current electromyographic data of the current individual, the input unit 101 automatically prompts the current individual to make gestures corresponding to a plurality of pre-set gesture labels in a certain order within a specified time through voice or text, records the electromyographic signals generated by the motor neuron of the individual when making different gestures through the electromyographic electrode, and encodes and stores the electromyographic signals and the gesture labels according to the time interval corresponding to each gesture. By obtaining the current electromyographic data and the gesture label of the current individual in the current state, the initial gesture recognition model can be fine-tuned in the subsequent process, and the current gesture recognition model for the current state of the current individual can be efficiently and accurately obtained.

[0030] The processing unit 102 is configured to obtain an initial gesture recognition model in the case that the current individual does not have a personal gesture recognition model, wherein the initial gesture recognition model is generated by meta-learning training based on a plurality of meta-learning tasks generated based on historical electromyographic data and gesture labels of a plurality of individuals, and each meta-learning task is generated based on historical electromyographic data and gesture labels of the same individual; and fine-tune the initial gesture recognition model according to the current electromyographic data and the gesture label of the current individual to obtain the current gesture recognition model of the current individual.

[0031] Specifically, the processing unit 102 acquires the current electromyography data of the current individual and the gesture label before or after the gesture label through the input unit 101, and according to the pre-configured program, first detects whether the personal gesture recognition model of the current individual is pre-stored according to the identity of the current individual. If it is detected that the personal gesture recognition model of the current individual does not exist, the processing unit 102 acquires the initial gesture recognition model. The initial gesture recognition model is generated by meta-learning training according to a plurality of meta-learning tasks generated based on historical electromyography data and gesture labels of a plurality of individuals, and each meta-learning task is generated based on historical electromyography data and gesture labels of the same individual. When acquiring the current gesture recognition model for the current individual, the processing unit 102 can read the pre-trained initial gesture recognition model from the storage address where the initial gesture recognition model is stored through communication. The processing unit 102 can also read the historical electromyography data and gesture labels of a plurality of individuals from a specified storage address, generate a plurality of meta-learning tasks based on the historical electromyography data and gesture labels of each individual, and then perform meta-learning training on an untrained meta-learner to generate the initial gesture recognition model. The processing unit 102 can also directly read a plurality of meta-learning tasks pre-generated based on historical electromyography data and gesture labels of different individuals to train an untrained meta-learner to generate the initial gesture recognition model. The embodiment does not limit the specific acquisition method of the initial gesture recognition model.

[0032] After acquiring the current electromyography data and gesture label of the current individual and the initial gesture recognition model, the processing unit 102 fine-tunes the initial gesture recognition model for the current individual according to the current electromyography data and gesture label of the current individual. For example, by taking the current electromyography data as input information and the gesture corresponding to the gesture label as supervision information, the initial gesture recognition model is supervised to learn, so that the initial gesture recognition model has better pertinence and adaptability to the current individual. The fine-tuned initial gesture recognition model is used as the current gesture recognition model of the current individual. By fine-tuning the initial gesture recognition model for the current individual according to the current electromyography data and gesture label of the current individual, the pertinence of the obtained current gesture recognition model is improved, and the accuracy of gesture recognition for the current individual is ensured.

[0033] Further, the initial gesture recognition model can be trained and generated in the following manner: based on historical electromyography data and gesture labels of a plurality of individuals, a network model with gradient back propagation is trained to obtain an original gesture recognition model; a plurality of meta-learning tasks are generated according to the historical electromyography data and gesture labels of the plurality of individuals, and each meta-learning task is generated based on the historical electromyography data and gesture labels of the same individual; and a meta-learner based on the original gesture recognition model is trained by using each meta-learning task to obtain the initial gesture recognition model.

[0034] Specifically, when training the initial gesture recognition model according to the historical electromyography data and gesture labels of multiple individuals, the processing unit 102 or other model training device can first train the network model with gradient back propagation based on the historical electromyography data and gesture labels of multiple individuals, in a training manner including supervised training, unsupervised training, etc., to obtain an original gesture recognition model, and then form a plurality of meta-learning tasks from the electromyography data set composed of the historical electromyography data and gesture labels of multiple individuals according to a pre-set constraint condition, and the data included in each meta-learning task comes from the same individual. For example, the historical electromyography data and gesture labels of multiple individuals are extracted by random sampling from the electromyography data set composed of the historical electromyography data and gesture labels of multiple individuals. Then, based on the historical electromyography data and gesture labels of each individual obtained by random sampling, one or more meta-learning tasks are generated. In this way, the data included in each meta-learning task in the plurality of tasks comes from the same individual. Subsequently, according to the plurality of generated meta-learning tasks, the meta-learner based on the original gesture recognition model is meta-learned and trained to optimize the meta-learner parameters, and then the initial gesture recognition model is obtained. Here, the original gesture recognition model can be obtained simultaneously with the formation of the meta-learning task, or the meta-learning task can be formed first, and then the original gesture recognition model is obtained. By obtaining the electromyography data of multiple individuals and the gesture labels corresponding to the electromyography data, and randomly selecting the electromyography data and gesture labels of multiple individuals according to a pre-set constraint condition, one or more meta-learning tasks are generated based on the electromyography data and gesture labels of each individual, and the meta-learner based on the original gesture recognition model is meta-learned and trained using the plurality of generated meta-learning tasks, so that the initial gesture recognition model trained has good generalization ability for different individuals and new individuals.

[0035] It is worth mentioning that the network model with gradient back propagation is any one of a convolutional neural network (CNN) model, a long short-term memory (LSTM) model, and a recurrent neural network (RNN) model.

[0036] For example, a neural network model that can be optimized using a random gradient descent-based optimization algorithm can be used as a base model, such as a 3-layer convolutional neural network model using an Adam optimizer, combined with a pre-set loss function, such as a cross-entropy loss function, for model training. The historical electromyography data in the electromyography data set is used as input, and the gesture to which the historical electromyography data belongs is used as a supervision signal. The loss value is calculated using the pre-set loss function, and the model parameters are optimized based on the loss value gradient update, the training of the model is completed, and the original gesture recognition model is obtained.

[0037] In addition, after the current electromyography data and gesture label of the current individual are collected each time, the electromyography data set containing the historical electromyography data and gesture label of a plurality of individuals can be expanded, and then the original gesture model is retrained based on the expanded electromyography data set, so as to improve the generalization effect of the original gesture recognition model. The retraining can be performed periodically according to a preset period, or can be performed after the electromyography data set is expanded to a certain extent, and the embodiment does not limit this.

[0038] In another example, the meta-learning task includes a support set and a query set, and the meta-learner based on the original gesture recognition model is trained by using each meta-learning task, including: for each meta-learning task, first assigning the parameters of the meta-learner to the base learner, and then training the base learner according to the electromyography data in the support set and the gesture label corresponding to the electromyography data to optimize the parameters of the base learner; obtaining the prediction error of the gesture label prediction result of the base learner after parameter optimization according to the electromyography data in the query set and the gesture label corresponding to the electromyography data; and performing gradient update on the meta-learner according to the prediction error to optimize the parameters of the meta-learner.

[0039] Specifically, the meta-learning task includes a support set and a query set, wherein the support set has an important setting of N-way K-shot, that is, there are N classes of samples in the support set, and each class of samples has K labeled data, and the query set can contain n classes of samples, and each class of samples has k labeled data, wherein N, K, n and k are positive integers. For example, the support set contains 6 gesture labels, and 3 electromyography data are collected for each gesture label, and the query set contains two gesture labels, and one electromyography data is collected for each gesture label. In this embodiment, the meta-learning task generated according to the historical electromyography data and gesture label of each individual can refer to Figure 2 The labeled data represents the obtained electromyography data, and the sample is each type of gesture label.

[0040] In the process of meta-learning training of the meta-learner based on the original gesture recognition model according to each meta-learning task, for each meta-learning task, the parameters in the meta-learner are first copied by the base learner, then the electromyographic data in the support set and the gesture labels corresponding to the electromyographic data are read, the electromyographic data are taken as input, the gesture labels are predicted by the base learner, and the parameters in the base learner are updated according to the loss value between the prediction result and the gesture label corresponding to the electromyographic data. Then read the electromyographic data in the query set and the gesture labels corresponding to the electromyographic data, take the electromyographic data corresponding to each gesture label in the query set as input, use the base learner with optimized parameters to predict the gesture label, and obtain the prediction error of the gesture label prediction result output by the base learner with optimized parameters according to the gesture label corresponding to the electromyographic data in the query set. Then, according to the obtained prediction error, the meta-learner is updated to obtain new parameters, and the new parameters are taken as the parameters of the meta-learner. In the case of the next round of training, the optimized parameters of the meta-learner are copied back to the base learner. In this way, multiple rounds of training are performed until the prediction error of the gesture label no longer decreases, and then the training is stopped, and the meta-learner with optimized parameters at this time is taken as the initial gesture recognition model. Through double-layer cyclic training and parameter optimization according to the support set and the query set in the meta-learning task, the initial gesture recognition model required based on the original gesture recognition model is accurately and efficiently obtained. Moreover, the obtained initial gesture recognition model only needs a small number of gradient update steps and a meta-learning task related to a specific task to fine-tune to adapt to a specific task.

[0041] In addition, the pre-set constraint condition can also include that in the process of generating the meta-learning task, the electromyographic data and gesture labels used in the generation of the support set can be taken from the relatively earlier data in the individual electromyographic data, and the electromyographic data and gesture labels used in the generation of the query set can be taken from the relatively later data in the individual electromyographic data. By generating the support set from the earlier electromyographic data of the individual and generating the query set from the later electromyographic data, the meta-learner trained is a model that predicts the later electromyographic data based on the earlier electromyographic data, thereby improving the consistency of the trained model with the actual gesture recognition process and further improving the accuracy of the model gesture prediction.

[0042] In another example, training the base learner according to the electromyographic data in the support set and the gesture labels corresponding to the electromyographic data to optimize the parameters of the base learner includes: taking the electromyographic data in the support set as input and the gesture labels corresponding to the electromyographic data as a supervision signal to obtain a loss value of the gesture label prediction result of the base learner; and updating the base learner according to the loss value to obtain new parameters of the base learner.

[0043] Specifically, when training the base learner, the electromyography data is taken as the input signal, and the gesture label corresponding to the electromyography data recorded in the support set is taken as the supervision signal in the training process. The base learner predicts the gesture label according to the input electromyography data, and calculates the loss value of the gesture label prediction result through a preset loss function, for example, a cross-entropy loss function, a square loss function or an exponential loss function, the gesture label prediction result and the supervision signal. Then, the base learner is updated according to the gradient of the loss value, so as to obtain the optimized parameters of the base learner. Through the supervised training, the parameter optimization of the base learner is accurately completed, and the effect of the meta-learning training is ensured.

[0044] In another example, the prediction error of the gesture label prediction result of the base learner after parameter optimization is obtained, including: obtaining the gesture label prediction result of the electromyography data according to the base learner after parameter optimization and the electromyography data in the query set; obtaining the loss value corresponding to the gesture label prediction result through a preset loss function, the gesture label prediction result and the gesture label corresponding to the electromyography data in the query set; and obtaining the prediction error according to the loss values corresponding to all electromyography data in the query set.

[0045] Specifically, the electromyography data in the query set is taken as the input, and the base learner after parameter optimization is used to predict the gesture label. The gesture label corresponding to the electromyography data in the query set is taken as the supervision signal. Combined with a preset loss function, for example, a cross-entropy loss function, a square loss function or an exponential loss function, the gesture label predicted by the base learner after parameter optimization and the gesture label corresponding to the electromyography data in the query set, the loss value of the gesture label prediction result of the base learner after parameter optimization is obtained, and based on the loss values corresponding to all electromyography data in the query set, the prediction error is obtained. Then, the meta-learner is updated according to the prediction error, and the parameter optimization of the meta-learner is completed. Through the preset loss function, the prediction error of the prediction result is accurately obtained, which is convenient for accurately adjusting the parameters of the meta-learner and completing the model optimization.

[0046] Further, the prediction error is obtained according to the loss values corresponding to all the electromyography data in the query set, including: obtaining the average value of the loss values corresponding to all the electromyography data in the query set, and taking the average value as the prediction error. The average value can be any one of an arithmetic average value, a weighted average value, a geometric average value, a root mean square average value, and a harmonic average value. Preferably, the average value is an arithmetic average value. Specifically, in the process of obtaining the prediction error of the base learner after parameter optimization, the loss values corresponding to each electromyography data in the query set can be obtained one by one, and then the loss values corresponding to each electromyography data are arithmetically averaged to obtain an arithmetic average value of the loss values corresponding to all the electromyography data, and the obtained arithmetic average value is taken as the prediction error. By arithmetically averaging the loss values corresponding to all the electromyography data in the query set, the arithmetic average value is used as the prediction error, so that the prediction error can be obtained as accurately as possible, and the influence of accidental factors of single prediction on the prediction result is avoided.

[0047] Further, in the process of meta-learning training of the meta-learner based on the original gesture recognition model by using each meta-learning task, the learning rate of the base learner when updating the gradient according to the electromyography data and the gesture labels corresponding to the electromyography data in the support set is less than the learning rate of the meta-learner when updating the gradient according to the electromyography data and the gesture labels corresponding to the electromyography data in the query set. By limiting the learning rates of the inner loop training according to the support set and the outer loop training according to the query set, the meta-learner can converge as soon as possible by combining the inner loop with a high learning rate and the outer loop with a low learning rate, thereby improving the training efficiency of the meta-learner.

[0048] In summary, in the training of the meta-learner based on the original gesture recognition model, the process of one round of meta-learning training based on one meta-learning task can refer to Figure 3 , including:

[0049] Step 301: obtaining the meta-learning task used in the current round of training.

[0050] Step 302: assigning the parameters of the meta-learner to the base learner.

[0051] Step 303: training the base learner according to the support set data of the meta-learning task used in the current round of training, and obtaining the optimized parameters of the base learner after gradient updating.

[0052] Step 304: based on the base learner after parameter optimization, calculating the loss value of the meta-learning task as the prediction error according to the query set data of the meta-learning task used in the current round of training, and calculating the corresponding gradient.

[0053] Step 305: updating the parameters of the meta-learner according to the calculated gradient, and completing the meta-learning training in the current round.

[0054] In another alternative example, the initial gesture recognition model can also be trained and generated by: generating a plurality of meta-learning tasks based on historical electromyography data and gesture labels of a plurality of individuals, and each meta-learning task being generated based on historical electromyography data and gesture labels of the same individual; and performing meta-learning training on a meta-learner based on the plurality of meta-learning tasks to obtain the initial gesture recognition model.

[0055] Specifically, when obtaining the initial gesture recognition model, historical electromyography data and gesture labels of a plurality of individuals can be randomly obtained, and one or more meta-learning tasks can be generated based on historical electromyography data and gesture labels of each individual, and finally a plurality of meta-learning tasks are obtained, and each meta-learning task is generated based on historical electromyography data and gesture labels of the same individual. Then, a network model with gradient back propagation is taken as a meta-learner, and the network model is trained by using each meta-learning task to obtain the initial gesture recognition model. The parameters of the network model with gradient back propagation at the beginning of training are randomly obtained. The way of meta-task training in meta-learning drives the model to optimize the parameters in the direction of stronger generalization, improving the accuracy of the model. By directly obtaining a network model with gradient back propagation with random parameters, and obtaining the initial gesture recognition model through meta-learning training, the process of obtaining the initial gesture recognition model is simplified, the model acquisition efficiency is improved, and only a small number of gradient update steps and a meta-learning task related to a specific task are needed to adapt to the specific task.

[0056] In another example, fine-tuning the initial gesture recognition model based on current electromyography data and gesture labels of the current individual includes: performing supervised training on the initial gesture recognition model based on the current electromyography data and gesture labels of the current individual; and performing gradient update on the initial gesture recognition model based on a preset gradient update step to obtain new parameters to form a current gesture recognition model of the current individual.

[0057] Specifically, when fine-tuning the initial gesture recognition model based on current electromyography data and gesture labels of the current individual, the fine-tuning can be achieved through supervised training, i.e., the current electromyography data of the current individual is taken as input data, and the gesture label corresponding to the current electromyography data is taken as a supervision signal for supervised training. In the process of gradient update, the initial gesture recognition model is updated based on the preset gradient update step, so as to obtain optimized parameters and form a current gesture recognition model of the current individual. Through supervised training and a preset gradient update step, fine-tuning of the initial gesture recognition model is accurately realized, and the recognition accuracy of the current gesture recognition model is ensured.

[0058] In another example, the processing unit 102 is further configured to generate a current meta-learning task of the current individual according to the current electromyography data and the gesture label of the current individual, and perform meta-learning training on the meta-learner based on the initial gesture recognition model according to the current meta-learning task to generate a personal gesture recognition model of the current individual.

[0059] Specifically, after obtaining the current electromyography data and the gesture label of the current individual, the processing unit 102 further generates a current meta-learning task of the current individual according to the current electromyography data and the gesture label of the current individual. Then, the meta-learner based on the initial gesture recognition model is trained according to the current meta-learning task of the current individual, so that the meta-learner after parameter optimization has better pertinence and adaptability to the current individual. The meta-learner after the meta-learning training is taken as the personal gesture recognition model of the current individual. For example, the electromyography data of the individual in the current state within 2 minutes when making gestures corresponding to different gesture labels is collected, the current electromyography data and the gesture label collected are used to fine-tune the initial gesture recognition model, and the electromyography data and the gesture label corresponding to the electromyography data are taken as the support set and the query set according to the pre-set constraint condition, the current meta-learning training task of the current individual is generated from the electromyography data and the gesture label corresponding to the electromyography data, and the meta-learner based on the initial gesture recognition model is trained according to the current meta-learning task to obtain the personal gesture recognition model of the current individual.

[0060] In addition, when the current meta-learning task is generated according to the current electromyography data and the gesture label of the current individual, electromyography data of the individual in the current state when the corresponding gesture label is made within 2 minutes can also be collected, a meta-learning training task is generated according to the electromyography data in the previous minute and the gesture label corresponding to the electromyography data, and a meta-learning verification task is generated according to the electromyography data in the next minute and the gesture label corresponding to the electromyography data. After the initial gesture recognition model is trained by cloud learning according to the meta-learning training task, the gesture prediction accuracy of the trained initial gesture recognition model is obtained and detected according to the meta-learning verification task. In the case where the gesture prediction accuracy reaches a preset threshold and the loss value of the gesture label prediction result no longer decreases, it is determined that the meta-learning training is completed, and there is no need to adjust, and the obtained initial gesture recognition model is taken as the personal gesture recognition model of the current individual. For example, the threshold of the prediction accuracy is set to 0.9, that is, in the case where the initial gesture recognition model after meta-learning training has a probability of 90% or above to correctly predict the gesture of the current individual, and the loss value of the gesture label prediction result no longer decreases, it is determined that the training is completed. In the case where the gesture prediction accuracy does not reach the preset threshold and / or the loss value of the gesture label prediction result is still decreasing, it is determined that the training is not completed, and the initial gesture recognition model is repeatedly trained and parameter adjusted according to the meta-learning training task, or electromyography data of the current individual within a longer time and gesture labels corresponding to the electromyography data are collected to generate a new meta-learning task to retrain and parameter adjust the initial gesture recognition model, until the gesture prediction accuracy reaches the preset threshold and the loss value of the gesture label prediction result no longer decreases. The prediction accuracy is detected by using the verification task, so as to ensure the accuracy of the obtained personal gesture recognition model.

[0061] It is worth mentioning that the preset time length and the preset threshold of the prediction accuracy can be set as needed. When the meta-learning task is generated, the training task can be generated according to the data collected earlier, the verification task can be generated according to the data collected later, or the training task can be generated according to the data collected later, the verification task can be generated according to the data collected earlier, all the collected data can be used for meta-learning task generation, or only part of the data can be selected for meta-learning task generation. The present embodiment does not limit this.

[0062] Further, the model obtaining system further comprises: a storage unit, configured to store the personal gesture recognition model; and the processing unit is further configured to: in the case where the current individual has a personal gesture recognition model, fine-tune the personal gesture recognition model according to the current electromyography data and the gesture label of the current individual to obtain a current gesture recognition model of the current individual; generate a current meta-learning task of the current individual according to the electromyography data and the gesture label of the current individual, and continue to perform meta-learning training on the personal gesture recognition model according to the current meta-learning task to generate a personal gesture recognition model with optimized parameters.

[0063] Specifically, the processing unit 102 stores the personal gesture recognition model of the current individual in the storage unit, reads the personal gesture recognition model of the current individual, and fine-tunes the personal gesture recognition model of the current individual according to the current electromyography data and the gesture label of the current individual, to obtain the current gesture recognition model of the current individual, after detecting whether the personal gesture recognition model of the current individual is stored in advance according to the identity of the current individual. That is, in the case of detecting that the current individual is not a new user, the pre-stored personal gesture recognition model of the current individual is read and used as the basis for obtaining the current gesture recognition model. Through the storage and reuse of the personal gesture recognition model of the individual, the waste of prior experience caused by the generation of the personal gesture recognition model in the generation process of the current gesture recognition model is avoided.

[0064] In addition, the processing unit 102 is also used to generate the current meta-learning task of the current individual according to the current electromyography data and the gesture label of the current individual, and then perform meta-learning training on the personal gesture recognition model of the current individual according to the current meta-learning task of the current individual, so that the personal gesture recognition model has better pertinence and adaptability to the current individual. Through the meta-learning training, the personal gesture recognition model is optimized in parameters, and the personal gesture recognition model optimized in parameters is stored through the storage unit.

[0065] The type of the processing unit is not particularly limited in the embodiment. The processing unit can be hardware for performing logical operations, such as a single-chip microcomputer, a microprocessor, a programmable logic controller (PLC) or a field-programmable gate array (FPGA), or a software program, a functional module, a function, an object library or a dynamic-link library for realizing the above functions on the basis of hardware. Alternatively, the processing unit can be a combination of the above two.

[0066] Another aspect of the embodiment of the application provides a gesture model generation method. The flow of the gesture model generation method can refer to Figure 4 , and includes the following steps:

[0067] In step 401, the current electromyography data and the gesture label of the current individual are obtained, and the gesture label contains gesture information.

[0068] At step 402, if the current individual does not have a personal gesture recognition model, an initial gesture recognition model is obtained, wherein the initial gesture recognition model is generated based on meta-learning training of a plurality of meta-learning tasks generated based on historical electromyography data and gesture labels of a plurality of individuals, and each meta-learning task is generated based on historical electromyography data and gesture labels of the same individual.

[0069] At step 403, the initial gesture recognition model is fine-tuned based on current electromyography data and gesture labels of the current individual to obtain a current gesture recognition model of the current individual.

[0070] It can be found that the embodiment is a method embodiment corresponding to the system embodiment, and the embodiment can be implemented in cooperation with the system embodiment. The related technical details mentioned in the system embodiment are still valid in the embodiment. In order to reduce repetition, they will not be described here. Accordingly, the related technical details mentioned in the embodiment can also be applied to the system embodiment.

[0071] Another aspect of the embodiment of the present application provides a gesture recognition method. The flow of the gesture recognition method can refer to Figure 5 , which includes the following steps:

[0072] At step 501, a current gesture recognition model of a current individual is obtained. Specifically, a terminal device for recognizing gestures of the current individual obtains the current gesture recognition model of the current individual obtained by fine-tuning the initial gesture recognition model through the model acquisition system.

[0073] At step 502, real-time electromyography data of the current individual is obtained. Specifically, when recognizing gestures, real-time electromyography data of the current individual is collected through electromyography electrodes, and the collected real-time electromyography data is input into the current gesture recognition model.

[0074] At step 503, the current gesture recognition model is used to obtain the gesture of the current individual based on the real-time electromyography data. Specifically, after obtaining the real-time electromyography data of the current individual, the current gesture recognition model predicts the gesture of the current individual based on the input real-time electromyography data, and outputs the predicted gesture of the current individual.

[0075] In one example, after the model acquisition system obtains the current gesture recognition model of the current individual, it further includes: obtaining the prediction accuracy of the current gesture recognition model; if the prediction accuracy does not meet a preset threshold, reacquiring current electromyography data and gesture labels of the current individual within a new preset time period; and re-fine-tuning the current gesture recognition model based on the reacquired current electromyography data and gesture labels.

[0076] Specifically, due to the characteristics of the electromyography data, the prediction accuracy of the current gesture recognition model for the current individual gradually decreases as the use time increases, therefore, after obtaining the current gesture recognition model of the current individual, the prediction accuracy of the current gesture recognition model is monitored, and in the case that the prediction accuracy of the current gesture recognition model is insufficient, the electromyography data and gesture labels corresponding to different gestures made by the current individual within a preset time period in the current state are reacquired, and the current gesture recognition model is re-tuned according to the reacquired current electromyography data and gesture labels, so that the current gesture recognition model can be more adapted to the current state of the current individual, and the gesture recognition accuracy is ensured. Considering the changeable characteristics of the electromyography signal, the gesture recognition model is periodically updated and iterated to ensure the accuracy of the gesture recognition result obtained by using the gesture recognition model.

[0077] Another aspect of the embodiment of the present application also provides a gesture recognition device, referring to Figure 6 , comprising:

[0078] The first acquisition module 601 is configured to acquire the current gesture recognition model of the current individual by using the model acquisition system.

[0079] The second acquisition module 602 is configured to acquire the real-time electromyography data of the current individual.

[0080] The recognition module 603 is configured to acquire the gesture of the current individual according to the real-time electromyography data by using the current gesture recognition model.

[0081] In one example, the gesture recognition device further comprises a third acquisition module, the third acquisition module is configured to acquire the prediction accuracy of the current gesture recognition model, and in the case that the prediction accuracy does not satisfy a preset threshold, the current electromyography data and gesture labels of the current individual within a new preset time period are reacquired, and the current gesture recognition model is re-tuned according to the reacquired current electromyography data and gesture labels.

[0082] It is not difficult to find that the embodiment is a device embodiment corresponding to the method embodiment, and the embodiment can be implemented in cooperation with the method embodiment. The related technical details mentioned in the method embodiment are still valid in the embodiment. In order to reduce repetition, they will not be described here. Accordingly, the related technical details mentioned in the embodiment can also be applied in the method embodiment.

[0083] Another aspect of the embodiment of the present application also provides an electronic device, referring to Figure 7The gesture recognition device includes: at least one processor 701; and a memory 702 connected with the at least one processor 701; wherein the memory 702 stores instructions executable by the at least one processor 701, and the instructions are executed by the at least one processor 701 to enable the at least one processor 701 to perform the gesture recognition method or the model obtaining method.

[0084] The memory 702 and the processor 701 are connected in a bus manner, the bus can include any number of interconnected buses and bridges, and the bus connects various circuits of the one or more processors 701 and the memory 702 together. The bus can also connect various other circuits such as peripheral devices, voltage stabilizers, and power management circuits together, which are well known in the art, and thus, further description thereof will not be given herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be one element or multiple elements such as multiple receivers and transmitters, and provides a unit for communicating with various other devices on a transmission medium. Data processed by the processor 701 is transmitted on a wireless medium through an antenna, and further, the antenna also receives data and transmits the data to the processor 701.

[0085] The processor 701 is responsible for managing the bus and general processing, and can also provide various functions including timing, peripheral interface, voltage regulation, power management, and other control functions. The memory 702 can be used to store data used by the processor 701 in performing operations.

[0086] Another aspect of the embodiments of the present application also provides a computer readable storage medium storing a computer program. The computer program is executed by a processor to implement the gesture recognition method or the model obtaining method.

[0087] That is, those skilled in the art can understand that all or part of the steps of the above-mentioned embodiment methods can be completed by programs instructing related hardware, the programs are stored in a storage medium, and include a plurality of instructions for causing a device (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the methods described in various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various storage medium capable of storing program codes.

[0088] Those skilled in the art can understand that the above-mentioned embodiments are specific embodiments for implementing the present application, and in actual applications, various changes can be made in form and details without departing from the spirit and scope of the present application.

Claims

1. A model acquisition system, characterized in that, include: The input unit is used to acquire the current electromyography data and gesture tags of the current individual, wherein the gesture tags contain gesture information; The processing unit is configured as follows: In the case where no personal gesture recognition model exists for the current individual, an initial gesture recognition model is obtained. The initial gesture recognition model is generated by meta-learning training based on several meta-learning tasks generated from historical electromyography data and gesture labels of multiple individuals. Each meta-learning task includes a support set and a query set. The support set is generated based on the individual's historical electromyography data and gesture labels at a first time. The query set is generated based on the individual's historical electromyography data and gesture labels at a second time. The first time is earlier than the second time. Based on the current electromyography data and gesture labels of the current individual, the initial gesture recognition model is fine-tuned to obtain the current gesture recognition model of the current individual.

2. The model acquisition system according to claim 1, characterized in that, The initial gesture recognition model can be trained and generated in the following way: Based on historical electromyography data and gesture labels of multiple individuals, a network model with gradient backpropagation is trained to obtain the original gesture recognition model. Based on the historical electromyography data and gesture labels of multiple individuals, several meta-learning tasks are generated, and each meta-learning task is generated based on the historical electromyography data and gesture labels of the same individual. The initial gesture recognition model is obtained by performing meta-learning training on the meta-learner based on the original gesture recognition model using the meta-learning tasks described above.

3. The model acquisition system according to claim 2, characterized in that, The network model with gradient backpropagation is any one of the following: Convolutional Neural Network (CNN), Long Short-Term Neural Network (LSTM), and Recurrent Neural Network (RNN).

4. The model acquisition system according to claim 2, characterized in that, The step of training the meta-learner based on the original gesture recognition model using each of the meta-learning tasks includes: For each meta-learning task, the parameters of the meta-learner are first assigned to the base learner. Then, the base learner is trained to optimize its parameters based on the electromyography data in the support set and the gesture labels corresponding to the electromyography data. Based on the electromyography (EMG) data and the corresponding gesture labels in the query set, obtain the prediction error of the gesture label prediction result of the base learner after parameter optimization; Based on the prediction error, the meta-learner is updated with gradients to optimize its parameters.

5. The model acquisition system according to claim 4, characterized in that, The step of training the base learner to optimize its parameters based on the electromyography (EMG) data and corresponding gesture labels in the support set includes: Using the electromyography (EMG) data in the support set as input and the gesture labels corresponding to the EMG data as supervision signals, the loss value of the gesture label prediction result of the base learner is obtained. Based on the loss value, the base learner is updated with gradients to obtain new parameters for the base learner.

6. The model acquisition system according to claim 4, characterized in that, The prediction error of the gesture label prediction result of the base learner after parameter optimization includes: Based on the optimized base learner and the electromyography data in the query set, obtain the gesture label prediction result of the electromyography data; By using a preset loss function, the gesture label prediction result, and the gesture label corresponding to the electromyography data in the query set, the loss value corresponding to the gesture label prediction result is obtained, and the prediction error is obtained based on the loss values ​​corresponding to all electromyography data in the query set.

7. The model acquisition system according to claim 6, characterized in that, The step of obtaining the prediction error based on the loss values ​​corresponding to all electromyographic data in the query set includes: Obtain the average value of the loss values ​​corresponding to all electromyography data in the query set, and use the average value as the prediction error.

8. The model acquisition system according to claim 4, characterized in that, During the meta-learning training process of the meta-learner based on the original gesture recognition model using each of the meta-learning tasks, the learning rate of the base learner when updating the gradient based on the electromyography data and the gesture labels corresponding to the electromyography data in the support set is less than the learning rate of the meta-learner when updating the gradient based on the electromyography data and the gesture labels corresponding to the electromyography data in the query set.

9. The model acquisition system according to claim 1, characterized in that, The initial gesture recognition model can also be trained and generated in the following ways: Based on the historical electromyography data and gesture labels of multiple individuals, several meta-learning tasks are generated, and each meta-learning task is generated based on the historical electromyography data and gesture labels of the same individual. The initial gesture recognition model is obtained by performing meta-learning training on the meta-learner based on the network model with gradient backpropagation using the meta-learning tasks.

10. The model acquisition system according to claim 1, characterized in that, Based on the current electromyography data and gesture labels of the current individual, the initial gesture recognition model is fine-tuned, including: The initial gesture recognition model is trained under supervision based on the current electromyography data and gesture labels of the current individual. According to the preset gradient update step number, the initial gesture recognition model is updated by gradient to obtain new parameters, so as to form the current gesture recognition model of the current individual.

11. The model acquisition system according to any one of claims 1 to 10, characterized in that, The processing unit is further configured to: Based on the current electromyography data and gesture labels of the current individual, generate the current meta-learning task for the current individual; Based on the current meta-learning task, a meta-learner based on the initial gesture recognition model is trained to generate the personal gesture recognition model for the current individual.

12. The model acquisition system according to claim 11, characterized in that, The model acquisition system further includes: a storage unit; The storage unit is used to store the personal gesture recognition model; The processing unit is further configured to: If the personal gesture recognition model exists for the current individual, the personal gesture recognition model is fine-tuned based on the current electromyography data and gesture labels of the current individual to obtain the current gesture recognition model for the current individual. Based on the electromyography data and gesture labels of the current individual, the current meta-learning task of the current individual is generated, and the personal gesture recognition model is further trained by meta-learning based on the current meta-learning task to generate the personal gesture recognition model with optimized parameters.

13. A model acquisition method, characterized in that, include: Obtain the current electromyography data and gesture tags of the current individual, wherein the gesture tags contain gesture information; In the case where no personal gesture recognition model exists for the current individual, an initial gesture recognition model is obtained. The initial gesture recognition model is generated by meta-learning training based on several meta-learning tasks generated from historical electromyography data and gesture labels of multiple individuals. Each meta-learning task includes a support set and a query set. The support set is generated based on the individual's historical electromyography data and gesture labels at a first time. The query set is generated based on the individual's historical electromyography data and gesture labels at a second time. The first time is earlier than the second time. Based on the current electromyography data and gesture labels of the current individual, the initial gesture recognition model is fine-tuned to obtain the current gesture recognition model of the current individual.

14. A gesture recognition method, characterized in that, include: The current gesture recognition model of the current individual is obtained through the model acquisition system according to any one of claims 1 to 12; Acquire the real-time electromyography data of the current individual; The gestures of the current individual are obtained based on the real-time electromyography data using the current gesture recognition model.

15. The gesture recognition method according to claim 14, characterized in that, After acquiring the current gesture recognition model through the model acquisition system of any one of claims 1 to 12, the method further includes: Obtain the prediction accuracy of the current gesture recognition model; If the prediction accuracy does not meet the preset threshold, the current electromyography data and gesture tags of the current individual are reacquired within a new preset time period. Based on the newly acquired current electromyographic data and gesture labels, the current gesture recognition model is fine-tuned.

16. A gesture recognition device, characterized in that, include: The first acquisition module is used to acquire the current gesture recognition model of the current individual through the model acquisition system according to any one of claims 1 to 12; The second acquisition module is used to acquire the real-time electromyography data of the current individual; The recognition module is used to obtain the gesture of the current individual based on the real-time electromyography data through the current gesture recognition model.

17. The gesture recognition device according to claim 16, characterized in that, It also includes the third acquisition model; The third acquisition module is used to acquire the prediction accuracy of the current gesture recognition model; If the prediction accuracy does not meet the preset threshold, the current electromyography data and gesture tags of the current individual are reacquired within a new preset time period. Based on the newly acquired current electromyographic data and gesture labels, the current gesture recognition model is fine-tuned.

18. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the model acquisition method as described in claim 13, or the gesture recognition method as described in claim 14 or 15.

19. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the model acquisition method as described in claim 13, or the gesture recognition method as described in claim 14 or 15.

Images