Source domain data missing type electromyography recognition method, device, equipment and medium

Abstract

This invention discloses a method, apparatus, device, and medium for electromyography (EMG) recognition in scenarios with missing source domain data. The method comprises: acquiring source domain EMG signal fragments, inputting them into a TCN feature extractor to extract features, and obtaining fixed-dimensional feature vectors; constructing a source domain gesture recognition model, inputting the feature vectors into a gesture classification head and a domain discriminator head, calculating cross-entropy loss and domain classification loss respectively, and freezing all parameters of the TCN feature extractor and the source domain gesture recognition model after adversarial training until convergence; guiding the user to complete a movement cycle in the target domain through a human-computer interaction interface, acquiring EMG signals and movement labels as calibration data pseudo-labels; constructing a nonlinear transformation module and inserting it into the backend of the frozen TCN feature extractor, generating calibration features aligned with the source domain by performing nonlinear mapping processing on the original feature vectors of the target domain; fine-tuning the nonlinear transformation module using calibration data, and constructing a target domain recognition model to achieve accurate recognition in scenarios with missing source domain data.

Description

Technical Field

[0001] This invention relates to the fields of biomedical signal processing and artificial intelligence pattern recognition technology, specifically to a method, device, equipment, and medium for recognizing electromyography (EMG) with missing source domain data. Background Technology

[0002] Surface electromyography (sEMG), as a non-invasive neural interface technology, has been widely used in various technical fields such as intelligent prosthetic control, virtual reality interaction, and rehabilitation medical training due to its advantage of being able to collect muscle electrical activity signals without invading the human body. It has provided important support for the technological upgrading and application of related fields and has extremely high practical value and market prospects.

[0003] However, in practical industrial applications, sEMG recognition systems generally face a severe "domain offset" technical challenge. This challenge directly leads to a significant decrease in the stability and recognition accuracy of sEMG signals, severely restricting their reliable application in real-world scenarios. Specifically, this is mainly reflected in the following three aspects:

[0004] First, the non-stationarity of the signal causes cross-time shift: due to factors such as muscle fatigue and dynamic changes in skin impedance during human muscle activity, the features of the acquired sEMG signal will slowly drift over time, i.e., cross-time domain shift, which in turn leads to a continuous decline in the recognition accuracy based on the initial training model.

[0005] Second, electrode position shift caused by physical interference: During the re-wearing of the sEMG device, the contact position between the electrodes and the skin is prone to slight shifts, and these slight shifts have a significant impact on the recognition effect. Relevant research data shows that an electrode position shift of only 1 cm can cause the sEMG signal recognition error rate to increase from 5% to over 20%, seriously affecting the practicality of the system.

[0006] Third, signal pattern interference caused by changes in limb posture: When the target limbs such as the human arm are in different postures in space, the geometry and force state of the muscles will be directly changed, which will lead to a significant change in the acquisition pattern of sEMG signals. This makes it impossible for recognition models trained based on fixed postures to effectively adapt to signals under different postures, further aggravating the problem of inaccurate recognition.

[0007] To address the domain offset problem in the sEMG system, several improvement schemes have been proposed in the existing technology. However, these schemes all have significant technical shortcomings and cannot meet the requirements of high performance, high reliability, and compliance in practical applications, as detailed below:

[0008] Traditional transfer learning / fine-tuning solutions rely on historical training data from the source domain to perform transfer learning or fine-tuning of the model, adapting it to signal changes in the target domain. However, this approach has a fatal flaw: in commercial applications, accessing and using historical training data from the source domain poses serious user privacy compliance risks. Furthermore, sEMG systems are primarily used on edge devices, which have limited storage resources and cannot store massive amounts of historical training data from the source domain, making this solution difficult to implement in real-world scenarios.

[0009] For the teacher-student model / DSDAN scheme, this approach achieves domain adaptation by constructing a dual teacher-student network architecture, which improves recognition performance to some extent, but still has significant shortcomings. On the one hand, the dual network architecture design leads to a significant increase in computational complexity, requiring high computing power from the device; on the other hand, the base models commonly used in this scheme, such as RNN and LSTM, are serial computation modes with high inference latency, which cannot meet the high-concurrency, low-latency response requirements of scenarios such as intelligent prosthetic limb control and real-time interaction.

[0010] For linear alignment schemes, which achieve signal adaptation between the source and target domains through linear transformations such as PCA and matrix alignment, their core drawback lies in their extremely poor ability to compensate for nonlinear interference. Due to the irregular anatomical structure of the human forearm muscles, the physical displacement of the electrode positions causes nonlinear distortion of the acquired sEMG signals in the feature space, rather than simple rotation or translation. Linear transformation methods cannot effectively compensate for the signal deviation caused by this nonlinear distortion. Existing research shows that, under sparse electrode configurations (such as the NinaProDB-1 dataset), the average recognition accuracy of sEMG signals using only linear adaptation is only 65.2%, far below the accuracy standards required for practical applications, making it difficult to meet the practical needs of scenarios such as intelligent prostheses and rehabilitation medicine.

[0011] In summary, current technical solutions for the sEMG system domain offset problem cannot simultaneously meet the three core requirements of "no need to access source domain data, low inference latency, and effective compensation for nonlinear interference." The deficiencies of existing technologies will severely restrict the industrial application of sEMG technology. Summary of the Invention

[0012] In order to overcome the shortcomings of the prior art, the present invention provides a method, device, equipment and medium for electromyography recognition with missing source domain data.

[0013] The technical solution of the present invention to solve the above-mentioned technical problems is:

[0014] A method for identifying electromyography (EMG) data missing from the source domain includes the following steps:

[0015] Step 1: Acquire source domain electromyography (EMG) signal segments with a predetermined number of channels N and a predetermined time window length T. Input the EMG signal segments into the TCN feature extractor and extract features from the EMG signal segments through the TCN feature extractor to obtain a fixed-dimensional feature vector.

[0016] Step 2: Construct a source domain gesture recognition model. Input the feature vector obtained in Step 1 into the gesture classification head and domain discriminator head of the source domain gesture recognition model, respectively. The gesture classification head outputs the gesture category probability and calculates the corresponding cross-entropy loss. The domain discriminator head is connected to the TCN feature extractor through a gradient inversion layer, outputs the domain label, and calculates the corresponding domain classification loss. Train the source domain gesture recognition model using an adversarial training method that minimizes the cross-entropy loss while maximizing the domain classification loss until the model converges. After the model converges, freeze all parameters of the TCN feature extractor and the source domain gesture recognition model.

[0017] Step 3: In the target domain scene, a dynamically changing intensity indicator is displayed through the human-computer interaction interface to guide the user to follow the intensity indicator to complete the entire action cycle. Electromyography (EMG) signals within the complete action cycle are collected and segmented into EMG signal segments according to a predetermined time window length T. Simultaneously, the action category label corresponding to the visual guidance is recorded and used as a pseudo label for the visual guidance calibration data, thereby completing the collection of visual guidance calibration data.

[0018] Step 4: Construct a trainable nonlinear transformation module and insert it into the back end of the frozen TCN feature extractor in Step 2; input the electromyographic signal fragments collected in the target domain into the frozen TCN feature extractor to obtain the original feature vector of the target domain; perform nonlinear mapping processing on the original feature vector of the target domain through the nonlinear transformation module to compensate for the nonlinear distortion of the electromyographic signal in the target domain and generate a calibrated feature vector aligned with the feature distribution of the source domain.

[0019] Step 5: Using the visual guidance calibration data obtained in Step 3, fine-tune only the parameters of the nonlinear transformation module in Step 4, keeping the parameters of the TCN feature extractor and the source domain gesture recognition model frozen, to obtain the nonlinear transformation module with optimal parameters. Combine this nonlinear transformation module with the TCN feature extractor and the source domain gesture recognition model frozen in Step 2 to construct the target domain gesture recognition model; and use the target domain gesture recognition model to accurately identify the target domain electromyographic signals in scenarios where source domain data is missing.

[0020] Preferably, in step 1, the TCN feature extractor comprises an input layer, a stacked residual convolutional layer, and an output layer, wherein,

[0021] The input layer is used to receive electromyographic signal segments with a predetermined number of channels N and a predetermined time window length T;

[0022] The stacked residual convolutional layer is configured with multiple residual blocks, each containing two causal-dilated convolutional layers with the same kernel size, which is used to extract variable-length temporal features of electromyography signals in parallel.

[0023] The output layer is used to receive the variable-length temporal features extracted in parallel by stacked residual convolutional layers, and aggregate them into a fixed-dimensional feature vector through global average pooling.

[0024] Weight normalization and Leaky ReLU activation functions are set between the input layer and the stacked residual convolutional layer, as well as between the stacked residual convolutional layer and the output layer; the TCN feature extractor also includes a Dropout mechanism.

[0025] Preferably, in step 2, the gesture classification head consists of a fully connected layer and a Softmax function; the domain discrimination head consists of a gradient inversion layer followed by a fully connected layer.

[0026] Preferably, in step 3, the complete action cycle includes a standstill, linear increase, holding maximum force, and linear relaxation.

[0027] Preferably, in step 4, the nonlinear transformation module includes an input layer, a hidden layer, and an output layer, wherein a nonlinear activation function is provided between the input layer and the hidden layer, and between the hidden layer and the output layer, for fitting and compensating for the nonlinear distortion of electromyographic signals caused by electrode offset; the number of trainable parameters of the nonlinear transformation module is less than 1% of the total number of parameters of the source domain gesture recognition model in step 2.

[0028] Preferably, in step 4, the mathematical model of the nonlinear transformation module is:

[0029] ;

[0030] In the formula: The alignment features are those after the adaptation module has been applied. This is the feature vector extracted from the raw surface electromyography signal by the TCN feature extractor; These are the adaptive feature transformation matrices; For the spatial transformation bias; It is a non-linear activation function.

[0031] Preferably, in step 5, the fine-tuning process sequentially includes forward propagation, multi-criteria loss optimization, and gradient freezing control, specifically as follows:

[0032] First, forward propagation is performed to input the target domain calibration signal into the frozen TCN feature extractor in step 2 to obtain the original feature vector. Then, the original feature vector is input into the nonlinear transformation module in step 4 to obtain the aligned feature vector. Finally, the aligned feature vector is sent into the frozen gesture classification head in step 2 to output the gesture category prediction probability.

[0033] Next, multi-criteria loss optimization is performed, which includes supervised loss and confidence regularization. The supervised loss is used to calculate the cross-entropy loss between the gesture category prediction probability and the action category label in step 3. The confidence regularization is used to calculate the information entropy loss of the output distribution. By minimizing the information entropy value, the target domain features are forced to move closer to the high-confidence cluster centers of the source domain, thereby achieving feature distribution alignment.

[0034] Finally, the gradient freezing strategy is executed. During the entire parameter update process, only the weight gradient of the nonlinear transformation module in step 4 is activated. The parameters of the TCN feature extractor and the source domain gesture recognition model that were frozen in step 2 remain unchanged and do not participate in the parameter update.

[0035] A source-domain data-deficient electromyography (EMG) recognition device, comprising:

[0036] The source domain electromyography acquisition module is used to acquire source domain electromyography signal segments with a predetermined number of channels N and a predetermined time window length T, and transmit the acquired source domain electromyography signal segments to the TCN feature extraction module.

[0037] The TCN feature extraction module is used to receive source domain electromyography signal segments transmitted by the source domain electromyography acquisition module, extract features from the source domain electromyography signal segments, and output a feature vector of fixed dimension.

[0038] The source domain model construction and training module is used to construct a source domain gesture recognition model. This model integrates a gesture classification head and a domain discriminator head. The gesture classification head receives a fixed-dimensional feature vector output by the TCN feature extraction module, outputs the gesture category probability, and calculates the corresponding cross-entropy loss. The domain discriminator head is connected to the TCN feature extraction module via a gradient inversion layer, receives the fixed-dimensional feature vector output by the TCN feature extraction module, outputs the domain label, and calculates the corresponding domain classification loss. The source domain gesture recognition model is trained using an adversarial training method that minimizes the cross-entropy loss while maximizing the domain classification loss until the model converges. After convergence, all parameters of the TCN feature extraction module and the source domain gesture recognition model are frozen.

[0039] The target domain calibration data acquisition module includes a human-computer interaction unit, a target domain electromyography (EMG) acquisition unit, and a label recording unit. The human-computer interaction unit displays dynamically changing intensity indicators, guiding the user to follow these indicators to complete a full movement cycle, and segments the EMG signal into segments according to a predetermined time window length T. The target domain EMG acquisition unit acquires the EMG signals during the user's complete movement cycle. The label recording unit synchronously records the movement category labels corresponding to the visual guidance, uses these labels as pseudo-labels for the visual guidance calibration data, and outputs complete visual guidance calibration data.

[0040] The nonlinear transformation module, located at the back end of the frozen TCN feature extraction module, is used to receive the feature vector extracted by the TCN feature extraction module from the original surface electromyography signal of the target domain, and perform nonlinear mapping processing on the feature vector to generate a calibrated feature vector aligned with the feature distribution of the source domain.

[0041] The target domain model fine-tuning and recognition module receives visually guided calibration data output from the target domain calibration data acquisition module, fine-tunes the parameters of the nonlinear transformation module to obtain the optimal nonlinear transformation module. The optimal nonlinear transformation module is then combined with the frozen TCN feature extraction module and the frozen source domain gesture recognition model to form the target domain gesture recognition model. The target domain gesture recognition model enables accurate recognition of target domain electromyographic signals in scenarios where source domain data is missing.

[0042] An electronic device includes a central processing unit and a memory, the central processing unit being configured to invoke and run a computer program stored in the memory to perform the steps of the source domain data missing type electromyography recognition method.

[0043] A computer-readable storage medium stores, in the form of computer-readable instructions, a computer program implementing the source domain data missing type electromyography recognition method, which, when called by a computer, executes the steps included in the corresponding method.

[0044] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0045] 1. This invention effectively solves the problem of cross-domain recognition in scenarios where source domain data is missing. It does not rely on the original electromyographic data of the source domain. By adopting the strategy of "freezing at both ends and fine-tuning in the middle", it can achieve the alignment of feature distribution between the target domain and the source domain by only adjusting the parameters of the nonlinear transformation module. This greatly reduces the dependence of cross-domain recognition on source domain data, improves the practicality and adaptability of this invention, and avoids the problem of recognition failure caused by the inability to obtain source domain data.

[0046] 2. This invention improves the accuracy and robustness of electromyography (EMG) signal feature extraction. The TCN feature extractor, through stacked residual convolutional layers and causal dilated convolutional structures, combined with gradient-increasing dilation coefficient settings, can exponentially expand the receptive field, thereby effectively capturing the long-term temporal dependence of EMG signals and extracting variable-length temporal features in parallel. With the synergistic effect of weight normalization, LeakyReLU activation function, and Dropout mechanism, the model is effectively regularized to prevent overfitting, thus ensuring the stability and comprehensiveness of feature extraction.

[0047] 3. This invention enhances the universality of cross-domain features. By designing adversarial training of the source domain gesture recognition model, it minimizes gesture classification loss and maximizes domain classification loss, forcing the TCN feature extractor to extract universal features that are sensitive to gesture categories and are not affected by domain differences such as user ID and arm posture. This reduces the interference caused by domain offset, lays a solid foundation for subsequent nonlinear adaptation, and improves the accuracy of cross-domain recognition.

[0048] 4. This invention reduces the complexity and computational cost of target domain adaptation. The nonlinear transformation module adopts a lightweight multilayer perceptron structure, and the number of trainable parameters is much lower than the total number of parameters of the source domain gesture recognition model. The fine-tuning process does not require modification of the frozen TCN feature extractor and source domain model parameters, which not only reduces computational overhead, but also avoids destroying the excellent performance of the source domain model that has been trained, thus adapting to the requirements of lightweight deployment.

[0049] 5. This invention improves the efficiency and accuracy of target domain calibration data acquisition. Through dynamic intensity indicators on the human-computer interaction interface, it guides users to complete a standardized full movement cycle. Simultaneously, it collects electromyographic signals and movement category labels as pseudo-labels to ensure the standardization and completeness of calibration data, thereby further improving the effect of nonlinear module fine-tuning and ensuring the accuracy of target domain identification.

[0050] 6. The present invention has stronger adaptability and can effectively compensate for nonlinear distortion of electromyographic signals caused by factors such as electrode offset and changes in limb posture. It is suitable for electromyographic recognition scenarios of different users and different postures. It does not require retraining a complete model for each target user, thereby greatly reducing deployment costs and having broad engineering application value. Attached Figure Description

[0051] Figure 1 This is a schematic flowchart of the source domain data missing type electromyography recognition method of the present invention. Detailed Implementation

[0052] The present invention will be further described in detail below with reference to the embodiments and accompanying drawings, but the embodiments of the present invention are not limited thereto.

[0053] Example 1

[0054] See Figure 1 The source domain data missing type electromyography identification method of the present invention includes the following steps:

[0055] Step 1: Acquire source domain electromyography (EMG) signal segments with a predetermined number of channels N and a predetermined time window length T. Input the EMG signal segments into the TCN feature extractor and extract features from the EMG signal segments through the TCN feature extractor to obtain a fixed-dimensional feature vector.

[0056] In this embodiment, the electromyography (EMG) acquisition module is powered by a lithium battery charge and discharge management circuit. During operation, it synchronously acquires the EMG signals of the human body through an 8-channel high-precision ADC. After the signals are amplified, filtered, and subjected to anti-interference processing by the front-end conditioning circuit, the original sEMG signal is obtained.

[0057] Furthermore, the TCN feature extractor comprises an input layer, stacked residual convolutional layers, and an output layer, wherein,

[0058] The input layer is used to receive electromyographic signal segments with a predetermined number of channels N and a predetermined time window length T, namely, the surface electromyographic signal tensor of Channels×Time_Steps;

[0059] The stacked residual convolutional layer contains multiple residual blocks, each containing two causal dilated convolutional layers with the same kernel size, used to extract variable-length temporal features of electromyography (EMG) signals in parallel. To achieve exponential expansion of the receptive field and effectively capture the long-term dependence of EMG signals, the dilation coefficients of each residual block are set to d=1, d=2, and d=4 respectively. Meanwhile, the number of filters in the stacked residual convolutional layer increases layer by layer along the feature extraction direction to improve the comprehensiveness and accuracy of feature extraction.

[0060] The output layer is used to receive the variable-length temporal features extracted in parallel by stacked residual convolutional layers, and aggregate them into a fixed-dimensional feature vector through global average pooling.

[0061] Weight normalization and Leaky ReLU activation functions are set between the input layer and the stacked residual convolutional layer, as well as between the stacked residual convolutional layer and the output layer. Furthermore, to avoid overfitting during model training, the TCN feature extractor also incorporates a Dropout mechanism. Through the synergistic effect of the aforementioned weight normalization, Leaky ReLU activation function, and Dropout mechanism, effective regularization of the TCN feature extractor is achieved, ensuring the stability and accuracy of feature extraction.

[0062] Step 2: Construct a source domain gesture recognition model. Input the feature vector obtained in Step 1 into the gesture classification head and domain discriminator head of the source domain gesture recognition model, respectively. The gesture classification head outputs the gesture category probability and calculates the corresponding cross-entropy loss. The domain discriminator head is connected to the TCN feature extractor through a gradient inversion layer, outputs the domain label, and calculates the corresponding domain classification loss. Train the source domain gesture recognition model using an adversarial training method that minimizes the cross-entropy loss while maximizing the domain classification loss until the model converges. After the model converges, freeze all parameters of the TCN feature extractor and the source domain gesture recognition model.

[0063] In this embodiment, to improve the robustness of the pre-trained model, the source domain gesture recognition model adopts a multi-task learning architecture. The gesture classification head in this architecture consists of a fully connected layer and a Softmax function. Its core function is to receive the feature vector output by the TCN feature extractor, output the probability of each gesture category, and calculate the cross-entropy loss based on these probability values. The domain discriminator consists of a gradient inversion layer followed by a fully connected layer. It is used to predict the domain labels (such as user ID or arm pose ID) of the input data, and then calculate the domain classification loss. During the backpropagation process of the model, the gradient reversal layer will convert the domain discrimination loss. The gradient is inverted and then passed to the TCN feature extractor; the model's total loss function... This is a combination of cross-entropy loss and domain classification loss, i.e.:

[0064] ;

[0065] By employing an adversarial training approach that minimizes cross-entropy loss while maximizing domain classification loss, the TCN feature extractor is forced to extract general features that are sensitive to gesture categories but cannot distinguish between different users and different arm postures, thus laying the foundation for subsequent cross-domain adaptation.

[0066] Step 3: In the target domain scenario, a dynamically changing intensity indicator (such as a progress bar or cursor) is displayed through the human-computer interaction interface to guide the user to follow the intensity indicator to complete a complete movement cycle including stillness, linear enhancement, holding maximum force, and linear relaxation. Electromyographic signals are collected during the complete movement cycle, and the movement category label corresponding to the visual guidance is recorded simultaneously. This label is used as a pseudo-label for the visual guidance calibration data, thereby completing the collection of visual guidance calibration data.

[0067] In this embodiment, by standardizing the intensity and speed of the actions, the data collected in a very short time (each action takes only 3 seconds, for a total of 27 seconds) contains rich dynamic features, which can greatly improve the information density of the visual guidance calibration data.

[0068] Step 4: Construct a trainable nonlinear transformation module and insert it into the back end of the frozen TCN feature extractor from Step 2; input the electromyographic signal fragments collected from the target domain into the frozen TCN feature extractor to obtain the original feature vector of the target domain; perform nonlinear mapping processing on the original feature vector of the target domain through the nonlinear transformation module to generate a calibrated feature vector aligned with the feature distribution of the source domain; wherein, the nonlinear transformation module includes an input layer, a hidden layer, and an output layer, wherein nonlinear activation functions are provided between the input layer and the hidden layer, and between the hidden layer and the output layer, to fit and compensate for the nonlinear distortion of the electromyographic signal caused by electrode offset; the number of trainable parameters of the nonlinear transformation module is less than 1% of the total number of parameters of the source domain gesture recognition model in Step 2.

[0069] In this embodiment, the nonlinear transformation module employs a lightweight multilayer perceptron structure to fit complex nonlinear distortions, and its mathematical model is as follows:

[0070] ;

[0071] In the formula: The alignment features are those after the adaptation module has been applied. The original sEMG feature vector extracted by the TCN feature extractor; These are the adaptive feature transformation matrices; For the spatial transformation bias; It is a non-linear activation function.

[0072] Unlike the linear transformations of existing technologies, the multi-layer nonlinear structure in the nonlinear transformation module can effectively fit the complex signal spatial distortion caused by electrode offset. The nonlinear transformation module is configured as a bottleneck structure, and its trainable parameters are strictly controlled within 1% of the total parameters of the source domain model, thereby ensuring that convergence in seconds can be achieved on edge devices with a very small number of samples.

[0073] Step 5: Using the visual guidance calibration data obtained in Step 3, fine-tune the parameters of the nonlinear transformation module in Step 4 to obtain the nonlinear transformation module with optimal parameters. Combine this nonlinear transformation module with the TCN feature extractor and source domain gesture recognition model frozen in Step 2 to construct the target domain gesture recognition model. The target domain gesture recognition model is used to accurately identify the target domain electromyographic signals in scenarios where source domain data is missing.

[0074] In this embodiment, the fine-tuning process sequentially includes forward propagation, multi-criteria loss optimization, and gradient freezing control. The specific steps are as follows:

[0075] First, forward propagation is performed to input the target domain calibration signal into the frozen TCN feature extractor in step 2 to obtain the original feature vector. Then, the original feature vector is input into the nonlinear transformation module in step 4 to obtain the aligned feature vector. Finally, the aligned feature vector is sent into the frozen gesture classification head in step 2 to output the gesture category prediction probability.

[0076] Next, multi-criteria loss optimization is performed, which includes supervised loss and confidence regularization. The supervised loss is calculated as the cross-entropy loss between the gesture category prediction probability and the action category label in step 3. This is used to ensure the accuracy of the feature mapping categories; the confidence regularization is used to calculate the information entropy loss of the output distribution. By minimizing the information entropy value, the target domain features are forced to move closer to the high-confidence cluster centers of the source domain, thus achieving feature distribution alignment.

[0077] Finally, the gradient freezing strategy is executed. During the entire parameter update process, only the weight gradient of the nonlinear transformation module in step 4 is activated. The parameters of the TCN feature extractor and the source domain gesture recognition model that have been frozen in step 2 remain unchanged and do not participate in parameter updates. The strategy of "freezing at both ends and fine-tuning in the middle" ensures that high-performance cross-domain adaptation can still be achieved without accessing the original data of the source domain.

[0078] Finally, compared with the prior art, the present invention has the following advantages:

[0079] Firstly, it boasts extremely high inference efficiency and response speed, thereby effectively reducing computational and memory overhead:

[0080] This invention uses a temporal convolutional network (TCN) to replace the traditional RNN / LSTM architecture, making full use of the parallel computing characteristics of TCN to significantly reduce the time overhead of a single inference. Experimental verification shows that when processing long sequence electromyographic signals, the TCN architecture not only has higher throughput than the RNN architecture, but also effectively reduces memory usage, making it suitable for edge devices and other scenarios with strict requirements for computing resources and response speed.

[0081] Secondly, it ensures privacy and security in scenarios where source data is missing, and addresses storage and compliance challenges:

[0082] In the target domain adaptation process, this invention only needs to download the pre-trained model parameters, without accessing or storing any original physiological data from the source domain. This effectively solves the storage bottleneck of edge devices, thereby avoiding the occupation of storage resources by a large amount of original physiological data, and fundamentally avoids the privacy leakage risks and related legal compliance risks brought about by cross-user data sharing, thus improving the practicality and security of this invention.

[0083] Thirdly, it exhibits strong robustness to nonlinear electrode offset, which can significantly improve recognition accuracy:

[0084] This invention, by setting a nonlinear transformation module, can effectively compensate for electromyographic signal distortion caused by changes in electrode wearing position. Experimental verification shows that in complex scenarios with cross-user and sparse electrode configurations (such as the NinaProDB-1 dataset), the average recognition accuracy of the traditional linear adaptation scheme (L-DA) is only 65.2%, while the average recognition accuracy of the nonlinear adaptation scheme (D-DA) proposed in this invention can reach 72.8%, an absolute improvement of 7.6%. This fully demonstrates that the multi-layer nonlinear structure of this invention has significant advantages in fitting complex signal spatial distortions caused by electrode offset, and has stronger adaptability.

[0085] Fourth, it has a pre-trained base that resists pose interference, ensuring recognition stability in complex scenarios:

[0086] This invention introduces domain adversarial training during the source model training stage, enabling the source domain gesture recognition model to be insensitive to "body posture" before freezing. Combined with the subsequent nonlinear transformation module, the entire recognition system can maintain high stability even in complex dynamic scenarios such as large arm swings, effectively reducing interference caused by changes in body posture and broadening the application scenarios of the invention.

[0087] Fifth, it achieves ultra-fast calibration in seconds, significantly improving the user experience:

[0088] This invention employs a visual target-guided micro-data acquisition protocol, requiring only about 30 seconds of calibration data to complete parameter fine-tuning and model calibration of the nonlinear transformation module. Compared to traditional technologies that require long-term data acquisition and training, this invention can significantly shorten calibration time, reduce user operating costs, and improve user experience.

[0089] Example 2

[0090] The source domain data missing type electromyography recognition device of the present invention includes:

[0091] The source domain electromyography acquisition module is used to acquire source domain electromyography signal segments with a predetermined number of channels N and a predetermined time window length T, and transmit the acquired source domain electromyography signal segments to the TCN feature extraction module.

[0092] The TCN feature extraction module is used to receive source domain electromyography signal segments transmitted by the source domain electromyography acquisition module, extract features from the source domain electromyography signal segments, and output a feature vector of fixed dimension.

[0093] The source domain model construction and training module is used to construct a source domain gesture recognition model. This model integrates a gesture classification head and a domain discriminator head. The gesture classification head receives a fixed-dimensional feature vector output by the TCN feature extraction module, outputs the gesture category probability, and calculates the corresponding cross-entropy loss. The domain discriminator head is connected to the TCN feature extraction module via a gradient inversion layer, receives the fixed-dimensional feature vector output by the TCN feature extraction module, outputs the domain label, and calculates the corresponding domain classification loss. The source domain gesture recognition model is trained using an adversarial training method that minimizes the cross-entropy loss while maximizing the domain classification loss until the model converges. After convergence, all parameters of the TCN feature extraction module and the source domain gesture recognition model are frozen.

[0094] The target domain calibration data acquisition module includes a human-computer interaction unit, a target domain electromyography (EMG) acquisition unit, and a label recording unit. The human-computer interaction unit displays dynamically changing intensity indicators, guiding the user to follow these indicators to complete a full movement cycle, and segments the EMG signal into segments according to a predetermined time window length T. The target domain EMG acquisition unit acquires the EMG signals during the user's complete movement cycle. The label recording unit synchronously records the movement category labels corresponding to the visual guidance, uses these labels as pseudo-labels for the visual guidance calibration data, and outputs complete visual guidance calibration data.

[0095] The nonlinear transformation module, located at the back end of the frozen TCN feature extraction module, is used to receive the feature vector extracted by the TCN feature extraction module from the original surface electromyography signal of the target domain, and perform nonlinear mapping processing on the feature vector to generate a calibrated feature vector aligned with the feature distribution of the source domain.

[0096] The target domain model fine-tuning and recognition module receives visually guided calibration data output from the target domain calibration data acquisition module, fine-tunes the parameters of the nonlinear transformation module to obtain the optimal nonlinear transformation module. The optimal nonlinear transformation module is then combined with the frozen TCN feature extraction module and the frozen source domain gesture recognition model to form the target domain gesture recognition model. The target domain gesture recognition model enables accurate recognition of target domain electromyographic signals in scenarios where source domain data is missing.

[0097] Example 3

[0098] The electronic device of the present invention includes a central processing unit and a memory, wherein the central processing unit is used to invoke and run a computer program stored in the memory to perform the steps of the source domain data missing type electromyography recognition method.

[0099] Example 4

[0100] The computer-readable storage medium of the present invention stores, in the form of computer-readable instructions, a computer program implementing the source domain data missing type electromyography recognition method, which, when called by a computer, executes the steps included in the corresponding method.

[0101] The above are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above content. Any changes, modifications, substitutions, combinations, or simplifications made without departing from the spirit and principle of the present invention shall be considered equivalent substitutions and shall be included within the protection scope of the present invention.

Claims

1. A method for recognizing electromyography (EMG) data missing from its source domain, characterized in that, Includes the following steps: Step 1: Acquire source domain electromyography (EMG) signal segments with a predetermined number of channels N and a predetermined time window length T. Input the EMG signal segments into the TCN feature extractor and extract features from the EMG signal segments through the TCN feature extractor to obtain a fixed-dimensional feature vector. Step 2: Construct a source domain gesture recognition model. Input the feature vector obtained in Step 1 into the gesture classification head and domain discriminator head of the source domain gesture recognition model, respectively. The gesture classification head outputs the gesture category probability and calculates the corresponding cross-entropy loss. The domain discriminator head is connected to the TCN feature extractor through a gradient inversion layer, outputs the domain label, and calculates the corresponding domain classification loss. Train the source domain gesture recognition model using an adversarial training method that minimizes the cross-entropy loss while maximizing the domain classification loss until the model converges. After the model converges, freeze all parameters of the TCN feature extractor and the source domain gesture recognition model. Step 3: In the target domain scene, a dynamically changing intensity indicator is displayed through the human-computer interaction interface to guide the user to follow the intensity indicator to complete the entire action cycle. Electromyography (EMG) signals within the complete action cycle are collected and segmented into EMG signal segments according to a predetermined time window length T. Simultaneously, the action category label corresponding to the visual guidance is recorded and used as a pseudo label for the visual guidance calibration data, thereby completing the collection of visual guidance calibration data. Step 4: Construct a trainable nonlinear transformation module and insert it into the back end of the frozen TCN feature extractor in Step 2; input the electromyographic signal fragments collected in the target domain into the frozen TCN feature extractor to obtain the original feature vector of the target domain; perform nonlinear mapping processing on the original feature vector of the target domain through the nonlinear transformation module to generate a calibrated feature vector aligned with the feature distribution of the source domain. Step 5: Using the visual guidance calibration data obtained in Step 3, fine-tune the parameters of the nonlinear transformation module in Step 4 to obtain the nonlinear transformation module with optimal parameters. Combine this nonlinear transformation module with the TCN feature extractor and source domain gesture recognition model frozen in Step 2 to construct the target domain gesture recognition model. The target domain gesture recognition model is used to accurately identify the target domain electromyographic signals in scenarios where source domain data is missing.

2. The method for identifying electromyography (EMG) data missing according to claim 1, characterized in that, In step 1, the TCN feature extractor comprises an input layer, stacked residual convolutional layers, and an output layer, wherein, The input layer is used to receive electromyographic signal segments with a predetermined number of channels N and a predetermined time window length T; The stacked residual convolutional layer is configured with multiple residual blocks, each containing two causal-dilated convolutional layers with the same kernel size, which is used to extract variable-length temporal features of electromyography signals in parallel. The output layer is used to receive the variable-length temporal features extracted in parallel by stacked residual convolutional layers, and aggregate them into a fixed-dimensional feature vector through global average pooling. Weight normalization and Leaky ReLU activation functions are set between the input layer and the stacked residual convolutional layer, as well as between the stacked residual convolutional layer and the output layer; the TCN feature extractor also includes a Dropout mechanism.

3. The method for identifying electromyography (EMG) data missing according to claim 1, characterized in that, In step 2, the gesture classification head consists of a fully connected layer and a Softmax function; the domain discrimination head consists of a gradient inversion layer followed by a fully connected layer.

4. The method for identifying electromyography (EMG) data missing according to claim 1, characterized in that, In step 3, the complete action cycle includes a standstill, linear increase, holding maximum force, and linear relaxation.

5. The method for identifying electromyography (EMG) data missing according to claim 1, characterized in that, In step 4, the nonlinear transformation module includes an input layer, a hidden layer, and an output layer. Nonlinear activation functions are provided between the input layer and the hidden layer, and between the hidden layer and the output layer, to fit and compensate for the nonlinear distortion of electromyographic signals caused by electrode offset. The number of trainable parameters of the nonlinear transformation module is less than 1% of the total number of parameters of the source domain gesture recognition model in step 2.

6. The method for identifying electromyography (EMG) data missing according to claim 5, characterized in that, In step 4, the mathematical model of the nonlinear transformation module is: ； In the formula: The alignment features are those after the adaptation module has been applied. This is the feature vector extracted from the raw surface electromyography signal by the TCN feature extractor; These are the adaptive feature transformation matrices; For the spatial transformation bias; It is a non-linear activation function.

7. The method for identifying electromyography (EMG) data missing according to claim 1, characterized in that, In step 5, the fine-tuning process sequentially includes forward propagation, multi-criteria loss optimization, and gradient freezing control. The specific steps are as follows: First, forward propagation is performed to input the target domain calibration signal into the frozen TCN feature extractor in step 2 to obtain the original feature vector. Then, the original feature vector is input into the nonlinear transformation module in step 4 to obtain the aligned feature vector. Finally, the aligned feature vector is sent into the frozen gesture classification head in step 2 to output the gesture category prediction probability. Next, multi-criteria loss optimization is performed, which includes supervised loss and confidence regularization. The supervised loss is used to calculate the cross-entropy loss between the gesture category prediction probability and the action category label in step 3. The confidence regularization is used to calculate the information entropy loss of the output distribution. By minimizing the information entropy value, the target domain features are forced to move closer to the high-confidence cluster centers of the source domain, thereby achieving feature distribution alignment. Finally, the gradient freezing strategy is executed. During the entire parameter update process, only the weight gradient of the nonlinear transformation module in step 4 is activated. The parameters of the TCN feature extractor and the source domain gesture recognition model that were frozen in step 2 remain unchanged and do not participate in the parameter update.

8. A source domain data missing type electromyography recognition device, characterized in that, include: The source domain electromyography acquisition module is used to acquire source domain electromyography signal segments with a predetermined number of channels N and a predetermined time window length T, and transmit the acquired source domain electromyography signal segments to the TCN feature extraction module. The TCN feature extraction module is used to receive source domain electromyography signal segments transmitted by the source domain electromyography acquisition module, extract features from the source domain electromyography signal segments, and output a feature vector of fixed dimension. The source domain model construction and training module is used to construct a source domain gesture recognition model. This model integrates a gesture classification head and a domain discriminator head. The gesture classification head receives a fixed-dimensional feature vector output by the TCN feature extraction module, outputs the gesture category probability, and calculates the corresponding cross-entropy loss. The domain discriminator head is connected to the TCN feature extraction module via a gradient inversion layer, receives the fixed-dimensional feature vector output by the TCN feature extraction module, outputs the domain label, and calculates the corresponding domain classification loss. The source domain gesture recognition model is trained using an adversarial training method that minimizes the cross-entropy loss while maximizing the domain classification loss until the model converges. After convergence, all parameters of the TCN feature extraction module and the source domain gesture recognition model are frozen. The target domain calibration data acquisition module includes a human-computer interaction unit, a target domain electromyography (EMG) acquisition unit, and a label recording unit. The human-computer interaction unit displays dynamically changing intensity indicators, guiding the user to follow these indicators to complete a full movement cycle, and segments the EMG signal into segments according to a predetermined time window length T. The target domain EMG acquisition unit acquires the EMG signals during the user's complete movement cycle. The label recording unit synchronously records the movement category labels corresponding to the visual guidance, uses these labels as pseudo-labels for the visual guidance calibration data, and outputs complete visual guidance calibration data. The nonlinear transformation module, located at the back end of the frozen TCN feature extraction module, is used to receive the feature vector extracted by the TCN feature extraction module from the original surface electromyography signal of the target domain, and perform nonlinear mapping processing on the feature vector to generate a calibrated feature vector aligned with the feature distribution of the source domain. The target domain model fine-tuning and recognition module receives visually guided calibration data output from the target domain calibration data acquisition module, fine-tunes the parameters of the nonlinear transformation module to obtain the optimal nonlinear transformation module. The optimal nonlinear transformation module is then combined with the frozen TCN feature extraction module and the frozen source domain gesture recognition model to form the target domain gesture recognition model. The target domain gesture recognition model enables accurate recognition of target domain electromyographic signals in scenarios where source domain data is missing.

9. An electronic device comprising a central processing unit and a memory, characterized in that, The central processing unit is used to invoke and run a computer program stored in the memory to perform the steps of the source domain data missing type electromyography recognition method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores, in the form of computer-readable instructions, a computer program implementing the source domain data missing type electromyography recognition method according to any one of claims 1 to 7, which, when called by a computer, executes the steps included in the corresponding method.

Images