Cross-domain hand action recognition method based on arm surface electromyogram signals
By combining the domain adaptation algorithm that aligns subspace alignment with second-order statistical distribution, the data distribution drift problem in transgender, cross-environment and cross-individual scenarios is solved, which significantly improves the accuracy of hand movement recognition and the generalization ability of classifiers.
Patent Information
- Application Number
- CN202510431840.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-08
- Publication Date
- 2025-07-22
AI Technical Summary
In the hand movement recognition based on sEMG signals, there is a problem of data distribution drift in transgender, cross-environment and cross-individual scenarios, which leads to limited generalization capabilities of classifiers. The existing field adaptive algorithms such as TCA and JDA have not been able to fully combine subspace alignment with multi-dimensional statistical optimization, and the recognition accuracy and robustness are insufficient.
The domain adaptation algorithm based on spatial and geometric alignment is adopted to generate the subspace substrates of the source domain and the target domain through principal component analysis, and the substrate differences are minimized using Frobenius norms, and combined with second-order statistical distribution alignment, the covariance matrix is optimized to reduce the data distribution differences.
The accuracy of hand movement recognition in transgender, cross-environment and cross-individual scenarios has been significantly improved, increasing by 26.66%, 14.43% and 25.00% respectively, enhancing the generalization ability and robustness of the classifier.
Smart Images
Figure CN120354097A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of body area network intelligent information systems, and relates to a method for recognizing hand movements based on surface electromyogram (sEMG) signals of the arm, which is particularly suitable for solving the problem of data distribution drift in cross-gender, cross-environment, and cross-individual scenarios and improving the generalization performance of classifiers. Background Art
[0002] In the prior art, hand movement recognition based on sEMG signals has important applications in the fields of human-computer interaction, robotic prosthetics, and rehabilitation medicine. However, traditional supervised learning methods rely on the assumption of independent and identically distributed data, and in practical applications, data distribution drift often occurs due to the following problems: 1) Cross-gender differences, where the sEMG signal representations are significantly different when different genders perform the same action; 2) Cross-environmental interference, where data distribution deviations are caused by differences in different acquisition devices or experimental scenarios; 3) Individual specificity, where signal distribution drift is caused by differences in muscle activity patterns of different subjects. Existing domain adaptation algorithms, such as Transfer Component Analysis (TCA) and Joint Distribution Alignment (JDA), although they can alleviate the distribution differences, do not fully combine subspace alignment and multi-dimensional statistic optimization, resulting in limited generalization ability of classifiers. For example, subspace alignment only focuses on geometric structure differences and ignores the multi-dimensional correlation of covariance distributions, and cannot completely eliminate cross-domain data differences. In addition, redundant feature interference and noise sensitivity further limit the recognition accuracy. Summary of the Invention
[0003] The object of the present invention is to propose a domain adaptation algorithm based on spatial and geometric alignment, which combines geometric alignment and statistical distribution alignment to effectively reduce cross-domain data differences and improve the accuracy and robustness of hand movement recognition.
[0004] To achieve the above object, the technical solution adopted by the present invention is: a cross-domain hand movement recognition method based on surface electromyogram signals of the arm:
[0005] Step 1: A surface electromyogram signal collector of the arm collects surface electromyogram signals sEMG and performs noise reduction processing. A Butterworth high-pass filter is used to eliminate baseline drift and action artifact interference, and the signal is standardized and segmented;
[0006] The collected surface electromyogram signals sEMG of the arm are subjected to noise reduction processing using a Butterworth high-pass filter to obtain a filtered signal with a frequency range of 20 - 500 Hz. The Butterworth filter has the characteristics of a flat response in the passband and a gentle attenuation in the stopband, and its amplitude-frequency response function is as follows:
[0007]
[0008] where Ω is the frequency; Ω0 is the cut-off frequency, and N is the system order of the filter.
[0009] Step 2: Extract time-domain, frequency-domain, and envelope signal features from the preprocessed sEMG signals;
[0010] The time-domain features include underlying descriptors and time-domain statistical features;
[0011] Among them, the underlying descriptors include integrated EMG value, mean absolute value, modified mean absolute value, simple square integral, sEMG variance, root mean square, Log detector, waveform length, mean amplitude change, differential absolute mean value, differential absolute standard deviation value, zero crossing rate, EMG pulse percentage, Willison amplitude, slope sign change, and maximum fractal length, a total of 16-dimensional features;
[0012] The time-domain statistical features include mean, maximum, minimum, standard deviation, range, skewness, and kurtosis, a total of 7-dimensional features;
[0013] The frequency-domain features include average power, total power, median frequency, peak frequency, average power, spectral moment, frequency ratio, power spectrum ratio, center frequency change, median spectral amplitude, and median spectral amplitude, a total of 11 dimensions;
[0014] The envelope signal features include envelope mean absolute value, envelope variance, envelope waveform length, envelope root mean square, envelope maximum value, and envelope minimum value, a total of 6 dimensions.
[0015] Step 3: Based on the time-domain, frequency-domain, and envelope signal features extracted in Step 2, obtain the feature spaces X of the training set and the test set, and generate the orthogonal bases X S and X T of the source domain and the target domain respectively through principal component analysis. Minimize the difference between the subspaces of the source domain and the target domain using the Frobenius norm, and achieve the geometric alignment of the source domain and the target domain by subspace alignment;
[0016] Given a training set with a known label L S =[y1,...] as the source domain S=[x S,1 ,...], x S,i ∈R N is a sample feature of it, and use the test set samples with unknown labels as the target domain T=[x T,1 ,...], x T,j ∈R N . Their feature spaces X and class spaces Y are the same;
[0017] In the first stage, generate the d-dimensional subspaces of the source domain and the target domain respectively, which are the d-dimensional feature vectors of the source domain and the target domain. Subsequently, find the orthogonal bases of the source domain subspace and the target domain subspace, denoted as X S and X TIt is represented that the orthogonal basis is generated by principal component analysis;
[0018] In the second stage, for subspace alignment, each source domain data S and target domain data T need to be projected onto their respective subspaces X S and TX T through operations SX S and X T respectively, so as to narrow the spatial distance between the source domain and the target domain, and make the bases of the two subspaces similar through transformation;
[0019] In the third stage, the Frobenius norm is used to measure the distance between the two bases, and the transformation matrix W acting on X S is used to align the bases of the two subspaces, resulting in the following formula:
[0020]
[0021] where is the Frobenius matrix norm, and F(W) is the distance between the basis X S of the source domain subspace after being transformed by W and the basis X T of the target domain subspace. Since X S and X T are orthogonal bases generated from the first d-dimensional eigenvectors, they have the property of inherent regularization, and the Frobenius norm is invariant under orthogonal operations. Therefore, formula (2) is rewritten as:
[0022]
[0023] where X S T represents the transpose matrix of X S . The optimal solution W * of the transformation matrix W is obtained by minimizing the Frobenius norm, as shown in formula (4):
[0024] W * = argmin W (F(W))
[0025] = X S T X T (4)
[0026] Since the transformation W acts on the basis X S of the source domain subspace, the newly obtained basis X a of the source domain subspace is:
[0027] X a = X S W * = X SX S T X T (5)
[0028] At this time, the data of the source domain on the new basis is expressed as:
[0029] S new = SX a = SX S W * = SX S X S T X T (6)
[0030] After the source domain S and the target domain T are respectively mapped by SX S W and TX T The new source domain S new is considered to be aligned with the target domain in space. Due to the situation of degenerate feature transformation, the feature divergence cannot be eliminated, and there are still differences in the statistical distributions of the source domain and the target domain. Therefore, the distribution differences between the source domain and the target domain data are narrowed through second-order statistical distribution alignment.
[0031] Step 4: Introduce second-order statistical distribution alignment, calculate the covariance matrices of the source domain and the target domain, and make the covariance distributions consistent through transformation;
[0032] Assume that μ S , μ T and C S , C T are the mean and covariance of the feature vectors of the source domain and the target domain respectively. After feature standardization, the means are usually equal, that is, μ S = μ T , so there is no need to align them; while in the second-order statistics, the covariance is the correlation between dimensions in a multi-dimensional space. After normalization and subspace alignment, there are still differences in the covariances of the source domain and the target domain, that is, C S ≠ C T ;
[0033] To minimize the distance between the second-order statistics of the source domain and the target domain, a transformation matrix M is introduced. Similar to subspace alignment, the Frobenius norm is still used as the measure of matrix distance, and the transformation matrix M is obtained by minimizing the Frobenius norm, as shown in formula (7):
[0034]
[0035] where is the covariance of the source domain after transformation. If rank(C S ) ≥ rank(C T) directly equates M to two parts of the structure. One part is the correlation between the bleaching source feature vectors, and the other part is to recolour using the covariance between the target feature vectors. By adding the new correlation to it, we get:
[0036]
[0037] The statistical distribution distance between the source domain and the target domain that has been subspace-aligned is narrowed through the linear transformation A. A contains the subspace transformation matrix W and the transformation matrix M used to align the statistical distribution, that is:
[0038]
[0039] Therefore, the data representation of the source domain after alignment is:
[0040] S new = SX a M = SX S WM = SX S A (10)
[0041] Through SX S The mapping of A projects the source domain S onto the source subspace aligned with the target domain. At the same time, the target domain T also needs to go through TX T and be projected onto the corresponding target subspace.
[0042] Step Five: Input the feature representation after domain adaptation into the classifier and output the hand gesture recognition result.
[0043] The beneficial effects of this invention are:
[0044] Significantly improve the cross-domain recognition accuracy: By combining subspace alignment and second-order statistical distribution alignment, the data distribution differences in cross-gender, cross-environment, and cross-individual scenarios are effectively reduced, and the average accuracy of hand gesture recognition is increased by 26.66%, 14.43%, and 25.00% respectively, which is significantly better than traditional domain adaptation algorithms (such as TCA, JDA, etc.).
[0045] Enhance the generalization ability of the classifier: By optimizing the correlation between multi-dimensional features through covariance alignment, the problem of ignoring statistical distribution differences in subspace alignment is solved. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] Figure 1 is a schematic diagram of the domain adaptation algorithm for space and geometric alignment;
[0047] Figure 2 are the wrist flexion and wrist extension hand gestures used in the experiment. DETAILED DESCRIPTION OF THE INVENTION
[0048] Example 1:
[0049] Implement cross - gender, cross - environment, and cross - individual gesture recognition in typical application scenarios of gesture recognition tasks. The steps are as follows:
[0050] (1) Use an arm surface electromyogram signal collector to collect arm surface electromyogram signals (sEMG) and perform noise reduction processing:
[0051] Gesture recognition uses multi - channel electromyogram signals EMG as a sensing device. The acquisition position is in the middle of the forearm above the wrist and below the elbow. Experimental dataset A is data of 7 hand movements of 36 subjects (aged from 18 to 41 with different hand training backgrounds), including hand at rest, wrist horizontal and radial flexion, etc. Dataset B is data of 6 subjects, 4 males (aged 21 - 26) and 2 females (aged 23 - 25), repeating 15 static gestures 8 times in different scenarios. Dataset C is 7 - gesture data of 40 participants (28 males and 12 females), with 7 gesture movement data. Pass the original sEMG signal through a Butterworth high - pass filter. The cut - off frequency in this range can significantly improve noise interference and make the deep information contained in the sEMG signal prominent.
[0052] (2) Extract time - domain, frequency - domain, and envelope signal features from the pre - processed sEMG signals:
[0053] 1) Time - domain features include low - level descriptors and time - domain statistical features. Among them, the low - level descriptors have 16 - dimensional features such as integrated electromyogram value, mean absolute value, modified mean absolute value, simple square integral, sEMG variance, root mean square, Log detector, waveform length, mean amplitude change, differential absolute mean value, differential absolute standard deviation value, zero - crossing rate, electromyogram pulse percentage, Willison amplitude, slope sign change, and maximum fractal length. The time - domain statistical features have 7 - dimensional features such as mean, maximum, minimum, standard deviation, range, skewness, and kurtosis.
[0054] 2) Frequency - domain features include 11 - dimensional features such as average power, total power, median frequency, peak frequency, average power, spectral moment, frequency ratio, power spectrum ratio, center frequency change, median spectral amplitude, and median spectral amplitude.
[0055] 3) Envelope signal features include 6 - dimensional features such as envelope mean absolute value, envelope variance, envelope waveform length, envelope root mean square, envelope maximum value, and envelope minimum value.
[0056] (3) Through subspace alignment, use the Frobenius norm to minimize the basis difference and achieve geometric alignment:
[0057] First, in the transgender scenario, using the data of 40 people in dataset C, the males are divided into two groups M1 and M2 (14 people in each group), and the female dataset is divided into two groups F1 and F2 (6 people in each group). First, take the two groups of males as the source domain S respectively, and the female data as the target domain T, and migrate them to the two groups of female data respectively. Subsequently, take the two groups of females as the source domain respectively, and the two groups of males as the target domain. In the cross-environment scenario, take dataset A, dataset B, and dataset C as the source domain and the target domain in turn. In the cross-individual scenario, take the data of 34 participants in dataset C as the source domain. Secondly, select the data of another 6 participants (3 males and 3 females) as the target domain separately in turn. Generate the subspace bases X S and X T of the source domain and the target domain respectively through principal component analysis, minimize the difference between the subspace bases of the source domain and the target domain using the Frobenius norm, and obtain the optimal solution of the transformation matrix W. After the source domain S and the target domain T pass through the mappings of SX S W and TX T respectively, the new source domain S new can be regarded as being aligned with the target domain in space.
[0058] (4) Introduce second-order statistical distribution alignment. Calculate the covariance matrices of the source domain and the target domain, and make the covariance distributions consistent through transformation:
[0059] μ S μ T and C S 、C T are the mean and covariance of the feature vectors of the source domain and the target domain respectively. After feature standardization, the means are usually equal, that is, so there is no need to align them; while in the second-order statistics, the covariance is the correlation between each dimension in the multi-dimensional space. After normalization and subspace alignment, there are still differences in the covariance between the source domain and the target domain. In order to minimize the distance between the second-order statistics (covariance) of the source domain and the target domain, introduce the transformation matrix M The data representation of the source domain after alignment is SX S A. The classifier trained by the new source domain data can be directly applied to the target domain, thereby improving the generalization performance of the classifier.
[0060] (5) Input the processed feature data into the support vector machine classifier to output the hand action recognition result.
[0061] In the recognition under the transgender scenario, the method proposed in the present invention achieves better performance compared with 7 baseline methods such as SDA, SA, GFK, TCA, JDA, MEDA, and WBDA, reaching an identification accuracy of 85.87%. There are performance improvements of 0.85%, 0.94%, 5.57%, 1.39%, 1.20%, 1.71%, and 1.31% compared with the comparative methods respectively. In the cross-environment scenario, the final average recognition accuracy of the base classifier is only 49.04%. After being processed by the spatial and geometric alignment algorithm, the recognition accuracy is 63.47%, improving the recognition accuracy of the classifier by 14.43%. This shows that the method proposed in the present invention can construct a more robust and effective classifier for sEMG signal data from different sources. In the cross-individual scenario, the result of the base classifier is only 66.41%. The recognition accuracies of the method proposed in the present invention and the 7 comparative methods are 91.41%, 88.80%, 91.15%, 85.94%, 83.85%, 84.64%, 87.50%, and 78.39%. This method has made a great contribution to reducing the generalization error of the classifier in the cross-individual case, resulting in a 25.00% increase in the accuracy of the final recognition result, proving the effectiveness of the method proposed in the present invention.
Claims
1. A cross-domain hand motion recognition method based on arm surface electromyography signals, characterized in that: Step 1: The arm surface electromyography signal collector collects the arm surface electromyography signal sEMG and performs noise reduction processing, uses a Butterworth high-pass filter to eliminate baseline drift and motion artifact interference, and standardizes and segments the signal; Step 2: Extract time domain, frequency domain and envelope signal features from the preprocessed sEMG signal; Step 3: Based on the time domain, frequency domain, and envelope signal features extracted in Step 2, obtain the feature space X of the training set and the test set, and generate the subspace orthogonal bases X S and X T of the source domain and the target domain respectively through principal component analysis. Minimize the difference between the source domain and the target domain subspace bases using the Frobenius norm, and achieve the geometric alignment between the source domain and the target domain by subspace alignment; Step 4: Introduce second-order statistical distribution alignment, calculate the covariance matrix of the source domain and the target domain, and make the covariance distribution consistent through transformation; Step 5: Input the domain-adapted feature representation into the classifier and output the hand action recognition result.
2. The cross-domain hand motion recognition method based on the surface electromyogram signal of the arm according to claim 1, wherein, In the step 1), the specific method is: The Butterworth high-pass filter is used to reduce the noise of the collected arm surface electromyography signal sEMG to obtain a filtered signal with a frequency range of 20-500Hz. The Butterworth filter has the characteristics of flat response in the passband and gentle attenuation in the stopband. Its amplitude-frequency response function is as follows: Where Ω is the frequency; Ω0 is the cutoff frequency, and N is the system order of the filter.
3. A cross-domain hand movement recognition method based on surface electromyogram signals of the arm according to claim 1, characterized in that In the step 2), the specific method is: The time domain features include low-level descriptors and time domain statistical features; The underlying descriptors include integrated EMG value, mean absolute value, corrected mean absolute value, simple square integral, sEMG variance, root mean square, Log detector, waveform length, average amplitude change, differential absolute mean, differential absolute standard deviation, zero crossing rate, EMG pulse percentage, Willison amplitude, slope sign change, and maximum fractal length, a total of 16 dimensions; The time domain statistical features include mean, maximum, minimum, standard deviation, range, skewness, and kurtosis, a total of 7 dimensions; Frequency domain features include average power, total power, median frequency, peak frequency, average power, spectral moment, frequency ratio, power spectrum ratio, center frequency change, spectrum amplitude median, and spectrum amplitude median, a total of 11 dimensions; The envelope signal features include envelope average absolute value, envelope variance, envelope waveform length, envelope root mean square, envelope maximum value, and envelope minimum value, a total of 6 dimensions.
4. A cross-domain hand motion recognition method based on surface electromyogram signals of the arm according to claim 1, wherein In the step 3), the specific method is: Given a known label L S = [y1,...] of the training set as the source domain S = [x S,1 ,...], x S,i ∈R N as one of its sample features, and taking the test set samples with unknown labels as the target domain T = [x T,1 ,...], x T,j ∈R N , and their feature space X and class space Y are the same; In the first stage, d-dimensional subspaces of the source domain and the target domain are generated, which are the d-dimensional feature vectors of the source domain and the target domain respectively. Subsequently, the orthogonal bases of the source domain subspace and the target domain subspace are found, denoted by X S and X T ; the orthogonal bases are generated by principal component analysis; In the second stage, for subspace alignment, each source domain data S and target domain data T need to be projected onto their respective subspaces X S and X T through operations SX S and X T respectively, so as to narrow the spatial distance between the source domain and the target domain, and make the bases of the two subspaces similar through transformation; In the third stage, the Frobenius norm is used to measure the distance between the two bases, and the transformation matrix W acting on X S is used to align the two subspace bases, resulting in the following formula: Among them is the Frobenius matrix norm, and F(W) is the basis X of the source domain subspace S after being transformed by W and the basis X of the target domain subspace T The distance of, because X S and X T are orthogonal bases generated by the first d-dimensional eigenvectors, so the two have the property of intrinsic regularization, and the Frobenius norm is invariant to orthogonal operations. Therefore, formula (2) is rewritten as: Among which X S T represents X S The transposed matrix of S obtains the optimal solution W of the transformation matrix W by minimizing the Frobenius norm * As shown in Equation (4): W * = argmin W (F(W)) = X S T X T (4) Since the transformation W acts on the basis X of the source domain subspace S and the newly obtained basis X a of the source domain subspace is as follows: X a = X S W * = X S X S T X T (5) At this point, the data in the source domain on the new basis is represented as: S new = SX a = SX S W * = SX S X S T X T (6) After the mappings of the source domain S and the target domain T through SX S W and TX T respectively, the new source domain S new is regarded as spatially aligned with the target domain. Due to the situation of degenerate feature transformation, the feature divergence cannot be eliminated, and there are still differences in the statistical distributions of the source domain and the target domain. Therefore, the distribution differences between the source domain and the target domain data are narrowed through second-order statistical distribution alignment.
5. A cross-domain hand motion recognition method based on surface electromyogram signals of the arm according to claim 1, characterized in that In the step 4), the specific method is: Suppose μ S , μ T and C S , C T are the means and covariances of the source domain and target domain feature vectors respectively. After feature standardization, the means are usually equal, i.e., μ S = μ T , so there is no need to align them; while in the second-order statistics, the covariance is the correlation between each dimension in the multi-dimensional space. After normalization and subspace alignment, there are still differences in the covariances of the source domain and target domain, i.e., C S ≠ C T ; In order to minimize the distance between the second-order statistics of the source domain and the target domain, the transformation matrix M is introduced. Similar to subspace alignment, the Frobenius norm is still used as the measure of matrix distance, and the transformation matrix M is obtained by the minimum Frobenius norm, as shown in formula (7): Among them, is the covariance of the transformed source domain. If rank(C S ) ≥ rank(C T ), directly equivalent M to two parts of the structure. One part is the correlation between the whitened source eigenvectors, and the other part is to re-color using the covariance between the target eigenvectors. Using to add the new correlation into it, we get: The linear transformation A is used to shorten the statistical distribution distance between the source domain and the target domain after subspace alignment. A contains the subspace transformation matrix W and the transformation matrix M used to align the statistical distribution, that is: Therefore, the source domain data after alignment is represented as: S new = SX a M = SX S WM = SX S A(10) Through SX S The mapping of A projects the source domain S onto the source subspace aligned with the target domain, while the target domain T also needs to go through TX T Projected onto the corresponding target subspace.