Hand motion recognition method and system

Abstract

The present invention discloses a hand motion recognition method and system. The method includes: extracting features from at least one target IMF component, combining the extracted time-domain features into a time-domain feature vector, and combining the extracted frequency-domain features into a frequency-domain feature vector; filtering the time-domain feature vector and the frequency-domain feature vector according to a preset feature scoring function to obtain a target time-domain feature vector and a target frequency-domain feature vector; normalizing the target time-domain feature vector and the target frequency-domain feature vector, and performing cross-weighted fusion on the normalized target time-domain feature vector and the normalized target frequency-domain feature vector to obtain a composite feature vector; inputting the composite feature vector into a preset improved CNN model, which outputs a hand motion recognition result corresponding to the strain signal. This method can effectively improve the accuracy of hand motion recognition.

Description

Technical Field

[0001] The present invention belongs to the technical field of flexible sensors, and in particular relates to a hand motion recognition method and system. Background Art

[0002] With the rapid development of artificial intelligence and smart wearable devices, hand motion recognition, as a crucial method for human-computer interaction, has garnered widespread attention. Hand motion recognition technology uses sensors to sense and analyze hand movements, effectively supporting applications in virtual reality, smart healthcare, robotic control, and other fields. However, the dynamic, complex, and high-precision nature of hand motions poses numerous challenges. In particular, in sensor data processing, efficient denoising, signal interference reduction, key feature extraction, and processing of multi-dimensional, heterogeneous sensor data have become key challenges in this field.

[0003] Existing methods for hand movement recognition often rely on a combination of traditional signal processing and deep learning. While these methods extract signal features through time-domain or frequency-domain analysis, while capable of achieving basic performance, they often encounter significant boundary effects, significant noise impact, and difficulty in feature selection when faced with complex movements and multi-sensor data fusion. Consequently, existing methods cannot meet the high-precision and robustness requirements of practical applications. Summary of the Invention

[0004] The present invention provides a hand motion recognition method and system for solving the technical problem of inaccurate hand motion recognition caused by obvious boundary effect, large noise influence, difficult feature selection and the like.

[0005] In a first aspect, the present invention provides a hand motion recognition method, comprising:

[0006] Acquire a strain signal from a hand sensor, and perform EWT transformation on the strain signal according to a boundary adaptive backfill mechanism to obtain a target strain signal;

[0007] Decomposing the target strain signal according to the EEMD to obtain at least one IMF component, calculating a multi-scale nonlinear relationship influence factor of the at least one IMF component, and selecting at least one target IMF component whose multi-scale nonlinear relationship influence factor is greater than a preset threshold;

[0008] Performing feature extraction on the at least one target IMF component, combining the extracted time domain features into a time domain feature vector, and combining the extracted frequency domain features into a frequency domain feature vector, and screening the time domain feature vector and the frequency domain feature vector according to a preset feature scoring function to obtain a target time domain feature vector and a target frequency domain feature vector;

[0009] Normalizing the target time domain feature vector and the target frequency domain feature vector, and performing cross-weighted fusion on the normalized target time domain feature vector and the normalized target frequency domain feature vector to obtain a composite feature vector;

[0010] The composite feature vector is input into a preset improved CNN model, and the improved CNN model outputs a hand motion recognition result corresponding to the strain signal.

[0011] In a second aspect, the present invention provides a hand motion recognition system, comprising:

[0012] an acquisition module configured to acquire a strain signal from a hand sensor and perform an EWT transformation on the strain signal according to a boundary adaptive backfill mechanism to obtain a target strain signal;

[0013] a calculation module configured to decompose the target strain signal according to the EEMD to obtain at least one IMF component, calculate a multi-scale nonlinear relationship influence factor of the at least one IMF component, and select at least one target IMF component whose multi-scale nonlinear relationship influence factor is greater than a preset threshold;

[0014] a screening module configured to perform feature extraction on the at least one target IMF component, combine the extracted time domain features into a time domain feature vector, and combine the extracted frequency domain features into a frequency domain feature vector, and screen the time domain feature vector and the frequency domain feature vector according to a preset feature scoring function to obtain a target time domain feature vector and a target frequency domain feature vector;

[0015] a fusion module configured to normalize the target time domain feature vector and the target frequency domain feature vector, and perform cross-weighted fusion on the normalized target time domain feature vector and the normalized target frequency domain feature vector to obtain a composite feature vector;

[0016] The output module is configured to input the composite feature vector into a preset improved CNN model, and the improved CNN model outputs a hand motion recognition result corresponding to the strain signal.

[0017] According to a third aspect, an electronic device is provided, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can perform the steps of the hand motion recognition method of any embodiment of the present invention.

[0018] In a fourth aspect, the present invention further provides a computer-readable storage medium having a computer program stored thereon, wherein when the program instructions are executed by a processor, the processor executes the steps of the hand motion recognition method of any embodiment of the present invention.

[0019] The hand motion recognition method and system of this application, BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0021] Figure 1 A flowchart of a hand motion recognition method provided by one embodiment of the present invention;

[0022] Figure 2 A structural block diagram of a hand motion recognition system provided by one embodiment of the present invention;

[0023] Figure 3 It is a structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0025] See also Figure 1 , which shows a flow chart of a hand motion recognition method of the present application.

[0026] like Figure 1 As shown, the hand motion recognition method specifically includes the following steps:

[0027] Step S101 : acquiring a strain signal from a hand sensor, and performing EWT transformation on the strain signal according to a boundary adaptive backfill mechanism to obtain a target strain signal.

[0028] In this step, the function expression of the boundary adaptive backfill mechanism is:

[0029] ,

[0030] Where, is the boundary weighting function, is the time point of the left boundary of the signal, For time, is the total time length of the signal, 、 are exponentials that control Gaussian attenuation. 、 Both control the rate at which the signal decays. 、 are factors that affect the polynomial. 、 、 、 The order of the polynomial, is the time point of the right boundary of the signal, Indicates the exponential operation.

[0031] Specifically, the strain signal is subjected to EWT (Empirical Wavelet Transform) according to the boundary adaptive backfill mechanism to obtain the target strain signal including:

[0032] The strain signal is subjected to boundary weighting processing according to the boundary adaptive backfilling mechanism to obtain a strain signal after boundary backfilling, which is expressed as:

[0033] ,

[0034] Where, is the strain signal after boundary backfilling, is the strain signal;

[0035] Perform EWT transformation on the strain signal after boundary backfilling to obtain the target strain signal, which is expressed as:

[0036] ,

[0037] Where, For the The target strain signal of each frequency band, is the inverse Fourier transform, for The inverse Fourier transform of For the The wavelet filter function of the frequency band.

[0038] Step S102: Decompose the target strain signal according to the EEMD to obtain at least one IMF component, calculate a multi-scale nonlinear relationship influence factor of the at least one IMF component, and select at least one target IMF component whose multi-scale nonlinear relationship influence factor is greater than a preset threshold.

[0039] In this step, the target strain signal is decomposed according to EEMD (Ensemble Empirical Mode Decomposition) to obtain at least one IMF component, which is expressed as:

[0040] ,

[0041] Where, For the The frequency band signal is decomposed by EEMD to obtain the The decomposition results, For the The first frequency band signal IMF components, For the The frequency band signal is decomposed by EEMD to obtain the The residual term, is the total number of IMFs for each frequency band signal;

[0042] Calculate the multi-scale nonlinear relationship influence factor of the at least one IMF component, the expression is:

[0043] ,

[0044] ,

[0045] ,

[0046] ,

[0047] Where, Indicates the The first frequency band signal The multi-scale nonlinear relationship impact factor of the IMF, express and The nonlinear mutual information between represents a nonlinear function, express and The weighting function of Indicates the The first frequency band signal The correlation coefficient of the IMF is Indicates the The first frequency band signal IMF correlation coefficients, express and Similarity in the frequency domain, represents the weighted time-domain correlation function, Indicates the The first frequency band signal IMF at the time The value at Indicates the The first frequency band signal IMF at the time The value at represents the similarity function of the spectrum, Indicates frequency, represents the polynomial coefficients, represents the order of the polynomial, Indicates from arrive The points, represents the frequency weighting factor, represents the exponent that controls the distance metric, 、 At different time points, For the The first frequency band signal IMF in frequency The value at For the The first frequency band signal IMF in frequency The value at is the total number of frequency bands, is the correlation weighting factor, is the total number of IMFs decomposed into a frequency band signal, is the correlation difference index;

[0048] At least one target IMF component whose multi-scale nonlinear relationship influence factor is greater than a preset threshold is selected.

[0049] Step S103, performing feature extraction on the at least one target IMF component, combining the extracted time domain features into a time domain feature vector, and combining the extracted frequency domain features into a frequency domain feature vector, and screening the time domain feature vector and the frequency domain feature vector according to a preset feature scoring function to obtain a target time domain feature vector and a target frequency domain feature vector.

[0050] In this step, feature extraction is performed on the at least one target IMF component to obtain time domain features, and the time domain features at multiple scales are combined into a time domain feature vector, which is expressed as:

[0051] ,

[0052] ,

[0053] Where, is the vector that combines the time domain features extracted from the filtered IMF. 、 、 Respectively The first frequency band signal IMF's first, second, and third The dynamic volatility index, For the The first frequency band signal The IMF's The dynamic volatility index, is the time length of the IMF, To support the IMF The translation of the scale factor, Indicates the The first frequency band signal IMF at the time The value at is the nonlinear index, Indicates scaling of the scale factor. is a constant;

[0054] Feature extraction is performed on the at least one target IMF component to obtain frequency domain features, and each frequency domain feature is combined with the weighted second-order moment of each frequency domain feature to obtain a frequency domain feature vector, which is expressed as:

[0055] ,

[0056] ,

[0057] ,

[0058] Where, For the The first frequency band signal The frequency domain eigenvector of an IMF, For the The first frequency band signal The frequency domain structure compression rate of an IMF is is the total number of frequency bands, is the frequency, For the The first frequency band signal IMF in frequency The spectrum power at For the The first frequency band signal The IMF The spectral power of the frequency, is the smoothing factor, Indicates the The first frequency band signal The IMF is Fourier transformed. Indicates the The first frequency band signal IMF at the time The value at is the Fourier transform;

[0059] The first feature score of each time domain feature vector is calculated according to a preset feature scoring function, and the second feature score of each frequency domain feature vector is calculated, wherein the expression of the feature scoring function is:

[0060] ,

[0061] Where, For the The first frequency band signal IMF characteristics The scoring function, Features The discreteness of For the The first frequency band signal The time domain characteristics of an IMF, For the The first frequency band signal The frequency domain characteristics of an IMF, Features The average across all IMFs, is the median of all feature means, 、 Both are skew penalty control parameters;

[0062] At least one target time domain feature vector having a first feature score greater than a first score threshold is selected, and at least one target frequency domain feature vector having a second feature score greater than a second score threshold is selected.

[0063] Step S104 , normalizing the target time domain feature vector and the target frequency domain feature vector, and cross-weighted fusion of the normalized target time domain feature vector and the normalized target frequency domain feature vector to obtain a composite feature vector.

[0064] In this step, the target time domain feature vector and the target frequency domain feature vector are normalized, and the expression is:

[0065] ,

[0066] ,

[0067] ,

[0068] Where, After normalization, The first frequency band signal The time domain characteristics of an IMF, After normalization, The first frequency band signal The frequency domain characteristics of an IMF, is the nonlinear normalized mapping function, For the The first frequency band signal The time domain characteristics of an IMF, For the The first frequency band signal The frequency domain characteristics of an IMF, is a feature set The standard deviation of is a feature set The mean of To adjust the parameters, is a constant;

[0069] The normalized target time domain feature vector and the normalized target frequency domain feature vector are cross-weighted fused to obtain a composite feature vector, which is expressed as:

[0070] ,

[0071] ,

[0072] ,

[0073] ,

[0074] Where, For the The first frequency band signal The composite eigenvector of IMFs, For the The first frequency band signal The weighting coefficients of the time domain and frequency domain characteristics of an IMF, For the The first frequency band signal The weighted coefficient of the time domain characteristics of an IMF, For the The first frequency band signal The weighting coefficient of the frequency domain characteristics of an IMF, for With the target variable The mutual information between is the target variable, for With the target variable The mutual information between them.

[0075] Step S105: input the composite feature vector into a preset improved CNN model, and the improved CNN model outputs a hand motion recognition result corresponding to the strain signal.

[0076] In this step, multi-dimensional influence factors are added before the convolutional layer of the CNN network, including the time influence factor that dynamically adjusts the feature importance of each time point according to the time domain part of the input composite feature vector, and the frequency domain influence factor that measures the importance of different frequency components to the final feature in the frequency domain. The expression of the time influence factor is:

[0077] ,

[0078] Where, Indicates the The first frequency band signal IMF at a point in time The time impact factor, represents the time domain features in the composite feature vector, represents the time domain variation adjustment factor, represents the time domain smoothing over-adjustment factor, Represents the time domain signal at the time point The instantaneous rate of change, represents the time domain balance term, which prevents excessive response to extreme changes. Indicates the total step length of the time point;

[0079] The expression of the domain impact factor is:

[0080] ,

[0081] Where, Indicates the The first frequency band signal IMF in frequency The frequency domain impact factor of represents the frequency domain features in the composite feature vector, represents the frequency domain change adjustment factor, represents the frequency domain smoothing over-regulation factor, represents the instantaneous rate of change of the frequency component, represents the frequency domain balance term, is the total number of frequency bands;

[0082] Combine the time impact factor and the frequency domain impact factor and apply them to the features of the convolution layer. The expression is:

[0083] ,

[0084] Where, represents the composite feature after adjustment, represents the input signal, where is the adjusted composite eigenvector, is the composite eigenvector of the mth IMF of the ith frequency band signal;

[0085] After the convolution layer, the improved nonlinear activation function is applied to adjust the composite feature vector adjusted by the influence factor, so that the model can learn more complex nonlinear relationships.

[0086] The expression of the nonlinear activation function is:

[0087] ,

[0088] Where, , , They represent different activation functions. , , Respectively represent the adjustable parameters of the corresponding activation function, controlling the contribution of each activation function, Represents the improved nonlinear activation function;

[0089] An attention mechanism is added after the output of the convolutional layer, including the time domain attention mechanism and the frequency domain attention mechanism, so that the network can adaptively focus on the most informative features and improve the network's ability to learn key features.

[0090] The expression of the temporal attention mechanism is:

[0091] ,

[0092] Where, Indicates the The first frequency band signal IMF at a point in time The temporal attention weight of Indicates the current time point, represents the total number of time steps, represents the number of time steps in the time window, Indicates the length of the time window, Indicates the The first frequency band signal IMF at a point in time The time domain features after activation function processing, Indicates at a point in time In the nearby time window, the past The cumulative sum of the time domain features at each time point, represents the adjustment parameter of the time domain attention weight, Indicates the time domain adjustment item parameters;

[0093] The expression of the frequency domain attention mechanism is:

[0094] ,

[0095] Where, Indicates the The first frequency band signal IMF at frequency point The frequency domain attention weight, represents the adjustment factor of frequency domain attention, Indicates the The first frequency band signal IMF at frequency point The frequency domain features after activation function processing, Indicates the frequency Nearby The cumulative sum of the frequency domain features of the frequency points, represents the size of the sliding window, Represents an index used to select the current frequency point Nearby frequency points, Indicates the frequency domain adjustment parameter.

[0096] The weighted operation is performed on the features after nonlinear activation. The expression is:

[0097] ,

[0098] ,

[0099] Where, Represents the time domain features after weighting by the time attention mechanism, represents element-wise multiplication, Represents the time domain features after being processed by the activation function, Represents the frequency domain features after weighting by the frequency domain attention mechanism, Represents the frequency domain features after being processed by the activation function.

[0100] In summary, the method of the present application performs an EWT transform on the original strain signal, and adds a boundary adaptive backfill mechanism to decompose the original strain signal into components of different frequency bands to remove noise and clutter. The boundary adaptive backfill mechanism is introduced. This method can effectively solve the boundary effect generated when using EWT to decompose the signal. Traditional backfill methods (such as zero filling) will produce mutations at the boundary of the signal, affecting subsequent feature extraction and model training. After using the boundary adaptive backfill mechanism, the boundary part of the signal will be smoothly transitioned according to the local mean and gradient, making the signal boundary more consistent with the internal features, thereby avoiding the negative impact of the boundary effect on the analysis results.

[0101] Perform EEMD decomposition on the EWT-transformed signal to filter out important IMF components and improve their quality: EEMD decomposition can decompose a complex signal into multiple IMF components with different frequency components. By filtering out the most important IMF components, useful information is retained while irrelevant or low-quality components are removed.

[0102] Dynamic screening of important IMFs: By applying nonlinear relationship influencing factors, the most representative IMF components can be automatically screened out, thereby improving the efficiency of feature selection and avoiding the interference of redundant information.

[0103] Time and frequency domain feature extraction and scoring function selection: In this step, a new time-frequency domain feature extraction method is proposed, which differs from traditional time-frequency domain feature extraction methods. The newly proposed time-frequency domain features can more accurately characterize the time and frequency domain characteristics of the signal and capture more complex signal variation patterns in the time and frequency domains.

[0104] Scoring function screening: Use a custom scoring function to screen the extracted time-frequency domain features, select the most important features, and obtain the final time-frequency domain feature set.

[0105] Innovative time-frequency domain features: Newly proposed time-frequency domain features are more refined and can capture the complex patterns of signals in the time and frequency domains. These features can provide more information than traditional features and help improve the accuracy of signal analysis.

[0106] Feature screening: By screening time-frequency features through a scoring function, irrelevant or redundant features can be effectively removed, retaining only the feature parts that best represent the signal, thereby improving the efficiency and accuracy of the model.

[0107] Normalization and weighted fusion of time-frequency domain features and their input into the improved CNN network. Normalization and weighted fusion of time-frequency domain features: By normalizing and weighted fusion of the filtered time-frequency domain features, the time and frequency domain features can be more balanced, eliminating the differences between different feature dimensions. Through weighted fusion, the expressive power of time-frequency features is enhanced. Balanced time-frequency features: Through normalization and weighted fusion, the balance of time and frequency domain features is ensured when they are input into the network, allowing the network to fairly utilize time-frequency feature information and avoiding overemphasis on features of a particular dimension.

[0108] The introduction of multidimensional influencing factors (TIF and FIF) and the combination of time-domain and frequency-domain attention mechanisms enable the signal's time-domain and frequency-domain features to more accurately reflect their contribution to the task, thereby improving the expressive power of features and the model's learning ability. In particular, in complex signals, the most informative components can be highlighted, enhancing the accuracy of hand movement recognition. This optimizes the model's processing of time-frequency features, avoiding the limitations of single time-domain or frequency-domain features. This enables the network to select and weight features at a higher level, thereby enhancing the model's recognition accuracy for hand movements, especially its robustness in complex scenarios.

[0109] The specific technical effects are:

[0110] Improve signal quality: effectively remove noise and clutter, reduce the impact of boundary effects, and ensure signal stability;

[0111] Enhanced feature selection and expression capabilities: By dynamically screening the most important IMF components and time-frequency features, the model's feature extraction capability and accuracy are improved;

[0112] Optimized feature weighted fusion: Through adaptive weighted fusion of time-frequency domain features, the model can better capture important information in the signal;

[0113] Improved classification accuracy: The optimized features are fed into an improved CNN network, which improves the accuracy and robustness of hand gesture recognition, especially in complex environments.

[0114] See also Figure 2 , which shows a structural block diagram of a hand motion recognition system of the present application.

[0115] like Figure 2 As shown, the hand motion recognition system 200 includes an acquisition module 210 , a calculation module 220 , a screening module 230 , a fusion module 240 and an output module 250 .

[0116] The acquisition module 210 is configured to acquire a strain signal from a hand sensor and perform an EWT transformation on the strain signal according to a boundary adaptive backfill mechanism to obtain a target strain signal.

[0117] a calculation module 220 configured to decompose the target strain signal according to the EEMD to obtain at least one IMF component, calculate a multi-scale nonlinear relationship influence factor of the at least one IMF component, and select at least one target IMF component whose multi-scale nonlinear relationship influence factor is greater than a preset threshold;

[0118] The screening module 230 is configured to perform feature extraction on the at least one target IMF component, combine the extracted time domain features into a time domain feature vector, and combine the extracted frequency domain features into a frequency domain feature vector, and screen the time domain feature vector and the frequency domain feature vector according to a preset feature scoring function to obtain a target time domain feature vector and a target frequency domain feature vector;

[0119] a fusion module 240 configured to normalize the target time domain feature vector and the target frequency domain feature vector, and perform cross-weighted fusion on the normalized target time domain feature vector and the normalized target frequency domain feature vector to obtain a composite feature vector;

[0120] The output module 250 is configured to input the composite feature vector into a preset improved CNN model, and the improved CNN model outputs a hand motion recognition result corresponding to the strain signal.

[0121] It should be understood that Figure 2 Modules and references documented in Figure 1 Therefore, the operations and features described above for the method and the corresponding technical effects also apply to Figure 2 The modules in it will not be described in detail here.

[0122] In some other embodiments, embodiments of the present invention further provide a computer-readable storage medium having a computer program stored thereon, wherein when the program instructions are executed by a processor, the processor is caused to execute the hand motion recognition method in any of the above method embodiments;

[0123] As an embodiment, the computer-readable storage medium of the present invention stores computer-executable instructions, and the computer-executable instructions are configured as follows:

[0124] Acquire a strain signal from a hand sensor, and perform EWT transformation on the strain signal according to a boundary adaptive backfill mechanism to obtain a target strain signal;

[0125] Decomposing the target strain signal according to the EEMD to obtain at least one IMF component, calculating a multi-scale nonlinear relationship influence factor of the at least one IMF component, and selecting at least one target IMF component whose multi-scale nonlinear relationship influence factor is greater than a preset threshold;

[0126] Performing feature extraction on the at least one target IMF component, combining the extracted time domain features into a time domain feature vector, and combining the extracted frequency domain features into a frequency domain feature vector, and screening the time domain feature vector and the frequency domain feature vector according to a preset feature scoring function to obtain a target time domain feature vector and a target frequency domain feature vector;

[0127] Normalizing the target time domain feature vector and the target frequency domain feature vector, and performing cross-weighted fusion on the normalized target time domain feature vector and the normalized target frequency domain feature vector to obtain a composite feature vector;

[0128] The composite feature vector is input into a preset improved CNN model, and the improved CNN model outputs a hand motion recognition result corresponding to the strain signal.

[0129] The computer-readable storage medium may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created based on the use of the hand motion recognition system, etc. In addition, the computer-readable storage medium may include a high-speed random access memory and may also include a memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state memory device. In some embodiments, the computer-readable storage medium may optionally include a memory remotely located relative to the processor, and these remote memories may be connected to the hand motion recognition system via a network. Examples of the aforementioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0130] Figure 3 Schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. Figure 3 As shown, the device includes: a processor 310 and a memory 320. The electronic device may also include: an input device 330 and an output device 340. The processor 310, the memory 320, the input device 330 and the output device 340 may be connected via a bus or other means. Figure 3The example of the connection via bus is taken. The memory 320 is the computer-readable storage medium mentioned above. The processor 310 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions and modules stored in the memory 320, that is, implements the hand motion recognition method of the above method embodiment. The input device 330 can receive input digital or character information, and generate key signal input related to user settings and function control of the hand motion recognition system. The output device 340 may include a display device such as a display screen.

[0131] The electronic device can execute the method provided by the embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not fully described in this embodiment, please refer to the method provided by the embodiment of the present invention.

[0132] As an embodiment, the electronic device is applied to a hand motion recognition system and is used as a client, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to:

[0133] Acquire a strain signal from a hand sensor, and perform EWT transformation on the strain signal according to a boundary adaptive backfill mechanism to obtain a target strain signal;

[0134] Decomposing the target strain signal according to the EEMD to obtain at least one IMF component, calculating a multi-scale nonlinear relationship influence factor of the at least one IMF component, and selecting at least one target IMF component whose multi-scale nonlinear relationship influence factor is greater than a preset threshold;

[0135] Performing feature extraction on the at least one target IMF component, combining the extracted time domain features into a time domain feature vector, and combining the extracted frequency domain features into a frequency domain feature vector, and screening the time domain feature vector and the frequency domain feature vector according to a preset feature scoring function to obtain a target time domain feature vector and a target frequency domain feature vector;

[0136] Normalizing the target time domain feature vector and the target frequency domain feature vector, and performing cross-weighted fusion on the normalized target time domain feature vector and the normalized target frequency domain feature vector to obtain a composite feature vector;

[0137] The composite feature vector is input into a preset improved CNN model, and the improved CNN model outputs a hand motion recognition result corresponding to the strain signal.

[0138] Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented using software plus a necessary general-purpose hardware platform, or of course, hardware. Based on this understanding, the essence of the above technical solution, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, or an optical disk, and includes a number of instructions for causing a computer device (such as a personal computer, server, or network device) to execute the methods of each embodiment or certain portions of the embodiments.

[0139] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A hand motion recognition method, characterized in that: include: The strain signal of the hand sensor is obtained, and the strain signal is subjected to EWT transformation according to the boundary adaptive backfill mechanism to obtain the target strain signal. The function expression of the boundary adaptive backfill mechanism is: ， Where, is the boundary weighting function, is the time point of the left boundary of the signal, For time, is the total time length of the signal, 、 are exponentials that control Gaussian attenuation. 、 Both control the rate at which the signal decays. 、 are factors that affect the polynomial. 、 、 、 The order of the polynomial, is the time point of the right boundary of the signal, Indicates the exponential operation; Decomposing the target strain signal according to the EEMD to obtain at least one IMF component, calculating a multi-scale nonlinear relationship influence factor of the at least one IMF component, and selecting at least one target IMF component whose multi-scale nonlinear relationship influence factor is greater than a preset threshold, wherein decomposing the target strain signal according to the EEMD to obtain at least one IMF component, calculating the multi-scale nonlinear relationship influence factor of the at least one IMF component, and selecting at least one target IMF component whose multi-scale nonlinear relationship influence factor is greater than the preset threshold includes: The target strain signal is decomposed according to EEMD to obtain at least one IMF component, which is expressed as: ， Where, For the The frequency band signal is decomposed by EEMD to obtain the The decomposition results, For the The first frequency band signal IMF components, For the The frequency band signal is decomposed by EEMD to obtain the The residual term, is the total number of IMFs for each frequency band signal; Calculate the multi-scale nonlinear relationship influence factor of the at least one IMF component, the expression is: ， ， ， ， Where, Indicates the The first frequency band signal The multi-scale nonlinear relationship impact factor of the IMF, express and The nonlinear mutual information between represents a nonlinear function, express and The weighting function of Indicates the The first frequency band signal The correlation coefficient of the IMF is Indicates the The first frequency band signal IMF correlation coefficients, express and Similarity in the frequency domain, represents the weighted time-domain correlation function, Indicates the The first frequency band signal IMF at the time The value at Indicates the The first frequency band signal IMF at the time The value at represents the similarity function of the spectrum, Indicates frequency, represents the polynomial coefficients, represents the order of the polynomial, Indicates from arrive The points, represents the frequency weighting factor, represents the exponent that controls the distance metric, 、 At different time points, For the The first frequency band signal IMF in frequency The value at For the The first frequency band signal IMF in frequency The value at is the total number of frequency bands, is the relevance weighting factor, is the total number of IMFs decomposed into a frequency band signal, is the correlation difference index; Selecting at least one target IMF component whose multi-scale nonlinear relationship impact factor is greater than a preset threshold; Performing feature extraction on the at least one target IMF component, combining the extracted time domain features into a time domain feature vector, and combining the extracted frequency domain features into a frequency domain feature vector, and screening the time domain feature vector and the frequency domain feature vector according to a preset feature scoring function to obtain a target time domain feature vector and a target frequency domain feature vector; Normalizing the target time domain feature vector and the target frequency domain feature vector, and performing cross-weighted fusion on the normalized target time domain feature vector and the normalized target frequency domain feature vector to obtain a composite feature vector; The composite feature vector is input into a preset improved CNN model, and the improved CNN model outputs a hand motion recognition result corresponding to the strain signal, wherein the improved CNN model includes: Convolutional layer including multi-dimensional impact factors, wherein the multi-dimensional impact factors include time impact factors and frequency domain impact factors, and the expression of the time impact factor is: ， Where, Indicates the The first frequency band signal IMF at a point in time The time impact factor, represents the time domain features in the composite feature vector, represents the time domain variation adjustment factor, represents the time domain smoothing over-adjustment factor, Represents the time domain signal at the time point The instantaneous rate of change, represents the time domain balance term, which prevents excessive response to extreme changes. Indicates the total step length of the time point; The expression of the frequency domain impact factor is: ， Where, Indicates the The first frequency band signal IMF in frequency The frequency domain impact factor of represents the frequency domain features in the composite feature vector, represents the frequency domain change adjustment factor, represents the frequency domain smoothing over-regulation factor, represents the instantaneous rate of change of the frequency component, represents the frequency domain balance term, is the total number of frequency bands; A nonlinear activation function connected to the convolutional layer, wherein the expression of the nonlinear activation function is: ， Where, They represent different activation functions. Respectively represent the adjustable parameters of the corresponding activation function, controlling the contribution of each activation function, Represents the improved nonlinear activation function, represents the input signal; The attention mechanism connected with the nonlinear activation function includes a time domain attention mechanism and a frequency domain attention mechanism. The expression of the time domain attention mechanism is: ， Where, Indicates the The first frequency band signal IMF at a point in time The temporal attention weight of represents the current time point, represents the total number of time steps, represents the number of time steps in the time window, Indicates the length of the time window, Indicates the first frequency band signal IMF at a point in time The time domain features after activation function processing, Indicates at a point in time In the nearby time window, the past The cumulative sum of the time domain features at each time point, represents the adjustment parameter of the time domain attention weight, Indicates the time domain adjustment item parameters; The expression of the frequency domain attention mechanism is: ， Where, Indicates the The first frequency band signal IMF at frequency point The frequency domain attention weight, represents the adjustment factor of frequency domain attention, Indicates the The first frequency band signal IMF at frequency point The frequency domain features after activation function processing, Indicates the frequency The cumulative sum of the frequency domain features of the nearby frequency points, represents the size of the sliding window, Represents an index used to select the current frequency point Nearby frequency points, Indicates the frequency domain adjustment parameter.

2. A hand motion recognition method according to claim 1, characterized in that: The step of extracting features from the at least one target IMF component, combining the extracted time domain features into a time domain feature vector, and combining the extracted frequency domain features into a frequency domain feature vector, and screening the time domain feature vector and the frequency domain feature vector according to a preset feature scoring function to obtain a target time domain feature vector and a target frequency domain feature vector includes: Feature extraction is performed on the at least one target IMF component to obtain a time domain feature, and the time domain features at multiple scales are combined into a time domain feature vector, which is expressed as: ， ， Where, is the vector that combines the time domain features extracted from the filtered IMF. 、 、 Respectively The first frequency band signal IMF's first, second, and third The dynamic volatility index, For the The first frequency band signal The IMF's The dynamic volatility index, is the time length of the IMF, To support the IMF The translation of the scale factor, Indicates the The first frequency band signal IMF at the time The value at is the nonlinear index, Indicates scaling of the scale factor. is a constant; Feature extraction is performed on the at least one target IMF component to obtain frequency domain features, and each frequency domain feature is combined with the weighted second-order moment of each frequency domain feature to obtain a frequency domain feature vector, which is expressed as: ， ， ， Where, For the The first frequency band signal The frequency domain eigenvector of an IMF, For the The first frequency band signal The frequency domain structure compression rate of an IMF is is the total number of frequency bands, is the frequency, For the The first frequency band signal IMF in frequency The spectrum power at For the The first frequency band signal The IMF The spectral power of the frequency, is the smoothing factor, Indicates the The first frequency band signal The IMF is Fourier transformed. Indicates the The first frequency band signal IMF at the time The value at is the Fourier transform; The first feature score of each time domain feature vector is calculated according to a preset feature scoring function, and the second feature score of each frequency domain feature vector is calculated, wherein the expression of the feature scoring function is: ， Where, For the The first frequency band signal The scoring function for the characteristics of an IMF, Features The discreteness of For the The first frequency band signal The time domain characteristics of an IMF, For the The first frequency band signal The frequency domain characteristics of an IMF, Features The average across all IMFs, is the median of all feature means, 、 Both are skew penalty control parameters; At least one target time domain feature vector having a first feature score greater than a first score threshold is selected, and at least one target frequency domain feature vector having a second feature score greater than a second score threshold is selected.

3. The hand motion recognition method according to claim 1, wherein: The target time domain feature vector and the target frequency domain feature vector are normalized, and the normalized target time domain feature vector and the normalized target frequency domain feature vector are cross-weightedly fused to obtain a composite feature vector. The target time domain feature vector and the target frequency domain feature vector are normalized, and the expression is: ， ， ， Where, After normalization, The first frequency band signal The time domain characteristics of an IMF, After normalization, The first frequency band signal The frequency domain characteristics of an IMF, is the nonlinear normalized mapping function, For the The first frequency band signal The time domain characteristics of an IMF, For the The first frequency band signal The frequency domain characteristics of an IMF, is the standard deviation of the feature set, is a feature set The mean of To adjust the parameters, is a constant; The normalized target time domain feature vector and the normalized target frequency domain feature vector are cross-weighted fused to obtain a composite feature vector, which is expressed as: ， ， ， ， Where, For the The first frequency band signal The composite eigenvector of IMFs, For the The first frequency band signal The weighting coefficients of the time domain and frequency domain characteristics of an IMF, For the The first frequency band signal The weighted coefficient of the time domain characteristics of an IMF, For the The first frequency band signal The weighting coefficient of the frequency domain characteristics of an IMF, for With the target variable The mutual information between is the target variable, for With the target variable The mutual information between them.

4. A hand motion recognition system, characterized in that: include: The acquisition module is configured to acquire the strain signal of the hand sensor and perform EWT transformation on the strain signal according to the boundary adaptive backfill mechanism to obtain a target strain signal, wherein the function expression of the boundary adaptive backfill mechanism is: ， Where, is the boundary weighting function, is the time point of the left boundary of the signal, For time, is the total time length of the signal, 、 are exponentials that control Gaussian attenuation. 、 Both control the rate at which the signal decays. 、 are factors that affect the polynomial. 、 、 、 The order of the polynomial, is the time point of the right boundary of the signal, Indicates the exponential operation; a calculation module configured to decompose the target strain signal according to the EEMD to obtain at least one IMF component, calculate a multi-scale nonlinear relationship influence factor of the at least one IMF component, and select at least one target IMF component whose multi-scale nonlinear relationship influence factor is greater than a preset threshold; The screening module is configured to perform feature extraction on the at least one target IMF component, combine the extracted time domain features into a time domain feature vector, and combine the extracted frequency domain features into a frequency domain feature vector, and screen the time domain feature vector and the frequency domain feature vector according to a preset feature scoring function to obtain a target time domain feature vector and a target frequency domain feature vector, wherein the decomposing the target strain signal according to the EEMD to obtain at least one IMF component, calculating a multi-scale nonlinear relationship influence factor of the at least one IMF component, and selecting at least one target IMF component whose multi-scale nonlinear relationship influence factor is greater than a preset threshold includes: The target strain signal is decomposed according to EEMD to obtain at least one IMF component, which is expressed as: ， Where, For the The frequency band signal is decomposed by EEMD to obtain the The decomposition results, For the The first frequency band signal IMF components, For the The frequency band signal is decomposed by EEMD to obtain the The residual term, is the total number of IMFs for each frequency band signal; Calculate the multi-scale nonlinear relationship influence factor of the at least one IMF component, the expression is: ， ， ， ， Where, Indicates the The first frequency band signal The multi-scale nonlinear relationship impact factor of the IMF, express and The nonlinear mutual information between represents a nonlinear function, express and The weighting function of Indicates the The first frequency band signal The correlation coefficient of the IMF is Indicates the The first frequency band signal IMF correlation coefficients, express and Similarity in the frequency domain, represents the weighted time-domain correlation function, Indicates the The first frequency band signal IMF at the time The value at Indicates the The first frequency band signal IMF at the time The value at represents the similarity function of the spectrum, Indicates frequency, represents the polynomial coefficients, represents the order of the polynomial, Indicates from arrive The points, represents the frequency weighting factor, represents the exponent that controls the distance metric, 、 At different time points, For the The first frequency band signal IMF in frequency The value at For the The first frequency band signal IMF in frequency The value at is the total number of frequency bands, is the relevance weighting factor, is the total number of IMFs decomposed into a frequency band signal, is the correlation difference index; Selecting at least one target IMF component whose multi-scale nonlinear relationship impact factor is greater than a preset threshold; a fusion module configured to normalize the target time domain feature vector and the target frequency domain feature vector, and perform cross-weighted fusion on the normalized target time domain feature vector and the normalized target frequency domain feature vector to obtain a composite feature vector; An output module is configured to input the composite feature vector into a preset improved CNN model, and the improved CNN model outputs a hand motion recognition result corresponding to the strain signal, wherein the improved CNN model includes: Convolutional layer including multi-dimensional impact factors, wherein the multi-dimensional impact factors include time impact factors and frequency domain impact factors, and the expression of the time impact factor is: ， Where, Indicates the The first frequency band signal IMF at a point in time The time impact factor, represents the time domain features in the composite feature vector, represents the time domain variation adjustment factor, represents the time domain smoothing over-adjustment factor, Represents the time domain signal at the time point The instantaneous rate of change, represents the time domain balance term, which prevents excessive response to extreme changes. Indicates the total step length of the time point; The expression of the frequency domain impact factor is: ， Where, Indicates the The first frequency band signal IMF in frequency The frequency domain impact factor of represents the frequency domain features in the composite feature vector, represents the frequency domain change adjustment factor, represents the frequency domain smoothing over-regulation factor, represents the instantaneous rate of change of the frequency component, represents the frequency domain balance term, is the total number of frequency bands; A nonlinear activation function connected to the convolutional layer, wherein the expression of the nonlinear activation function is: ， Where, They represent different activation functions. Respectively represent the adjustable parameters of the corresponding activation function, controlling the contribution of each activation function, Represents the improved nonlinear activation function, represents the input signal; The attention mechanism connected with the nonlinear activation function includes a time domain attention mechanism and a frequency domain attention mechanism. The expression of the time domain attention mechanism is: ， Where, Indicates the The first frequency band signal IMF at a point in time The temporal attention weight of represents the current time point, represents the total number of time steps, represents the number of time steps in the time window, Indicates the length of the time window, Indicates the first frequency band signal IMF at a point in time The time domain features after activation function processing, Indicates at a point in time In the nearby time window, the past The cumulative sum of the time domain features at each time point, represents the adjustment parameter of the time domain attention weight, Indicates the time domain adjustment item parameters; The expression of the frequency domain attention mechanism is: ， Where, Indicates the The first frequency band signal IMF at frequency point The frequency domain attention weight, represents the adjustment factor of frequency domain attention, Indicates the The first frequency band signal IMF at frequency point The frequency domain features after activation function processing, Indicates the frequency The cumulative sum of the frequency domain features of the nearby frequency points, represents the size of the sliding window, Represents an index used to select the current frequency point Nearby frequency points, Indicates the frequency domain adjustment parameter.

5. An electronic device, characterized in that: include: At least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 3.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 3 is implemented.

Images