Signal Enhancement and Gesture Recognition Method and System for Low-Cost Inertial Sensors

Abstract

The present invention relates to a signal enhancement and gesture recognition method and system for low-cost inertial sensors, belonging to the field of inertial sensors. By using the determinism index of recurrence plots, the randomness of the multi-resolution modal components obtained after empirical mode decomposition is evaluated, and at the same time, wavelet threshold denoising is used to effectively control the random errors in the modal components, so as to realize the signal enhancement of low-cost IMU data, greatly narrowing the gap between the signal quality of low-cost IMUs and that of high-cost IMUs; in order to fully exploit the correlation information of multi-source IMU signals that is not easily affected by time-frequency errors, for the first time, the time-frequency characteristics of IMU motion data are fused with the complex network topology characteristics reflecting the dynamic and structural information of gesture actions, improving the accuracy of gesture recognition and achieving high-precision recognition of various three-dimensional gesture actions.

Description

Technical Field

[0001] The present invention relates to the field of inertial sensors, and particularly to a method and system for signal enhancement and gesture recognition of a low-cost inertial sensor. Background Art

[0002] With the rise and development of Augmented Reality (AR), Virtual Reality (VR), and the metaverse, human-computer interaction through gesture recognition has become a hot spot and essential element in the intelligent terminal industry. Since the data acquisition process of inertial sensors is not affected by external environments such as light, occlusion, and noise, gesture recognition tasks based on inertial sensors have very broad application prospects. Among them, low-cost inertial sensors have the advantages of small size, low power consumption, and strong wearability, and can be widely used in various life scenarios. However, due to their low cost, their quality and performance vary, and the IMU (Inertial Measurement Unit) data collected often contains serious error interference, which in turn results in poor effects in tasks such as trajectory restoration, motion tracking, and gesture recognition. In addition, existing research on inertial sensor gesture recognition tasks often focuses on simple functions such as classifying specific gestures, and their recognition methods cannot adapt to different gesture movement habits, with poor scalability. Therefore, how to enhance the signal of low-cost inertial sensors and extract effective features from the limited information they contain to perform high-precision gesture recognition has become an urgent technical problem to be solved. Summary of the Invention

[0003] The purpose of the present invention is to provide a method and system for signal enhancement and gesture recognition of a low-cost inertial sensor, and perform high-precision gesture recognition on various three-dimensional gesture actions measured by the low-cost inertial sensor.

[0004] To achieve the above purpose, the present invention provides the following solutions:

[0005] A method for signal enhancement and gesture recognition of a low-cost inertial sensor, including:

[0006] Obtain six-axis IMU data of a three-dimensional gesture action measured by an inertial sensor; the price of the inertial sensor is lower than a price threshold;

[0007] Perform empirical mode decomposition on each-axis IMU data to obtain multiple IMF components of each-axis IMU data;

[0008] Convert each IMF component of each-axis IMU data into a recurrence plot, and calculate the determinism of each recurrence plot;

[0009] According to the determinacy of each recurrence plot of the IMU data per axis, the IMF components to be removed of random error are selected using the inflection point method;

[0010] Perform wavelet threshold denoising on the IMF components to be removed of random error, and merge the denoised IMF components in the IMU data per axis with the undenoised IMF components in the IMU data per axis, and then obtain the IMU data with enhanced signal per axis through the inverse transform of empirical mode decomposition;

[0011] Convert the IMU data with enhanced six-axis signal into a complex network, and extract the topological features of the complex network;

[0012] Adopt the tsfresh method to extract the time-frequency features of the IMU data with enhanced six-axis signal;

[0013] Input the time-frequency features and the topological features together into a random forest classifier, and output the three-dimensional gesture recognition result.

[0014] Optionally, the conversion of each IMF component of the IMU data per axis into a recurrence plot and the calculation of the determinacy of each recurrence plot specifically include:

[0015] Adopt the mutual information method and the equal-spacing lattice method, and calculate the time delay of each IMF component of the IMU data per axis by minimizing the mutual information;

[0016] Use the pseudo-nearest neighbor point improvement method to calculate the embedding dimension of each IMF component of the IMU data per axis;

[0017] Construct the regression matrix of the IMU data per axis according to the time delay and the embedding dimension of each IMF component of the IMU data per axis;

[0018] According to the regression matrix of the IMU data per axis, convert each IMF component of the IMU data per axis into a recurrence plot;

[0019] According to the recurrence plot, use the formula to calculate the determinacy of each recurrence plot of the IMU data per axis; in the formula, DET represents determinacy, P(l) represents the diagonal structure distribution probability with length equal to l, l min represents the minimum diagonal length, R i,j represents the regression matrix, and N represents the total number of time points.

[0020] Optionally, the IMF components to be removed of random error are the IMF components with determinacy less than the inflection point.

[0021] Optionally, the conversion of the IMU data with enhanced six-axis signal into a complex network specifically includes:

[0022] The mutual information method and the equal-spacing grid method are adopted to calculate the time delay of the IMU data with enhanced six-axis signals by minimizing the mutual information;

[0023] The false nearest neighbor improvement method is used to calculate the embedding dimension of the IMU data with enhanced six-axis signals;

[0024] Phase space reconstruction is performed on the six-axis IMU data according to the time delay and embedding dimension of the IMU data with enhanced six-axis signals;

[0025] The representation vectors of the IMU data for each axis are determined in the reconstructed phase space;

[0026] According to the representation vectors of the IMU data for each axis, the autoregressive matrix of the IMU data for each axis is calculated;

[0027] According to the representation vectors of the six-axis IMU data, the cross-regressive matrix of the six-axis IMU data is calculated;

[0028] The autoregressive matrix and the cross-regressive matrix of the six-axis IMU data are combined to obtain the regression matrix of the multivariate time series;

[0029] The IMU data with enhanced six-axis signals is converted into a complex network according to the regression matrix of the multivariate time series.

[0030] Optionally, the calculation formula for the autoregressive matrix of the IMU data for each axis is In the formula, is the autoregressive matrix of any axis of IMU data X A , Θ is the Heaviside function, and ε A is the threshold of any axis of IMU data X A , is the representation vector of the time series X A at time i; is the representation vector of the time series X A at time j;

[0031] The calculation formula for the cross-regressive matrix of the six-axis IMU data is In the formula, is the cross-regressive matrix of any axis of IMU data X A and another axis of IMU data X B , ε AB is the threshold when constructing a complex network for the time series A and the time series B, is the representation vector of the time series X B at time j.

[0032] Optionally, the self-recurrence rate of any-axis IMU data is higher than the cross-recurrence rate between any-axis IMU data and IMU data of other axes.

[0033] A signal enhancement and gesture recognition system for a low-cost inertial sensor, comprising:

[0034] A six-axis IMU data acquisition module, configured to acquire six-axis IMU data of an inertial sensor for measuring three-dimensional gesture actions;

[0035] An empirical mode decomposition module, configured to perform empirical mode decomposition on each-axis IMU data to obtain multiple IMF components of each-axis IMU data;

[0036] A recurrence plot conversion module, configured to convert each IMF component of each-axis IMU data into a recurrence plot and calculate the determinism of each recurrence plot;

[0037] An IMF component selection module, configured to select, according to the determinism of each recurrence plot of each-axis IMU data, the IMF components to be removed of random error by using the inflection point method;

[0038] A wavelet threshold denoising module, configured to perform wavelet threshold denoising on the IMF components to be removed of random error, and merge the denoised IMF components in each-axis IMU data with the undenoised IMF components in each-axis IMU data, and then obtain the IMU data with enhanced signal of each axis through inverse empirical mode decomposition;

[0039] A complex network conversion module, configured to convert the six-axis IMU data with enhanced signal into a complex network and extract topological features of the complex network;

[0040] A time-frequency feature extraction module, configured to extract time-frequency features of the six-axis IMU data with enhanced signal by using the tsfresh method;

[0041] An identification result output module, configured to input the time-frequency features and the topological features into a random forest classifier together and output a three-dimensional gesture recognition result.

[0042] An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein when the processor executes the computer program, the signal enhancement and gesture recognition method of the low-cost inertial sensor as described above is implemented.

[0043] A computer-readable storage medium, on which a computer program is stored, and when the computer program is executed, the signal enhancement and gesture recognition method of the low-cost inertial sensor as described above is implemented.

[0044] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0045] The present invention discloses a method and system for signal enhancement and gesture recognition of a low-cost inertial sensor. The inertial sensor for measuring six-axis IMU data of three-dimensional gesture actions is a low-cost inertial sensor. By using the determinism index of recurrence plots, the randomness of the multi-resolution modal components obtained after empirical mode decomposition is evaluated, and at the same time, wavelet threshold denoising is used to effectively control the random errors in the modal components, thereby realizing the signal enhancement of low-cost IMU data and greatly narrowing the gap between the signal quality of low-cost IMUs and that of high-cost IMUs. To fully exploit the correlation information of multi-source IMU signals that is not easily affected by time-frequency errors, for the first time, the time-frequency characteristics of IMU motion data are fused with the complex network topological characteristics reflecting the structural information and dynamic information of gesture actions, improving the accuracy of gesture recognition and realizing the high-precision recognition of various three-dimensional gesture actions. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0047] Figure 1 It is a flowchart of a method for signal enhancement and gesture recognition of a low-cost inertial sensor provided by an embodiment of the present invention;

[0048] Figure 2 It is a schematic diagram of the principle of a three-stage signal enhancement strategy provided by an embodiment of the present invention;

[0049] Figure 3 It is a schematic diagram of the empirical mode decomposition of the x-axis component of the acceleration signal of an inertial sensor provided by an embodiment of the present invention;

[0050] Figure 4 It is a schematic diagram of converting a time series into a recurrence plot provided by an embodiment of the present invention; Figure 4 In (a) is a schematic diagram of converting a random time series into a recurrence plot, Figure 4 In (b) is a schematic diagram of converting a periodic time series into a recurrence plot;

[0051] Figure 5 It is a schematic diagram of converting an IMF component into a recurrence plot provided by an embodiment of the present invention; Figure 5 In (a) is a schematic diagram of converting IMF1 into a recurrence plot, Figure 5 In (b) is a schematic diagram of converting IMF2 into a recurrence plot, Figure 5 In (c) is a schematic diagram of converting IMF3 into a recurrence plot, Figure 5In (d), it is a schematic diagram of converting IMF4 into a recurrence plot. Figure 5 In (e), it is a schematic diagram of converting IMF5 into a recurrence plot.

[0052] Figure 6 It is a schematic diagram of the changing trend of the deterministic index provided by the embodiment of the present invention.

[0053] Figure 7 It is a comparison diagram of the trajectory restoration effects of different-cost IMUs before and after signal enhancement provided by the embodiment of the present invention.

[0054] Figure 8 It is a flow chart of a low-cost inertial sensor gesture recognition model provided by the embodiment of the present invention.

[0055] Figure 9 It is a schematic diagram of determining the time delay by the mutual information method provided by the embodiment of the present invention.

[0056] Figure 10 It is a schematic diagram of determining the embedding dimension by the pseudo-nearest neighbor point improvement method provided by the embodiment of the present invention. Detailed implementation manners

[0057] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0058] The purpose of the present invention is to provide a signal enhancement and gesture recognition method and system for a low-cost inertial sensor, and to perform high-precision gesture recognition on various three-dimensional gesture actions measured by the low-cost inertial sensor.

[0059] To make the above objects, features, and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific implementation manners.

[0060] As an IMU signal is a time series data, most of the existing time series processing techniques process the signal in the time domain or frequency domain, such as empirical mode decomposition, fast Fourier transform, wavelet transform and other methods. However, inertial sensors have low cost and poor performance, and their time-frequency characteristics are greatly interfered by random errors. Therefore, traditional time-frequency processing techniques often cannot achieve good results. However, the topological characteristics of complex networks have the advantage of being not easily interfered by random errors in the time-frequency domain, and the relevant statistical indicators of complex networks can well evaluate the randomness of the original IMU data, and then reflect the magnitude of the random errors contained in each modal component. Therefore, combining complex network methods to remove random errors in low-cost IMUs to achieve signal enhancement, and extracting topological characteristics from the enhanced IMU signals through complex networks to improve the accuracy of gesture recognition will be a very valuable solution in the field of low-cost inertial sensors.

[0061] In order to remove the random errors of low-cost IMUs, the present invention designs a three-stage signal enhancement strategy of "empirical mode decomposition - complex network - wavelet transform", which can well measure the magnitude of random errors and remove random errors to achieve signal enhancement.

[0062] In order to improve the accuracy of the gesture recognition model, the present invention converts the time series into a complex network to extract complex network features reflecting the dynamic properties of the motion process, and combines them with time-frequency features to achieve high-precision recognition of 62 three-dimensional gesture actions. The 62 three-dimensional gesture actions include the numbers 0 to 9, lowercase letters a to z, and uppercase letters A to Z.

[0063] The implementation process of the present invention can be divided into two major parts:

[0064] I. The three-stage signal enhancement strategy of "empirical mode decomposition - complex network - wavelet transform";

[0065] II. The high-precision gesture recognition model combining complex network topological characteristics;

[0066] A signal enhancement and gesture recognition method for a low-cost inertial sensor provided by an embodiment of the present invention, as Figure 1 described, includes:

[0067] The first part: The three-stage signal enhancement strategy of "empirical mode decomposition - complex network - wavelet transform"

[0068] The first part includes steps S1 to S5.

[0069] Figure 2Schematic diagram of the three-stage signal enhancement strategy shown. The three-stage signal enhancement strategy realizes signal enhancement and reconstruction of inertial sensor signals by means of empirical mode decomposition, wavelet transform, and conversion of time series into recurrence plots. This reconstruction process can be divided into three steps: empirical mode decomposition of six-axis IMU data, conversion of time series into recurrence plots to determine IMU random errors, and wavelet threshold denoising of high-frequency mode components. The specific process is as follows:

[0070] Step S1: Obtain six-axis IMU data of an inertial sensor measuring three-dimensional gesture actions.

[0071] The six-axis IMU data consists of 3-axis acceleration data and 3-axis angular velocity data.

[0072] The price of the inertial sensor is lower than the price threshold, that is, the inertial sensor used in the present invention is a low-cost inertial sensor. The models and prices of low-cost inertial sensors on the market are shown in Table 1.

[0073] Table 1 Low-cost inertial sensors

[0074]

[0075] Step S2: Perform empirical mode decomposition on each axis of IMU data to obtain multiple IMF components of each axis of IMU data.

[0076] The low-cost IMU data is a signal with strong non-stationarity and large random errors, and empirical mode decomposition can decompose the IMU data into multiple intrinsic mode functions representing its internal characteristics based on local features. Therefore, the present invention first uses the method of empirical mode decomposition to decompose the original IMU signal into a series of intrinsic mode functions to achieve multi-scale and refined analysis of the data, providing a basis for subsequent signal enhancement processing. Here, taking the acceleration x(t) in the x-axis direction of the IMU signal as an example, the specific process of its empirical mode decomposition is as follows:

[0077] (1) Determine the local maximum and minimum points of the IMU signal x(t), and obtain the upper envelope x max (t) and the lower envelope x min (t) of the IMU signal by interpolation fitting

[0078] (2) Calculate the mean m1(t) of the upper and lower envelopes x max (t) and x min (t) of the IMU time series:

[0079]

[0080] (3) Subtract the mean m1(t) from the original IMU time series x(t) to obtain the processed IMU signal h1(t)

[0081] h1(t) = x(t) - m1(t)

[0082] (4) Check whether h1(t) satisfies the conditions to become an Intrinsic Mode Function (IMF). If not, take h1(t) as the original time series and repeat the above process until it meets the conditions. If it meets the conditions, regard h1(t) as a new IMF component, and calculate the residual r1(t) by subtracting this IMF component from the original IMU time series, that is:

[0083] r1(t) = x(t) - h1(t)

[0084] (5) Then repeat the above process with the residual r1(t) as the original time series until the residual is only a trend term (the number of extreme points is less than two), and output all N IMF components and the final residual Res(t).

[0085] Through the above steps, the original IMU signal can be decomposed into:

[0086]

[0087] Among them, each IMF component needs to meet two conditions: (1) The difference between the number of extreme points and the number of zero-crossing points in the IMF component should not exceed 1; (2) At any point, the mean value of the upper and lower envelope lines formed by the maximum and minimum values must be zero.

[0088] The empirical mode decomposition effect on the x-axis component of the inertial sensor acceleration signal is as Figure 3 shown.

[0089] Step S3: Convert each IMF component of the IMU data for each axis into a recurrence plot and calculate the determinism of each recurrence plot.

[0090] After the empirical mode decomposition of the low-cost IMU data, since the inertial sensor contains a large amount of random error during the data acquisition process, the present invention needs to determine the magnitude of the random error in each IMF component for refined processing. Existing solutions often set threshold functions for each IMF component, construct sample entropy, etc. to achieve the purpose of reducing random error. However, such methods often cause a large amount of loss of effective information in each IMF component, which is not conducive to further downstream tasks such as trajectory reconstruction and gesture recognition, and has obvious limitations.

[0091] The present invention converts each IMF component of the IMU into a complex network, and uses the characteristics of the complex network topology that is not easily disturbed by random errors in the time and frequency domains. By statistically analyzing the deterministic indicators of the complex network, the randomness of the original data is evaluated, and then the size of the random errors contained in each IMF is measured, and each random error is removed in a targeted manner to avoid a large loss of effective information. The complex network reflects the most essential dynamic properties of the motion process, and the characterization of the dynamic characteristics of the inertial sensor is achieved through the complex network.

[0092] Recursion graph is a special form of complex network representation. By observing the distribution of recursion points in the recursion graph, we can study the randomness and certainty hidden in the complex system. Among them, certainty is a common indicator in the quantitative analysis of recursion graph, which is defined as:

[0093]

[0094] Among them, P(l) represents the distribution probability of a diagonal structure with a length equal to l, l min Represents the minimum diagonal length, so the deterministic value range is between 0 and 1, R i,j represents the regression matrix, and N represents the total number of time points.

[0095] like Figure 4 As shown in (a) of FIG, the IMU sequence with high randomness is converted into a recursive graph, and it can be found that the distribution of the recursive points is disorderly and presents obvious random characteristics. At this time, according to the above formula, the present invention can calculate the DET value of the IMU sequence with more random errors to be: 0.08. Figure 4 As shown in (b) of Figure 1, the sinusoidal signal is converted into a recursive graph, and it can be found that the distribution of the recursive points in the graph has an obvious regularity: the recursive points form a line segment parallel to the main diagonal. Therefore, the recursive graph converted from the sinusoidal signal is more deterministic and less random, and its DET value is 0.97.

[0096] It can be found that by calculating the certainty of the recursive graph, the present invention can well measure the size of the random error in each IMF component. Therefore, the present invention converts the IMF components obtained by decomposing the original IMU into a recursive graph, and then selects the IMF components containing large random errors for further signal enhancement processing, wherein the specific selection process of the IMF containing large random errors is as follows:

[0097] (1) The mutual information method and the equally spaced grid method are used to calculate the time delay of each IMF component of each axis IMU data by minimizing the mutual information;

[0098] (2) Using the pseudo-nearest neighbor point improvement method to calculate the embedding dimension of each IMF component of each axis IMU data;

[0099] (3) Construct the regression matrix of the IMU data for each axis based on the time delay and embedding dimension of each IMF component of the IMU data for each axis, as shown in (a) to Figure 5 (e) in Figure 5 Convert each IMF component into a recurrence plot, as shown in (e) in

[0100] (4) Calculate the determinism of each recurrence plot.

[0101] In step S4, according to the determinism of each recurrence plot of the IMU data for each axis, use the inflection point method to select the IMF components to be removed of random error.

[0102] Figure 6 As shown, the inflection point is 0.95923. The IMF components with determinism less than 0.95923 have low determinism and high randomness. The IMF components with determinism greater than 0.95923 have high determinism and low randomness. Therefore, select the IMF components with higher randomness for removing random error, that is, select the IMF components with determinism less than the inflection point for further signal enhancement processing.

[0103] In step S5, perform wavelet threshold denoising on the IMF components to be removed of random error, and merge the denoised IMF components in the IMU data for each axis with the undenoised IMF components in the IMU data for each axis, and then obtain the IMU data with enhanced signal for each axis through inverse empirical mode decomposition.

[0104] The present invention utilizes the characteristic that the difference in wavelet decomposition coefficients of the semantic information of the IMU data and random error at each scale is relatively large, and performs threshold quantization processing on the wavelet coefficients to ensure that the semantic information in the signal is better preserved. Among them, the low-frequency coefficients usually have larger amplitudes, fewer numbers, and contain more semantic information, while the high-frequency coefficients have smaller amplitudes, more numbers, and usually contain a large amount of random error. The wavelet threshold denoising mainly includes the following three steps:

[0105] (1) Select a suitable wavelet basis and decomposition level according to the characteristics of the IMU data, and perform wavelet decomposition on the IMF with larger random error.

[0106] (2) Select a suitable threshold and threshold function, and perform threshold quantization processing on the high-frequency coefficients at each scale without changing the low-frequency coefficients to remove the random error in each IMF.

[0107] (3) Perform wavelet inverse transform on the processed high-frequency wavelet coefficients and low-frequency wavelet coefficients to obtain the signal after wavelet processing.

[0108] Through wavelet threshold denoising, the high-frequency IMF components containing large random errors can be refined. While reducing random errors, a large amount of semantic information loss in the IMF components is avoided. The high-frequency IMF components after wavelet processing are merged with the low-frequency IMF components that do not need to be processed, and the enhanced IMU data is obtained through the inverse empirical mode decomposition transformation.

[0109] In summary, aiming at the problem that low-cost IMU data contains a large amount of random errors, the present invention proposes a three-stage signal enhancement strategy of "empirical mode decomposition - complex network - wavelet transform". This strategy uses the determinism index of the complex network to evaluate the randomness of the IMF components obtained by the empirical mode decomposition of low-cost IMU data, and then uses wavelet transform to effectively control random errors to improve the data quality of low-cost inertial sensors and realize signal enhancement of low-cost IMUs. The complete calculation process of the three-stage signal enhancement strategy is shown in Algorithm 1.

[0110] Algorithm 1 Three-stage signal enhancement strategy:

[0111] Input: Original sensor data

[0112] Output: Sensor data after signal enhancement

[0113] 1. Perform empirical mode decomposition on the original IMU data

[0114] 2. Convert each decomposed IMF component into a recurrence plot

[0115] 2.1 Calculate the time delay of each IMF component

[0116] 2.2 Calculate the embedding dimension of each IMF component

[0117] 2.3 Construct an adjacency matrix according to the time delay and embedding dimension, and convert each IMF component into a recurrence plot

[0118] 3. Calculate the determinism of each recurrence plot

[0119] 4. Perform wavelet threshold denoising on the IMF components with low determinism and strong randomness

[0120] 5. Combine the IMF components after wavelet threshold denoising with the IMF components that do not need to be processed

[0121] 6. Empirical mode reconstruction, and output the sensor data after signal enhancement

[0122] The "Empirical Mode Decomposition - Complex Network - Wavelet Transform" three-stage signal enhancement strategy proposed in the present invention effectively guarantees that the semantic information in each high-frequency component is not lost while removing a large amount of random errors in low-cost IMUs, providing a basis and guarantee for realizing a high-precision gesture recognition model. At the same time, for the IMU signals processed by the present invention, their random errors are significantly weakened and the data quality is significantly enhanced, which helps to carry out downstream tasks such as trajectory restoration, gesture recognition, and human-computer interaction.

[0123] To verify the effectiveness of the signal enhancement strategy, as Figure 7 shown, the present invention specifically compares the trajectory restoration effects of low-cost IMUs and industrial-grade high-cost IMUs before and after signal enhancement, as well as the changes in the IMF components of inertial sensors with different costs before and after wavelet processing. It can be clearly found that the determinacy of the IMF after wavelet processing is improved, the randomness is weakened, and the random error is suppressed. At the same time, the trajectory restoration effect of the low-cost IMU after signal enhancement is similar to that of the high-cost IMU, further demonstrating that the three-stage signal enhancement strategy has great advantages in processing low-cost IMU signals.

[0124] Part Two: High-Precision Gesture Recognition Model Combining Complex Network Topological Features

[0125] Part Two includes steps S6 to S8.

[0126] Traditional gesture recognition models mainly extract features from IMU data based on time domain and frequency domain. However, due to the characteristics of strong nonlinearity and large random errors in low-cost IMU data, when extracting features in the time domain and frequency domain, it is extremely vulnerable to outliers and random errors. Compared with time-frequency domain features, the statistical indicators of complex networks are based on the topological structure features of the network, are less affected by the above factors, and can reflect the internal laws hidden in complex systems from a new perspective. Therefore, the present invention first converts the multi-dimensional time series after signal enhancement into a complex network to extract the internal structure information in IMU data, combines the complex network structure features with time-frequency features, and uses a random forest model for classification to achieve high-precision gesture recognition of low-cost inertial sensors. The specific implementation process is as Figure 8 shown.

[0127] Step S6: Convert the six-axis signal-enhanced IMU data into a complex network and extract the topological features of the complex network.

[0128] IMU data consists of 3-axis acceleration data and 3-axis angular velocity data, that is, IMU data is multivariate time series data. Most of the existing schemes for converting time series into complex networks convert six-axis IMU data into six independent complex networks. However, such conversion methods often result in the loss of the correlation between multi-axis IMU data, which is not conducive to extracting the topological features of IMU data. Therefore, this specification adopts the method of converting multivariate IMU data into a single complex network to ensure that the structured information of IMU data is not lost. Multivariate IMU data X A ,X B …X F The specific process of converting into a single complex network is as follows:

[0129] (1) As Figure 9 shown, according to the mutual information method proposed by Fraser and Swinney and combined with the equal-spacing grid method proposed by Yang Zhi'an et al., the time delay τ is determined by minimizing the mutual information.

[0130] The mutual information method is used to obtain Figure 9 the framework, and the equal-spacing grid method is used to solve Figure 9 each time point in the abscissa. After obtaining Figure 9 using the mutual information method and the equal-spacing grid method, the time delay τ = 30 is determined by minimizing the mutual information.

[0131] (2) As Figure 10 shown, according to the multivariate time series, the phase space is unfolded dimension by dimension using the pseudo-nearest neighbor point improvement method until the number of pseudo-nearest neighbor points no longer increases, and the embedding dimension d is determined.

[0132] (3) The phase space reconstruction of the original time series is completed according to the time delay and the embedding dimension.

[0133] Among them, after the complete phase space reconstruction, any axis data x(t) (t = 1, 2.., N) of the IMU data can be represented as the following vector:

[0134]

[0135] Among them, t represents the time stamp, d represents the embedding dimension, and N represents the total time length.

[0136] (4) For the vectors from the IMU data of the same axis (taking X A as an example), the autoregressive matrix is calculated according to the following formula

[0137]

[0138] where, ε Ais the threshold value, Θ is the Heaviside function, is the time series X A is the representation vector corresponding to the time point j, is the time series X A is the representation vector corresponding to the time point j.

[0139] (5) For vectors from different time series (taking X A , X B as an example), calculate the cross-regression matrix according to the following formula

[0140]

[0141] In the formula, is the cross-regression matrix of the IMU data X A on any axis and the IMU data X B on another axis, ε AB is the threshold value when constructing a complex network for time series A and time series B, is the time series X B is the representation vector corresponding to the time point j.

[0142] (6) Combine the autoregressive matrix and the cross-regression matrix to obtain the regression matrix of the multivariate time series:

[0143]

[0144] In the formula, R X represents the regression matrix, represents the autoregressive matrix of time series A, represents the cross-regression matrix of time series A and time series B, represents the cross-regression matrix of time series A and time series F, represents the cross-regression matrix of time series B and time series A, represents the cross-regression matrix of time series F and time series A, represents the autoregressive matrix of time series F.

[0145] It should be noted that in order to ensure stronger correlation between two vectors from the same-axis IMU sequences, it is necessary to ensure that the self-recursion rate α auto between the same IMU sequences is higher than the cross-recursion rate α cross between the IMU sequences of other axes, that is:

[0146]

[0147] Among them, N A represents the total number of time points contained in time series A, NB represents the total number of time points contained in time series B.

[0148] The elements of the regression matrix are 0 or 1. An element of 1 indicates that two vectors are connected in the recurrence plot, and an element of 0 indicates that two vectors are not connected in the recurrence plot. Therefore, a complex network can be directly constructed according to the regression matrix.

[0149] Through the above method, the conversion of multi-source IMU data to a complex network can be realized, and then the topological features in the complex network can be extracted. In addition, to extract the time-frequency features of IMU data, the present invention uses the tsfresh method to process the data after signal enhancement to achieve multi-angle and multi-dimensional feature extraction and analysis of IMU data. Finally, the present invention combines the extracted time-frequency features and topological features and inputs them into a relatively interpretable random forest model to achieve high-precision gesture recognition.

[0150] Step S7: Extract the time-frequency features of the six-axis signal-enhanced IMU data by using the tsfresh method.

[0151] Step S8: Input the time-frequency features and the topological features into a random forest classifier to output the three-dimensional gesture recognition result.

[0152] Meanwhile, the present invention selects multiple deep learning models for gesture action recognition and compares them with the method proposed in this paper. It can be found that when the selected features are appropriate and sufficient, the model proposed in this paper not only has stronger interpretability, but also the recognition accuracy of the present invention (92.41%) can exceed the recognition accuracy of the deep learning model (91.42%). This result further reflects the important value of complex network topological features in feature engineering in the fields of motion state recognition and time series analysis, providing a new reference for pattern recognition.

[0153] In summary, the process of the high-precision gesture recognition model that fully combines complex network topological features is as follows:

[0154] (1) Convert the six-axis IMU signal after signal enhancement into a complex network

[0155] (2) Extract topological features such as clustering coefficient, average path length, and node degree in the complex network

[0156] (3) Use methods such as tsfresh to extract the time-frequency features of the six-axis IMU signal

[0157] (4) Combine the time-frequency features and topological features of the IMU signal

[0158] (5) Construct a random forest classifier for gesture recognition

[0159] The core of the present invention lies in taking advantage of the fact that the topological features of complex networks are not easily interfered by random errors in the time-frequency domain to enhance the signal of low-cost IMU data and thus achieve high-precision gesture recognition. In order to enhance the signal of low-cost IMU data, a three-stage signal enhancement strategy of "empirical mode decomposition-complex network-wavelet transform" is proposed. In order to recognize 62 kinds of three-dimensional gesture actions of low-cost IMUs, based on the data with enhanced signal, the IMU data is further processed. The present invention not only extracts the time-frequency features of IMU data, but also converts the IMU time series into a complex network to extract topological features reflecting the internal laws of complex systems from a new perspective, and uses a random forest model for classification. Through the above method, the present invention realizes high-precision gesture recognition of low-cost inertial sensors under the conditions of different user movement habits, various IMU models and low costs.

[0160] The main innovation points of the technology of the present invention are as follows:

[0161] 1. A three-stage signal enhancement strategy of "empirical mode decomposition-complex network-wavelet transform" is proposed. This strategy uses the "deterministic" index of complex networks to evaluate the randomness of the multi-resolution modal components obtained after empirical mode decomposition, and at the same time uses wavelet transform to effectively control the random errors in the modal components, so as to realize the signal enhancement of low-cost IMU data. This strategy realizes the evaluation of IMU random errors and has high application value in many tasks.

[0162] 2. In order to fully exploit the correlation information of multi-source IMU signals (accelerometer, gyroscope) that is not easily interfered by time-frequency errors, the present invention first fuses the time-frequency features of IMU motion data with the topological features of complex networks that reflect gesture action structure information and dynamic information. This feature selection method has important value in motion recognition and feature engineering, providing new references for pattern recognition.

[0163] 3. Considering comprehensively the different user movement habits and various low-cost IMU models, for the first time, high-precision recognition of 62 kinds of three-dimensional gesture actions is realized, improving the robustness of the low-cost IMU gesture recognition model and having wide application value.

[0164] The embodiment of the present invention also provides a signal enhancement and gesture recognition system for low-cost inertial sensors, including:

[0165] A six-axis IMU data acquisition module for acquiring six-axis IMU data of an inertial sensor measuring three-dimensional gesture actions;

[0166] An empirical mode decomposition module for performing empirical mode decomposition on each-axis IMU data to obtain multiple IMF components of each-axis IMU data;

[0167] A recursive graph conversion module, which is used to convert each IMF component of the IMU data per axis into a recursive graph and calculate the determinism of each recursive graph;

[0168] An IMF component selection module, which is used to select the IMF components to remove random errors by using the inflection point method according to the determinism of each recursive graph of the IMU data per axis;

[0169] A wavelet threshold denoising module, which is used to perform wavelet threshold denoising on the IMF components to remove random errors, and merge the denoised IMF components in the IMU data per axis with the undenoised IMF components in the IMU data per axis, and then obtain the IMU data with enhanced signal per axis through the inverse empirical mode decomposition;

[0170] A complex network conversion module, which is used to convert the IMU data with enhanced six-axis signal into a complex network and extract the topological features of the complex network;

[0171] A time-frequency feature extraction module, which is used to extract the time-frequency features of the IMU data with enhanced six-axis signal by using the tsfresh method;

[0172] An identification result output module, which is used to input the time-frequency features and the topological features into a random forest classifier together and output the three-dimensional gesture recognition result.

[0173] The signal enhancement and gesture recognition system of the low-cost inertial sensor provided by the embodiment of the present invention and the signal enhancement and gesture recognition method of the low-cost inertial sensor described in the above embodiment have similar working principles and beneficial effects, so they will not be elaborated here. For specific content, please refer to the introduction of the above method embodiment.

[0174] In addition, the embodiment of the present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the signal enhancement and gesture recognition method of the low-cost inertial sensor as described above.

[0175] When the computer program in the above-mentioned memory is implemented in the form of software functional units and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard disks, read-only memories, random access memories, magnetic disks, or optical discs.

[0176] Further, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed, it implements the signal enhancement and gesture recognition method of the low-cost inertial sensor as described above.

[0177] In the present specification, the various embodiments are described in a progressive manner. The key point of each embodiment is to illustrate the differences from other embodiments. For the same or similar parts among the various embodiments, reference can be made to each other. For the system disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and reference can be made to the description of the method part for the relevant parts.

[0178] In this article, specific examples are used to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present invention.

Claims

1. A signal enhancement and gesture recognition method for a low-cost inertial sensor, characterized in that, Including: Obtain six-axis IMU data of inertial sensors measuring three-dimensional gesture actions; the price of the inertial sensors is lower than the price threshold; Perform empirical mode decomposition on the IMU data of each axis to obtain multiple IMF components of the IMU data of each axis; Adopt the mutual information method and the equal-spacing lattice method to calculate the time delay of each IMF component of the IMU data of each axis by minimizing the mutual information; Use the pseudo-nearest neighbor point improvement method to calculate the embedding dimension of each IMF component of the IMU data of each axis; Construct a regression matrix of the IMU data of each axis according to the time delay and embedding dimension of each IMF component of the IMU data of each axis; Convert each IMF component of the IMU data of each axis into a recurrence plot according to the regression matrix of the IMU data of each axis; According to the recursive graph, using the formula calculate the determinacy of each recursive graph of the IMU data for each axis; in the formula, DET represents determinacy, P(l) represents the diagonal structure distribution probability with length equal to l, l min represents the minimum diagonal length, R i,j represents the regression matrix, N represents the total number of time points; the subscript i represents the moment i, and the subscript j represents the moment j; Select the IMF components to be removed of random error by using the inflection point method according to the determinism of each recurrence plot of the IMU data of each axis; Perform wavelet threshold denoising on the IMF components to be removed of random error, and merge the denoised IMF components in the IMU data of each axis with the undenoised IMF components in the IMU data of each axis, and then obtain the IMU data with enhanced signal of each axis through the inverse transform of empirical mode decomposition; Convert the six-axis IMU data with enhanced signal into a complex network, and extract the topological features of the complex network; The conversion of the six-axis IMU data with enhanced signal into a complex network specifically includes: adopting the mutual information method and the equal-spacing lattice method to calculate the time delay of the six-axis IMU data with enhanced signal by minimizing the mutual information; using the pseudo-nearest neighbor point improvement method to calculate the embedding dimension of the six-axis IMU data with enhanced signal; perform phase space reconstruction on the six-axis IMU data according to the time delay and embedding dimension of the six-axis IMU data with enhanced signal; determine the representation vector of the IMU data of each axis in the reconstructed phase space; calculate the autoregressive matrix of the IMU data of each axis according to the representation vector of the IMU data of each axis; calculate the cross-regressive matrix of the six-axis IMU data according to the representation vector of the six-axis IMU data; merge the autoregressive matrix and the cross-regressive matrix of the six-axis IMU data to obtain the regression matrix of the multivariate time series; convert the six-axis IMU data with enhanced signal into a complex network according to the regression matrix of the multivariate time series; Adopt the tsfresh method to extract the time-frequency features of the six-axis IMU data with enhanced signal; Input the time-frequency features and the topological features into a random forest classifier together, and output the three-dimensional gesture recognition result.

2. The signal enhancement and gesture recognition method for the low-cost inertial sensor according to claim 1, characterized in that The IMF components to be removed of random error are the IMF components with determinism less than the inflection point.

3. The signal enhancement and gesture recognition method for the low-cost inertial sensor according to claim 1, wherein The calculation formula for the autoregressive matrix of the IMU data per axis is In the formula, is the autoregressive matrix of the IMU data X A for any axis, Θ is the Heaviside function, and ε A is the threshold of the IMU data X A for any axis, is the representation vector corresponding to the time series X A at time i; is the representation vector corresponding to the time series X A at time j; The calculation formula for the cross-regression matrix of six-axis IMU data is In the formula, is the cross-regression matrix of the IMU data X of any axis A and the IMU data X of another axis B , ε AB is the threshold when constructing a complex network for time series A and time series B, is the representation vector corresponding to the time series X B at time j.

4. The signal enhancement and gesture recognition method for the low-cost inertial sensor according to claim 1, characterized in that, The self-recurrence rate of the IMU data of any axis is higher than the cross-recurrence rate between the IMU data of any axis and the IMU data of other axes.

5. A signal enhancement and gesture recognition system for a low-cost inertial sensor, characterized in that, The system is used to implement the signal enhancement and gesture recognition method of the low-cost inertial sensor as described in any one of claims 1-4; the system includes: A six-axis IMU data acquisition module for acquiring six-axis IMU data of inertial sensors measuring three-dimensional gesture actions; An empirical mode decomposition module for performing empirical mode decomposition on the IMU data of each axis to obtain multiple IMF components of the IMU data of each axis; A recursive graph conversion module for converting each IMF component of the IMU data per axis into a recursive graph and calculating the determinacy of each recursive graph; An IMF component selection module for selecting the IMF components to remove random errors using the inflection point method according to the determinacy of each recursive graph of the IMU data per axis; A wavelet threshold denoising module for performing wavelet threshold denoising on the IMF components to remove random errors, and merging the denoised IMF components in the IMU data per axis with the undenoised IMF components in the IMU data per axis, and obtaining the IMU data with enhanced signals per axis through inverse empirical mode decomposition; A complex network conversion module for converting the six-axis IMU data with enhanced signals into a complex network and extracting the topological features of the complex network; A time-frequency feature extraction module for extracting the time-frequency features of the six-axis IMU data with enhanced signals using the tsfresh method; An identification result output module for inputting the time-frequency features and the topological features into a random forest classifier together and outputting the three-dimensional gesture recognition result.

6. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it implements the signal enhancement and gesture recognition method of the low-cost inertial sensor according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, A computer program is stored thereon, and when the computer program is executed, it implements the signal enhancement and gesture recognition method of the low-cost inertial sensor according to any one of claims 1 to 4.

Images