Neural network gait recognition system and method based on array data acquisition strategy

Abstract

The invention discloses a neural network gait recognition system and method based on an array data acquisition strategy. The recognition system comprises a gait data acquisition module and a gait recognition module. The gait data acquisition module acquires gait data through 12 IMU (Inertial Measurement Unit) sensors in array layout; the gait recognition module carries out gait mode recognition through a parallel attention convolutional network based on coding-decoding; the parallel attention convolutional network based on coding-decoding comprises three parallel coding-decoding structures and a connection structure of multi-receptive field feature extraction and a fusion attention mechanism. The problem that a single sensor drifts in the installation and wearing process is solved through array sensing collection, and meanwhile the influence of the number difference of testers is made up. The accuracy and efficiency of gait recognition are improved through coding-decoding parallel attention convolutional network gait recognition.

Description

Technical Field

[0001] The present invention belongs to the field of gait recognition, and specifically relates to a neural network gait recognition system and a recognition method based on an array data acquisition strategy, which can solve the problem of sensor drift during installation and wear and improve the accuracy and efficiency of gait recognition. Background Art

[0002] Gait recognition technology plays an important role in many fields such as wearable exoskeleton robots, rehabilitation and elderly care, intelligent wearable systems, and modern human-computer interaction recognition. Gait signal acquisition is one of the important means to evaluate and monitor human motion posture, and is of great significance for the research of wearable exoskeleton robots, rehabilitation and elderly care, intelligent wearable systems, and modern human-computer interaction recognition. At present, gait signal acquisition technology mainly includes visual signal acquisition, electromyographic signal acquisition, and inertial signal acquisition. The acquisition method based on inertial signals can monitor human motion in real time and provide stable and continuous motion data without considering the influence of environmental factors such as light and occlusion. In addition, inertial sensors are easy to integrate into embedded gait recognition systems and are not restricted by external installation environments. However, single inertial sensors will drift and sensor failure during installation and wearing. Gait recognition methods mainly include sensor data-based methods, computer vision-based methods, machine learning-based methods, and deep learning-based methods. However, deep learning models have poor generalization ability for new gait data and low accuracy in multi-modal gait recognition.

[0003] Aiming at the drift and sensor failure of a single sensor during installation and wearing, the poor generalization ability of the deep learning model for new gait data, and the low accuracy of multi-modal gait recognition, the present invention designs a neural network gait recognition system and method based on array data acquisition strategy. In the gait recognition method, the IMU sensor adopts a cross-distributed five-point array layout and adopts a parallel attention convolutional network model based on encoding-decoding. Summary of the invention

[0004] The purpose of the present invention is to solve the problems of drift and sensor failure of a single sensor during installation and wearing, poor generalization ability of the deep learning model for new gait data, and low accuracy of multi-modal gait recognition. By adopting a cross-distributed five-point array layout method for the sensor, a parallel attention convolutional network model based on encoding and decoding is built, and a neural network gait recognition method based on array data acquisition strategy is designed.

[0005] The neural network gait recognition system based on array data acquisition strategy of the present invention comprises a gait data acquisition module and a gait recognition module; the gait data acquisition module comprises a plurality of IMU sensors that collect gait data through an array layout; the gait recognition module carries out gait pattern recognition through a parallel attention convolutional network based on encoding-decoding; the parallel attention convolutional network based on encoding-decoding comprises a parallel encoding-decoding structure, a multi-receptive field feature extraction and a connection structure of a fusion attention mechanism.

[0006] Furthermore, the IMU sensors are respectively attached to the waist, thigh, calf and ankle; the IMU sensors are attached to the leggings and protective gear in a multi-point array layout at the thigh and calf.

[0007] Furthermore, there are 12 IMU sensors, of which 1 is located at the waist, 5 are located at the thighs, 5 are located at the calves, and 1 is located at the ankle.

[0008] Furthermore, the IMU sensors located at the thigh and calf are arranged in a cross-distributed five-point array.

[0009] The present invention also discloses a neural network gait recognition method based on array data acquisition strategy, comprising the following steps:

[0010] s1. The gait data acquisition module collects the gait data of the subject through an array of IMU sensors, and each IMU sensor collects the pitch angle Pitch, yaw angle Yaw, and roll angle Roll of the corresponding part;

[0011] s2. Perform preprocessing operations such as normalization and data set segmentation on the collected raw gait data; the data range is mapped to the standard range of -1 to 1 through normalization processing to avoid the difference between data affecting the training and network of the model. The processing formula is as follows:

[0012]

[0013] In the formula, y i is the i-th sampling value of the sampling information; y max is the maximum value of the sampled data vector; y min is the minimum value of the sampled data vector; Y i is the normalized sampling information;

[0014] The data set is segmented using a sliding window segmentation method, with a sliding window size w of 25 and a step size of 1;

[0015] s3. Building a parallel attention convolutional network model based on encoding-decoding, inputting the data set into the parallel attention convolutional network model based on encoding-decoding for training, so that the trained neural network model recognizes the corresponding gait pattern according to the characteristics of the gait data;

[0016] s4. When performing gait recognition, the gait data of the subject is collected through the data preprocessing method in s2, and the data is input into the parallel attention convolutional network model based on encoding-decoding to obtain the gait recognition result.

[0017] Furthermore, when collecting gait data using an IMU sensor with a cross-distributed five-point array layout, if the IMU sensor drifts laterally and longitudinally, for the central sensor, the laterally symmetrical sensor performs lateral offset compensation on its data, and the longitudinally symmetrical sensor performs longitudinal data enhancement on its data, thereby improving the robustness of the collected data.

[0018] Furthermore, when a parallel encoding-decoding structure is used to extract features of the original data, the original image is simultaneously input into three encoding-decoding modules, where the encoder part includes a convolutional layer, batch normalization, an activation function, and a maximum pooling layer; the decoder part includes a deconvolutional layer, a convolutional layer, a batch normalization, and an activation function.

[0019] Furthermore, three encoding-decoding modules with different receptive field sizes are used to extract raw data features from different scales.

[0020] Furthermore, the connection structure of the fusion attention mechanism first flattens the data at the jump connection, then passes it to the attention module for weighting, and finally performs channel fusion of features with the corresponding decoder end data through full connection processing.

[0021] The beneficial effects of the present invention are:

[0022] 1. The cross-distributed array layout of the present invention can achieve compensation and enhancement effects on the collected data.

[0023] 2. The thigh and calf of the present invention 5 × The data feature combination of 5 can effectively suppress sensor drift, increase the amount of gait data collected, and enhance the generalization ability for different groups of people.

[0024] 3. The array sensor layout of the present invention improves the robustness of subsequent network hardware deployment by simulating sensor drift.

[0025] 4. The encoding-decoding based parallel attention convolutional network model of the present invention has strong generalization ability, and gait recognition is more stable and accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The technical solution of the present invention is further described below in conjunction with the accompanying drawings and embodiments:

[0027] Figure 1 Schematic diagram of a neural network gait recognition system based on array data acquisition strategy.

[0028] Figure 2 Schematic diagram of the IMU sensor array layout.

[0029] Figure 3 Figure 2 is the structure diagram of the encoder-decoder based parallel attention convolutional network.

[0030] Figure 4 This is the gait recognition result output by the neural network. DETAILED DESCRIPTION

[0031] Embodiment 1 A neural network gait recognition system based on array data acquisition strategy

[0032] This embodiment provides a neural network gait recognition system based on an array data acquisition strategy, including a gait data acquisition module and a gait recognition module; the gait data acquisition module includes a plurality of IMU sensors that collect gait data through an array layout; the gait recognition module recognizes gait patterns through a parallel attention convolutional network based on encoding-decoding; the parallel attention convolutional network based on encoding-decoding includes a parallel encoding-decoding structure, a multi-receptive field feature extraction, and a connection structure of a fusion attention mechanism.

[0033] In this embodiment, the IMU sensors are respectively attached to the waist, thigh, calf and ankle; the IMU sensors are attached to the leggings and protective gear in a multi-point array layout at the thigh and calf;

[0034] In this embodiment, Figure 1 As shown, there are 12 IMU sensors, which are attached to the waist (1), thighs (5), calves (5) and ankles (1). The angle values ​​of the roll angle, yaw angle and pitch angle at the waist, thighs, calves and ankles can be obtained as human gait feature data.

[0035] In this embodiment, the IMU sensors located at the thigh and calf adopt a cross-distributed five-point array layout to achieve compensation and enhancement effects on the data.

[0036] Embodiment 2 A neural network gait recognition method based on array data acquisition strategy

[0037] This embodiment provides a neural network gait recognition method based on array data acquisition strategy, comprising the following steps:

[0038] s1. The gait data acquisition module collects the gait data of the subject through the array-arranged IMU sensors. Each IMU sensor collects the pitch angle Pitch, yaw angle Yaw, and roll angle Roll of the corresponding part. Table 1 is part of the actual data collected by the IMU sensor in this embodiment.

[0039] Table 1 Some actual data collected by IMU sensor

[0040]

[0041]

[0042] s2. Perform preprocessing operations such as normalization and data segmentation on the collected raw gait data. The normalization process maps the data range to the standard range of -1 to 1 to avoid the difference between the data affecting the training and network of the model. The formula is as follows:

[0043]

[0044] In the formula, y i is the i-th sampling value of the sampling information; y max is the maximum value of the sampled data vector; y min is the minimum value of the sampled data vector; Y i is the normalized sampling information.

[0045] The data set is segmented using a sliding window segmentation method, the sliding window size w used is 25, and the step length step is 1. Table 2 shows part of the data set after preprocessing in this embodiment.

[0046] Table 2 Part of the data set after preprocessing

[0047]

[0048]

[0049] s3. Build a parallel attention convolutional network model based on encoding and decoding, input the data set into the parallel attention convolutional network model based on encoding and decoding for training, so that the trained neural network model can recognize the corresponding gait pattern according to the characteristics of the gait data.

[0050] S4. When performing gait recognition, the gait data of the subject is collected and processed by the data preprocessing method in S2, and the data is input into the parallel attention convolutional network model based on encoding-decoding to obtain the gait recognition result. Figure 4 Shown is the gait recognition result output by the neural network in this embodiment.

[0051] In this embodiment, when an IMU sensor with a cross-distributed five-point array layout is used to collect gait data, if the IMU sensor drifts laterally and longitudinally, for the central sensor, the laterally symmetrical sensor performs lateral offset compensation on its data, and the longitudinally symmetrical sensor performs longitudinal data enhancement on its data, thereby improving the robustness of the collected data.

[0052] In this embodiment, when a parallel encoding-decoding structure is used to extract raw data features, the raw image is simultaneously input into three encoding-decoding modules, wherein the encoder part includes a convolution layer, batch normalization, activation function, and a maximum pooling layer; and the decoder part includes a deconvolution layer, a convolution layer, batch normalization, and an activation function. The parallel encoding-decoding structure achieves more comprehensive and efficient feature extraction and learning.

[0053] In this embodiment, three encoding-decoding modules with different receptive field sizes are used to extract original data features from different scales to realize multi-receptive field feature extraction. Three encoding-decoding modules with different receptive field sizes extract original data features from different scales to obtain more comprehensive multi-scale information and improve the model's ability to understand input data.

[0054] In this embodiment, the connection structure of the fusion attention mechanism first flattens the jump connection data, then passes it to the attention module for weighting, and finally performs channel fusion of features with the corresponding decoder end data through full connection processing. This significantly improves the generalization ability and recognition accuracy in multi-modal gait recognition.

[0055] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solution of the present invention can be modified or replaced by equivalents without departing from the purpose and scope of the technical solution of the present invention, which should be included in the scope of the claims of the present invention.

Claims

1. A neural network gait recognition system based on array data acquisition strategy, characterized in that: It includes a gait data acquisition module and a gait recognition module; the gait data acquisition module includes a plurality of IMU sensors that collect gait data through an array layout; the gait recognition module recognizes gait patterns through a parallel attention convolutional network based on encoding-decoding; the parallel attention convolutional network based on encoding-decoding includes a parallel encoding-decoding structure, multi-receptive field feature extraction and a connection structure of a fusion attention mechanism.

2. The neural network gait recognition system based on array data acquisition strategy according to claim 1, characterized in that: The IMU sensors are respectively attached to the waist, thigh, calf and ankle; the IMU sensors are attached to the leggings and protective gear in a multi-point array layout at the thigh and calf.

3. The neural network gait recognition system based on array data acquisition strategy according to claim 2, characterized in that: There are 12 IMU sensors, 1 at the waist, 5 at the thighs, 5 at the calves, and 1 at the ankle.

4. The neural network gait recognition system based on array data acquisition strategy according to claim 3 is characterized in that: The IMU sensors located at the thigh and calf are arranged in a cross-distributed five-point array.

5. A method for gait recognition using the neural network gait recognition system based on array data acquisition strategy according to any one of claims 1 to 4, characterized in that: The following steps are involved: s1. The gait data acquisition module collects the gait data of the subject through an array of IMU sensors, and each IMU sensor collects the pitch angle Pitch, yaw angle Yaw, and roll angle Roll of the corresponding part; s2. Perform preprocessing operations such as normalization and data set segmentation on the collected raw gait data; the data range is mapped to the standard range of -1 to 1 through normalization processing to avoid the difference between data affecting the training and network of the model. The processing formula is as follows: In the formula, y i is the i-th sampling value of the sampling information; y max is the maximum value of the sampled data vector; y min is the minimum value of the sampled data vector; Y i is the normalized sampling information; The data set is segmented using a sliding window segmentation method, with a sliding window size w of 25 and a step size of 1; s3. Building a parallel attention convolutional network model based on encoding-decoding, inputting the data set into the parallel attention convolutional network model based on encoding-decoding for training, so that the trained neural network model recognizes the corresponding gait pattern according to the characteristics of the gait data; s4. When performing gait recognition, the gait data of the subject is collected through the data preprocessing method in s2, and the data is input into the parallel attention convolutional network model based on encoding-decoding to obtain the gait recognition result.

6. The method for gait recognition according to claim 5, characterized in that: When using IMU sensors with a cross-distributed five-point array layout to collect gait data, if the IMU sensor drifts laterally and longitudinally, for the central sensor, the laterally symmetrical sensors will perform lateral offset compensation on its data, and the longitudinally symmetrical sensors will perform longitudinal data enhancement on its data, thereby improving the robustness of the collected data.

7. The method for gait recognition according to claim 5, characterized in that: When using a parallel encoding-decoding structure to extract raw data features, the original image is simultaneously input into three encoding-decoding modules, where the encoder part includes a convolutional layer, batch normalization, an activation function, and a maximum pooling layer; the decoder part includes a deconvolutional layer, a convolutional layer, a batch normalization, and an activation function.

8. The method for gait recognition according to claim 7, characterized in that: Three encoding-decoding modules with different receptive field sizes are used to extract raw data features from different scales.

9. The method for gait recognition according to claim 5, characterized in that: The connection structure of the fusion attention mechanism first flattens the data at the jump connection, then passes it to the attention module for weighting, and finally performs channel fusion of features with the corresponding decoder-side data through full connection processing.

Images