Ankle joint torque prediction method and electronic equipment oriented to complex movement of human body

Abstract

The invention belongs to the field of algorithms, and particularly relates to an ankle joint torque prediction method for human body complex movement and electronic equipment. The method comprises the following steps: acquiring IMU signals at two positions of a shank and a tiptoe of a single-side lower limb, and corresponding ankle joint torque data; carrying out data segmentation by adopting a sliding time window to obtain training data in various motion modes; training to obtain a pre-training model in each motion mode; and receiving the motion mode set by the user and IMU data of two positions of the shank and the tip of the single-side lower limb, and inputting the received IMU data into the pre-training model in the corresponding motion mode to obtain prediction data of the corresponding ankle joint torque. According to the method, the same deep learning regression model architecture is used, and a group of optimal network parameters are trained for each motion mode, so that the network is more focused on a data fitting task of the current motion mode, the joint torque prediction is more accurate, and the method has more advantages in the aspects of network parameters and calculation real-time performance.

Description

Technical Field

[0001] The invention belongs to the field of algorithms, and in particular relates to an ankle joint torque prediction method and electronic equipment for complex human body movements. Background Art

[0002] Using an inertial measurement unit (IMU) to estimate the ankle torque of the lower limb joint can provide valuable information for exoskeleton control. Existing methods mainly calculate joint torque by combining wearable sensors with musculoskeletal models. Existing methods based on skeletal muscle models have the following main disadvantages: (1) They require multiple types of sensors, such as surface electromyography sensors, IMUs, and foot pressure sensors; (2) They require a complex sensor calibration process before collecting data; (3) Surface electromyography signals have individual differences and are very sensitive to skin impedance and the location of attached muscles; (4) They are limited by the modeling accuracy of the musculoskeletal model and the accuracy of the solution is not high. Summary of the invention

[0003] In view of the above problems, the present invention proposes a technical solution:

[0004] A method for predicting ankle joint torque for complex human motions comprises the following steps:

[0005] S1: Collect IMU signals of the calf and leg tip of the human body under different motion modes, as well as the corresponding ankle joint torque data;

[0006] S2: Use a sliding time window to segment the IMU signal and the corresponding ankle joint torque data to obtain training data under various motion modes;

[0007] S3: input the training data of various motion modes into respective TCN-BiGRU models for training, and obtain pre-trained models of various motion modes. The number of pre-trained models obtained is equal to the number of motion modes.

[0008] The motion mode set by the user and the IMU data of the calf and leg tip of the unilateral lower limb are received, and the received IMU data is input into the pre-trained model under the corresponding motion mode to obtain the corresponding ankle joint torque prediction data.

[0009] Preferably, the movement modes include: walking on flat ground, climbing stairs, descending stairs, climbing a slope, and descending a slope.

[0010] Preferably, the IMU signals of the unilateral lower limb calf and leg tip moving in different motion modes include three-axis acceleration data and three-axis angular velocity data, with a total of 12 channels; the ankle joint torque data includes data in the sagittal plane, with a total of 1 channel.

[0011] Preferably, the time window length of the sliding time window is set to 300 ms, and the time window increment of the sliding time window is set to 50 ms.

[0012] Preferably, the TCN-BiGRU model includes a TCN network and a BiGRU network, wherein the number of residual blocks in the TCN network is set to 3, the expansion exponents of the three residual blocks are set to 1, 2, and 4, respectively, the convolutional layer input channels of the three residual blocks are set to 12, 30, and 60, respectively, the convolutional layer output channels of the three residual blocks are set to 30, 60, and 120, respectively, the convolution kernel size of each convolutional layer of the three residual blocks is set to 3, and the dropout of the TCN network is set to 0.2.

[0013] Preferably, the TCN-BiGRU model includes a TCN network and a BiGRU network, wherein the number of GRU layers of the BiGRU network is set to 10, the input size of the BiGRU network is set to 120, the number of hidden units of the BiGRU network is set to 100, and the dropout of the BiGRU network is set to 0.2.

[0014] Preferably, the method further comprises the steps of:

[0015] S4: Based on the training data under various motion modes, the pre-trained model of the corresponding motion mode is evaluated using the ten-fold cross-validation method.

[0016] The present invention also provides an electronic device,

[0017] include:

[0018] Memory, used to store computer programs;

[0019] The processor is used to implement the above method steps when executing the program stored in the memory.

[0020] Beneficial effects:

[0021] (1) This method uses TCN-BiGRU as a deep learning regression model. The temporal convolutional network (TCN) is a method based on the convolutional neural network structure. Due to the use of causal convolution and dilated convolution, it can capture the relationship between data in a longer range in the time dimension, so it performs well in time series data processing. The gated recurrent unit network (GRU) is developed from the recurrent neural network (RNN), which can solve the problem that RNN cannot have long-term memory. Compared with the long short-term memory network (LSTM), the network architecture is simpler and has fewer parameters. By combining the two, the model's understanding of time series data can be greatly improved, thereby achieving better fitting performance.

[0022] (2) This method uses the same deep learning regression model architecture to train a set of optimal network parameters for each motion mode, so that the network can focus more on the data fitting task of the current motion mode, the joint torque prediction is more accurate, and it has more advantages in terms of network parameters and real-time calculation. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a flow chart of the method of the present invention;

[0024] Figure 2 This is a schematic diagram of data segmentation with a sliding time window according to the present invention;

[0025] Figure 3 This is a diagram of the residual block structure in the present invention. DETAILED DESCRIPTION

[0026] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] like Figure 1-Figure 2 A method for predicting ankle joint torque for complex human motion is shown, comprising the following steps:

[0028] S1: Collect IMU signals of the lower leg and leg tip of a human body under different motion modes, as well as corresponding ankle joint torque data; the motion modes include: walking on flat ground, climbing stairs, descending stairs, climbing slopes, and descending slopes; the IMU signals of the lower leg and leg tip of a human body under different motion modes include three-axis acceleration data and three-axis angular velocity data, with a total of 12 channels; the ankle joint torque data includes data in the sagittal plane, with a total of 1 channel;

[0029] In this embodiment, the data collection directly uses the open source data set of lower limb biomechanics published by Camargo et al., which includes three-dimensional biomechanics and wearable sensor data of 22 healthy adults. The motion modes of the data set include: walking on flat ground, walking on a treadmill, going up and down stairs, and going up and down slopes. Under the conditions of walking on flat ground and treadmill, 3 and 28 different walking speeds are included respectively, and under the conditions of going up and down stairs and going up and down slopes, 4 different step heights and 6 different slope inclinations are included respectively. The wearable sensor data of the data set include: surface electromyography data of 11 surface electromyography sensors, joint angle data of 3 angle sensors, limb kinematics data of 4 inertial measurement units, 32 optical motion capture markers and ground reaction force solution to obtain inverse kinematics and inverse dynamics data and corresponding motion mode label data at each moment. This method only uses the IMU data of the unilateral lower limb calf and leg tip, ankle joint torque data and motion mode data. Since the exoskeleton system only assists the ankle joint in the sagittal plane, the ankle joint torque data only selects the data in the sagittal plane, and the data channel is 1 channel. The target joint is the ankle joint, so the IMU data selects the 3-axis acceleration and angular velocity data collected by two sensors located at the calf and toes, with a total of 12 channels.

[0030] S2: Use a sliding time window to segment the IMU signal and the corresponding ankle joint torque data to obtain training data under various motion modes;

[0031] In this embodiment, the sampling rate of IMU data and ankle joint torque data is 100Hz, the time window length of the sliding time window is set to 300ms, and the corresponding number of sampling points is 30; the time window increment of the sliding time window is set to 50ms, and the corresponding number of sampling points is 5; therefore, the size of the sliding time window of the IMU data is 30×12, where 30 represents the number of points corresponding to the time window length, and 12 represents the number of channels; similarly, the size of the sliding time window of the ankle joint torque data is 30×1, and the schematic diagram of the sliding time window segmentation data is as shown in Figure 2 shown.

[0032] S3: input the training data of various motion modes into respective TCN-BiGRU models for training, and obtain pre-trained models of various motion modes. The number of pre-trained models obtained is equal to the number of motion modes.

[0033] The motion mode set by the user and the IMU data of the calf and leg tip of the unilateral lower limb are received, and the received IMU data is input into the pre-trained model under the corresponding motion mode to obtain the corresponding ankle joint torque prediction data.

[0034] Specifically, the TCN-BiGRU model includes a TCN network and a BiGRU network, wherein the number of residual blocks in the TCN network is set to 3, the expansion exponents of the three residual blocks are set to 1, 2, and 4 respectively, the convolution layer input channels of the three residual blocks are set to 12, 30, and 60 respectively, the convolution layer output channels of the three residual blocks are set to 30, 60, and 120 respectively, the convolution kernel size of each convolution layer of the three residual blocks is set to 3, and the dropout of the TCN network is set to 0.2; the number of GRU layers of the BiGRU network is set to 10 layers, the input size of the BiGRU network is set to 120, the number of hidden units of the BiGRU network is set to 100, and the dropout of the BiGRU network is set to 0.2;

[0035] The TCN network has the advantages of parallel computing, capturing long-term correlation, avoiding gradient disappearance and gradient explosion, etc. Its basic structure is the residual block, and the structure is as follows Figure 3 As shown in the figure, the residual block includes the main path, the residual path, and the fusion. The main path performs a series of operations on the input (causal dilation convolution, batch normalization, ReLU activation function, and Dropout random discarding layer); the residual path determines whether to perform a one-dimensional convolution operation on the input based on whether the dimensions of the input and output are equal, thereby making the dimensions consistent and facilitating addition operations, which can avoid problems such as gradient disappearance and performance degradation; the fusion adds the outputs of the main path and the residual path in the depth direction to form the output of the residual block. Its core is the causal dilation convolution, which can increase the receptive field of the convolution layer, thereby enabling the top layer of the network to capture very long time dependencies.

[0036] The GRU network model is developed from RNN. It solves the gradient vanishing problem by introducing a gating mechanism, improves the performance of the model, and merges the forget gate and input gate in the LSTM model structure into an update gate, which simplifies the network structure and improves computational efficiency. The BiGRU network model introduces a bidirectional structure based on the GRU model, which can better capture the bidirectional dependency of sequence data.

[0037] S4: Based on the training data under various motion modes, the pre-trained model of the corresponding motion mode is evaluated using the ten-fold cross-validation method.

[0038] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for predicting ankle joint torque for complex human motion, characterized in that: The following steps are involved: S1: Collect IMU signals of the calf and leg tip of the human body under different motion modes, as well as the corresponding ankle joint torque data; S2: Use a sliding time window to segment the IMU signal and the corresponding ankle joint torque data to obtain training data under various motion modes; S3: input the training data of various motion modes into respective TCN-BiGRU models for training, and obtain pre-trained models of various motion modes. The number of pre-trained models obtained is equal to the number of motion modes. The motion mode set by the user and the IMU data of the calf and leg tip of the unilateral lower limb are received, and the received IMU data is input into the pre-trained model under the corresponding motion mode to obtain the corresponding ankle joint torque prediction data.

2. The ankle joint torque prediction method for complex human motion according to claim 1, characterized in that: The motion modes include: walking on flat ground, climbing stairs, descending stairs, climbing a slope, and descending a slope.

3. The ankle joint torque prediction method for complex human motion according to claim 1, characterized in that: The IMU signals of the unilateral lower limb calf and leg tip moving in different motion modes include three-axis acceleration data and three-axis angular velocity data, with a total of 12 channels; the ankle joint torque data includes data in the sagittal plane, with a total of 1 channel.

4. The ankle joint torque prediction method for complex human motion according to claim 1, characterized in that: The time window length of the sliding time window is set to 300ms, and the time window increment of the sliding time window is set to 50ms.

5. The ankle joint torque prediction method for complex human motion according to claim 1, characterized in that: The TCN-BiGRU model includes a TCN network and a BiGRU network. The number of residual blocks in the TCN network is set to 3, the expansion exponents of the three residual blocks are set to 1, 2, and 4, respectively, the convolutional layer input channels of the three residual blocks are set to 12, 30, and 60, respectively, the convolutional layer output channels of the three residual blocks are set to 30, 60, and 120, respectively, the convolution kernel size of each convolutional layer of the three residual blocks is set to 3, and the dropout of the TCN network is set to 0.

2.

6. The ankle joint torque prediction method for complex human motion according to claim 1, characterized in that: The TCN-BiGRU model includes a TCN network and a BiGRU network, where the number of GRU layers of the BiGRU network is set to 10, the input size of the BiGRU network is set to 120, the number of hidden units of the BiGRU network is set to 100, and the dropout of the BiGRU network is set to 0.

2.

7. The ankle joint torque prediction method for complex human motion according to claim 1, characterized in that: Also includes the steps: S4: Based on the training data under various motion modes, the pre-trained model of the corresponding motion mode is evaluated using the ten-fold cross-validation method.

8. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, for implementing the method steps described in any one of claims 1 to 7 when executing a program stored in a memory.

Images