A motion pattern transfer recognition method for exoskeleton in complex terrain

Abstract

A method for identifying motion pattern transitions in an exoskeleton under complex terrain includes: using multiple sensors to collect angular velocity and angular acceleration in the Z-axis direction, the pressure of the human body and exoskeleton on the ground, and complex terrain images; using a posture solution method to calculate the posture state of the mechanical legs, using a Kalman filter to process the sensor data, and preprocessing the complex terrain images; dividing the gait cycle of a single leg into sub-phases, using a bidirectional long short-term memory network to extract multidimensional data features, and using a gradient boosting decision tree to detect gait phases; inputting the gait phases into a deep separable convolutional network and the complex terrain image into a convolutional neural network, extracting multidimensional data features from each, and obtaining motion pattern recognition results; focusing on the sub-phase pre-swing in the gait cycle through a self-attention mechanism, and identifying motion pattern transitions using a temporal convolutional network. This invention improves the real-time and stability of motion pattern transition recognition, providing strong support for human-machine interaction with exoskeletons.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a method for identifying motion pattern transfer of an exoskeleton under complex terrain. Background Art

[0002] An exoskeleton is a wearable robotic system that combines the stability of a rigid structure with the flexibility of a flexible structure to provide improved adaptability, comfort, and efficiency. Overcoming the limitations of traditional assistive devices, exoskeletons offer excellent human-machine interaction and natural movement, providing a comfortable wear experience, better adapting to human movement needs, and demonstrating excellent performance and effectiveness in weight-bearing assistance.

[0003] Exoskeleton wearers participate in various sports indoors, using biosensors to collect human motion data and quickly and accurately identify movement patterns. Currently, biosensors are a hot topic in research, but after extensive research, several issues have emerged: They typically collect and preprocess EEG and EMG signals. EMG signal acquisition requires specialized equipment, limiting its use in outdoor exercise scenarios. During limb movement, the relative motion between the electrodes and the skin causes sweat to form, which can easily cause the electrodes to fall off, impacting the effectiveness of EMG signal acquisition. Furthermore, when using this method to identify movement pattern shifts with an exoskeleton, the wearer cannot effectively collect sensor data while walking outdoors due to the complex and changing terrain. Summary of the Invention

[0004] In view of the above problems, the present invention is proposed to provide a method for identifying motion pattern transfer of an exoskeleton in complex terrain, which overcomes the above problems or at least partially solves the above problems.

[0005] In order to solve the above technical problems, the embodiments of the present application disclose the following technical solutions:

[0006] A method for identifying motion pattern transfer of an exoskeleton in complex terrain, comprising:

[0007] S100. The inertial sensor collects angular velocity and angular acceleration data in the Z-axis direction, the force sensor collects pressure data of the human body and the exoskeleton on the ground, and the image sensor collects raw image data of complex terrain;

[0008] S200. Preprocess the data collected in S100 according to the first preset rule to obtain the posture state of the mechanical leg, the ground contact force and the preprocessed complex terrain image data;

[0009] S300. Divide the gait cycle of a single leg into sub-phases, input the posture state and ground contact force data of the mechanical leg pre-processed in S200 into a bidirectional long short-term memory network, extract multidimensional data features through the bidirectional long short-term memory network, and use a gradient boosting decision tree to detect gait phases;

[0010] S400. Input the detected gait phase into a deep separable convolutional network, input the preprocessed complex terrain image into a convolutional neural network, extract the multidimensional data features of each, and put them into the fully connected layer in parallel. After passing through the Softmax output layer, the motion pattern recognition result is obtained;

[0011] S500. Based on the motion pattern recognition result, a self-attention mechanism is used to focus on the sub-phase front swing in the gait cycle, and the motion pattern transfer is identified through a temporal convolutional network.

[0012] Furthermore, in S200, the data collected in S100 is preprocessed according to a first preset rule to obtain the posture state of the robotic leg. The first preset rule includes: solving the body quaternion based on the posture fusion, and the recursive equation for solving the posture by the quaternion differential equation is as shown in Formula 1:

[0013]

[0014] Among them, the subscript b represents the body coordinate system, the subscript n represents the navigation coordinate system, w represents the angular velocity, t represents the sampling time, and b w represents the angular velocity in the body coordinate system, represents quaternion multiplication, △t represents rotation time, and q0, q1, q2, and q3 represent quaternions.

[0015] Furthermore, in S200, the data collected in S100 is preprocessed according to a first preset rule to obtain the posture state of the mechanical leg, and the posture state of the mechanical leg includes at least: hip joint angle, hip joint angular velocity, hip joint angular acceleration, knee joint angle, knee joint angular velocity, knee joint angular acceleration, ankle joint angle, ankle joint angular velocity and ankle joint angular acceleration.

[0016] Furthermore, in S300, the sub-phases in the gait cycle of the single leg are divided, and the gait cycle of the single leg is divided into five sub-phases, namely heel strike, mid-stance, heel lift-off, backswing and frontswing.

[0017] Furthermore, in S300, the posture state and ground contact force data of the robotic leg preprocessed in S200 are input into a bidirectional long short-term memory network, and multidimensional data features are extracted through the bidirectional long short-term memory network, wherein the input matrix Y is as shown in Formula 2:

[0018]

[0019] Among them, the feature dimensions H, K, FA, and FF are hip joint, knee joint, ankle joint, and sole, respectively, and L is the label value corresponding to the sampling point.

[0020] Furthermore, the bidirectional long short-term memory network extracts multidimensional data features through the hidden layer to obtain an output matrix f(G,w). The output matrix f(G,w) is as shown in Formula 3:

[0021]

[0022] Among them, w is the weight value of the feature, and G is the label value corresponding to the gait sub-phase.

[0023] Furthermore, in S400, the detected gait phase is input into a depthwise separable convolutional network to extract multidimensional data features in the gait phase, which is divided into a two-step convolution process of channel-by-channel convolution and point-by-point convolution; wherein,

[0024] The channel-by-channel convolution process is as shown in Equations 4 and 5:

[0025] CK_s1=CK_w*CK_h*Input_c (4)

[0026] Output_c=CK_w*CK_h*(P_w-CK_w+1)*(P_h-CK_h+1)*Input_c (5)

[0027] Among them, CK_s represents the shape of the convolution kernel, CK_w represents the width of the convolution kernel, CK_h represents the height of the convolution kernel, Input_c represents the number of input channels, Output_c represents the number of output channels, P_w represents the array width, and P_h represents the array height.

[0028] The point-by-point convolution process is as shown in Equations 6 and 7:

[0029] CK_s 2=Input_c*Output_c (6)

[0030] CP=F_w*F_h*Input_c*Output_c (7)

[0031] Among them, CK_s represents the convolution kernel shape, Input_c represents the number of input channels, Output_c represents the number of output channels, CP represents the feature map, F_w represents the feature layer width, and F_h represents the feature layer height.

[0032] Furthermore, in S400, the pre-processed complex terrain image is input into a convolutional neural network to extract multi-dimensional data features of the complex terrain image, and the pooling layer reduces the data dimension; wherein,

[0033] The formula of convolutional neural network is as follows:

[0034] Output=(Input-CK+2*P) / S+1 (8)

[0035] Among them, Output represents the output size, Input represents the input size, CK represents the convolution kernel size, P represents the padding value, and S represents the step size.

[0036] Furthermore, in S500, the self-attention mechanism calculates attention and weights by mapping queries, keys, and values. The input sequence passes through the self-attention mechanism to calculate the query, key, and value of each element. The similarity between the query and the key is calculated to obtain the attention weight of each element to other elements. The calculation process of the self-attention mechanism is shown in Equation 9:

[0037]

[0038] Among them, Q, K and V represent query, key and value respectively, d_k represents the dimension of query and key, and Softmax is the normalization function.

[0039] Furthermore, in S500, the calculation process of the temporal convolutional network includes: first obtaining the product accumulation term of the input sequence and the convolution operation, then adding the input sequence x to the accumulation term, and then processing it through the activation function Activation to obtain the final output o; the specific calculation process and results are shown in Equations 10, 11, and 12:

[0040]

[0041] o=Activation(x+F(x)) (11)

[0042]

[0043] Where x is the input sequence, (x*df)(s) means one-dimensional convolution at sequence position s, using convolution kernel df for calculation, sequence f(i) and input sequence x s-di The sum of products within a certain range, di is a constant, Activation is the activation function, o is the output sequence of the activation function, wc is the weight coefficient corresponding to the multidimensional feature, and T is the result of motion mode transfer recognition.

[0044] The beneficial effects of the above technical solutions provided by the embodiments of the present invention include at least:

[0045] The present invention discloses a method for identifying motion mode transfer of an exoskeleton under complex terrain, comprising: S100. collecting angular velocity and angular acceleration data in the Z-axis direction through an inertial sensor, collecting pressure data of the human body and the exoskeleton on the ground through a force sensor, and collecting original image data of the complex terrain through an image sensor; S200. pre-processing the data collected in S100 according to a first preset rule to obtain the posture state of the mechanical leg, the ground contact force and the pre-processed image data of the complex terrain; S300. dividing the sub-phases in the gait cycle of a single leg, and converting the posture state of the mechanical leg and the ground contact force data pre-processed in S200 into the sub-phases in the gait cycle of a single leg. The data is input into a bidirectional long short-term memory network, multidimensional data features are extracted through the bidirectional long short-term memory network, and gait phase is detected by using a gradient boosting decision tree; S400. The detected gait phase is input into a depthwise separable convolutional network, and the image after complex terrain preprocessing is input into the convolutional neural network, and the respective multidimensional data features are extracted respectively, and put into the fully connected layer in parallel, and the motion pattern recognition result is obtained through the Softmax output layer; S500. According to the motion pattern recognition result, a self-attention mechanism is used to focus on the sub-phase front swing in the gait cycle, and the motion pattern transfer is identified through a temporal convolutional network.

[0046] The beneficial effects produced by the present invention are:

[0047] 1. The exoskeleton's mechanical legs in the present invention collect and process angles, angular velocities, and angular accelerations using an inertial measurement unit, while using a CCD image sensor to obtain complex terrain images.

[0048] 2. In the present invention, the depthwise separable convolution and convolutional neural network are placed in parallel in the fully connected layer, and the classification results are obtained through the Softmax output layer, which improves the effect of motion pattern recognition.

[0049] 3. In the present invention, the motion pattern recognition results are input into the self-attention mechanism, and the motion pattern transfer is identified through the temporal convolutional network, which improves the real-time and stability of motion pattern transfer recognition and provides strong support for human-computer interaction of the exoskeleton.

[0050] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0052] Figure 1 This is a flow chart of a method for identifying motion pattern transfer of an exoskeleton in complex terrain in Example 1 of the present invention;

[0053] Figure 2 This is a schematic diagram of an exoskeleton wearer walking in Example 1 of the present invention;

[0054] Figure 3 This is a schematic diagram of motion mode transfer recognition in Example 1 of the present invention. DETAILED DESCRIPTION

[0055] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0056] In order to solve the problems existing in the prior art, an embodiment of the present invention provides a method for identifying motion pattern transfer of an exoskeleton under complex terrain.

[0057] Example 1

[0058] The present invention discloses a method for identifying the movement pattern transfer of an exoskeleton under complex terrain. Figure 1 ,include:

[0059] S100. The inertial sensor collects angular velocity and angular acceleration data in the Z-axis direction, the force sensor collects pressure data of the human body and the exoskeleton on the ground, and the image sensor collects raw image data of complex terrain;

[0060] Specifically, such as Figure 2 The inertial sensors are fixed on the thighs, calves and soles of the human body to collect angular velocity and angular acceleration in the Z-axis direction; the force sensor is fixed on the sole of the foot to collect the pressure of the human body and the exoskeleton on the ground; the CCD image sensor is fixed on one thigh to collect images of complex terrain; the complex terrains are flat ground, stairs, ramps, grass and sand.

[0061] S200. Preprocess the data collected in S100 according to the first preset rule to obtain the posture state of the mechanical leg, the ground contact force and the preprocessed complex terrain image data;

[0062] Specifically, the posture solver method is used to calculate the position of the robotic leg. A Kalman filter is used to process continuous-time and linear dynamic sensor data to achieve linear smoothing. After deburring, filling in blank values, and eliminating fault points, the hip joint angle, hip joint angular velocity, hip joint angular acceleration, knee joint angle, knee joint angular velocity, knee joint angular acceleration, ankle joint angle, ankle joint angular velocity, ankle joint angular acceleration, and ground contact force are obtained. Complex terrain images are preprocessed using Gaussian filtering to remove image noise, histogram equalization to enhance the image, and image resizing.

[0063] In S200 of this embodiment, the data collected in S100 is preprocessed according to a first preset rule to obtain the posture state of the robotic leg. The first preset rule includes: calculating the body quaternion based on the posture fusion. The recursive equation for solving the posture using the quaternion differential equation is as shown in Formula 1:

[0064]

[0065] Among them, the subscript b represents the body coordinate system, the subscript n represents the navigation coordinate system, w represents the angular velocity, t represents the sampling time, and b w represents the angular velocity in the body coordinate system, represents quaternion multiplication, △t represents rotation time, and q0, q1, q2, and q3 represent quaternions.

[0066] S300. Divide the sub-phases in the gait cycle of a single leg, input the posture state of the mechanical leg and the ground contact force data pre-processed in S200 into a bidirectional long short-term memory network, extract multidimensional data features through the bidirectional long short-term memory network, and use a gradient boosting decision tree to detect gait phases;

[0067] In this embodiment, the gait cycle of a single leg is divided into five sub-phases, namely heel strike, mid-stance, heel lift-off, backswing, and frontswing.

[0068] The bidirectional long short-term memory network processes long sequence data, prevents gradient disappearance and gradient explosion, introduces a gating mechanism, effectively utilizes the feature information of the forward and backward sequences, and better captures long-term dependencies.

[0069] The parameters of the bidirectional long short-term memory network are as follows: sequence length is 8, input size is 10, cell array is 8*10, number of hidden layers is 16, batch size is 64, artificial neural network is bidirectional, and packet loss rate is 0.2.

[0070] In S300 of this embodiment, the posture state and ground contact force data of the robotic leg preprocessed in S200 are input into a bidirectional long short-term memory network, and multidimensional data features are extracted through the bidirectional long short-term memory network, where the input matrix Y is as shown in Formula 2:

[0071]

[0072] Among them, the feature dimensions H, K, FA, and FF are the hip joint, knee joint, ankle joint, and sole, respectively, and L is the label value corresponding to the sampling point. Specifically, Equation 2 has a total of 10 feature dimensions: hip angle, hip angular velocity, hip angular acceleration, knee angle, knee angular velocity, knee angular acceleration, ankle angle, ankle angular velocity, ankle angular acceleration, and ground contact force. If the lower limb joints are divided into left and right joints, then there are a total of 20 feature dimensions. The sensor's acquisition board collects data every 10ms, using row-by-row data points as sampling points.

[0073] In this embodiment, the hidden layer of the bidirectional long short-term memory network is as shown in Equations 3, 4, and 5:

[0074]

[0075]

[0076]

[0077] Among them, H represents the hidden layer, h represents the number of hidden layers, represents the XOR operation, w represents the forward hidden weight matrix, v represents the backward hidden weight matrix, b represents the bias vector, and t represents the calculation time.

[0078] In this embodiment, the bidirectional long short-term memory network extracts multidimensional data features through the hidden layer to obtain an output matrix f(G, w). The output matrix f(G, w) is as shown in Formula 6:

[0079]

[0080] Among them, w is the weight value of the feature, and G is the label value corresponding to the gait sub-phase.

[0081] In this embodiment, the gradient boosting decision tree adopts a gradient descent optimization strategy, takes the negative gradient of the loss function as the approximate value of the residual, uses a new decision tree model to fit the residual, and repeatedly iterates the new decision tree after weighting and putting it into the model, gradually reducing the value of the loss function and the deviation of the model to obtain an integrated model.

[0082] The parameters of the gradient boosting decision tree are as follows: the number of iterations is 60, the loss function is deviance, the maximum depth is 5, the minimum number of samples a parent node has is 5, the minimum number of samples a leaf node has is 4, the minimum sum of weights of leaf nodes is 0.4, and the number of leaf nodes is 8.

[0083] The output matrix is ​​input into the gradient boosting decision tree to detect gait phases. A set of multiple consecutive gait cycles, each containing 5 gait sub-phases, is shown in Equations 7 and 8:

[0084] C={a,b,c,d,e} (7)

[0085]

[0086] Among them, C is the gait cycle, which includes heel strike, mid-stance, heel lift-off, backswing and frontswing; G is multiple consecutive gait cycles.

[0087] S400. Input the detected gait phase into a deep separable convolutional network, input the preprocessed complex terrain image into a convolutional neural network, extract the multidimensional data features of each, and put them into the fully connected layer in parallel. After passing through the Softmax output layer, the motion pattern recognition result is obtained;

[0088] Specifically, the motion pattern is a collection of multiple consecutive gait cycles when the wearer walks on complex terrain, including walking on flat ground, walking on stairs, walking on slopes, walking on grass, and walking on sand.

[0089] The gait phase is input into the deep separable convolutional network, and the complex terrain image is input into the convolutional neural network. Multi-dimensional data features are extracted from each of them and put into the fully connected layer in parallel. The motion pattern recognition results are obtained through the Softmax output layer.

[0090] The collection of various motion modes under complex terrain is as shown in Equation 9:

[0091] M={A,B,C,D,E} (9)

[0092] In this embodiment S400, the detected gait phase is input into a depthwise separable convolutional network to extract multidimensional data features in the gait phase, which is divided into two convolution processes: channel-by-channel convolution and point-by-point convolution; wherein,

[0093] The channel-by-channel convolution process is as shown in Equations 10 and 11:

[0094] CK_s1=CK_w*CK_h*Input_c (10)

[0095] Output_c=CK_w*CK_h*(P_w-CK_w+1)*(P_h-CK_h+1)*Input_c (11)

[0096] Among them, CK_s represents the shape of the convolution kernel, CK_w represents the width of the convolution kernel, CK_h represents the height of the convolution kernel, Input_c represents the number of input channels, Output_c represents the number of output channels, P_w represents the array width, and P_h represents the array height;

[0097] The point-by-point convolution process is as shown in Equations 12 and 13:

[0098] CK_s 2=Input_c*Output_c (12)

[0099] CP=F_w*F_h*Input_c*Output_c (13)

[0100] Among them, CK_s represents the convolution kernel shape, Input_c represents the number of input channels, Output_c represents the number of output channels, CP represents the feature map, F_w represents the feature layer width, and F_h represents the feature layer height.

[0101] In S400 of this embodiment, the pre-processed complex terrain image is input into a convolutional neural network to extract multi-dimensional data features of the complex terrain image, and the pooling layer reduces the data dimension; wherein,

[0102] The formula of convolutional neural network is as follows:

[0103] Output=(Input-CK+2*P) / S+1 (14)

[0104] Where Output represents the output size, Input represents the input size, CK represents the convolution kernel size, P represents the padding value, and S represents the stride. The parameters of the convolutional neural network are as follows: the number of input channels is 3, the number of output channels is 64, the convolution kernel size is 12, the stride is 2, and the padding value is 2.

[0105] S500. Based on the motion pattern recognition result, a self-attention mechanism is used to focus on the sub-phase front swing in the gait cycle, and the motion pattern transfer is identified through a temporal convolutional network.

[0106] Specifically, such as Figure 3 As shown, motion mode transfer is the switch between flat walking and other motion modes when the wearer walks on complex terrain, including It appears in the sub-phase pre-sway of the gait cycle. The self-attention mechanism focuses on the sub-phase pre-sway in the gait cycle, and the temporal convolutional network is used to identify the movement pattern shift.

[0107] The set of multiple motion mode transfers under complex terrain is as shown in Equation 15:

[0108] Tran={A→B,A→C,A→D,A→E,A←B,A←C,A←D,A←E} (15)

[0109] When the self-attention mechanism processes sequence data, each element can be associated with other elements. It not only depends on the elements in the adjacent positions, but also can adaptively capture the long-term dependencies between elements by calculating the relative importance between elements.

[0110] In S500 of this embodiment, the self-attention mechanism calculates attention and weight by mapping queries, keys, and values. The input sequence is passed through the self-attention mechanism to calculate the query, key, and value of each element; the similarity between the query and the key is calculated to obtain the attention weight of each element to other elements. The calculation process of the self-attention mechanism is shown in Equation 16:

[0111]

[0112] Among them, Q, K and V represent query, key and value respectively, d_k represents the dimension of query and key, and Softmax is the normalization function.

[0113] The parameters of the self-attention mechanism are as follows: the input dimension is 10 and the number of hidden layers is 8.

[0114] The temporal convolutional network uses causal convolution and dilated convolution to process time series, obtains long-term dependencies through multi-layer stacking and residual connections, and has parallel computing capabilities.

[0115] Temporal convolutional networks (TCNs) include causal convolution, dilated convolution, residual connections, and activation functions. Causal convolution considers past information; the more historical information it can trace, the more hidden layers it requires. Dilated convolutions are characterized by each layer extracting information from the previous layer, expanding exponentially by 2. Each dilated convolution layer requires zero padding to maintain consistent input and output dimensions. Each weight vector is normalized, and the ReLU nonlinear activation function and Dropout regularization are used. Residual connections are implemented, with a residual connection performed every two dilated convolution layers.

[0116] The parameters of the temporal convolutional network are as follows: the number of input channels is 8, the number of output channels is 8, the convolution kernel size is 2, the batch size is 32, the packet loss rate is 0.2, the stride is 2, the padding value is 2, and the dilation size is 1.

[0117] In S500 of this embodiment, the calculation process of the temporal convolutional network includes: first obtaining the product accumulation term of the input sequence and the convolution operation, then adding the input sequence x to the accumulation term, and then processing it through the activation function Activation to obtain the final output o; the specific calculation process and results are shown in Equations 17, 18, and 19:

[0118]

[0119] o=Activation(x+F(x)) (18)

[0120]

[0121] Where x is the input sequence, (x*df)(s) means one-dimensional convolution at sequence position s, using convolution kernel df for calculation, sequence f(i) and input sequence x s-di The sum of products within a certain range, di is a constant, Activation is the activation function, o is the output sequence of the activation function, wc is the weight coefficient corresponding to the multidimensional feature, and T is the result of motion mode transfer recognition.

[0122] The present embodiment discloses a method for identifying motion mode transfer of an exoskeleton under complex terrain, comprising: S100. collecting angular velocity and angular acceleration data in the Z-axis direction through an inertial sensor, collecting pressure data of the human body and the exoskeleton on the ground through a force sensor, and collecting original image data of the complex terrain through an image sensor; S200. pre-processing the data collected in S100 according to a first preset rule to obtain the posture state of the mechanical leg, the ground contact force and the pre-processed complex terrain image data; S300. dividing the sub-phases in the gait cycle of a single leg, and converting the posture state of the mechanical leg and the ground contact force data pre-processed in S200 into the sub-phases; The data is input into a bidirectional long short-term memory network, multidimensional data features are extracted through the bidirectional long short-term memory network, and gait phase is detected by using a gradient boosting decision tree; S400. The detected gait phase is input into a depthwise separable convolutional network, and the preprocessed complex terrain image is input into a convolutional neural network, and the respective multidimensional data features are extracted respectively, and put into the fully connected layer in parallel, and the motion pattern recognition result is obtained through the Softmax output layer; S500. According to the motion pattern recognition result, a self-attention mechanism is used to focus on the sub-phase front swing in the gait cycle, and the motion pattern transfer is identified through a temporal convolutional network.

[0123] The beneficial effects produced by this embodiment are:

[0124] 1. The exoskeleton's mechanical legs in the present invention collect and process angles, angular velocities, and angular accelerations using an inertial measurement unit, while using a CCD image sensor to obtain complex terrain images.

[0125] 2. In the present invention, the depthwise separable convolution and convolutional neural network are placed in parallel in the fully connected layer, and the classification results are obtained through the Softmax output layer, which improves the effect of motion pattern recognition.

[0126] 3. In the present invention, the motion pattern recognition results are input into the self-attention mechanism, and the motion pattern transfer is identified through the temporal convolutional network, which improves the real-time and stability of motion pattern transfer recognition and provides strong support for human-computer interaction of the exoskeleton.

[0127] It should be understood that the specific order or hierarchy of steps in the disclosed processes is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process can be rearranged without departing from the scope of the present disclosure. The accompanying method claims present elements of the various steps in an exemplary order and are not intended to be limited to the specific order or hierarchy described.

[0128] In the foregoing detailed description, various features are grouped together in a single embodiment to simplify the disclosure. This method of disclosure should not be interpreted as reflecting an intention that embodiments of the claimed subject matter require more features than are expressly recited in each claim. On the contrary, as reflected in the appended claims, the invention comprises less than all the features of any individual disclosed embodiment. The appended claims are therefore hereby expressly incorporated into the detailed description, with each claim standing on its own as a separate preferred embodiment of the invention.

[0129] Those skilled in the art will also appreciate that the various illustrative logic blocks, modules, circuits, and algorithmic steps described in conjunction with the embodiments herein may be implemented as electronic hardware, computer software, or a combination thereof. In order to clearly illustrate the interchangeability between hardware and software, the various illustrative components, blocks, modules, circuits, and steps described above are generally described around their functions. Whether such functions are implemented as hardware or software depends on the specific application and the design constraints imposed on the entire system. A skilled person may implement the described functions in an adaptable manner for each specific application, but such implementation decisions should not be interpreted as departing from the scope of protection of this disclosure.

[0130] The steps of the methods or algorithms described in conjunction with the embodiments herein may be directly embodied as hardware, software modules executed by a processor, or a combination thereof. The software module may be located in a RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, register, hard disk, removable disk, CD-ROM, or any other form of storage medium well known in the art. An exemplary storage medium is connected to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium may also be an integral part of the processor. The processor and storage medium may be located in an ASIC. The ASIC may be located in a user terminal. Of course, the processor and storage medium may also be present in a user terminal as discrete components.

[0131] For software implementation, the techniques described in this application can be implemented using modules (e.g., procedures, functions, etc.) that perform the functions described in this application. These software codes can be stored in a memory unit and executed by a processor. The memory unit can be implemented within the processor or external to the processor. In the latter case, it is communicatively coupled to the processor via various means, which are well known in the art.

[0132] The foregoing description includes examples of one or more embodiments. Of course, it is not possible to describe all possible combinations of components or methods for the purposes of describing the above embodiments, but one of ordinary skill in the art will recognize that the various embodiments may be further combined and arranged. Therefore, the embodiments described herein are intended to encompass all such changes, modifications and variations that fall within the scope of the appended claims. Furthermore, to the extent the term "comprising" is used in the specification or claims, the term is intended to be encompassed in a manner similar to the term "including," as explained in terms of "including," used as a transitional word in the claims. Furthermore, any use of the term "or" in the specification of the claims is intended to mean a "non-exclusive or."

Claims

1. A method for identifying motion pattern transfer of an exoskeleton in complex terrain, characterized in that: include: S100. The inertial sensor collects angular velocity and angular acceleration data in the Z-axis direction, the force sensor collects pressure data of the human body and the exoskeleton on the ground, and the image sensor collects raw image data of complex terrain; S200. Preprocess the data collected in S100 according to the first preset rule to obtain the posture state of the mechanical leg, the ground contact force and the preprocessed complex terrain image data; S300. Divide the sub-phases in the gait cycle of a single leg, input the posture state and ground contact force data of the mechanical leg pre-processed in S200 into a bidirectional long short-term memory network, extract multidimensional data features through the bidirectional long short-term memory network, and detect gait phases using a gradient boosting decision tree. In S300, the sub-phases in the gait cycle of a single leg are divided into five sub-phases, namely heel strike, mid-stance, heel lift-off, backswing, and frontswing. S400. The detected gait phase is input into a deep separable convolutional network, and the preprocessed complex terrain image is input into a convolutional neural network. The multidimensional data features of each are extracted and put into the fully connected layer in parallel. The motion pattern recognition result is obtained through the Softmax output layer; S500. Based on the motion pattern recognition result, a self-attention mechanism is used to focus on the sub-phase front swing in the gait cycle, and the motion pattern transfer is identified through a temporal convolutional network.

2. The method for identifying motion pattern transfer of an exoskeleton in complex terrain according to claim 1, wherein: In S200, the data collected in S100 is preprocessed according to a first preset rule to obtain the posture state of the robotic leg. The first preset rule includes: solving the body quaternion based on the posture fusion, and the recursive equation for solving the posture using the quaternion differential equation is as shown in Formula 1: ； Among them, the subscript b represents the body coordinate system, the subscript n represents the navigation coordinate system, w represents the angular velocity, and t represents the sampling time. b w represents the angular velocity in the body coordinate system, represents quaternion multiplication, represents the rotation time, 、 、 、 Represents a quaternion.

3. The method for identifying motion pattern transfer of an exoskeleton in complex terrain according to claim 1, wherein: In S200, the data collected in S100 is preprocessed according to a first preset rule to obtain the posture state of the mechanical leg, which at least includes: hip joint angle, hip joint angular velocity, hip joint angular acceleration, knee joint angle, knee joint angular velocity, knee joint angular acceleration, ankle joint angle, ankle joint angular velocity and ankle joint angular acceleration.

4. The method for identifying motion pattern transfer of an exoskeleton in complex terrain according to claim 1, wherein: In S300, the posture state of the mechanical leg and the ground contact force data pre-processed in S200 are input into the bidirectional long short-term memory network, and the multidimensional data features are extracted through the bidirectional long short-term memory network, wherein the input matrix As shown in formula 2: (2)； Among them, the feature dimensions H, K, FA, and FF are hip joint, knee joint, ankle joint, and sole, respectively, and L is the label value corresponding to the sampling point.

5. The method for identifying motion pattern transfer of an exoskeleton in complex terrain according to claim 4, wherein: The bidirectional long short-term memory network extracts multidimensional data features through the hidden layer to obtain the output matrix , the output matrix As shown in formula 3: ； in, is the weight value of the feature, is the label value corresponding to the gait sub-phase; Indicates that there are n labels in the first column, and each row represents a predicted label value corresponding to each dimension feature. represents n hip joint angle eigenvalues, represents n hip joint angular velocity eigenvalues, represents the n hip joint angular acceleration eigenvalues, represents n knee joint angle eigenvalues, represents n knee joint angular velocity eigenvalues, represents the n knee joint angular acceleration eigenvalues, represents n plantar angle eigenvalues, represents n plantar angular velocity eigenvalues, represents n plantar angular acceleration eigenvalues, Represents n plantar pressure eigenvalues.

6. The method for identifying motion pattern transfer of an exoskeleton in complex terrain according to claim 1, wherein: In S400, the detected gait phase is input into a depth-wise separable convolutional network to extract multidimensional data features in the gait phase, which is divided into two convolution processes: channel-by-channel convolution and point-by-point convolution; wherein, The channel-by-channel convolution process is as shown in Equations 4 and 5: (4)； (5)； in, represents the shape of the convolution kernel, represents the computational amount of channel-by-channel convolution, represents the convolution kernel width, represents the height of the convolution kernel, Indicates the number of input channels, Indicates the number of output channels, Indicates the array width, Indicates the height of the array; The point-by-point convolution process is as shown in Equations 6 and 7: (6)； (7)； in, represents the shape of the convolution kernel, represents the computational cost of point-by-point convolution, Indicates the number of input channels, Indicates the number of output channels, CP indicates the feature map, represents the feature layer width, Indicates the feature layer height.

7. The method for identifying motion pattern transfer of an exoskeleton in complex terrain according to claim 1, wherein: In S400, the pre-processed complex terrain image is input into a convolutional neural network to extract multi-dimensional data features of the complex terrain image, and the pooling layer reduces the data dimension; wherein, The formula of convolutional neural network is as follows: (8)； in, Indicates the output size, Indicates the input size, CK indicates the size of the convolution kernel, P indicates the padding value, and S indicates the step size.

8. The method for identifying motion pattern transfer of an exoskeleton in complex terrain according to claim 1, wherein: In S500, the self-attention mechanism calculates attention and weights by mapping queries, keys, and values. The input sequence is passed through the self-attention mechanism to calculate the query, key, and value of each element. Calculate the similarity between the query and the key to obtain the attention weight of each element to other elements; the calculation process of the self-attention mechanism is as shown in Equation 9: (9)； Among them, Q, K and V represent query, key and value respectively. Represents the dimensions of query and key, and Softmax is a normalization function.

9. The method for identifying motion pattern transfer of an exoskeleton in complex terrain according to claim 1, wherein: In S500, the calculation process of the temporal convolutional network includes: first obtaining the product accumulation term of the input sequence and the convolution operation, and then Added to the accumulated item, and then through the activation function Processing to get the final output ; The specific calculation process and results are shown in Equations 10, 11, and 12: (10) (11) (12) in, is the input sequence, Indicates the position in the sequence Perform one-dimensional convolution at , using the convolution kernel Perform calculations, sequence and the input sequence The sum of products within a certain range, is a constant, is the activation function, is the output sequence of the activation function, wc is the weight coefficient corresponding to the multidimensional data feature, T is the result of motion mode transfer recognition, Indicates that there are n labels in the first column, and each row represents a predicted label value corresponding to each dimension feature. represents n hip joint angle eigenvalues, represents n hip joint angular velocity eigenvalues, represents the n hip joint angular acceleration eigenvalues, represents n knee joint angle eigenvalues, represents n knee joint angular velocity eigenvalues, represents the n knee joint angular acceleration eigenvalues, represents n plantar angle eigenvalues, represents n plantar angular velocity eigenvalues, represents n plantar angular acceleration eigenvalues, Represents n plantar pressure eigenvalues.

Images