Hip exoskeleton torque prediction and auxiliary control method based on space-time fusion network

Abstract

The application relates to the technical field of intelligent exoskeleton control, and particularly discloses a hip joint exoskeleton torque prediction and auxiliary control method based on a space-time fusion network. The method collects sensor data of a waist and back IMU and bilateral motor encoders, constructs a time sequence sliding window input, and inputs the time sequence sliding window into a double-channel space-time fusion network STFNet. The STFNet comprises a parallel TCN channel and a Bi-LSTM channel, respectively extracts local space time sequence patterns and long-range forward and backward dependence features, outputs an action probability vector through an action classification network, and inputs the action probability vector and shared features after splicing into a torque regression network. The fusion features are estimated to be hip joint biological torques through a double-path regression network, and exoskeleton instruction torques are generated through a relay control layer composed of scaling, delay and filtering. The application only depends on light sensors, realizes high-precision torque estimation and adaptive assistance in a multi-action scene, reduces metabolic energy consumption, and has strong real-time performance and generalization ability.

Description

Technical Field

[0001] This invention relates to the field of intelligent exoskeleton control technology, and in particular to a method for predicting and assisting the control of hip joint exoskeleton torque based on a spatiotemporal fusion network. Background Technology

[0002] As wearable robots, lower limb exoskeletons work in conjunction with the human musculoskeletal system and have long been expected to enhance users' mobility and rehabilitation. The hip joint is the core joint for lower limb movement, bearing a significant amount of power output in daily activities such as walking, running, and climbing stairs. Hip-assisted exoskeletons have been shown to significantly reduce users' metabolic energy consumption. However, existing hip exoskeleton control strategies face two major challenges in real-world outdoor applications.

[0003] Firstly, traditional exoskeleton controllers typically rely on predefined torque trajectories or task classification methods, requiring separate torque curves for different actions such as walking on flat ground, going up and down slopes, and climbing stairs. Task classifiers are needed to handle action transitions, making them ill-suited for continuous and complex outdoor environments. Secondly, traditional active exoskeletons' time-based (gait cycle) or kinematic (e.g., joint angle) torque-assisted control strategies cannot effectively adapt to changes in outdoor walking patterns. These controllers typically generate discrete auxiliary torque trajectories based on gait phase, and these trajectories can only change at the beginning of the next discrete segment, making it difficult to adapt to different walking patterns (e.g., speed changes and sudden stops).

[0004] Secondly, existing deep learning-based torque estimation methods suffer from complex sensor configurations and insufficient generalization ability across multiple actions. Some methods rely on multimodal sensors such as plantar pressure and electromyography signals, which are burdensome to wear and difficult to deploy. While some methods based on a single IMU reduce sensor complexity, they share a unified kinematic-torque mapping across multiple actions, ignoring the differences in human biomechanical characteristics under different actions, thus limiting the accuracy of torque estimation.

[0005] Existing technologies attempt to address multi-motion adaptability issues through activity classification and gait phase detection. However, these methods have significant shortcomings: First, they rely on multiple IMUs and additional sensors such as foot switches, increasing the wearable burden and hardware complexity of the system. Second, they are essentially based on a rule-switching "classification-computation" model, which first determines the type of movement and then calls the corresponding human dynamics model to calculate torque. This is limited by model accuracy and cannot achieve end-to-end torque regression. Finally, their control output is merely a proportional scaling of bio-torque, lacking active optimization of assist phase and human metabolic energy consumption.

[0006] Therefore, there is an urgent need for a method that relies solely on lightweight sensor inputs, can accurately estimate hip joint torque in multi-motion scenarios, and provide adaptive assist control accordingly. Summary of the Invention

[0007] The present invention aims to solve the problems existing in the prior art and provides a method for predicting and assisting control of hip joint exoskeleton torque based on spatiotemporal fusion network. It adopts a dual-channel spatiotemporal fusion network combined with motion condition feature fusion module to realize high-precision real-time estimation of hip joint bio-torque and provides adaptive assist control based on torque estimation results.

[0008] To achieve the above objectives, this invention provides a method for predicting and assisting the control of hip exoskeleton torques based on a spatiotemporal fusion network, comprising the following steps: S1. Data Acquisition: Acquire sensor data from the hip exoskeleton. The sensors include a first inertial measurement unit installed in the lower back or pelvis and dual motor encoders in the hip joint drive unit. S2. Feature Extraction: Construct a time-series sliding window to extract and standardize features from the sensor data, forming an input feature matrix; S3. Dual-channel feature extraction: Construct a dual-channel spatiotemporal fusion network STFNet, including a TCN channel, a Bi-LSTM channel and a feature fusion layer. Input the input feature matrix into the TCN channel and the Bi-LSTM channel respectively to extract temporal features. Then, concatenate the feature vectors output by the TCN channel and the Bi-LSTM channel through the feature fusion layer to obtain the first fused feature vector. S4. Action condition feature fusion: The first fused feature vector is processed by an independently trained action classification network to output an action probability vector; the action probability vector is concatenated with the shared feature vector extracted from the first fused feature vector through a fully connected layer to obtain a second fused feature vector; S5. Torque Regression: Input the second fused feature vector into the hip joint torque regression network to obtain the estimated value of the hip joint bio-torque at the current moment; S6. Auxiliary Torque Generation: Based on the estimated value of the hip joint bio-torque, the command torque of the exoskeleton is generated and output through the relay control layer. The command torque is sent to the motor controller of the hip joint drive unit so that the exoskeleton provides auxiliary torque according to the command torque.

[0009] Furthermore, in step S3, the TCN channel uses multi-layer dilated convolution to extract the local spatial features and temporal dependencies of the input data, which sequentially includes three residual blocks. Each residual block includes two dilated convolutional layers with increasing dilation rates layer by layer. The data is then reduced in dimensionality by a global average pooling layer and a Flatten layer, and finally the output dimension is mapped to a specified dimension by a fully connected layer. Within each residual block, the first layer has 96 convolutional kernels and the second layer has 48 convolutional kernels. The Bi-LSTM channel uses a bidirectional long short-term memory network to capture the long-range forward and backward dependencies of time-series data, and then connects to the Bi-LSTM layer to output the forward and backward merged features of the last time step.

[0010] Furthermore, in step S3, the first fused feature vector obtained by concatenating the feature vectors output by the TCN channel and the Bi-LSTM channel through the feature fusion layer is regularized by the Dropout layer; the dropout rate is 0.1.

[0011] Furthermore, in step S4, the action condition feature fusion specifically includes: The first fused feature vector is passed through an action classification network to output action probability vectors for six types of actions: walking on flat ground (LG), going uphill (RA), going downhill (RD), going up stairs (SA), going down stairs (SD), and running (RUN). The first fused feature vector is input into the fully connected layer to extract the shared feature vector; The shared feature vector is concatenated with the action probability vector to obtain the second fused feature vector.

[0012] Furthermore, in step S5, the hip joint torque regression network adopts a dual-path structure, including a dynamic torque prediction path, a motion bias path, and a torque synthesis layer, specifically: The dynamic torque prediction path inputs the first fused feature vector into a fully connected layer to extract shared features. The shared feature vector is then concatenated with the action probability vector to obtain fused features. These features are then passed through two fully connected layers to output the dynamic torque components. The action bias path takes the action probability vector as input to an unbiased fully connected layer and outputs action bias torque components, which are used to provide independent baseline torque bias for different actions. The torque synthesis layer adds the dynamic torque component and the action bias torque bias through an additive layer to obtain the estimated value of the hip joint bio-torque.

[0013] Furthermore, in step S6, the relay control layer includes: The scaling unit is used to multiply the estimated value of the hip joint bio-torque by a scaling factor to obtain a scaled torque signal. The delay unit is used to apply a fixed delay to the scaled torque signal to obtain the delayed torque signal. A low-pass filter unit is used to perform low-pass filtering on the delayed torque signal to obtain the command torque.

[0014] Furthermore, the scaling factor ranges from 0.1 to 0.4; the fixed delay ranges from 50 to 150 milliseconds; and the low-pass filter unit uses a second-order Butterworth low-pass filter with a cutoff frequency of 10 Hz. The scaling factor in the relay control layer is adaptively adjusted according to user preferences or action type. For high-torque-requirement actions such as going uphill or climbing stairs, a scaling factor of 0.3 to 0.4 is used; for low-torque-requirement actions such as walking on flat ground, a scaling factor of 0.15 to 0.2 is used; and for fast movements such as running, a scaling factor of 0.1 to 0.15 is used.

[0015] Furthermore, the method also includes a model lightweighting step, specifically: Before model deployment, the trained STFNet model is compressed using mixed precision quantization and channel pruning techniques to generate a lightweight version of the model.

[0016] Furthermore, the TCN channel and Bi-LSTM channel of the model are quantized with 8-bit and 4-bit mixed precision, respectively; the contribution of feature channels is evaluated using interpretability analysis methods, and redundant feature channels are pruned or fused.

[0017] The present invention employs the above-mentioned method for predicting and assisting the control of hip exoskeleton torque based on spatiotemporal fusion network, and its beneficial effects are as follows: (1) This invention employs a dual-channel spatiotemporal fusion network (STFNet) architecture. It efficiently extracts local spatial-temporal patterns through dilated convolution in the TCN channel, while simultaneously capturing the long-range anteroposterior dependencies of hip movement using the Bi-LSTM channel. Parallel fusion of the two channels achieves high-precision estimation of the hip joint's biomechanical torque. Online deployment on a publicly available dataset validates the torque estimation, showing an RMSE ≤ 0.15 Nm / kg and R0.05. 2 With an accuracy of ≥0.8, it significantly improves the estimation accuracy compared to a single TCN network or a single Bi-LSTM network, laying a reliable foundation for subsequent precise assisted control. (2) This invention introduces a motion classification and motion condition feature fusion module. By classifying various common daily actions (walking on flat ground, going up and down slopes, going up and down stairs, etc.), and splicing the motion category probability as condition information into the torque regression network, it solves the problem of decreased estimation accuracy caused by sharing a unified kinematic-torque mapping among multiple actions in the traditional end-to-end regression model, and significantly improves the generalization ability of torque estimation in multi-action scenarios. (3) This invention constructs a complete control link from bio-torque estimation to exoskeleton auxiliary torque output. Through three adjustable links—scaling, fixed delay, and low-pass filtering—the auxiliary torque output by the exoskeleton actively matches the biomechanical characteristics of the human hip joint in terms of amplitude and phase. Experimental verification shows that this relay control strategy can effectively increase the positive power output of the exoskeleton and reduce the wearer's metabolic energy consumption, achieving a good balance between assist effect and user comfort. Furthermore, the scaling factor can be adaptively adjusted according to the action type, further optimizing the assist efficiency under different working conditions. (4) The present invention relies only on the lumbar IMU and the dual-sided motor encoder as sensor inputs, without the need for additional sensors on the knee or ankle joints, electromyography signals, or plantar pressure sensors, which significantly reduces the physical burden on the wearer and the complexity of system deployment. Attached Figure Description

[0018] Figure 1 This is an overall flowchart of a method for predicting and assisting the control of hip exoskeleton torque based on a spatiotemporal fusion network according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the overall architecture of the STFNet network for a method of predicting and assisting control of hip exoskeleton torque based on a spatiotemporal fusion network according to an embodiment of the present invention. Figure 3 This is a schematic diagram of data preprocessing and sliding window feature extraction for a method of hip exoskeleton torque prediction and auxiliary control based on spatiotemporal fusion network according to an embodiment of the present invention. Figure 4 This is a schematic diagram of the TCN channel network structure of a method for predicting and assisting control of hip exoskeleton torque based on spatiotemporal fusion network according to an embodiment of the present invention. Figure 5 This is a schematic diagram of the Bi-LSTM channel network structure of a method for predicting and assisting the torque of a hip exoskeleton based on a spatiotemporal fusion network according to an embodiment of the present invention. Figure 6 This is a schematic diagram of the motion classification and motion condition feature fusion module structure of a hip joint exoskeleton torque prediction and auxiliary control method based on a spatiotemporal fusion network according to an embodiment of the present invention. Figure 7 This is a flowchart of the relay control layer processing of a method for predicting and assisting the torque of a hip exoskeleton based on a spatiotemporal fusion network according to an embodiment of the present invention. Figure 8This is a schematic diagram comparing the torque estimation results and the actual torque of the STFNet model of a method for predicting and assisting control of hip exoskeleton torque based on spatiotemporal fusion network under different action types according to an embodiment of the present invention; wherein, (a) is a time series comparison diagram of the actual torque and the predicted torque under each action type, and (b) is the residual between the actual torque and the predicted torque under each action type. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments and the accompanying drawings. It should be understood that these descriptions are merely exemplary and not intended to limit the scope of the invention. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0020] Example 1, such as Figure 1 As shown in the figure, an embodiment of the present invention provides a method for predicting and assisting the control of hip exoskeleton torque based on a spatiotemporal fusion network, comprising the following steps: 1. Sensor data acquisition: Acquire sensor data from the hip exoskeleton. The sensors include a first inertial measurement unit installed in the lower back or pelvis and dual motor encoders in the hip joint drive unit. The first inertial measurement unit is used to collect triaxial acceleration and triaxial angular velocity data of the lower back; the dual-side motor encoders are used to measure the angle and angular velocity of the hip joint. All sensor data are sampled at a frequency of 200Hz.

[0021] 2. Feature Extraction: A time-series sliding window is constructed to extract and standardize features from sensor data, forming an input feature matrix; specifically: The raw time-series data acquired by the sensors is divided into sliding window samples using a fixed-length time window T (T=40 sampling points, corresponding to 200ms of historical data). Each window contains: bilateral hip joint angles, bilateral hip joint angular velocities, triaxial acceleration of the lumbar and back IMU, and triaxial angular velocity. The data channels of each sensor are arranged in chronological order within the window, forming an input feature matrix with a dimension of T×10.

[0022] 3. Dual-channel feature extraction: Construct a dual-channel spatiotemporal fusion network STFNet, including a TCN channel, a Bi-LSTM channel, and a feature fusion layer. Input the input feature matrix into the TCN channel and the Bi-LSTM channel respectively to extract temporal features. Then, concatenate the feature vectors output by the TCN channel and the Bi-LSTM channel through the feature fusion layer to obtain the first fused feature vector.

[0023] The TCN channel employs multi-layer dilated convolution to extract local spatial features and temporal dependencies from the input data. It consists of three residual blocks, each containing two dilated convolutional layers with increasing dilation rates (2, 4, and 8 respectively). The kernel size is 3×3. Within each residual block, the first layer has 96 kernels and the second layer has 48 kernels. The output dimension is then reduced by a global average pooling layer and a Flatten layer, and finally mapped to 256 dimensions by a fully connected layer.

[0024] The Bi-LSTM channel uses a bidirectional long short-term memory network to capture the long-range forward and backward dependencies of time-series data, and then connects to a Bi-LSTM layer (with 64 hidden units). Only the forward and backward merged features of the last time step are output, resulting in a 128-dimensional feature vector.

[0025] The feature fusion layer concatenates the 256-dimensional feature vector output from the TCN channel and the 128-dimensional feature vector output from the Bi-LSTM channel to obtain a first fused feature vector with a dimension of 384.

[0026] The first fused feature vector is obtained by concatenating the feature vectors output from the TCN channel and the Bi-LSTM channel through a feature fusion layer, and then regularized by a Dropout layer with a dropout rate of 0.2.

[0027] 4. Action Conditional Feature Fusion: Construct an action classification network and an action conditional feature fusion module. The action classification network adopts a two-stage approach of independent training: first, the action classification network is trained separately, and then the action probability vector output by the trained classification network is used as the conditional input to the torque regression network.

[0028] The first fused feature vector (384 dimensions) is passed sequentially through two fully connected layers (128 and 64 neurons, ReLU activation function), and then through a Softmax layer to output action probability vectors for six types of actions. The six types of movements are: walking on flat ground (LG), going uphill (RA), going downhill (RD), going up stairs (SA), going down stairs (SD), and running (RUN).

[0029] The first fused feature vector is input into a fully connected layer (64 neurons, ReLU activation function) to extract a shared feature vector. This shared feature vector is then combined with the action probability vector. (6 dimensions) are concatenated to obtain the second fused feature vector (70 dimensions).

[0030] S5. Torque Regression: The second fused feature vector is input into the hip joint torque regression network to obtain the estimated hip joint bio-torque at the current moment. The hip joint torque regression network adopts a dual-path structure, including a dynamic torque prediction path, a motion bias path, and a torque synthesis layer, specifically: The dynamic torque prediction path inputs the first fused feature vector (384 dimensions) into a fully connected layer (64 neurons, ReLU activation function), extracts 64-dimensional shared features, and then combines the 64-dimensional shared feature vector with the action probability vector. (6-dimensional) splicing is performed to obtain 70-dimensional fusion features, which are then passed through two fully connected layers (with 64 neurons and 64 neurons respectively, and ReLU activation function for both) to output dynamic torque components; Action bias path, which converts the action probability vector Input a fully connected layer without bias (Dense(1,use_bias=False)) and output action bias torque components to provide independent baseline torque biases for different actions; The torque synthesis layer adds the dynamic torque components and the motion bias torque bias through an additive layer to obtain an estimate of the hip joint bio-torque. The unit is Nm / kg.

[0031] S6. Model Training: The model was trained using the multi-motion hip exoskeleton dataset (Molinaro et al., Science Robotics 2024) released by Georgia Institute of Technology. This dataset contains time-series sensor data and corresponding time-synchronized inverse dynamic hip joint torque labels for 34 participants in six motion categories (LG, RUN, RA, RD, SA, SD). Training employed a leave-one-out cross-validation protocol. The regression loss function was the mean squared error (MSE) between the predicted and actual torques. The classification loss function was sparse cross-entropy. The optimizer was Adam, with an initial learning rate of 0.001. A learning rate decay strategy (ReduceLROnPlateau, patience=5, factor=0.5) was implemented, halving the learning rate if the validation loss did not decrease for five consecutive epochs.

[0032] S7. Auxiliary Torque Generation: Deploy the STFNet model in the embedded controller of the hip exoskeleton for real-time operation: online inference, relay control, and torque output.

[0033] Online inference: Real-time sensor data is acquired, processed through a sliding window, and then input into the STFNet model to output the estimated hip joint bio-torque at the current moment. .

[0034] Relay control: Based on hip joint biotorque estimation The command torque is converted into an exoskeleton through relay control. The relay control layer includes three processing steps: scaling: scaling M est Multiply by scaling factor ( The value range is 0.1 to 0.4, preferably... =0.2), to obtain the scaled torque signal; Delay: Apply a fixed delay to the scaled torque signal. ( The range of values ​​is Preferred The delayed torque signal is obtained by measuring the torque output in milliseconds (ms). Low-pass filtering: A second-order Butterworth low-pass filter (cutoff frequency 10Hz) is used to filter the delayed torque signal, removing high-frequency noise and estimation jitter. The delay stage aims to shift the phase of the exoskeleton's output torque relative to the estimated bio-torque, maximizing the exoskeleton's positive power output and reducing user metabolic costs. The scaling factor in the relay control layer is adaptively adjusted based on user preferences or action type, including higher scaling factors for high-torque-demand actions such as climbing hills and stairs, and lower scaling factors for low-torque-demand actions such as walking on flat ground.

[0035] Torque output: outputs the command torque. The motor controller, which sends the command to the hip joint drive unit, causes the exoskeleton to provide auxiliary torque to the wearer's hip joint according to the commanded torque.

[0036] The method of this invention also includes a model lightweighting step, specifically: Before model deployment, the trained STFNet model is compressed using mixed-precision quantization and channel pruning techniques to generate a lightweight version. The TCN and Bi-LSTM channels are quantized using 8-bit and 4-bit mixed precision, respectively. Interpretive analysis methods are used to evaluate the contribution of feature channels, and redundant feature channels are pruned or fused.

[0037] Example 2: STFNet Model Construction and Training 1. Data Acquisition and Preprocessing: like Figure 3 As shown, sensor data is collected first. The sensor configuration is as follows: One IMU for the lower back: installed in the lower back (posterior pelvis or lower back), outputting three-axis acceleration, three-axis angular velocity and Euler angle; Two motor encoders are installed in the left and right hip joint drive units to output hip joint angle and angular velocity.

[0038] All sensors were sampled at a frequency of 200Hz, and temporal features were extracted using the sliding window method. The sliding window length was specified. Number of sampling points (corresponding to 200ms of historical data), window stride Each sampling point. The integrated sensor data channels within each window include: left hip angle. Left hip angular velocity Right hip angle Right hip angular velocity , lumbar and back IMU acceleration a x a y a z Angular velocity g of IMU in the lower back x g y g z The channels are arranged in chronological order within the window, forming... The input feature matrix X is dimensional.

[0039] Z-score normalization is applied to the input feature matrix: ,in and These are the mean and standard deviation of each channel in the training set, respectively.

[0040] 2. STFNet Network Construction: like Figure 2 As shown, STFNet contains two parallel branches and an action condition feature fusion module.

[0041] 2.1, TCN channel, such as Figure 4 As shown; The TCN channel uses dilated convolution to extract local spatial-temporal patterns. Input feature X norm (Dimensionality is T×10, where T=40 is the sliding window length and 10 is the sensor feature dimension.) First, the channel dimension is increased through a Lambda layer, then sequentially through three residual blocks. The dilation rate of each residual block increases progressively, at 2, 4, and 8 respectively. Each residual block contains two Conv2D dilated convolutional layers (each with a 3×3 kernel). The number of filters is set as follows: 96 kernels in the first layer and 48 kernels in the second layer. A BatchNormalization layer is applied after each residual block for normalization. After dimensionality reduction through a global average pooling layer and a Flatten layer, the TCN channels are then multiplied by a fully connected layer to increase the output dimension F. TCN , mapped to 256 dimensions.

[0042] 2.2, Bi-LSTM channels, such as Figure 5 As shown; Bi-LSTM channels are used to capture bidirectional long-range dependencies in time-series data. Input features (Xnorm) are first processed through BatchNormalization and Dropout (0.1), then directly fed into a Bi-LSTM layer (64 hidden units, return_sequences=False), outputting a 128-dimensional bidirectional fused feature. The Bi-LSTM layer is followed by a Dropout layer (dropout rate 0.2) for regularization, and then a Flatten layer flattens the features into a one-dimensional vector (FBiLSTM).

[0043] 2.3, Action classification and action condition feature fusion module, such as... Figure 6 As shown; Using sensor features extracted through a sliding window as input, an action classification network is constructed. The feature extraction part of this classification network is consistent with the TCN and Bi-LSTM channel structures in steps 2.1 and 2.2. The extracted fusion features (384 dimensions) are sequentially passed through two fully connected layers (128 and 64 neurons, ReLU activation function, dropout rate 0.2), and finally output as probability vectors for six action categories via a softmax layer. There are six action types: LG (0), RA (1), RD (2), SA (3), SD (4), and RUN (5).

[0044] The action classification network is trained independently, and its parameters are fixed after training. It is used to output action probability vectors during the inference phase. The action conditional feature fusion module takes the first fused feature vector and the action probability vector as input: the first fused feature vector is passed through a fully connected layer (64 neurons, ReLU activation function) to extract a shared feature vector. The shared feature vector is concatenated with the action probability vector to obtain the second fused feature vector, which is then input into the torque regression network.

[0045] 2.4 Torque Regression Output; The second fused feature vector (70 dimensions) is input into the hip joint torque regression network. The torque regression network employs a dual-path structure: Dynamic torque prediction path: The second fused feature vector is passed through two fully connected layers (64 neurons each, ReLU activation function) in sequence, and then through the output layer (1 neuron, linear activation function) to output the dynamic torque components. Action bias path: Input the action probability vector into an unbiased fully connected layer (Dense(1, use)). bias =False)), outputs the motion bias torque component, which is used to provide independent baseline torque bias for different motions; Torque synthesis layer: The dynamic torque component and the motion bias torque component are added together through an additive layer to obtain the estimated value of the hip joint bio-torque M at the current moment. est The unit is Nm / kg.

[0046] 3. Model Training: The model was trained using a publicly available multi-motion dataset of a hip exoskeleton. The dataset contains sensor data from 34 participants across six motion categories, along with corresponding inverse dynamic hip joint torque labels. A leave-one-out cross-validation protocol was employed. The model loss function was the mean squared error of torque prediction. ; in, For the actual hip joint torque, This represents the number of samples in the batch.

[0047] The optimizer uses Adam, with an initial learning rate of Add the ReduceLROnPlateau strategy (patience=5, factor=0.5). Training batch size. Maximum number of iterations During training, an early stopping strategy (patience=10) is incorporated to automatically retain the optimal model weights.

[0048] Validation on public datasets shows that, Figure 8 As shown, the torque estimation curves of the STFNet model under various motion types are in high agreement with the actual torque curves. The STFNet model achieves a hip joint torque estimation RMSE ≤ 0.15 Nm / kg, R 2 ≥0.8.

[0049] Example 3: Online generation of exoskeleton-assisted torque; 1. Model Deployment: The trained STFNet model is deployed on an embedded controller of a hip exoskeleton (such as a Raspberry Pi4B + NVIDIA Jetson Nano). Before deployment, the model can be lightweighted: mixed-precision quantization is used to quantize the TCN and Bi-LSTM channels to 8-bit / 4-bit, and interpretability analysis is used to evaluate the contribution of feature channels before channel pruning or fusion. The model is then compressed to run in real time on the embedded platform (inference latency <20ms).

[0050] 2. Online Inference and Torque Generation: During online operation, the data acquisition module collects sensor data from the lumbar IMU and the motor encoders at both thighs at a frequency of 200Hz. The sliding window moves forward in 10-sampling-point steps, updating the input feature matrix in real time. The STFNet model outputs a hip joint bio-torque estimate M at each control cycle (55–200Hz). est .

[0051] 3. Relay Control Layer: like Figure 7 As shown, M est The command torque M is converted into an exoskeleton command through the relay control layer. cmd : (1) Scaling: M scaled = ×M est Scaling factor Used to control the assist force. Different scaling factors can be used for different motion types: for high-torque-requirement motions such as going uphill or climbing stairs, a scaling factor can be selected. =0.3~0.4, for actions requiring low torque, such as walking on flat ground or going downhill. =0.15~0.2, for fast-paced activities such as running. =0.1~0.15.

[0052] (2) Delay: M delayed =delay(M scaled , ).Delay This is used to adjust the phase of the hip joint auxiliary torque relative to the estimated torque, so that the peak auxiliary torque is better aligned with the peak range of the hip joint's positive power, thereby increasing the exoskeleton's positive power output and reducing the user's metabolic costs. For walking on flat ground, the delay... The optimal delay is 100–125 ms. For high-load actions such as uphill movements, the delay is [not specified]. It can be appropriately shortened to 50-75ms.

[0053] (3) Low-pass filter: M cmd =lowpass(M delayed f c A second-order Butterworth low-pass filter is used, with a cutoff frequency f. c =10Hz, used to smooth the command torque curve, filter out high-frequency estimation jitter and noise, and improve control stability. The filtering stage introduces an additional delay of approximately 25ms, which needs to be included in the total delay calculation.

[0054] 4. Torque output: Command torque M cmd The motor controller of the hip joint drive unit sends the auxiliary torque to the drive motor via a current loop.

[0055] 5. Cross-motion adaptive adjustment: When the wearer's action type changes, the action classification module automatically detects the new action and outputs the corresponding action probability vector. The action conditional feature fusion module then dynamically adjusts the feature weights, resulting in a smooth transition in the torque estimation output. Simultaneously, the scaling factor of the relay control layer... It can switch according to the action type to achieve adaptive assistance across actions.

[0056] Example 4: Lightweight Model and Embedded Deployment; This embodiment describes a lightweight method for the STFNet model.

[0057] 1. Mixed precision quantization: Mixed-precision quantization is used for the TCN-channel dilated convolutional layers and the Bi-LSTM-channel Bi-LSTM layers of the STFNet model. Specifically, 8-bit fixed-point representations are retained for precision-sensitive parts such as the input layer, output layer, and residual block skip connections, while 4-bit low-precision representations are used for intermediate feature mapping layers and auxiliary convolutional layers. Quantization-aware training (QAT) is employed during quantization to simulate quantization errors and ensure that the model's accuracy loss after quantization is less than 2%.

[0058] 2. Redundant channel pruning: A gradient contribution analysis-based method was used to evaluate the marginal contribution of each input feature channel to the torque prediction results. Experiments showed that the triaxial data from the lumbar IMU contributed the most to the hip joint torque estimation, followed by the encoders from the motors on both thighs, which contributed moderately. Redundant feature channels with a contribution of less than 2% were pruned or subjected to motion condition feature fusion to further reduce the computational burden during model inference.

[0059] After being lightweighted, the STFNet model parameter size is reduced by about 40%, and the inference latency is reduced by about 35%. It can run in real time on the NVIDIA Jetson Nano platform at a frequency of about 100 to 200 Hz, meeting the real-time requirements of the exoskeleton control system.

[0060] Therefore, this invention adopts the aforementioned method for hip exoskeleton torque prediction and auxiliary control based on a spatiotemporal fusion network. 1) It employs a dual-channel spatiotemporal fusion network (STFNet) architecture, efficiently extracting local spatial-temporal patterns through dilated convolution of the TCN channel, while simultaneously capturing the long-range anteroposterior dependency of hip motion using the Bi-LSTM channel. Parallel fusion of the two channels achieves high-precision estimation of hip joint bio-torque. Online deployment and verification on a publicly available dataset show that the torque estimation RMSE of this invention is ≤0.15 Nm / kg, R... 2≥0.8, which significantly improves the estimation accuracy compared to a single TCN network or a single Bi-LSTM network, laying a reliable foundation for subsequent precise assisted control; 2) This invention introduces a motion classification and motion condition feature fusion module, which classifies various common daily actions (walking on flat ground, going up and down slopes, going up and down stairs, etc.) and injects the motion category probability as conditional information into the torque regression network, solving the problem of decreased estimation accuracy caused by sharing a unified kinematic-torque mapping among multiple actions in traditional end-to-end regression models, and significantly improving the generalization ability of torque estimation in multi-action scenarios; 3) This invention constructs a complete control link from bio-torque estimation to exoskeleton auxiliary torque output. Through three adjustable links of scaling, fixed delay and low-pass filtering, the auxiliary torque output by the exoskeleton actively matches the biomechanical characteristics of the human hip joint in terms of amplitude and phase. Experimental verification shows that the relay control strategy can effectively increase the positive power output of the exoskeleton and reduce the wearer's metabolic energy consumption, achieving a good balance between assist effect and user comfort. Moreover, the scaling factor can be adaptively adjusted according to the type of action, further optimizing the assist efficiency under different working conditions. 4) This invention only relies on the lumbar and back IMU and the dual-side motor encoder as sensor inputs, without the need for additional sensors on the knee or ankle joints, electromyographic signals, or plantar pressure sensors, which significantly reduces the wearer's physical burden and the complexity of system deployment.

[0061] It is worth noting that all contents not described in detail in this invention are existing technologies and are well known to those skilled in the art.

[0062] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for predicting and assisting the control of hip exoskeleton torques based on spatiotemporal fusion networks, characterized in that, Includes the following steps: S1. Data Acquisition: Acquire sensor data from the hip exoskeleton. The sensors include a first inertial measurement unit installed in the lower back or pelvis and dual motor encoders in the hip joint drive unit. S2. Feature Extraction: Construct a time-series sliding window to extract and standardize features from the sensor data, forming an input feature matrix; S3. Dual-channel feature extraction: Construct a dual-channel spatiotemporal fusion network STFNet, including a TCN channel, a Bi-LSTM channel and a feature fusion layer. Input the input feature matrix into the TCN channel and the Bi-LSTM channel respectively to extract temporal features. Then, concatenate the feature vectors output by the TCN channel and the Bi-LSTM channel through the feature fusion layer to obtain the first fused feature vector. S4. Action condition feature fusion: The first fused feature vector is processed by an independently trained action classification network to output an action probability vector; the action probability vector is concatenated with the shared feature vector extracted from the first fused feature vector through a fully connected layer to obtain a second fused feature vector; S5. Torque Regression: Input the second fused feature vector into the hip joint torque regression network to obtain the estimated value of the hip joint bio-torque at the current moment; S6. Auxiliary Torque Generation: Based on the estimated value of the hip joint bio-torque, the command torque of the exoskeleton is generated and output through the relay control layer. The command torque is sent to the motor controller of the hip joint drive unit so that the exoskeleton provides auxiliary torque according to the command torque.

2. The method for predicting and assisting control of hip exoskeleton torque based on spatiotemporal fusion network according to claim 1, characterized in that, In step S3, the TCN channel uses multi-layer dilated convolution to extract the local spatial features and temporal dependencies of the input data. It sequentially contains three residual blocks, each containing two dilated convolutional layers with increasing dilation rates. The layers are then dimensionality-reduced by a global average pooling layer and a Flatten layer, and finally the output dimension is mapped to a specified dimension by a fully connected layer. Within each residual block, the first layer has 96 convolutional kernels and the second layer has 48 convolutional kernels. The Bi-LSTM channel uses a bidirectional long short-term memory network to capture the long-range forward and backward dependencies of time-series data, and then connects to the Bi-LSTM layer to output the forward and backward merged features of the last time step.

3. The method for predicting and assisting control of hip exoskeleton torque based on spatiotemporal fusion network according to claim 1, characterized in that, In step S3, the first fused feature vector obtained by concatenating the feature vectors output by the TCN channel and the Bi-LSTM channel through the feature fusion layer is regularized by the Dropout layer; the dropout rate is 0.

1.

4. The method for predicting and assisting control of hip exoskeleton torque based on spatiotemporal fusion network according to claim 1, characterized in that, In step S4, the action condition feature fusion specifically includes: The first fused feature vector is passed through an action classification network to output action probability vectors for six types of actions: walking on flat ground (LG), going uphill (RA), going downhill (RD), going up stairs (SA), going down stairs (SD), and running (RUN). The first fused feature vector is input into the fully connected layer to extract the shared feature vector; The shared feature vector is concatenated with the action probability vector to obtain the second fused feature vector.

5. The method for predicting and assisting control of hip exoskeleton torque based on spatiotemporal fusion network according to claim 1, characterized in that, In step S5, the hip joint torque regression network adopts a dual-path structure, including a dynamic torque prediction path, a motion bias path, and a torque synthesis layer, specifically: The dynamic torque prediction path inputs the first fused feature vector into a fully connected layer to extract shared features. The shared feature vector is then concatenated with the action probability vector to obtain fused features. These features are then passed through two fully connected layers to output the dynamic torque components. The action bias path takes the action probability vector as input to an unbiased fully connected layer and outputs action bias torque components, which are used to provide independent baseline torque bias for different actions. The torque synthesis layer adds the dynamic torque component and the action bias torque bias through an additive layer to obtain the estimated value of the hip joint bio-torque.

6. The method for predicting and assisting control of hip exoskeleton torque based on spatiotemporal fusion network according to claim 1, characterized in that, In step S6, the relay control layer includes: The scaling unit is used to multiply the estimated value of the hip joint bio-torque by a scaling factor to obtain a scaled torque signal. The delay unit is used to apply a fixed delay to the scaled torque signal to obtain the delayed torque signal. A low-pass filter unit is used to perform low-pass filtering on the delayed torque signal to obtain the command torque.

7. The method for predicting and assisting control of hip exoskeleton torque based on spatiotemporal fusion network according to claim 6, characterized in that, The scaling factor ranges from 0.1 to 0.4; the fixed delay ranges from 50 to 150 milliseconds; the low-pass filter unit uses a second-order Butterworth low-pass filter with a cutoff frequency of 10 Hz. The scaling factor in the relay control layer is adaptively adjusted according to user preferences or action type. For high-torque-requirement actions such as going uphill or climbing stairs, a scaling factor of 0.3 to 0.4 is used; for low-torque-requirement actions such as walking on flat ground, a scaling factor of 0.15 to 0.2 is used; and for fast movements such as running, a scaling factor of 0.1 to 0.15 is used.

8. The method for predicting and assisting control of hip exoskeleton torque based on spatiotemporal fusion network according to any one of claims 1 to 7, characterized in that, It also includes a model lightweighting step, specifically: Before model deployment, the trained STFNet model is compressed using mixed precision quantization and channel pruning techniques to generate a lightweight version of the model.

9. The method for predicting and assisting control of hip exoskeleton torque based on spatiotemporal fusion network according to claim 8, characterized in that, The TCN and Bi-LSTM channels of the model are quantized using 8-bit and 4-bit mixed precision, respectively; the contribution of feature channels is evaluated using interpretability analysis methods, and redundant feature channels are pruned or fused.

Images