A method, system, terminal, and storage medium for hip exoskeleton control based on gait pattern prediction and gait phase estimation.

Abstract

The application relates to the technical field of intelligent control, and discloses a hip exoskeleton control method and system based on gait pattern prediction and gait phase estimation, a terminal and a storage medium, the method comprising the following steps: constructing an initial unsupervised terrain detection model and training the model to obtain a target unsupervised terrain detection model; inputting an environment binary image into the target unsupervised terrain detection model to obtain a terrain detection result, and obtaining a gait pattern prediction result based on a majority voting strategy; obtaining a target gait phase estimation based on an adaptive oscillator system according to a hip joint angular velocity; obtaining a net auxiliary torque based on an expected auxiliary torque curve, a gait pattern prediction result and the target gait phase estimation, and switching a hip exoskeleton assistance mode according to the net auxiliary torque. The unsupervised terrain detection model avoids labeling a large amount of data, the terrain detection is combined with the exoskeleton control, and the terrain self-adaptive walking capability of the exoskeleton is enhanced.

Description

Technical Field

[0001] This invention relates to the field of intelligent control technology, and in particular to a method, system, terminal, and computer-readable storage medium for controlling a hip exoskeleton based on gait pattern prediction and gait phase estimation. Background Technology

[0002] In recent years, lower limb exoskeletons have received widespread attention for their role in assisting human movement. Unlike steady-state walking in laboratory or fixed application scenarios, walking in daily life involves different time-state patterns corresponding to various terrains and the switching between these different time-state patterns. Therefore, terrain adaptive control is crucial for the practical application of lower limb exoskeletons in daily life. Inspired by humans using vision to guide their movement, researchers have employed visual sensors and deep learning models to capture and detect approaching terrain.

[0003] However, these vision sensor- and deep learning-based methods primarily focus on accurately detecting the terrain in front of the user and have not yet been successfully integrated with lower limb exoskeleton control, failing to provide adequate assistance in multimodal movement across various terrains (especially during gait pattern transitions). Furthermore, supervised learning-based terrain detection models require significant human and financial resources for data annotation to ensure a sufficient number of real-world labeled samples for model training. These limitations hinder the widespread application of lower limb exoskeletons in assisting daily human walking.

[0004] Therefore, existing technologies still need to be improved and developed. Summary of the Invention

[0005] The main objective of this invention is to provide a hip exoskeleton control method, system, terminal, and computer-readable storage medium based on gait pattern prediction and gait phase estimation. This invention aims to address the problems of existing lower limb exoskeletons, such as the time-consuming and laborious nature of terrain detection supervised learning data annotation and the lack of integration of terrain detection with lower limb exoskeleton control, resulting in insufficient terrain-adaptive walking ability of the lower limb exoskeleton.

[0006] To achieve the above-mentioned objectives, this invention provides a hip exoskeleton control method based on gait pattern prediction and gait phase estimation, wherein the hip exoskeleton control method based on gait pattern prediction and gait phase estimation includes:

[0007] An initial unsupervised terrain detection model is constructed, and the initial unsupervised terrain detection model is trained and tested using a dataset to obtain the target unsupervised terrain detection model;

[0008] Acquire 3D environment point cloud data and preprocess the 3D environment point cloud data to obtain a binary environment image;

[0009] The binary environmental image is input into the unsupervised target terrain detection model, which outputs terrain detection results and smooths the results to obtain gait pattern prediction results.

[0010] The hip joint angular velocity is obtained, and the hip joint angular velocity is used as the teaching signal of the adaptive oscillator system to obtain an initial gait phase estimate. The initial gait phase estimate is then corrected to obtain a target gait phase estimate.

[0011] The net assist torque is calculated based on the gait pattern prediction results and the target gait phase estimation, and the assist mode of the hip exoskeleton is switched according to the net assist torque.

[0012] Optionally, the step of constructing an initial unsupervised terrain detection model, and training and testing the initial unsupervised terrain detection model using a dataset to obtain a target unsupervised terrain detection model, specifically includes:

[0013] Construct an initial unsupervised terrain detection model, wherein the initial unsupervised terrain detection model includes a feature generator, a first classifier, and a second classifier;

[0014] Acquire unlabeled real environment point cloud data and labeled simulated environment point cloud data, and preprocess the real environment point cloud data and the simulated environment point cloud data to obtain a target domain dataset and a source domain dataset. The target domain dataset includes a real environment binary image, and the source domain dataset includes a simulated environment binary image and labels.

[0015] Randomly select a first predetermined proportion of target domain data from the target domain dataset for manual annotation to obtain a validation set, and combine the second predetermined proportion of target domain data from the target domain dataset with the source domain dataset to form a training set, wherein the validation set and the training set constitute the dataset;

[0016] The feature generator, the first classifier, and the second classifier are trained using the source domain dataset in the training set, and a first objective function is constructed with the goal of minimizing the softmax cross-entropy loss on the source domain data.

[0017] ;

[0018] Where A represents the first objective function, This indicates taking the minimum value. Represents a feature generator. This represents the first classifier. This represents the second classifier. For the expectation operator, Represents the source domain dataset. This represents a set of binary images representing the simulation environment. Represents a set of labeled elements. This represents the source domain data in the source domain dataset. Represents a binary image of the simulation environment. Indicates labeling, Indicates the total number of terrain categories. It is a binary operator. Indicates the output terrain type. Indicates that given input Time output For category The probability of;

[0019] With the parameters of the feature generator fixed, the first classifier and the second classifier are trained using the training set to construct a second objective function that maximizes the output difference between the first classifier and the second classifier on the target domain data.

[0020] ;

[0021] in, This represents the second objective function. Represents the target domain dataset. This represents the target domain data in the target domain dataset. This indicates that the first classifier performs well on a given input. Time output For category The probability, This indicates that the second classifier performs well on a given input. Time output For category The probability of;

[0022] With the parameters of the first and second classifiers fixed, the feature generator is trained using the target domain data in the training set to construct a third objective function that minimizes the classification difference between the first and second classifiers on the target domain data.

[0023] ;

[0024] in, Indicate the third objective function;

[0025] The performance of the initial unsupervised terrain detection model after training is tested using the validation set. After the test is completed, the initial unsupervised terrain detection model is trained again until the maximum number of training rounds is reached to obtain the target unsupervised terrain detection model.

[0026] Optionally, the step of acquiring three-dimensional environment point cloud data and preprocessing the three-dimensional environment point cloud data to obtain a binary environment image specifically includes:

[0027] Acquire 3D environmental point cloud data from an RGB-D camera mounted on the waist of the human body;

[0028] The 3D environment point cloud data is transformed from the camera coordinate system to the ground coordinate system using a rotation matrix to obtain the 3D environment point cloud data in the ground coordinate system.

[0029] The dimensionality of the three-dimensional environmental point cloud data in the ground coordinate system is reduced to obtain two-dimensional environmental point cloud data.

[0030] The two-dimensional environmental point cloud data is converted into a binary environmental image using a point cloud rasterization method.

[0031] Optionally, the step of inputting the binary environmental image into the unsupervised target terrain detection model, the unsupervised target terrain detection model outputting terrain detection results, and smoothing the terrain detection results to obtain gait pattern prediction results specifically includes:

[0032] The binary environmental image is input into the unsupervised target terrain detection model. The first classifier of the unsupervised target terrain detection model outputs the first classification score corresponding to each terrain category, and the second classifier of the unsupervised target terrain detection model outputs the second classification score corresponding to each terrain category.

[0033] By fusing the first classification score and the second classification score corresponding to various terrain categories, the target classification score corresponding to each terrain category is obtained. The terrain category corresponding to the maximum value among the multiple target classification scores is taken as the terrain detection result output by the unsupervised terrain detection model.

[0034] Based on the majority voting strategy, the terrain detection results are smoothed by a sliding window. The terrain category that appears most frequently within a set window length is taken as the target terrain detection result, and the gait pattern corresponding to the target terrain detection result is taken as the gait pattern prediction result.

[0035] The terrain categories include flat land, uphill, downhill, up stairs, and down stairs.

[0036] Optionally, the step of acquiring the hip joint angular velocity, using the hip joint angular velocity as a teaching signal for the adaptive oscillator system to obtain an initial gait phase estimate, and correcting the initial gait phase estimate to obtain a target gait phase estimate specifically includes:

[0037] Acquire the hip joint angular velocity collected by the inertial measurement unit installed on the human leg;

[0038] Based on the difference between the hip joint angular velocity and the reconstructed angular velocity reconstructed through finite-term Fourier decomposition, the state variables of the adaptive oscillator system are updated in real time:

[0039] ;

[0040] ;

[0041] ;

[0042] ;

[0043] ;

[0044] ;

[0045] in, Represents hip joint angular velocity With reconstructed angular velocity The difference, Indicates time step, and These represent the phase and natural frequency of the oscillator in the adaptive oscillator system, respectively. Indicates the rate of change of phase. Represents the rate of change of frequency. This represents the adjustable coefficients that determine the phase and frequency learning speed. Indicates the first The first Fourier coefficient corresponding to the subharmonic Learning update rate Indicates the first The second Fourier coefficient corresponding to the subharmonic Learning update rate This represents the adjustable coefficient that determines the learning speed of the first and second Fourier coefficients. Indicates the maximum harmonic order. This indicates the reconstructed angular velocity value;

[0046] Based on the rate of change of the phase, the phase signal that varies with time is obtained through numerical iteration. The time-varying phase signal As an initial gait phase estimate;

[0047] The initial gait phase estimate is normalized to [0, 1], corresponding to 0–100% of the gait cycle, to obtain the intermediate gait phase estimate:

[0048] ;

[0049] in, This indicates the intermediate gait phase estimation. Indicates normalization;

[0050] The hip joint angle is acquired by an inertial measurement unit installed on the human leg. Maximum hip flexion is detected based on this hip joint angle. At the moment of detection of maximum hip flexion, the deviation between the intermediate gait phase estimate and a reference value is calculated.

[0051] ;

[0052] in, This indicates the moment when maximum hip flexion was detected. Indicates in The deviation between the estimated gait phase at time intervals and the reference value. Indicates in Gait phase estimation at intermediate time points;

[0053] Based on the deviation between the intermediate gait phase estimate and the reference value, the phase offset correction term is updated using an exponential decay control method in the form of a first-order derivative:

[0054] ;

[0055] in, This indicates the phase offset correction term. This represents the rate of change of the phase offset correction term. Indicates proportional gain. This represents a frequency signal that changes over time.

[0056] Subtract the phase offset correction term from the intermediate gait phase estimate and normalize it to obtain the target gait phase estimate:

[0057] ;

[0058] in, This represents the target gait phase estimation.

[0059] Optionally, the step of calculating the net assist torque based on the gait pattern prediction result and the target gait phase estimation, and switching the assist mode of the hip exoskeleton based on the net assist torque, specifically includes:

[0060] Based on different gait phases, the expected auxiliary torque curves for hip extension and flexion are constructed using the hyperbolic tangent function:

[0061] ;

[0062] in, This represents the desired auxiliary torque curve. Indicates gait phase, Indicates the initial phase of the assist. Indicates peak torque. Represents the hyperbolic tangent function. A positive coefficient indicating the steepness of the adjustment curve. Indicates the predefined duration of the boost;

[0063] Multiple assist parameters corresponding to different gait patterns were obtained through pre-trial walking experiments on subjects. Among them, the multiple assist parameters corresponding to different gait patterns include the assist initiation phase of hip extension. Peak torque of hip extension Assisted starting phase of hip flexion Peak torque of hip flexion ;

[0064] The moment of maximum hip flexion is taken as the starting point of the gait cycle. Within the gait cycle, based on the expected assist torque curves of hip extension and flexion, the net assist torque is calculated according to the assist parameters corresponding to the gait pattern prediction results and the target gait phase estimation.

[0065] ;

[0066] in, Indicates the net auxiliary torque. This represents the expected auxiliary torque for hip extension. This represents the desired auxiliary torque for hip flexion;

[0067] Using a finite state machine, the assist mode of the hip exoskeleton is switched according to the net assist torque.

[0068] Optionally, the process includes acquiring labeled point cloud data of the simulation environment, followed by:

[0069] Random size noise and geometric noise are added to the simulated environment point cloud data to obtain noisy simulated environment point cloud data;

[0070] The appropriate angle for the RGB-D camera to acquire three-dimensional environmental point cloud data when installed on the waist of a human body is obtained through visual pre-experiment. The simulated environmental point cloud data with added noise is then mapped to the virtual camera viewpoint corresponding to the appropriate angle to obtain the mapped simulated environmental point cloud data.

[0071] To achieve the above-mentioned objectives, the present invention also provides a hip exoskeleton control system based on gait pattern prediction and gait phase estimation, wherein the hip exoskeleton control system based on gait pattern prediction and gait phase estimation includes:

[0072] Terrain detection model training module: used to build an initial unsupervised terrain detection model, train and test the initial unsupervised terrain detection model using a dataset, and obtain the target unsupervised terrain detection model;

[0073] Visual perception module: used to acquire 3D environment point cloud data and preprocess the 3D environment point cloud data to obtain a binary image of the environment;

[0074] Gait pattern prediction module: used to input the binary image of the environment into the unsupervised terrain detection model of the target, the unsupervised terrain detection model of the target outputs the terrain detection result, and smooths the terrain detection result to obtain the gait pattern prediction result;

[0075] Gait phase estimation module: used to acquire hip joint angular velocity, use the hip joint angular velocity as a teaching signal for the adaptive oscillator system to obtain an initial gait phase estimate, and correct the initial gait phase estimate to obtain a target gait phase estimate;

[0076] Exoskeleton assist control module: used to calculate the net assist torque based on the gait pattern prediction result and the target gait phase estimation, and to switch the assist mode of the hip joint exoskeleton based on the net assist torque.

[0077] To achieve the above-mentioned objectives, the present invention also provides a terminal, the terminal comprising: a memory, a processor, and a hip exoskeleton control program based on gait pattern prediction and gait phase estimation stored in the memory and executable on the processor, wherein the hip exoskeleton control program based on gait pattern prediction and gait phase estimation, when executed by the processor, implements the steps of the hip exoskeleton control method based on gait pattern prediction and gait phase estimation as described above.

[0078] To achieve the above-mentioned objectives, the present invention also provides a computer-readable storage medium storing a hip exoskeleton control program based on gait pattern prediction and gait phase estimation. When the hip exoskeleton control program based on gait pattern prediction and gait phase estimation is executed by a processor, it implements the steps of the hip exoskeleton control method based on gait pattern prediction and gait phase estimation as described above.

[0079] In this invention, an initial unsupervised terrain detection model is constructed, and the initial unsupervised terrain detection model is trained and tested using a dataset to obtain a target unsupervised terrain detection model; three-dimensional environmental point cloud data is acquired and preprocessed to obtain a binary environmental image; the binary environmental image is input into the target unsupervised terrain detection model, which outputs terrain detection results, and the terrain detection results are smoothed to obtain gait pattern prediction results; hip joint angular velocity is acquired and used as a teaching signal for an adaptive oscillator system to obtain an initial gait phase estimate, and the initial gait phase estimate is corrected to obtain a target gait phase estimate; a net assist torque is calculated based on the gait pattern prediction results and the target gait phase estimate, and the assist mode of the hip exoskeleton is switched according to the net assist torque. This invention utilizes unsupervised adaptive terrain detection, i.e., gait pattern prediction, to detect changes in a user's gait pattern in advance while avoiding the need for extensive data labeling required for training traditional supervised learning models. Through gait phase estimation based on an adaptive oscillator system, it provides accurate gait phase estimation for multimodal movements on various terrains, and uses hip joint angular velocity as the teaching signal for the adaptive oscillator system, improving the adaptability of the lower limb exoskeleton to gait pattern changes. Based on the expected assist torque curves of hip joint extension and flexion, it combines terrain detection (gait pattern prediction) with lower limb exoskeleton control, enabling the exoskeleton to provide appropriate assistance during multimodal movements on various terrains, thus enhancing the terrain-adaptive walking capability of the lower limb exoskeleton. Attached Figure Description

[0080] Figure 1 This is a first flowchart of a preferred embodiment of the hip exoskeleton control method based on gait pattern prediction and gait phase estimation of the present invention;

[0081] Figure 2 This is a second flowchart of a preferred embodiment of the hip exoskeleton control method based on gait pattern prediction and gait phase estimation of the present invention;

[0082] Figure 3 This is a third flowchart of a preferred embodiment of the hip exoskeleton control method based on gait pattern prediction and gait phase estimation of the present invention;

[0083] Figure 4 This is a schematic diagram of the structure of the unsupervised terrain detection model of the present invention;

[0084] Figure 5 This is a structural diagram of a preferred embodiment of the hip exoskeleton control system based on gait pattern prediction and gait phase estimation of the present invention;

[0085] Figure 6This is a structural diagram of a preferred embodiment of the terminal of the present invention. Detailed Implementation

[0086] To make the objectives, technical solutions, and advantages of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0087] In recent years, lower limb exoskeletons have received widespread attention for their role in assisting human movement. Unlike steady-state walking in laboratory or fixed application scenarios, walking in daily life involves different time-mode patterns corresponding to various terrains (such as flat ground, going up and down slopes, and going up and down stairs) and the switching between these different time-mode patterns. Therefore, terrain-adaptive control is crucial for the practical application of lower limb exoskeletons in daily life.

[0088] Currently, most existing exoskeleton controls typically detect the user's gait pattern and use a finite state machine (FSM) to integrate different control algorithms to help the wearer walk on various terrains (such as walking from flat ground to climbing stairs). However, accurately and timely identifying the user's gait pattern remains a significant challenge. While methods using mechanical sensors, bioelectric sensors, or a combination thereof to detect changes in the user's gait pattern have high recognition accuracy (greater than 95%), they primarily focus on identifying movement patterns after the gait cycle begins, resulting in recognition delays. Furthermore, sensor placement, noise, and differences between subjects pose significant challenges to universality. This can lead to sudden changes in exoskeleton-assisted behavior during walking, causing gait instability and increasing the user's risk of falls.

[0089] Inspired by humans' use of vision to guide their movement, researchers have employed visual sensors to capture and detect approaching terrain, potentially enabling the control of lower limb exoskeletons / prosthetics across various terrains for multimodal motion. Currently, terrain detection typically uses supervised deep learning methods (such as convolutional neural networks) to classify generated images. Compared to motion pattern detection techniques relying on proprioceptive sensors, these methods generally exhibit higher robustness and versatility. However, these visual sensor-based and deep learning-based methods primarily focus on accurately detecting terrain in front of the user and have not yet been successfully integrated with lower limb exoskeleton control, failing to provide adequate assistance (i.e., appropriate assist torque timing and magnitude) during multimodal motion across various terrains, especially during gait pattern transitions. Furthermore, supervised learning-based terrain detection models require significant human and financial investment in data annotation to ensure a sufficient number of real-world labeled samples for model training. These limitations hinder the widespread application of lower limb exoskeletons in assisting daily human walking.

[0090] To address the aforementioned technical problems, this invention provides a hip exoskeleton control method based on gait pattern prediction and gait phase estimation. The method involves constructing an initial unsupervised terrain detection model, training and testing the initial unsupervised terrain detection model using a dataset to obtain a target unsupervised terrain detection model; acquiring 3D environmental point cloud data and preprocessing the 3D environmental point cloud data to obtain a binary environmental image; inputting the binary environmental image into the target unsupervised terrain detection model, which outputs terrain detection results, and smoothing the terrain detection results to obtain gait pattern prediction results; acquiring hip joint angular velocity, using the hip joint angular velocity as a teaching signal for an adaptive oscillator system to obtain an initial gait phase estimate, and correcting the initial gait phase estimate to obtain a target gait phase estimate; calculating a net assist torque based on the gait pattern prediction results and the target gait phase estimate, and switching the assist mode of the hip exoskeleton according to the net assist torque. This invention utilizes unsupervised adaptive terrain detection, i.e., gait pattern prediction, to detect changes in a user's gait pattern in advance while avoiding the need for extensive data labeling required for training traditional supervised learning models. Through gait phase estimation based on an adaptive oscillator system, it provides accurate gait phase estimation for multimodal movements, including gait pattern transitions, across various terrains. Using hip joint angular velocity as the teaching signal for the adaptive oscillator system improves the adaptability of the lower limb exoskeleton to gait pattern changes. By combining terrain detection (gait pattern prediction) with lower limb exoskeleton control based on the expected assist torque curves of hip extension and flexion, the lower limb exoskeleton can provide appropriate assistance for multimodal movements, including gait pattern transitions, across various terrains, enhancing its terrain-adaptive walking capability.

[0091] The application content will be further explained below with reference to the accompanying drawings and the description of the embodiments.

[0092] A preferred embodiment of the hip exoskeleton control method based on gait pattern prediction and gait phase estimation of the present invention is as follows: Figure 1 , Figure 2 and Figure 3 As shown, it specifically includes:

[0093] S1. Construct an initial unsupervised terrain detection model, and use the dataset to train and test the initial unsupervised terrain detection model to obtain the target unsupervised terrain detection model.

[0094] In one embodiment of the present invention, the step of constructing an initial unsupervised terrain detection model, training and testing the initial unsupervised terrain detection model using a dataset, and obtaining a target unsupervised terrain detection model specifically includes:

[0095] Construct an initial unsupervised terrain detection model, wherein the initial unsupervised terrain detection model includes a feature generator, a first classifier, and a second classifier;

[0096] Acquire unlabeled real environment point cloud data and labeled simulated environment point cloud data, and preprocess the real environment point cloud data and the simulated environment point cloud data to obtain a target domain dataset and a source domain dataset. The target domain dataset includes a real environment binary image, and the source domain dataset includes a simulated environment binary image and labels.

[0097] Randomly select a first predetermined proportion of target domain data from the target domain dataset for manual annotation to obtain a validation set, and combine the second predetermined proportion of target domain data from the target domain dataset with the source domain dataset to form a training set, wherein the validation set and the training set constitute the dataset;

[0098] The feature generator, the first classifier, and the second classifier are trained using the source domain dataset in the training set, and a first objective function is constructed with the goal of minimizing the softmax cross-entropy loss on the source domain data.

[0099] ;

[0100] Where A represents the first objective function, This indicates taking the minimum value. Represents a feature generator. This represents the first classifier. This represents the second classifier. For the expectation operator, Represents the source domain dataset. This represents a set of binary images representing the simulation environment. Represents a set of labeled elements. This represents the source domain data in the source domain dataset. Represents a binary image of the simulation environment. Indicates labeling, This represents the total number of terrain categories in daily life; here, we take 5. This is a binary operator; it equals 1 only if the result is true, otherwise it equals 0. Indicates the output terrain type. Indicates that given input Time output For category The probability of;

[0101] With the parameters of the feature generator fixed, the first classifier and the second classifier are trained using the training set to construct a second objective function that maximizes the output difference between the first classifier and the second classifier on the target domain data.

[0102] ;

[0103] in, This represents the second objective function. Represents the target domain dataset. This represents the target domain data in the target domain dataset. This indicates that the first classifier performs well on a given input. Time output For category The probability, This indicates that the second classifier performs well on a given input. Time output For category The probability; it should be noted that the target domain dataset and target domain data here refer to the target domain dataset and target domain data in the training set;

[0104] With the parameters of the first and second classifiers fixed, the feature generator is trained using the target domain data in the training set to construct a third objective function that minimizes the classification difference between the first and second classifiers on the target domain data.

[0105] ;

[0106] in, Indicate the third objective function;

[0107] The performance of the initial unsupervised terrain detection model after training is tested using the validation set. After the test is completed, the initial unsupervised terrain detection model is trained again until the maximum number of training rounds is reached to obtain the target unsupervised terrain detection model.

[0108] Specifically, the training process of an unsupervised, domain-adaptive terrain detection model, such as... Figure 2 and Figure 3As shown, the specific steps include: 1) Acquiring real environment point cloud data (i.e., acquiring 3D environment point cloud data of the real physical environment through an RGB-D camera), and converting the real environment point cloud data into a binary environment image through rotation and dimensionality reduction methods to obtain unlabeled real environment data (i.e., the target domain dataset, including the real environment binary image). 2) Acquiring simulated environment point cloud data (i.e., generating labeled simulated environment point cloud data based on the characteristics of five common terrains (flat ground, stairs, slopes), adding random size and geometric noise to the simulated environment point cloud data, and finding a suitable angle for the RGB-D camera to acquire environmental information when fixed at the user's waist through pre-experiments, mapping the generated simulated environment point cloud data to a virtual camera viewpoint with the same angle as the real camera, and finally converting the simulated environment point cloud data into a binary environment image to obtain labeled simulated environment data (i.e., the source domain dataset, including the simulated environment binary image and labels). 3) Building an unsupervised terrain detection model, such as... Figure 4As shown, the system consists of a feature generator based on a convolutional neural network and two classifiers. The feature generator contains three convolutional layers and corresponding batch normalization, max pooling, and activation layers. The number of channels in the convolutional layers are 64, 128, and 384, respectively, and the kernel sizes are 5×5, 3×3, and 3×3, respectively. The extracted deep features are then fed into the classifiers after being unfolded. The classifier structure is a multilayer perceptron with two hidden layers, where the hidden layers include fully connected layers, batch normalization layers, and activation layers. 4) Set the training-related hyperparameters. The Adam optimizer is selected for training, with a learning rate of 0.0002, a batch size of 128, and a maximum number of training epochs of 100. 5) Create the dataset. Randomly extract 30% of the real-world data for manual annotation as the validation set. The remaining 70% of the unlabeled real-world data and all simulated environment data form the training set. 6) Initialize the feature generator and classifiers. 7) Train the feature generator and classifier using labeled source domain simulation environment data to enable the model to correctly classify source domain data. The goal is to minimize the softmax cross-entropy loss in the source domain to ensure the network can correctly identify different terrains. 8) Fix the parameters of the feature generator and train the classifier using the source domain dataset and the target domain dataset (unlabeled real-world environment data in the training set). The goal is to maximize the output difference between the two classifiers on the target domain data to detect target domain data with distributional differences from the source domain as much as possible. Furthermore, the classification loss from step 7) on the source domain data is retained to maintain accurate classification of environmental terrain. 9) Fix the classifier parameters and train the feature generator using the target domain dataset to minimize the classification difference between the two classifiers on the target domain data. 10) Test the model performance using a validation set. After testing, repeat the training until the maximum number of training epochs is reached to complete the training. It should be noted that... Figure 2 The unsupervised domain adaptation model training is also known as the unsupervised terrain detection model training.

[0109] This invention employs an unsupervised domain adaptive algorithm based on maximum classifier difference. It trains a model consisting of a feature generator and two classifiers using simulated terrain data with varying terrain types and unlabeled real-world environmental data. Through three training phases, the model aligns features between the source domain (simulated terrain) and the target domain (real-world terrain), generalizing the model trained on simulated terrain data to real-world environmental terrain. Ultimately, it achieves accurate detection of five common everyday walking terrain types.

[0110] S2. Acquire three-dimensional environment point cloud data and preprocess the three-dimensional environment point cloud data to obtain a binary environment image.

[0111] In one embodiment of the present invention, the step of acquiring three-dimensional environment point cloud data and preprocessing the three-dimensional environment point cloud data to obtain a binary environment image specifically includes:

[0112] Acquire 3D environmental point cloud data from an RGB-D camera mounted on the waist of the human body;

[0113] The 3D environment point cloud data is transformed from the camera coordinate system to the ground coordinate system using a rotation matrix to obtain the 3D environment point cloud data in the ground coordinate system.

[0114] The dimensionality of the three-dimensional environmental point cloud data in the ground coordinate system is reduced to obtain two-dimensional environmental point cloud data.

[0115] The two-dimensional environmental point cloud data is converted into a binary environmental image using a point cloud rasterization method.

[0116] Specifically, such as Figure 3 As shown, this invention uses an RGB-D camera and an IMU (Inertial Measurement Unit) fixed to the user's waist to acquire a 3D environmental point cloud (referring to the 3D environmental point cloud data to be measured) and camera orientation information of the upcoming environment terrain. Then, using rotation and dimensionality reduction methods, the environmental point cloud is converted into a binary environmental image (referring to the binary image of the environment to be measured). Because the model uses a binary image as input, the computational load is reduced, which is beneficial for its application in wearable robots.

[0117] S3. Input the binary environmental image into the unsupervised target terrain detection model. The unsupervised target terrain detection model outputs the terrain detection result and smooths the terrain detection result to obtain the gait pattern prediction result.

[0118] In one embodiment of the present invention, the step of inputting the binary environmental image into the unsupervised target terrain detection model, the unsupervised target terrain detection model outputting terrain detection results, and smoothing the terrain detection results to obtain gait pattern prediction results specifically includes:

[0119] The binary environmental image is input into the unsupervised target terrain detection model. The first classifier of the unsupervised target terrain detection model outputs the first classification score corresponding to each terrain category, and the second classifier of the unsupervised target terrain detection model outputs the second classification score corresponding to each terrain category.

[0120] By fusing the first classification score and the second classification score corresponding to various terrain categories, the target classification score corresponding to each terrain category is obtained. The terrain category corresponding to the maximum value among the multiple target classification scores is taken as the terrain detection result output by the unsupervised terrain detection model.

[0121] Based on the majority voting strategy, the terrain detection results are smoothed by a sliding window. The terrain category that appears most frequently within a set window length is taken as the target terrain detection result, and the gait pattern corresponding to the target terrain detection result is taken as the gait pattern prediction result.

[0122] The terrain categories include flat land, uphill, downhill, up stairs, and down stairs.

[0123] Specifically, the gait pattern prediction process based on the unsupervised terrain detection model includes the following steps: 1) The binary environmental image (referring to the binary image of the environment to be tested) is fed into the feature generator of the trained unsupervised terrain detection model to extract features, and the classification scores of the two classifiers are output. 2) The classification scores of the two classifiers on five terrain types are fused, and the maximum value is taken as the terrain detection result output by the model. 3) To reduce classification errors caused by noise, data synchronization errors, etc., the terrain detection results are smoothed using a sliding window based on a majority voting strategy, with a window length of 5. Specifically, the model prediction results of the most recent 5 frames are statistically analyzed, and the category with the most occurrences is taken as the final terrain detection result in front of the user, and this result is used as the final gait pattern prediction result. It should be noted that there is a one-to-one correspondence between terrain and gait patterns, and the terrain detection results also correspond one-to-one with the gait pattern prediction results. That is, flat terrain corresponds to flat gait patterns, sloping terrain corresponds to sloping gait patterns, and staircase terrain corresponds to staircase gait patterns.

[0124] This invention feeds a binary environmental image into a pre-trained unsupervised domain adaptive network. The network's two classifiers output environmental terrain classification results, which are then fused. To reduce classification errors caused by obstacles, noise, and other factors, a majority voting strategy is used to smooth the fused classification results. The processed output is then considered the final predicted gait pattern.

[0125] S4. Obtain the hip joint angular velocity, use the hip joint angular velocity as the teaching signal of the adaptive oscillator system to obtain the initial gait phase estimate, and correct the initial gait phase estimate to obtain the target gait phase estimate.

[0126] In one embodiment of the present invention, the step of acquiring the hip joint angular velocity, using the hip joint angular velocity as a teaching signal for the adaptive oscillator system to obtain an initial gait phase estimate, and correcting the initial gait phase estimate to obtain a target gait phase estimate specifically includes:

[0127] Acquire the hip joint angular velocity collected by the inertial measurement unit installed on the human leg;

[0128] Based on the difference between the hip joint angular velocity and the reconstructed angular velocity reconstructed through finite-term Fourier decomposition, the state variables of the adaptive oscillator system are updated in real time:

[0129] ;

[0130] ;

[0131] ;

[0132] ;

[0133] ;

[0134] ;

[0135] in, Represents hip joint angular velocity With reconstructed angular velocity The difference, Indicates time step, and These represent the phase and natural frequency of the oscillator in the adaptive oscillator system, respectively. Indicates the rate of change of phase. Represents the rate of change of frequency. This represents the adjustable coefficients that determine the phase and frequency learning speed. Indicates the first The first Fourier coefficient corresponding to the subharmonic Learning update rate Indicates the first The second Fourier coefficient corresponding to the subharmonic Learning update rate This represents the adjustable coefficient that determines the learning speed of the first and second Fourier coefficients. Indicates the maximum harmonic order. This represents the reconstructed angular velocity value; it should be noted that a dot above the variable sign indicates differentiation;

[0136] Based on the rate of change of the phase, the phase signal that varies with time is obtained through numerical iteration. The time-varying phase signal As an initial gait phase estimate;

[0137] The initial gait phase estimate is normalized to [0, 1], corresponding to 0–100% of the gait cycle, to obtain the intermediate gait phase estimate:

[0138] ;

[0139] in, This indicates the intermediate gait phase estimation. Indicates normalization;

[0140] The hip joint angle is acquired by an inertial measurement unit installed on the human leg. Maximum hip flexion is detected based on this hip joint angle. At the moment of detection of maximum hip flexion, the deviation between the intermediate gait phase estimate and a reference value is calculated.

[0141] ;

[0142] in, This indicates the moment when maximum hip flexion was detected. Indicates in The deviation between the estimated gait phase at time intervals and the reference value. Indicates in Gait phase estimation at intermediate time points;

[0143] Based on the deviation between the intermediate gait phase estimate and the reference value, the phase offset correction term is updated using an exponential decay control method in the form of a first-order derivative:

[0144] ;

[0145] in, This indicates the phase offset correction term. This represents the rate of change of the phase offset correction term. Indicates proportional gain. This represents a frequency signal that changes over time.

[0146] Subtract the phase offset correction term from the intermediate gait phase estimate and normalize it to obtain the target gait phase estimate:

[0147] ;

[0148] in, This represents the target gait phase estimation.

[0149] Specifically, based on the gait phase estimation of the adaptive oscillator system, such as Figure 2 and Figure 3 As shown, the specific steps include: 1) Real-time acquisition of the user's hip joint angular velocity using an IMU fixed to the user's thigh. And reconstructing angular velocity through Fourier decomposition of finite terms. The difference is calculated and used as the input to the adaptive oscillator system. 2) The state variables of the adaptive oscillator system are updated in real time based on the instantaneous difference, where the Fourier coefficients are... and The update enables frequency domain learning of the input signal, and then the angular velocity is reconstructed in real time based on the currently learned spectral characteristics. 3) Normalize the gait phase estimated by the oscillator to [0,1], corresponding to 0–100% of the gait cycle. 4) Obtain the user's hip joint angle in real time through an IMU fixed to the user's thigh. 5) After detecting maximum hip flexion (i.e., gait event), use it as the starting point for each gait cycle. Ideally, the gait phase should linearly increase from 0 to 1. Ideally, the gait phase should be 0 when a gait event is detected. However, in practice, there is a deviation between the gait phase estimated by the oscillator and this reference value. 6) To synchronize the gait phase estimation with the user's gait, deviations are corrected by locking the starting point of each gait cycle at maximum hip flexion. Specifically, after calculating the deviation, the phase offset correction term is updated using an exponential decay control method with first-order differential form. 7) Estimate the current gait phase. Subtract the correction term The corrected gait phase estimate is then normalized. .

[0150] This invention employs two inertial measurement units mounted on the thigh to measure the angle and angular velocity of the hip joint in the sagittal plane. The measured hip joint angular velocity is used as a training signal for an adaptive oscillator dynamic system (i.e., an adaptive oscillator system) to extract continuous phases of the gait cycle in real time. Maximum hip flexion (MHF) is then defined as the starting point of each gait cycle. Finally, gait phase estimation is synchronized with the user's gait by correcting mismatches (i.e., biases).

[0151] S5. Calculate the net assist torque based on the gait pattern prediction result and the target gait phase estimation, and switch the assist mode of the hip exoskeleton based on the net assist torque.

[0152] In one embodiment of the present invention, the step of calculating the net assist torque based on the gait pattern prediction result and the target gait phase estimation, and switching the assist mode of the hip exoskeleton based on the net assist torque, specifically includes:

[0153] Based on different gait phases, the expected auxiliary torque curves for hip extension and flexion are constructed using the hyperbolic tangent function:

[0154] ;

[0155] in, This represents the desired auxiliary torque curve. Indicates gait phase, This indicates the initial phase of the assist (i.e., the phase at which assist begins). Indicates peak torque. Represents the hyperbolic tangent function. A positive coefficient indicating the steepness of the adjustment curve. Indicates the predefined duration of the boost;

[0156] Multiple assist parameters corresponding to different gait patterns were obtained through pre-trial walking experiments on subjects. Among them, the multiple assist parameters corresponding to different gait patterns include the assist initiation phase of hip extension. Peak torque of hip extension Assisted starting phase of hip flexion Peak torque of hip flexion ;

[0157] The moment of maximum hip flexion is taken as the starting point of the gait cycle. Within the gait cycle, based on the expected assist torque curves of hip extension and flexion, the net assist torque is calculated according to the assist parameters corresponding to the gait pattern prediction results and the target gait phase estimation.

[0158] ;

[0159] in, Indicates the net auxiliary torque. This represents the expected auxiliary torque for hip extension. This represents the desired auxiliary torque for hip flexion;

[0160] Using a finite state machine, the assist mode of the hip exoskeleton is switched according to the net assist torque. When the net assist torque is greater than 0, the assist mode of the hip exoskeleton is switched to the assist hip extension mode; when the net assist torque is less than 0, the assist mode of the hip exoskeleton is switched to the assist hip flexion mode; and when the net assist torque is equal to 0, the assist mode of the hip exoskeleton is switched to the no-assist mode.

[0161] Specifically, the exoskeleton-assisted control process based on the desired assist torque curve of hip extension / flexion, such as... Figure 2 and Figure 3As shown, the specific steps include: 1) Constructing a parameterized single-peak function based on the hyperbolic tangent function according to different gait phases, serving as the expected auxiliary torque curve for hip extension / flexion. This allows the auxiliary torque output by the exoskeleton to gradually increase and reach a peak at a specific stage in the gait cycle, then naturally decay to zero, thereby avoiding the impact of sudden start and stop, and improving the comfort and naturalness of wearing the device. 2) Optimizing multiple feature assistance parameters under different movement modes through pre-experiment walking experiments with subjects, including the assistance start phase and peak torque for hip extension, and the assistance start phase and peak torque for hip flexion. 3) At the beginning of each new gait cycle, switching the exoskeleton's assistance mode using the FSM. The FSM selects the corresponding extension and flexion assistance parameters according to the different user gait modes predicted in step S3, and updates the assistance output strategy at the beginning of each cycle. Within a new gait cycle, based on the selected assist parameters and the gait phase estimated in step S4, the expected assist torque for hip extension and flexion is generated using the expected assist torque generation formula (i.e., the expected assist torque curve), and the difference between the two is taken as the net assist torque of the exoskeleton. The assist mode of the exoskeleton is switched according to the net assist torque.

[0162] like Figure 2 As shown, the present invention obtains gait pattern prediction results through the gait pattern prediction module of the high-level controller. Gait phase estimation is obtained through the gait phase estimation module of the high-level controller. , combined and Then, through the expected net auxiliary torque curve of the middle layer controller. Obtain the desired net auxiliary torque Finally, the actual output net auxiliary torque function of the underlying controller is used. ,in, This represents the hardware feedback quantity, from which the actual output net auxiliary torque is obtained. It acts on the hip-assisted exoskeleton system. In this embodiment, considering the influence of the exoskeleton hardware state, it is based on the desired net assist torque and The actual net auxiliary torque is obtained. When the actual net auxiliary torque is greater than 0, the assist mode of the hip exoskeleton is switched to assisted hip extension mode; when the actual net auxiliary torque is less than 0, the assist mode of the hip exoskeleton is switched to assisted hip flexion mode; when the actual net auxiliary torque is equal to 0, the assist mode of the hip exoskeleton is switched to no assist mode. It should be noted that the net auxiliary torque mentioned above... This refers to the expected net assist torque. The previous discussion of switching the hip exoskeleton's assist mode based on the net assist torque did not consider the influence of the exoskeleton's hardware condition.

[0163] This invention uses a parameterized single-peak curve as the expected auxiliary torque (i.e., moment) curve for hip extension / flexion. The corresponding expected auxiliary torque is calculated according to different gait phases. At the beginning of a new gait cycle, the exoskeleton assistance mode is switched using the FSM according to the predicted gait pattern.

[0164] In one embodiment of the present invention, after acquiring labeled simulation environment point cloud data, the method further includes:

[0165] Random size noise and geometric noise are added to the simulated environment point cloud data to obtain noisy simulated environment point cloud data;

[0166] The appropriate angle for the RGB-D camera to acquire three-dimensional environmental point cloud data when installed on the waist of a human body is obtained through visual pre-experiment. The simulated environmental point cloud data with added noise is then mapped to the virtual camera viewpoint corresponding to the appropriate angle to obtain the mapped simulated environmental point cloud data.

[0167] Specifically, this invention adds random size noise and geometric noise (such as adding noise to the height of stairs and the slope of slope and performing small rigid transformations) to the generated simulated environment point cloud, conducts preliminary experiments, finds a suitable angle for the RGB-D camera to acquire environmental information when fixed at the user's waist, and maps the generated simulated environment point cloud to a virtual camera viewpoint with the same angle as the real camera.

[0168] In summary, this invention provides a hip exoskeleton control method based on gait pattern prediction and gait phase estimation for walking assistance on various terrains in daily life. This method mainly consists of unsupervised, domain-adaptive gait pattern prediction and adaptive oscillator-based gait phase estimation. It can detect changes in the user's movement pattern in advance while avoiding the need for large amounts of data annotation required for training traditional supervised learning models, and provides accurate gait phase estimation during multimodal movement, including gait pattern transitions. Furthermore, since the model uses binary images as input, the computational load is reduced, which is beneficial for applications in wearable robots. Unlike most existing adaptive oscillator-based methods, gait phase estimation uses hip joint angular velocity instead of angle as the teaching signal to improve adaptability to gait pattern changes and enhance the exoskeleton's terrain-adaptive walking capability. Combining terrain detection with lower limb exoskeleton control can provide appropriate assistance during multimodal movement on various terrains, including during gait pattern transitions.

[0169] Furthermore, based on the aforementioned hip exoskeleton control method based on gait pattern prediction and gait phase estimation, this invention also provides a hip exoskeleton control system based on gait pattern prediction and gait phase estimation, wherein a preferred embodiment of the hip exoskeleton control system based on gait pattern prediction and gait phase estimation is as follows: Figure 5 As shown, it specifically includes:

[0170] Terrain detection model training module 01: used to construct an initial unsupervised terrain detection model, and to train and test the initial unsupervised terrain detection model using a dataset to obtain a target unsupervised terrain detection model;

[0171] Visual perception module 02: used to acquire three-dimensional environment point cloud data and preprocess the three-dimensional environment point cloud data to obtain a binary image of the environment;

[0172] Gait pattern prediction module 03: is used to input the binary environmental image into the target unsupervised terrain detection model, the target unsupervised terrain detection model outputs the terrain detection result, and the terrain detection result is smoothed to obtain the gait pattern prediction result;

[0173] Gait phase estimation module 04: used to acquire hip joint angular velocity, use the hip joint angular velocity as a teaching signal of the adaptive oscillator system to obtain initial gait phase estimation, and correct the initial gait phase estimation to obtain target gait phase estimation;

[0174] Exoskeleton assist control module 05: used to calculate the net assist torque based on the gait pattern prediction result and the target gait phase estimation, and to switch the assist mode of the hip joint exoskeleton based on the net assist torque.

[0175] Furthermore, based on the aforementioned hip exoskeleton control method and system based on gait pattern prediction and gait phase estimation, this invention also provides a terminal, wherein a preferred embodiment of the terminal is as follows: Figure 6 As shown, it specifically includes a processor 10, a memory 20, and a display 30. Figure 6 Only some of the terminal components are shown; however, it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0176] In some embodiments, the memory 20 may be an internal storage unit of the terminal, such as a hard disk or memory. In other embodiments, the memory 20 may be an external storage device of the terminal, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, or flash card. Furthermore, the memory 20 may include both internal and external storage units. The memory 20 is used to store application software and various types of data installed on the terminal, such as the terminal's program code. The memory 20 may also be used to temporarily store data that has been output or will be output. In one embodiment, the memory 20 stores a hip exoskeleton control program 40 based on gait pattern prediction and gait phase estimation. This hip exoskeleton control program 40 based on gait pattern prediction and gait phase estimation can be executed by the processor 10 to implement the steps of the hip exoskeleton control method based on gait pattern prediction and gait phase estimation in this application.

[0177] In some embodiments, processor 10 may be a central processing unit (CPU), microprocessor, or other data processing chip, used to run program code stored in memory 20 or process data, such as executing a hip exoskeleton control program 40 based on gait pattern prediction and gait phase estimation.

[0178] In some embodiments, display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. Display 30 is used to display information on the terminal and to display a visual user interface.

[0179] In one embodiment, when the processor 10 executes the hip exoskeleton control program 40 based on gait pattern prediction and gait phase estimation stored in the memory 20, it implements the steps of the hip exoskeleton control method based on gait pattern prediction and gait phase estimation as described above.

[0180] The present invention also provides a computer-readable storage medium storing a hip exoskeleton control program based on gait pattern prediction and gait phase estimation. When executed by a processor, the hip exoskeleton control program based on gait pattern prediction and gait phase estimation implements the steps of the hip exoskeleton control method based on gait pattern prediction and gait phase estimation as described above.

[0181] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal that includes that element.

[0182] Of course, those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.). This program can be stored in a computer-readable storage medium, and when executed, it can include the processes described in the above method embodiments. The computer-readable storage medium can be a memory, magnetic disk, optical disk, etc.

[0183] It should be understood that the application of the present invention is not limited to the examples above. Those skilled in the art can make improvements or modifications based on the above description, and all such improvements and modifications should fall within the protection scope of the appended claims.

Claims

1. A method of controlling a hip exoskeleton based on gait pattern prediction and gait phase estimation, characterized by, The hip exoskeleton control method based on gait pattern prediction and gait phase estimation comprises: An initial unsupervised terrain detection model is constructed, the initial unsupervised terrain detection model is trained and tested by using a data set, and a target unsupervised terrain detection model is obtained; Three-dimensional environment point cloud data is acquired, and the three-dimensional environment point cloud data is preprocessed to obtain an environment binary image; The environment binary image is input into the target unsupervised terrain detection model, the target unsupervised terrain detection model outputs a terrain detection result, and the terrain detection result is smoothed to obtain a gait pattern prediction result; A hip joint angular velocity is acquired, the hip joint angular velocity is used as a teaching signal of an adaptive oscillator system, an initial gait phase estimation is obtained, and the initial gait phase estimation is corrected to obtain a target gait phase estimation; A net assist torque is calculated according to the gait pattern prediction result and the target gait phase estimation, and a power assisting mode of a hip exoskeleton is switched according to the net assist torque.

2. The method of hip exoskeleton control based on gait pattern prediction and gait phase estimation according to claim 1, characterized in that, The construction of the initial unsupervised terrain detection model, the training and testing of the initial unsupervised terrain detection model by using a data set, and the obtaining of the target unsupervised terrain detection model specifically comprise: An initial unsupervised terrain detection model is constructed, wherein the initial unsupervised terrain detection model comprises a feature generator, a first classifier and a second classifier; Unlabeled real environment point cloud data and labeled simulation environment point cloud data are acquired, and the real environment point cloud data and the simulation environment point cloud data are preprocessed to obtain a target domain data set and a source domain data set, wherein the target domain data set comprises real environment binary images, and the source domain data set comprises simulation environment binary images and labels; A first set proportion of target domain data in the target domain data set is randomly selected for manual labeling to obtain a verification set, and a second set proportion of target domain data in the target domain data set and the source domain data set form a training set, wherein the verification set and the training set form the data set; The source domain data set in the training set is used to train the feature generator, the first classifier and the second classifier, a first target function is constructed to minimize the softmax cross-entropy loss on the source domain data: ； wherein A represents a first objective function, represents taking a minimum value, represents a feature generator, represents a first classifier, represents a second classifier, is a desired calculator, represents a source domain dataset, represents a set of simulation environment binary images, represents a set of labels, represents source domain data in the source domain dataset, represents a simulation environment binary image, represents a label, represents a total number of terrain classes, is a binary operator, represents an output terrain class, represents an output when a given input is inputted; is a probability of a class. The parameters of the feature generator are fixed, the first classifier and the second classifier are trained by using the training set, a second target function is constructed to maximize the output difference of the first classifier and the second classifier on the target domain data: ； wherein, represents a second objective function, represents a target domain dataset, represents a target domain data in the target domain dataset, represents a probability that the first classifier outputs when given an input is of class , represents a probability that the second classifier outputs when given an input is of class ; The parameters of the first classifier and the second classifier are fixed, the feature generator is trained by using the target domain data in the training set, and a third target function is constructed to minimize the classification difference of the first classifier and the second classifier on the target domain data: ； wherein, denotes the third objective function; The performance of the trained initial unsupervised terrain detection model is tested by using the verification set, and after the testing is completed, the initial unsupervised terrain detection model is repeatedly trained until the maximum training round is reached, and a target unsupervised terrain detection model is obtained.

3. The method of hip exoskeleton control based on gait pattern prediction and gait phase estimation according to claim 2, characterized in that, The three-dimensional environment point cloud data is acquired and preprocessed to obtain an environment binary image, specifically comprising: Acquiring three-dimensional environment point cloud data collected by an RGB-D camera installed on the waist of a human body; Converting the three-dimensional environment point cloud data from a camera coordinate system to a ground coordinate system by using a rotation matrix to obtain three-dimensional environment point cloud data in the ground coordinate system; Dimension reduction is performed on the three-dimensional environment point cloud data in the ground coordinate system to obtain two-dimensional environment point cloud data; The two-dimensional environment point cloud data is converted into an environment binary image by using a point cloud gridding method.

4. The method of hip exoskeleton control based on gait pattern prediction and gait phase estimation according to claim 2, wherein, The environment binary image is input into the target unsupervised terrain detection model, and the target unsupervised terrain detection model outputs a terrain detection result, and the terrain detection result is smoothed to obtain a gait pattern prediction result, specifically comprising: The environment binary image is input into the target unsupervised terrain detection model, and the first classifier of the target unsupervised terrain detection model outputs a first classification score corresponding to each terrain category, and the second classifier of the target unsupervised terrain detection model outputs a second classification score corresponding to each terrain category; Fusing the first classification score and the second classification score corresponding to each terrain category to obtain a target classification score corresponding to each terrain category, and taking the terrain category corresponding to the maximum value in the plurality of target classification scores as the terrain detection result output by the target unsupervised terrain detection model; Based on the majority voting strategy, the terrain detection result is smoothed by a sliding window, and the terrain category with the highest frequency within a set window length is taken as the target terrain detection result, and the gait pattern corresponding to the target terrain detection result is taken as the gait pattern prediction result; The terrain categories include flat ground, uphill, downhill, upstairs, and downstairs.

5. The method of hip exoskeleton control based on gait pattern prediction and gait phase estimation according to claim 1, wherein, The hip joint angular velocity is acquired, the hip joint angular velocity is taken as a teaching signal of an adaptive oscillator system, an initial gait phase estimation is obtained, and the initial gait phase estimation is corrected to obtain a target gait phase estimation, specifically comprising: Acquiring the hip joint angular velocity collected by an inertial measurement unit installed on the leg of a human body; According to the difference between the hip joint angular velocity and the reconstructed angular velocity reconstructed by finite term Fourier decomposition, the state variables of the adaptive oscillator system are updated in real time: ； ； ； ； ； ； wherein, represents the hip joint angular velocity and the difference between the reconstructed angular velocity , represents the time step, and respectively represent the phase and the natural frequency of the oscillators in the adaptive oscillator system, represents the rate of change of the phase, represents the rate of change of the frequency, represents the adjustable coefficient that determines the learning speed of the phase and the frequency, represents the learning update rate of the first Fourier coefficient corresponding to the first harmonic, represents the learning update rate of the second Fourier coefficient corresponding to the second harmonic, represents the learning update rate of the second Fourier coefficient corresponding to the second harmonic, represents the adjustable coefficient that determines the learning speed of the first Fourier coefficient and the second Fourier coefficient, represents the maximum harmonic number, represents the reconstructed angular velocity value; from the rate of change of the phase, a time-varying phase signal is calculated by numerical iteration the time-varying phase signal is used as an initial gait phase estimate; The initial gait phase estimation is normalized to [0, 1], corresponding to 0-100% of the gait cycle, to obtain an intermediate gait phase estimation: ； wherein, denotes an intermediate gait phase estimate, denotes a normalization; Acquiring the hip joint angle collected by an inertial measurement unit installed on the leg of a human body, detecting the maximum hip joint flexion according to the hip joint angle, and calculating the deviation between the intermediate gait phase estimation and a reference value at the moment when the maximum hip joint flexion is detected: ； wherein, denotes the moment at which the maximum hip flexion is detected, denotes the deviation between the mid gait phase estimate and the reference value at the moment denotes the deviation between the mid gait phase estimate and the reference value at the moment denotes the mid gait phase estimate at the moment denotes the mid gait phase estimate at the moment According to the deviation between the intermediate gait phase estimation and the reference value, the phase offset correction term is updated by a first-order differential form exponential decay control method: ； wherein denotes a phase offset correction term, denotes a rate of change of the phase offset correction term, denotes a proportional gain, denotes a frequency signal that changes over time; The intermediate gait phase estimation is subtracted by the phase offset correction term and normalized to obtain the target gait phase estimation: ； wherein, denotes the target gait phase estimate.

6. The method of hip exoskeleton control based on gait pattern prediction and gait phase estimation according to claim 5, wherein, The net assist torque is calculated according to the gait pattern prediction result and the target gait phase estimation, and the hip exoskeleton assistance mode is switched according to the net assist torque, and specifically comprises: According to different gait phases, the desired assist torque curves of hip joint extension and flexion are constructed based on hyperbolic tangent function: ； wherein denotes a desired assist torque curve, denotes a gait phase, denotes an assist start phase, denotes a peak torque, denotes a hyperbolic tangent function, denotes a positive coefficient that regulates the steepness of the curve, denotes a predefined assist rise duration; A plurality of assistance parameters corresponding to different gait patterns optimized by a pre-experiment of the subject walking, wherein the plurality of assistance parameters corresponding to the different gait patterns include an assistance start phase of hip joint extension , a peak torque of hip joint extension , an assistance start phase of hip joint flexion , and a peak torque of hip joint flexion ; The time point of detecting the maximum hip joint flexion is taken as the starting point of the gait cycle, and within the gait cycle, the net assist torque is calculated according to the corresponding assistance parameters of the gait pattern prediction result and the target gait phase estimation based on the desired assist torque curves of hip joint extension and flexion: ； wherein, represents the net assist torque, represents the desired assist torque for hip extension, represents the desired assist torque for hip flexion; The finite state machine is used to switch the assistance mode of the hip exoskeleton according to the net assist torque.

7. The method of hip exoskeleton control based on gait pattern prediction and gait phase estimation according to claim 3, wherein, The labeled simulation environment point cloud data is obtained, and then further comprises: Random size noise and geometric noise are added to the simulation environment point cloud data to obtain noise-added simulation environment point cloud data; Through visual pre-experiment, the appropriate angle of the RGB-D camera for acquiring three-dimensional environment point cloud data when installed on the human waist is obtained, and the noise-added simulation environment point cloud data is mapped to the virtual camera view angle corresponding to the appropriate angle to obtain the mapped simulation environment point cloud data.

8. A hip exoskeleton control system based on gait pattern prediction and gait phase estimation, characterized by, The hip exoskeleton control system based on gait pattern prediction and gait phase estimation comprises: A terrain detection model training module is configured to construct an initial unsupervised terrain detection model, train and test the initial unsupervised terrain detection model using a data set, and obtain a target unsupervised terrain detection model; A visual perception module is configured to acquire three-dimensional environment point cloud data and preprocess the three-dimensional environment point cloud data to obtain an environment binary image; A gait pattern prediction module is configured to input the environment binary image into the target unsupervised terrain detection model, and the target unsupervised terrain detection model outputs a terrain detection result, which is smoothed to obtain a gait pattern prediction result; A gait phase estimation module is configured to acquire hip joint angular velocity, use the hip joint angular velocity as a teaching signal of an adaptive oscillator system to obtain an initial gait phase estimation, and correct the initial gait phase estimation to obtain a target gait phase estimation; An exoskeleton assistance control module is configured to calculate a net assist torque according to the gait pattern prediction result and the target gait phase estimation, and switch the assistance mode of the hip exoskeleton according to the net assist torque.

9. A terminal, characterized by comprising: The terminal comprises a memory, a processor, and a hip exoskeleton control program based on gait pattern prediction and gait phase estimation stored on the memory and executable on the processor, and when the hip exoskeleton control program based on gait pattern prediction and gait phase estimation is executed by the processor, the steps of the hip exoskeleton control method based on gait pattern prediction and gait phase estimation according to any one of claims 1-7 are implemented.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a hip exoskeleton control program based on gait pattern prediction and gait phase estimation, which, when executed by a processor, implements the steps of the hip exoskeleton control method based on gait pattern prediction and gait phase estimation according to any one of claims 1-7.

Images