Gait prediction method and wearable exoskeleton device

Abstract

The application discloses a gait prediction method and a wearable exoskeleton device. The method comprises obtaining first data information and second data information, the first data information comprising a thigh movement state of a target object, and the second data information comprising current environment characteristics; inputting the first data information and the second data information into a pre-trained gait prediction model, and obtaining a gait prediction result output by the gait prediction model. The method of the embodiment of the application predicts the gait by using an expert hybrid model, the model takes the thigh movement state of the target object and the current environment characteristics as input, can explicitly introduce the environment characteristics such as terrain conditions into the model structure, realizes collaborative modeling of human historical movement information and environment condition information, and thus can realize gait prediction for different environment characteristic conditions, and through the expert hybrid mechanism, the adaptability of the model to different environment characteristics such as terrain conditions is enhanced, so that the accuracy and generalization performance of the gait prediction are improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a gait prediction method, a wearable exoskeleton device, a computing device, and a storage medium. Background Technology

[0002] Lower limb exoskeletons are important wearable exoskeleton devices that are tightly coupled to the hip, knee, and ankle joints through mechanical structures, providing users with assistance or enhancement for walking, standing, and climbing stairs. In the field of medical rehabilitation, they can provide repetitive gait training for patients with motor dysfunction caused by spinal cord injury, stroke, and other reasons, effectively promoting the reconstruction of neural function. Current mainstream technologies are evolving from rigid structures to lightweight, flexible exoskeletons, with actuation methods covering active, passive, and semi-active, and widely integrating various sensors such as inertial measurement units and electromyography (EMG), combined with advanced algorithms such as neural network technology and model predictive control to improve the stability and adaptability of human-machine collaboration. In lower limb exoskeletons, gait prediction is a crucial link connecting human motion state, environmental information, and auxiliary control strategies, and is also an important foundation for achieving coordinated human-machine-environment interaction. For walking tasks in complex terrain, lower limb exoskeletons not only need to sense the current motion state of the human body, but also need to combine changes in the environment ahead to make reasonable predictions of short-term future motion trends, thereby providing forward-looking information for reference trajectory generation, auxiliary torque distribution, and mode switching. Especially in scenarios involving stairs and steps, lower limb exoskeletons that rely solely on current state feedback often struggle to respond promptly to terrain changes, impacting the accuracy and coordination of assisted control. Therefore, research on gait prediction in complex terrain conditions is crucial for improving the naturalness, safety, and stability of lower limb exoskeleton-assisted control. Extensive research has been conducted both domestically and internationally in recent years. For example, patent application number 202510761127.3 discloses a personalized gait prediction method integrating self-attention mechanisms and Bayesian regression to address the issue of poor gait speed prediction accuracy. However, most existing methods still primarily rely on historical information of human proprioceptive motion parameters, with insufficient explicit utilization of environmental constraints, particularly terrain geometry. Their generalization ability across users and terrain conditions still has room for improvement. Therefore, there is an urgent need to propose a precise gait prediction solution for lower limb exoskeletons in complex terrain conditions, particularly in scenarios such as stairs. Summary of the Invention

[0003] To address at least one of the problems existing in the prior art, such as the limited generalization ability of gait prediction methods across users and terrain conditions, embodiments of the present invention provide a gait prediction method and a wearable exoskeleton device.

[0004] According to a first aspect of the present invention, a gait prediction method is provided, comprising: Acquire first data information and second data information, wherein the first data information includes the thigh movement state of the target object, and the second data information includes current environmental features; Input the first data information and the second data information into the pre-trained gait prediction model, and obtain the gait prediction result output by the gait prediction model; The gait prediction model employs an expert hybrid model, comprising an input mapping layer for mapping the input first data information into an initial feature representation, a temporal feature backbone network for feature extraction based on the initial feature representation input from the input mapping layer, an environmental condition encoding branch for mapping the current environmental features into a high-dimensional condition vector, a routing branch for weighting each expert prediction branch, and an expert prediction branch for feature fusion of the feature data output from the temporal feature backbone network and the high-dimensional condition vector output from the environmental condition encoding branch. The high-dimensional condition vector obtained from the environmental condition encoding branch is used as input to both the temporal feature backbone network and each expert prediction branch. The output of the expert hybrid model is obtained by weighted fusion of the prediction results of each expert prediction branch based on the gating weights output from the routing branch.

[0005] According to a second aspect of the present invention, a wearable exoskeleton device is provided, comprising: The information acquisition module is used to acquire first data information and second data information, wherein the first data information includes the thigh movement state of the target object, and the second data information includes the current environmental features; The gait prediction module is used to input the first data information and the second data information into a pre-trained gait prediction model and obtain the gait prediction result output by the gait prediction model. The gait prediction model employs an expert hybrid model, comprising an input mapping layer for mapping the input first data information to an initial feature representation, a temporal feature backbone network for feature extraction based on the initial feature representation input from the input mapping layer, an environmental condition encoding branch for mapping the current environmental features to a high-dimensional condition vector, a routing branch for weighting each expert prediction branch, and an expert prediction branch for feature fusion of the feature data output from the temporal feature backbone network and the high-dimensional condition vector output from the environmental condition encoding branch. The high-dimensional condition vector obtained from the environmental condition encoding branch is used as input to both the temporal feature backbone network and each expert prediction branch. The output of the expert hybrid model is obtained by weighted fusion of the prediction results from each expert prediction branch using the gating weights output from the routing branch.

[0006] According to a third aspect of the present invention, a computing device is provided, comprising: a processor and a memory communicatively connected, the memory storing instructions executable by the processor, the instructions being executed by the processor to enable the processor to perform the steps of the method described in the first aspect.

[0007] According to a fourth aspect of the present invention, a storage medium is provided thereon storing a computer program that, when executed by a processor, implements the steps of the method of the first aspect described above.

[0008] The method of this invention utilizes an expert hybrid model to predict gait. This model takes the thigh movement state of the target object and the current environmental features as input, and can explicitly introduce environmental features such as terrain conditions into the model structure, realizing the collaborative modeling of human historical movement information and environmental condition information. Furthermore, the model extracts human movement features from historical movement information through a temporal feature backbone network, maps the current environmental features into a high-dimensional conditional vector through a conditional encoding branch, and fuses the human movement features and the high-dimensional conditional vector through an expert prediction branch. The high-dimensional conditional vector obtained from the environmental conditional encoding branch is simultaneously used for conditional modulation of the temporal feature backbone network and feature fusion of the expert prediction branch. Therefore, the expert hybrid model of this invention can achieve gait prediction for different environmental feature conditions, and the expert hybrid mechanism enhances the model's adaptability to different environmental features such as terrain conditions, thereby improving the accuracy and generalization performance of gait prediction. Furthermore, to enable the model to adaptively select appropriate prediction strategies based on different environmental conditions, the expert hybrid model in this embodiment of the invention is further designed with routing branches to allocate weights to each expert prediction branch. This structure enables different expert prediction branches to learn prediction mappings under different environmental conditions based on shared input representations, and to adaptively select more appropriate prediction strategies and weights based on different environmental features such as terrain conditions, thereby enhancing the model's ability to express multiple environmental constraints and improving the accuracy and generalization ability of gait prediction in complex scenarios. Attached Figure Description

[0009] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 A flowchart illustrating a gait prediction method according to one embodiment of the present invention is shown schematically. Figure 2 The diagram schematically illustrates the network structure of an expert hybrid gait prediction model according to one embodiment of the present invention. Figure 3 The network structure diagram of the LSTM gait prediction model in the prior art is schematically shown; Figure 4 The network structure diagram of the TCN gait prediction model in the prior art is schematically shown; Figure 5 The illustration schematically shows the prediction results of the expert hybrid gait prediction model of this invention under LOSO and LOHO conditions; Figure 6 The diagram schematically illustrates the prediction mean error curve and its 95% confidence interval of the expert hybrid gait prediction model of this invention under LOSO and LOHO conditions; Figure 7 A schematic block diagram of a wearable exoskeleton device according to an embodiment of the present invention is shown. Figure 8 A schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention is shown. Detailed Implementation

[0011] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0012] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other.

[0013] In the description of this application, it should be understood that the use of terms such as "center," "middle," "longitudinal," "transverse," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "axial," "radial," and "circumferential" to indicate orientation or positional relationships based on the orientation or positional relationships shown in the accompanying drawings is solely for the convenience of describing this application and simplifying the description, and does not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of this application. Features defined with "first" and "second" are used to distinguish feature names and do not have special meanings. Furthermore, features defined with "first" and "second" may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "multiple" means two or more.

[0014] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection between two components. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0015] It should also be noted that, in this document, the terms "comprising" or "including" include not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. The terminology used herein is generally that commonly used by those skilled in the art; in case of any discrepancy with commonly used terminology, the terminology used herein shall prevail.

[0016] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] The present invention will now be described in further detail with reference to the accompanying drawings.

[0018] Figure 1 The illustration schematically shows the flow of a gait prediction method according to one embodiment of the present invention. The execution entity of this method can be, for example, a processor of a computer, cloud server, server in a cluster, PC, smart mobile terminal, robot, wearable exoskeleton device, etc. This embodiment of the invention does not limit this. Figure 1 As shown, the method includes: Operation S1 acquires first data information and second data information, wherein the first data information includes the thigh movement state of the target object and the second data information includes the current environmental features; Operation S2 involves inputting the first and second data information into a pre-trained gait prediction model and obtaining the gait prediction result output by the gait prediction model.

[0019] In operation S1, the thigh motion state can, for example, include thigh angle, angular velocity, and angular acceleration. The current environmental feature can be terrain constraint information, preferably terrain constraint information characterizing different stair conditions. For example, if the current environmental feature is terrain constraint information characterizing different stair conditions, it can be a normalized parameter determined by the stair height of the current environment and the target's height. Designing the environmental feature as a normalized parameter determined by the stair height and the target's height allows the trained gait prediction model to learn the ratio of step height to its own leg length, rather than just learning the absolute physical height of the stair. This eliminates data distribution drift caused by differences in user height, ensuring that the environmental features input to the model network have a unified physical semantics regardless of who wears the device, thus enabling the trained gait prediction model to generalize across users. For example, if the current environmental feature is a normalized parameter determined by the stair height of the current environment and the target's height, the calculation form of the current environmental feature can be:

[0020] in, Given the current environmental characteristics, Indicates the current stair height. This represents the height of the target person, and 0.53 is the ratio coefficient between the length of the human leg and the height.

[0021] Specifically, taking the thigh movement state, including hip joint angle, hip joint angular velocity, and hip joint angular acceleration, as an example, the acquisition method can be as follows: Hip joint angular acceleration and hip joint angular velocity are obtained through real-time measurement data from two IMUs (Inertial Measurement Units) installed in the mid-thigh and waist, and the hip joint angle is calculated. The specific calculation method can refer to relevant existing technologies. Specifically, the hip joint angular acceleration sequence and hip joint angular velocity sequence can be obtained through continuous IMU measurement, and the corresponding hip joint angle sequence can be calculated. The hip joint angle, hip joint angular velocity, and hip joint angular acceleration from 24 consecutive steps are then used as the first data information and input into the trained gait prediction model.

[0022] Taking the normalized parameters of the current environment, determined by the stair height and the height of the target object, as an example, the stair height can be obtained in real-time, while the target object's height can be retrieved from a storage module or obtained in real-time through pre-entry. For instance, a user can input the stair height and the target object's height using an input module provided by a wearable device such as a wearable exoskeleton. Based on this real-time input, the stair height and the target object's height can be obtained, thus calculating the current environmental characteristics.

[0023] In operation S2, the gait prediction result output by the gait prediction model can be a sequence of knee joint angles including a preset number of steps, for example, a sequence of knee joint angles for the next five steps. The gait prediction model preferably employs an expert hybrid model. Taking an expert hybrid model as an example, the following combines... Figure 2 The network structure of the gait prediction model in this embodiment of the invention will be described.

[0024] in, Figure 2 The network structure of an expert hybrid model according to one embodiment of the present invention is illustrated schematically, such as... Figure 2 As shown, the expert hybrid model of this embodiment includes an input mapping layer (labeled as Input Projection in the figure) for mapping the first input data information to an initial feature representation. The initial feature representation obtained by the input mapping layer is output to a temporal feature backbone network (composed of Conditional ResBlock in the figure); a temporal feature backbone network for extracting motion features based on the initial input feature representation; and a network for extracting environmental features (…). Figure 2 The network structure consists of: an environmental conditional encoding branch (labeled Condition MLP) that maps conditional features (also known as conditional features) to a high-dimensional conditional vector; a routing branch (labeled Router MLP) that assigns weights to each expert prediction branch (labeled Expert); and an expert prediction branch that fuses the feature data output from the temporal feature backbone network with the high-dimensional conditional vector output from the environmental conditional encoding branch. In this network structure, the high-dimensional conditional vector obtained from the environmental conditional encoding branch is used simultaneously for conditional modulation of the temporal feature backbone network and feature fusion of the expert prediction branches. The output of the expert hybrid model is obtained by weighted fusion of the prediction results of each expert prediction branch based on the gating weights output from the routing branch. Based on this network structure design, the expert hybrid model of this invention can use human historical kinematic information (corresponding to the first data information) and environmental features (corresponding to the second data information) as inputs to predict future knee joint movement trends through human-machine-environment collaborative modeling. By explicitly introducing environmental features, especially terrain constraints used to characterize different stair conditions, into the model structure, and enhancing the model's adaptability to different stair conditions through the expert hybrid mechanism, the hybrid expert model of this invention can effectively improve the accuracy and generalization performance of gait prediction.

[0025] As one possible implementation, the input mapping layer can be implemented by flattening the motion parameter sequence in the input first data information and then linearly mapping it into the latent space to obtain the initial feature representation. Here, the latent space refers to a high-level semantic coordinate system suitable for deep learning models to understand and compute. In this latent space, the values ​​of each dimension no longer represent measurable angles but rather feature codes of the motion state, enabling the deep learning model to complete subsequent complex temporal modeling and generation tasks in a smoother, more fundamental feature coordinate system. For example, the mapping method of the input mapping layer can be represented as follows:

[0026] in, This represents the initial feature representation. Represents the mapping weight matrix. The term represents the bias term, vec represents the matrix vectorization operation (expanding the input matrix column-wise into a one-dimensional column vector), and X is the sequence of motion parameters in the first input data. For the motion parameter sequence X, there exists... ,in, This indicates the length of the time window for collecting the first data information. The first data information represents the feature dimensions, which in a preferred embodiment of the invention include three dimensions: hip joint angle, hip joint angular velocity, and hip joint angular acceleration. That is, in the input mapping layer, the matrix composed of the motion parameter sequences of these three dimensions is expanded and mapped using the above formula, thus obtaining the initial feature representation. This is a feature vector obtained by unifying and fusing the three-dimensional motion parameter sequences. In this embodiment of the invention, the input mapping layer effectively decouples the physical correlation between hip joint angle, angular velocity, and angular acceleration through learnable linear recombination, thereby extracting potential motion factors that are more discriminative for downstream tasks. Furthermore, the continuous value distribution after this transformation is smoother than the coarse original physical quantities, which is beneficial for subsequent normalization operations and significantly accelerates the convergence process of the overall model.

[0027] As one possible implementation, the temporal feature backbone network can be constructed by consisting of multiple layers of conditional residual blocks, for example, such as... Figure 2 As shown, the temporal feature backbone network consists of three layers of conditional residual blocks (where each layer of conditional residual blocks is in...). Figure 2 (marked as Conditional ResBlock). Preferably, such as... Figure 2 As shown, each layer of conditional residual block can be implemented as including normalized units ( Figure 2 The middle is marked as LayerNorm), the first layer fully connected mapping (such as Figure 2The diagram shows the use of a linear layer), GELU activation, and a second fully connected output (e.g., a linear layer). Figure 2 The diagram shows a linear layer (using a normalization unit) that normalizes the initial feature representation of the input. The normalized result is then fused with the high-dimensional conditional vector output from the environmental conditional encoding branch and input into the first fully connected layer. After activation by GELU, it is output through the second fully connected layer. This temporal feature backbone network structure can maintain the stability of residual optimization while enabling the layer-by-layer injection of environmental information during feature extraction. This layer-by-layer injection allows environmental information to continuously guide the feature extraction of each layer's conditional residual block, preventing dilution as the network deepens after shallow injection and ensuring that deeper layers remain subject to environmental regulation. Simultaneously, the residual structure makes the network more sensitive to environmental changes and enhances its generalization ability. For example, the first... The update form of the layer conditional residual block can be expressed as follows:

[0028] in, Indicates the first Layer conditional residual block, Indicates the first Normalized results of layer-conditional residual blocks. This represents a mapping between linear layers (i.e., fully connected layers) and nonlinear activation functions. The high-dimensional condition vector output by the environmental condition encoding branch.

[0029] As one possible implementation, the environmental conditional coding branch can be implemented as consisting of two fully connected layers and an intermediate activation function, such as... Figure 2 As shown, the fully connected layer can be a linear layer, and the intermediate activation function can be GELU activation. The environmental condition encoding branch encodes the input environmental features, such as normalized parameters determined by the height of the stairs in the current environment and the height of the target object. Mapped to a high-dimensional conditional vector and the high-dimensional condition vector ( Figure 2 The high-dimensional conditional vector (denoted as cond) is injected layer by layer into the temporal feature backbone network so that it can participate in the conditional modulation of the temporal feature backbone network. Simultaneously, the high-dimensional conditional vector is also input into each expert prediction branch for feature fusion. For example, the computational form of the environmental conditional coding branch can be expressed as follows: ; ;

[0030] in, Represents the hidden layer feature vector. Using the GELU activation function, This represents the weight matrix of the first fully connected layer. This represents the bias vector of the first fully connected layer. The environmental features are input into the environmental condition coding branch. The high-dimensional conditional vector obtained by mapping. This represents the weight matrix of the second fully connected layer. This represents the bias vector of the second fully connected layer.

[0031] To enable the expert hybrid model to adaptively select appropriate prediction strategies based on different terrain conditions, such as staircases, this embodiment of the invention further designs a routing branch for weight allocation among the experts. As one possible implementation, the routing branch can be implemented as consisting of two fully connected layers and an intermediate activation function, such as... Figure 2 As shown, the fully connected layer can be a linear layer, and the intermediate activation function can be Tanh activation. Preferably, the calculation process of the routing branch can be expressed as follows:

[0032]

[0033] in, This represents the hidden layer feature vector, where tanh is the activation function. This represents the weight matrix of the first fully connected layer in the routing branch. This represents the bias vector of the first fully connected layer of the routing branch. As an environmental feature, The unnormalized scores for each expert's prediction branch are given. This represents the weight matrix of the second fully connected layer in the routing branch. This represents the bias vector of the second fully connected layer of the routing branch. In this embodiment of the invention, environmental features... The input used as the routing branch is the unnormalized score for each expert. The core function of this approach is to act as an "expert selection switch for environmental perception," thereby enabling adaptive routing. This means the model no longer relies solely on motion data to assign experts, but dynamically selects the expert best suited to the current scenario based on environmental conditions such as stair height, thus improving prediction accuracy in complex terrain. If the model relies solely on motion history sequences to implicitly infer terrain, it not only requires a large amount of additional parameters to separate environmental factors from temporal drivers, but is also prone to fluctuations in routing decisions due to sensor noise, directly incorporating explicit environmental features... Injecting routing branches allows the model to proactively and pre-select expert assignments based on physical priors in the early stages of feature extraction, avoiding blind competition among experts without prior information. Furthermore, since the relative ratio of stair height to leg length directly determines the most suitable gait strategy, expert routing decisions based on this ratio can significantly improve the model's generalization ability and response accuracy in unfamiliar stair scenarios.

[0034] The unnormalized score of each expert prediction branch is obtained through the routing branch calculation. ,like Figure 2 As shown, in a preferred embodiment, the expert hybrid model of this invention further normalizes the calculation results of the routing branches. After softmax normalization, the gating weights of each expert prediction branch are finally obtained, so that the gait prediction model of this invention can predict the gait cloud measurement results based on the gating weights assigned to each expert prediction branch. For example, the softmax normalization can be calculated using the following formula:

[0035] in, Let represent the gating weight of the k-th expert prediction branch, K represent the number of expert prediction branches, exp be the natural exponential function with base e, and k represent the number of the expert prediction branch. This represents the unnormalized score of the k-th expert prediction branch, calculated from the routing branch.

[0036] As one possible implementation, the expert prediction branches are designed to include at least two, each expert prediction branch being mapped by a first-layer fully connected map (e.g., ...). Figure 2 The diagram shows options for linear layers, GELU activation, and a second fully connected output (e.g., a linear layer). Figure 2 The diagram shows a composition that can be made using linear layers, for example, such as... Figure 2 As shown, the expert prediction branch includes three parts (labeled Expert1, Expert2, and Expert3). In each expert prediction branch, the output of the temporal feature backbone network is first concatenated with the high-dimensional conditional vector to obtain a fused representation. The concatenation form can be expressed by the following formula:

[0037] in, This represents the output of the last layer of the backbone network. It is a high-dimensional conditional vector. The fused representation obtained by splicing together the expert prediction branches.

[0038] After obtaining the fusion representation, each expert prediction branch will continue to calculate the branch gait prediction result based on the obtained fusion representation. Preferably, in order to enable different expert prediction branches to learn prediction mappings under different stair conditions based on the shared input fusion representation, thereby enhancing the model's ability to express multiple environmental constraints, the calculation form of the branch gait prediction result calculated by the k-th expert prediction branch based on the obtained fusion representation can be designed as follows:

[0039]

[0040] in, Using the GELU activation function, This represents the intermediate hidden features of the k-th expert prediction branch after the first fully connected layer mapping and activation. This represents the weight matrix of the first fully connected layer of the k-th expert prediction branch. For fusion representation, Let represent the bias vector of the first fully connected layer mapping for the k-th expert prediction branch. This represents the weight matrix of the second fully connected layer output of the k-th expert prediction branch. Let represent the bias vector of the second fully connected layer output of the k-th expert prediction branch. This represents the branch gait prediction result output by the k-th expert prediction branch. Since all expert prediction branches obtain their fusion representations through concatenation, they all receive the same fusion representation, ensuring a unified judgment benchmark. However, because each expert prediction branch in this embodiment has completely independent fully connected parameters and receives differentiated gradient backpropagations of the gating weights assigned by the routing branch during training, this forces different expert prediction branches to focus on samples within different stair height ranges. Furthermore, since a high-dimensional conditional vector has been explicitly incorporated into the fusion representation... This means that each expert prediction branch learns the mapping rules in a specific terrain context. By having multiple independent expert prediction branches work together to cover complex terrain spaces, the overall expressive power and generalization performance of the model can be improved.

[0041] After obtaining the gait prediction results of each branch, the expert hybrid model ultimately uses the gating weights given by the routing branch to perform a weighted fusion of the gait prediction results of each branch, in order to obtain the output of the entire model, i.e., the gait prediction result. The weighted fusion of the gait prediction results of each branch using the gating weights given by the routing branch can be achieved by the following formula:

[0042] in, This represents the gait prediction result output by the expert hybrid model, where K is the number of expert prediction branches and k is the branch number. The branch gait prediction result output by the k-th expert prediction branch. Let be the gating weight of the k-th expert prediction branch.

[0043] Therefore, by designing expert hybrid models such as Figure 2 The network structure shown in this embodiment of the invention enables the expert hybrid model to collaboratively model human historical motion information and environmental condition information. Furthermore, the expert hybrid model in this embodiment of the invention does not treat the stair height as a discrete category label (such as "high / medium / low") for rote memorization, but rather treats it as a continuous high-dimensional conditional vector. By injecting data layer by layer into the temporal backbone network and the expert prediction branch, this design allows the model network to learn "what changes with environmental characteristics." The mapping pattern of "how gait features smoothly evolve with continuous changes" is observed, and when encountering intermediate heights not present in the training set, the routing branch will adjust according to the environmental features. The system dynamically assigns expert weights, and the outputs of different experts are softly interpolated and fused to smoothly generalize to unseen terrain. It also adaptively selects more suitable prediction strategies based on different terrain conditions, such as staircases, thereby improving the accuracy and generalization ability of gait prediction in complex terrain scenarios such as staircases.

[0044] The training process of the expert hybrid model in this embodiment of the invention will be described below.

[0045] The expert hybrid model in this invention can be trained based on supervised learning. Taking the input as a sequence of hip joint angles, hip joint angular velocities, and hip joint angular accelerations from the past 24 steps, along with environmental features, and the output as a sequence of knee joint angles for the next 5 steps, as an example, three sets of inertial measurement units (IMUs) (such as Xsens MTw motion sensors) can be worn on the test subject's waist, thigh, and calf respectively. A staircase scenario with different desired heights can be set up to collect continuous kinematic signals. Then, based on the continuously collected kinematic signals, a sliding window is used to construct supervised learning samples on the continuous sequence to form at least one set of sample data. Each set of sample data includes an input sequence X, a target sequence Y, and environmental features. The input sequence X consists of the hip joint angle sequence, hip joint angular velocity sequence, and hip joint angular acceleration sequence from the past 24 steps, while the target sequence Y consists of the knee joint angle sequence from the next 5 steps. The input sequence X, target sequence Y, and environmental features are also considered. There is a correspondence, that is, the target sequence Y is the input sequence X in a certain environmental feature. The measurement results are as follows. After the sample data is formed, the expert hybrid model can be trained based on supervised learning. For example, during training, the model parameters can be set as follows: conditional encoding dimension cond_dim=8, layer feature dimension hidden_dim=160, backbone network depth depth=3, and number of experts n_experts=3. Training uses the AdamW optimizer, with an initial learning rate set to... Weight decay is set to The batch size is set to 64, the maximum number of training epochs is set to 80, and gradient pruning, adaptive learning rate decay, and early stopping strategies are adopted. The loss function is uniformly the mean squared error (MSE). Preferably, after constructing supervised learning sample data based on a sliding window, the sample data can be further divided. For example, within each training pool, the sample data can be divided into training and validation sets according to trial. The validation set ratio can be set to, for example, 0.2. During model training, the validation set RMSE is used as the optimal model selection criterion, and the model weight with the smallest compromise validation RMSE is saved. By training the above expert hybrid model, the gait prediction model of this invention, which takes the first data information and the second data information as input and the gait prediction result as output, can be obtained.

[0046] Once the model is trained, it can be used in practical applications, such as... Figure 1 The method shown inputs the acquired first and second data information into the trained gait prediction model to obtain gait prediction results. As a result, wearable exoskeleton devices such as lower limb exoskeletons and wearable exoskeleton robots can use the obtained gait prediction results for precise auxiliary control or trajectory planning.

[0047] The following combination Figures 3 to 6 The performance of the expert hybrid gait prediction model in this embodiment of the invention will be described.

[0048] To clearly illustrate the performance of the expert hybrid gait prediction model in this embodiment of the invention, preferably, two types of baseline network models (such as LSTM (Long Short-Term Memory Network) and TCN (Temporal Convolutional Network) can be constructed simultaneously. Figure 3 , Figure 4 In contrast, in the comparative experiment, the expert hybrid gait prediction model and its baseline model of this invention adopted a unified training configuration and evaluation protocol. All models were trained based on supervised learning, with the input being the human kinematic history sequence of the past 24 steps and the output being the knee joint angle sequence of the next 5 steps. Without using the knee joint history angle setting, the input feature dimension of the human kinematic history sequence of the past 24 steps was 3, specifically including thigh angle, angular velocity, and angular acceleration, combined with environmental condition features. To evaluate the model's predictive performance under different generalization conditions, this invention employs two types of cross-validation protocols: (1) Leave-One-Subject-Out (LOSO): Data collected from 3 subjects are used as the test set in turn, and the remaining subjects form the training pool. The validation set is then divided from the training pool to evaluate the model’s cross-individual generalization ability.

[0049] (2) Leave-One-Height-Out (LOHO): Five different stair height conditions are used as the test set in turn, and the remaining heights form the training pool. The validation set is then divided from the training pool to evaluate the generalization ability of the model under the cross-height conditions.

[0050] The root mean square error (RMSE) and mean absolute error (MAE) are used to measure the overall prediction error level.

[0051] Three healthy participants were recruited for the experiment. All participants had no history of lower limb motor dysfunction or neurological diseases and were able to independently perform daily walking and stair climbing. Basic information about the participants is shown in Table 1.

[0052] Table 1 Basic Information of Subjects

[0053] The prediction results of the expert hybrid gait prediction model in this embodiment of the invention under LOSO and LOHO conditions are as follows: Figure 5 , Figure 6 As shown. Among them, Figure 5 The black curve in the graph represents the actual knee joint angle change curve, and the red curve represents the model prediction result. From Figure 5 It can be seen that the expert hybrid gait prediction model of this invention maintains a high degree of consistency with the actual curve in most scenarios, especially near the main trend and peak region of knee joint movement, and can perform predictions relatively stably; at the same time, as Figure 6 As shown, the mean error of the expert hybrid gait prediction model in this embodiment of the invention is relatively small overall, and the 95% confidence interval is relatively concentrated, indicating that the output of the expert hybrid gait prediction model in this embodiment of the invention has good stability. This shows that the expert hybrid gait prediction model based on human-machine-environment information in this embodiment of the invention can provide relatively accurate and stable gait prediction results in complex staircase scenarios.

[0054] In addition, the following experiments were conducted based on two baseline models (LSTM and TCN): Model comparison experiment: The expert hybrid gait prediction model of the present invention is compared with four baseline models, namely the LSTM model with environmental feature input, the LSTM model without environmental feature input, the TCN model with environmental feature input, and the TCN model without environmental feature input.

[0055] Ablation experiments: Four ablation models were set up, namely a complete model that retains both environmental feature input and expert mixing mechanism, a model that retains environmental feature input but removes expert mixing mechanism, a model that removes environmental feature input but retains expert mixing mechanism, and a model that removes both environmental feature input and expert mixing mechanism.

[0056] The experimental results are shown in Tables 2 and 3, respectively.

[0057] Table 2 Comparison of prediction results from different models

[0058] Table 3 Comparison of ablation test results

[0059] The model comparison experiments show that the expert hybrid gait prediction model of this invention achieved the best prediction performance under both LOSO and LOHO protocols. Under the LOSO protocol, its RMSE and MAE were 8.6489 ± 0.1099 and 6.8048 ± 0.0828, respectively; under the LOHO protocol, its RMSE and MAE were 5.3072 ± 0.1350 and 3.5706 ± 0.1244, respectively, both superior to the baseline models. (Compared to the best-performing baseline model...) In comparison, the RMSE of TC-MoE decreased from 9.2466 to 8.6489 under LOSO and from 6.0356 to 5.3072 under LOHO, indicating that the expert hybrid gait prediction model of this embodiment has better generalization ability under both cross-subject and cross-stair height conditions. Especially under the LOHO protocol, the expert hybrid gait prediction model of this embodiment shows a more significant performance advantage compared to the TCN model and LSTM model without environmental feature input, indicating that explicitly introducing terrain constraint information plays an important role in improving the model's prediction ability when the stair height is not seen. Further comparison of baseline models with and without environmental feature input reveals that, regardless of whether it is an LSTM or TCN structure, introducing environmental features... The performance of the models has been improved. For example, in the TCN structure, adding... Afterwards, LOHO RMSE decreased from 6.2996 to 6.0356, and LOHOMAE decreased from 4.1268 to 3.9428; under the LSTM structure, adding Subsequently, the LOSO RMSE decreased from 10.2643 to 9.2426, and the LOHO RMSE decreased from 7.4310 to 7.2071. This result indicates that terrain information can provide additional environmental constraints for the model, enabling it to more accurately adjust the prediction of knee joint movement trends under different stair height conditions.

[0060] The ablation experiments further validated the role of environmental feature input and the expert mixture mechanism. The complete model achieved RMSEs of 8.5545 ± 0.6025 and 5.1949 ± 0.2180 under LOSO and LOHO conditions, respectively, both outperforming other ablation models. When environmental feature input was retained but the expert mixture mechanism was removed, the LOHO RMSE increased to 5.6181 ± 0.3022; when the expert mixture mechanism was retained but environmental feature input was removed, the LOHO RMSE increased to 6.4717 ± 0.1657. In contrast, removing both environmental feature input and the expert mixture mechanism simultaneously further increased the LOHO RMSE to 6.6322 ± 0.1297. This indicates that environmental feature input has a more significant impact on model performance, while the expert mixture mechanism further enhances the model's adaptability to different scene variations. The combination of the two can more fully characterize gait changes under complex stair conditions and individual differences.

[0061] In practice, the expert hybrid gait prediction model described above in this embodiment of the invention can be applied to wearable exoskeleton devices such as lower limb exoskeletons and wearable exoskeleton robots for accurate gait prediction. Figure 7 The wearable exoskeleton device according to one embodiment of the present invention is schematically shown in the figure, and is implemented by including: The information acquisition module 100 is used to acquire first data information and second data information, wherein the first data information includes the thigh movement state of the target object and the second data information includes the current environmental features; The gait prediction module 200 is used to input the first data information and the second data information into a pre-trained gait prediction model and obtain the gait prediction result output by the gait prediction model.

[0062] The gait prediction module employs an expert hybrid model, which includes an input mapping layer for mapping the first input data information to an initial feature representation, a temporal feature backbone network for feature extraction based on the initial feature representation input from the input mapping layer, an environmental condition encoding branch for mapping the second data information to a high-dimensional condition vector, a routing branch for weighting each expert prediction branch, and an expert prediction branch for feature fusion of the feature data output from the temporal feature backbone network and the high-dimensional condition vector output from the environmental condition encoding branch. The high-dimensional condition vector obtained from the environmental condition encoding branch is used for both conditional modulation of the temporal feature backbone network and feature fusion of the expert prediction branch. The output of the expert hybrid model is obtained by weighted fusion of the prediction results of each expert prediction branch using the gating weights output from the routing branch.

[0063] It should be noted that the specific implementation processes of the information acquisition module 100 and the gait prediction module 200 in this embodiment of the invention, as well as the network structure and training process of the expert hybrid model, can be referred to the method section above, and will not be repeated here. Specifically, the information acquisition module 100 may include an IMU measurement unit, a sensor, a user input unit, etc., and the gait prediction module 200 may include a processor, a controller, etc.

[0064] In some embodiments, the present invention provides a non-volatile computer-readable storage medium storing one or more programs including execution instructions, which can be read and executed by an electronic device (including but not limited to a computer, server, or network device, etc.) to perform the gait prediction method of any of the above embodiments of the present invention.

[0065] In some embodiments, the present invention also provides a computer program product, the computer program product including a computer program stored on a non-volatile computer-readable storage medium, the computer program including program instructions, which, when executed by a computer, cause the computer to perform the gait prediction method of any of the above embodiments.

[0066] In some embodiments, the present invention also provides an electronic device comprising: at least one processor and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the gait prediction method of any of the above embodiments.

[0067] In some embodiments, the present invention also provides a computing device comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the gait prediction method of any of the above embodiments.

[0068] In some embodiments, the present invention also provides a storage medium storing a computer program that, when executed by a processor, implements the gait prediction method of any of the above embodiments.

[0069] Figure 8 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. The wearable exoskeleton device described above can be implemented using the structure shown in the diagram, as follows: Figure 8 As shown, the electronic device includes: One or more processors 610 and memory 620, Figure 8 Take the 610 processor as an example.

[0070] The electronic device may also include an input device 630 and an output device 640.

[0071] The processor 610, memory 620, input device 630, and output device 640 can be connected via a bus or other means. Figure 8 Taking the example of a connection between China and Israel via a bus.

[0072] The memory 620, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the gait prediction method in the embodiments of the present invention. The processor 610 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions, and modules stored in the memory 620, thereby implementing the gait prediction method of the above-described method embodiments.

[0073] The memory 620 may include a program storage area and a data storage area, wherein the program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the gait prediction method, etc. Furthermore, the memory 620 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 620 may optionally include memory remotely located relative to the processor 610, and these remote memories may be connected to the electronic device via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0074] Input device 630 can receive input digital or character information and generate signals related to user settings and function control of the image processing device. Output device 640 may include a display device such as a display screen.

[0075] The one or more modules are stored in the memory 620, and when executed by the one or more processors 610, they execute the gait prediction method in any of the above method embodiments.

[0076] The above-described product can execute the method provided in the embodiments of the present invention, and has the corresponding functional modules and beneficial effects for executing the method. Technical details not described in detail in this embodiment can be found in the method provided in the embodiments of the present invention.

[0077] The electronic devices of this invention exist in various forms, including but not limited to: (1) Mobile communication devices: These devices are characterized by their mobile communication capabilities and primarily aim to provide voice and data communication. These terminals include: smartphones (e.g., iPhones), multimedia phones, feature phones, and low-end phones, etc.

[0078] (2) Ultra-mobile personal computer devices: These devices fall under the category of personal computers, possessing computing and processing capabilities, and generally also have mobile internet access features. These terminals include PDAs, MIDs, and UMPCs, such as the iPad.

[0079] (3) Portable entertainment devices: These devices can display and play multimedia content. This category includes audio and video players (such as iPods), handheld game consoles, e-book readers, as well as smart toys and portable car navigation devices.

[0080] (4) Server: A device that provides computing services. The components of a server include a processor, hard disk, memory, system bus, etc. Servers are similar to general computer architectures, but because they need to provide highly reliable services, they have higher requirements in terms of processing power, stability, reliability, security, scalability, and manageability.

[0081] (5) Other electronic devices with data interaction functions.

[0082] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0083] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented using software plus a general-purpose hardware platform, or of course, using hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0084] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A gait prediction method, characterized in that, The method includes: Acquire first data information and second data information, wherein the first data information includes the thigh movement state of the target object, and the second data information includes current environmental features; Input the first data information and the second data information into the pre-trained gait prediction model, and obtain the gait prediction result output by the gait prediction model; The gait prediction model employs an expert hybrid model, comprising an input mapping layer for mapping the input first data information into an initial feature representation, a temporal feature backbone network for feature extraction based on the initial feature representation input from the input mapping layer, an environmental condition encoding branch for mapping the current environmental features into a high-dimensional condition vector, a routing branch for weighting each expert prediction branch, and an expert prediction branch for feature fusion of the feature data output from the temporal feature backbone network and the high-dimensional condition vector output from the environmental condition encoding branch. The high-dimensional condition vector obtained from the environmental condition encoding branch is used as input to both the temporal feature backbone network and each expert prediction branch. The output of the expert hybrid model is obtained by weighted fusion of the prediction results of each expert prediction branch based on the gating weights output from the routing branch.

2. The method according to claim 1, characterized in that, The temporal feature backbone network consists of multiple layers of conditional residual blocks; the environmental condition coding branch consists of two fully connected layers and an intermediate activation function; the routing branch consists of two fully connected layers and an intermediate activation function. The expert prediction branch includes at least two branches, each consisting of a first-layer fully connected mapping, GELU activation, and a second-layer fully connected output.

3. The method according to claim 2, characterized in that, The thigh motion state includes hip joint angle, hip joint angular velocity, and hip joint angular acceleration; the current environmental features are normalized parameters determined by the height of the stairs in the current environment and the height of the target object; the gait prediction results include a sequence of knee joint angles for a preset number of steps.

4. The method according to claim 3, characterized in that, The first of the time-series feature backbone network The update form of the layer conditional residual block is as follows: ； in, Indicates the first Layer conditional residual block, Indicates the first Normalized results of layer-conditional residual blocks. This represents a mapping consisting of linear layers and nonlinear activation functions. The high-dimensional condition vector output by the environmental condition encoding branch.

5. The method according to claim 3, characterized in that, Each expert prediction branch is implemented by concatenating the output of the temporal feature backbone network with a high-dimensional conditional vector to obtain a fused representation, and then calculating the branch prediction result based on the obtained fused representation. The calculation form for the k-th expert prediction branch's prediction result based on the obtained fused representation is as follows: ； ； in, Using the GELU activation function, This represents the intermediate hidden features of the k-th expert prediction branch after the first fully connected layer mapping and activation. This represents the weight matrix of the first fully connected layer of the k-th expert prediction branch. For fusion representation, Let represent the bias vector of the first fully connected layer mapping for the k-th expert prediction branch. This represents the weight matrix of the second fully connected layer output of the k-th expert prediction branch. Let represent the bias vector of the second fully connected layer output of the k-th expert prediction branch. This represents the branch prediction result output by the k-th expert prediction branch.

6. The method according to claim 3, characterized in that, The current environmental characteristics are calculated in the following form: ； in, Indicates the current stair height. This represents the height of the target person, and 0.53 is the ratio coefficient between the length of the human leg and the height.

7. The method according to claim 3, characterized in that, The current environmental features are also input into the routing branch, which determines the weights assigned to each expert prediction branch based on the input current environmental features.

8. A wearable exoskeleton device, characterized in that, include: The information acquisition module is used to acquire first data information and second data information, wherein the first data information includes the thigh movement state of the target object, and the second data information includes the current environmental features; The gait prediction module is used to input the first data information and the second data information into a pre-trained gait prediction model and obtain the gait prediction result output by the gait prediction model. The gait prediction model employs an expert hybrid model, comprising an input mapping layer for mapping the input first data information to an initial feature representation, a temporal feature backbone network for feature extraction based on the initial feature representation input from the input mapping layer, an environmental condition encoding branch for mapping the current environmental features to a high-dimensional condition vector, a routing branch for weighting each expert prediction branch, and an expert prediction branch for feature fusion of the feature data output from the temporal feature backbone network and the high-dimensional condition vector output from the environmental condition encoding branch. The high-dimensional condition vector obtained from the environmental condition encoding branch is used as input to both the temporal feature backbone network and each expert prediction branch. The output of the expert hybrid model is obtained by weighted fusion of the prediction results from each expert prediction branch using the gating weights output from the routing branch.

9. A storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the steps of the method described in any one of claims 1 to 7.

10. A computing device, characterized in that, include: A processor and a memory are communicatively connected, the memory storing instructions executable by the processor to enable the processor to perform the steps of the method according to any one of claims 1 to 7.

Images