Gait sub-phase prediction method and device based on deep learning model fusion

Abstract

A gait sub-phase prediction method and device based on deep learning model fusion, the method comprising: synchronously collecting multi-source gait data of a wearer walking at multiple speeds; preprocessing the collected multi-source gait data, dividing time windows by using a sliding overlapping window technique, and constructing a supervised learning training data set based on double labels; hierarchically fusing CNN, Bi-LSTM and MHA to construct a gait sub-phase prediction model; using a Bayesian optimization algorithm to automatically search for an optimal hyperparameter combination; defining a stable prediction time (P stable） metric, combining classical metrics, and comprehensively evaluating the model performance through leave-one-subject-out cross-validation and speed transfer robustness experiments; constructing a modular gait sub-phase prediction system architecture based on the Simulink platform, and integrating a multi-thread buffer mechanism to process data streams in real time; solidifying model parameters into an online inference module to dynamically output prediction results.

Description

Technical Field

[0001] The present invention belongs to the field of lower limb rehabilitation exoskeleton robot perception technology, and specifically relates to a gait sub-phase prediction method and device based on deep learning model fusion. Background Art

[0002] Lower limb rehabilitation exoskeleton robots need to sense and predict sub-phases in the gait cycle in real time to ensure appropriate assistance timing and support strength. However, the gait cycle exhibits complex nonlinear dynamic characteristics in practical applications due to individual differences, speed changes and other factors, making traditional prediction methods face severe challenges in prediction time range, robustness and generalization. As the prediction time range increases, the lack of long-term dependence and error accumulation problems limit the stability and accuracy of existing methods in practical scenarios. Secondly, the significant differences in height, stride, weight and exercise habits among different individuals make it difficult for existing algorithms to maintain consistent prediction performance in cross-individual applications. In addition, the sensitivity of the gait cycle to walking speed further exacerbates the complexity of prediction. Traditional models based on linear assumptions or fixed cycles show poor robustness under speed changes.

[0003] The single application of deep learning technology in existing methods has alleviated these problems to a certain extent, but there are still significant limitations. For example, the Convolutional Neural Network (CNN) has advantages in local feature extraction, but it has limitations in processing the long-term dependence and nonlinear dynamic characteristics of time series; although the Long Short-Term Memory (LSTM) can capture the long-term dependence of time series, its ability to capture nonlinear dynamic characteristics is limited; the simple application of the existing attention mechanism (Attention) is difficult to fully extract the key features in the gait signal, and fails to reasonably allocate weights in complex and changeable gait scenarios. Therefore, it is urgent to design a gait sub-phase prediction method based on deep learning model fusion to effectively make up for the shortcomings of a single model in practical applications, thereby enhancing the model's dynamic adaptability to multiple scenarios. The breakthrough of this problem not only has important research value, but will also provide solid technical support for the practical application of lower limb rehabilitation exoskeleton robots, thereby significantly improving the effect and experience of patient rehabilitation training. Summary of the invention

[0004] In order to overcome the shortcomings of existing gait sub-phase prediction technology in terms of prediction time range, speed change robustness and cross-individual generalization, the present invention proposes a gait sub-phase prediction method and device based on deep learning model fusion, which significantly improves the real-time and accuracy of the model while maintaining excellent cross-individual generalization ability and speed change robustness.

[0005] To achieve the above objectives, the first aspect of the present invention relates to a gait sub-phase prediction method based on deep learning model fusion, comprising the following steps:

[0006] S1: Through variable speed walking experiments, multi-source gait data of different individuals wearing lower limb rehabilitation exoskeleton robots at various speeds are collected, including ground contact force (GCF), surface electromyography (sEMG), and inertial measurement unit (IMU) signals. At the same time, a camera is used to record the walking process video; a unified marking event is used to mark the start of the experiment to accurately align the timestamps of the multi-source gait data and the walking process video.

[0007] S2: Preprocess the collected multi-source gait data, including filtering, denoising, and normalization, and assign corresponding gait sub-phase labels to the preprocessed multi-source gait data based on GCF and the walking process video.

[0008] S3: The sliding overlapping window technique is used to divide the multi-source gait data into time windows of fixed length and assign a true label (T) to each window. label ) and the predicted label (P label ), and set the prediction time advance (P w ) to construct a standardized supervised learning training dataset.

[0009] S4: By hierarchically fusing convolutional neural networks (CNN), bidirectional long-short-term memory networks (Bi-LSTM), and multi-head attention mechanisms (MHA), a deep learning model fusion architecture is formed to construct a gait sub-phase prediction model.

[0010] S5: The Bayesian optimization algorithm is used to automatically optimize the model hyperparameters to explore the best hyperparameter combination; then, the stratified K-fold cross-validation strategy is used for model training and evaluation, and the model checkpoint mechanism is enabled to prevent overfitting.

[0011] S6: Introducing multiple classic metrics (including accuracy, precision, recall, F1 score, confusion matrix) and a custom metric: stable prediction time (P stable ), and comprehensively evaluate the prediction performance of the gait sub-phase prediction model.

[0012] S7: Statistical prediction accuracy and P of gait sub-phase prediction model under different prediction time advance conditions stable ; Visual comparison through line chart, showing different P settings w The performance of the models under different conditions.

[0013] S8: Conduct subject-by-subject elimination cross-validation and speed migration robustness validation experiments to evaluate the adaptability and stability of the gait sub-phase prediction model under conditions of cross-individual and speed changes, and draw box plots to visualize the experimental results.

[0014] S9: Build a modular gait sub-phase prediction system architecture on the Simulink platform, integrate multi-source gait data acquisition, signal preprocessing, online reasoning and result output modules, and solidify the pre-trained and cross-validated gait sub-phase prediction model parameters into the online reasoning module.

[0015] S10: Real-time acquisition and storage of multi-source gait data, input of sEMG and IMU signals into the modular gait sub-phase prediction system architecture, and real-time processing of data streams using a multi-threaded buffering mechanism, execution of signal processing and online reasoning modules, and dynamic output of gait sub-phase prediction results.

[0016] The second aspect of the present invention relates to a gait sub-phase prediction device based on deep learning model fusion, including a display interface, a memory and one or more processors, wherein the memory stores executable code, and when the one or more processors execute the executable code, they are used to implement the gait sub-phase prediction method based on deep learning fusion of the present invention.

[0017] The present invention constructs a high-quality supervised learning training data set by synchronously collecting multi-source gait data and combining data preprocessing, gait sub-phase label assignment and sliding overlapping window technology; forms a deep learning model fusion architecture by hierarchical fusion of convolutional neural network (CNN), bidirectional long short-term memory network (Bi-LSTM) and multi-head attention mechanism (MHA) to construct a gait sub-phase prediction model; uses Bayesian optimization algorithm to automatically optimize the model hyperparameters and explore the best hyperparameter combination; introduces classic evaluation indicators (including accuracy, precision, recall rate, F1 score, confusion matrix) and custom evaluation indicators: stable prediction time (P stable), comprehensively evaluate the performance of the gait sub-phase prediction model; evaluate the adaptability and stability of the model under cross-individual and speed change conditions through subject elimination cross-validation and speed migration robustness verification experiments; build a modular gait sub-phase prediction system architecture on the Simulink platform, integrate multi-source gait data acquisition, signal preprocessing, online reasoning and result output modules, and solidify the pre-trained and cross-validated model parameters into the online reasoning module; collect and store multi-source gait data in real time, input surface electromyography (sEMG) and inertial measurement unit (IMU) signals into the modular gait sub-phase prediction system architecture, use a multi-threaded buffer mechanism to process data streams in real time, execute signal processing and online reasoning modules, and dynamically output the prediction results of gait sub-phases.

[0018] The innovation of the present invention lies in: forming a deep learning model fusion architecture by hierarchically fusing convolutional neural networks (CNN), bidirectional long short-term memory networks (Bi-LSTM) and multi-head attention mechanisms (MHA) to construct a gait sub-phase prediction model; automatically optimizing the hyperparameters of the gait sub-phase prediction model through the Bayesian optimization algorithm, exploring the best hyperparameter combination, and significantly improving the model performance; defining the stable prediction time P stable , and combined with classic evaluation indicators such as accuracy, precision, recall rate, F1 score and confusion matrix, a multi-dimensional evaluation indicator system was constructed to comprehensively evaluate the prediction performance of the gait sub-phase prediction model; a modular gait sub-phase prediction system architecture was built based on the Simulink platform, and data streams were processed in real time through a multi-threaded buffer mechanism; the pre-trained and cross-validated model parameters were solidified into the online inference module, and the prediction results of the gait sub-phase were dynamically output.

[0019] The working principle of the present invention is as follows: multi-source gait data of different individuals wearing lower limb rehabilitation exoskeleton robots are synchronously collected through variable speed walking experiments, and the multi-source gait data are accurately aligned with the timestamps of the walking process video based on unified marking events; a high-quality supervised learning training data set is constructed through sliding overlapping window technology and dual-label mechanism; a deep learning model fusion architecture is formed through hierarchical fusion of CNN, Bi-LSTM and MHA, and a gait sub-phase prediction model is constructed; the hyperparameters of the gait sub-phase prediction model are automatically optimized through the Bayesian optimization algorithm, the optimal hyperparameter combination is explored, and the model generalization ability and overfitting risk are balanced; a modular gait sub-phase prediction system architecture is constructed based on the Simulink platform, and data streams are processed in real time through a multi-threaded buffering mechanism; the pre-trained and cross-validated model parameters are solidified into the online reasoning module, and the prediction results of the gait sub-phases are dynamically output.

[0020] The advantages of the present invention are: accurately aligning the timestamps of multi-source gait data and walking process videos by uniformly marking events; using sliding overlapping window technology and combining with P-based w The dual-label mapping architecture is combined to realize the nonlinear mapping of the gait characteristics of the current time window and the future gait sub-phase, and effectively model the temporal evolution law of the gait sub-phase; by hierarchically fusing CNN, Bi-LSTM and MHA, a deep learning model fusion architecture is formed to construct a gait sub-phase prediction model, which can effectively extract the complex multi-dimensional features of gait, model temporal dependencies, and realize the fusion of global and local features; the Bayesian optimization algorithm is used to automatically optimize the model hyperparameters, avoiding the complexity of manual parameter adjustment and further improving the model performance; the stable prediction time P is defined stable , and combined with classic evaluation indicators such as accuracy, precision, recall rate, F1 score and confusion matrix, a multi-dimensional evaluation index system was constructed to quantify the comprehensive performance of the gait sub-phase prediction model in terms of prediction accuracy, timeliness and stability; through cross-validation and speed migration robustness verification experiments with subject elimination one by one, the stable prediction performance of the model under the conditions of individual differences and speed changes was verified to support the personalized adaptation needs of lower limb rehabilitation exoskeleton robots; based on the Simulink platform, a modular gait sub-phase prediction system architecture was constructed, integrating multi-threaded buffering and pre-trained model solidification technology to achieve pipeline parallelism of multi-source gait data acquisition, preprocessing, and online reasoning, and dynamically output gait sub-phase prediction results to perform real-time prediction of gait sub-phases. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a schematic diagram of the overall process in a specific embodiment of the present invention;

[0022] FIG2 (a) and FIG2 (b) are schematic diagrams of an experimental setup in a specific embodiment of the present invention, wherein FIG2 (a) is a schematic diagram of a constructed lower limb rehabilitation exoskeleton robot experimental platform, and FIG2 (b) is an overall schematic diagram of a variable speed walking experimental scheme;

[0023] Figure 3 This is an overall architecture diagram of a gait sub-phase prediction model constructed in a specific embodiment of the present invention;

[0024] Figures 4 (a) to 4 (e) are comparison diagrams of confusion matrices of different models in a specific embodiment of the present invention, wherein Figure 4 (a) is a diagram of the prediction result of CNN, Figure 4 (b) is a diagram of the prediction result of Bi-LSTM, Figure 4 (c) is a diagram of the prediction result of MHA, Figure 4 (d) is a diagram of the prediction result of CNN-BiLSTM, and Figure 4 (e) is a diagram of the prediction result of the gait sub-phase prediction model proposed in the present invention;

[0025] FIG. 5( a ) and FIG. 5( b ) are diagrams showing different models at different P values ​​in a specific embodiment of the present invention. w The line comparison chart of the prediction performance under the setting conditions, where Figure 5 (a) is P stable About P w Figure 5(b) is the trend diagram of the prediction accuracy with respect to P w ;

[0026] Figure 6 (a) to Figure 6 (c) are box plots of the cross-individual generalization ability evaluation of different models at three speeds in a specific embodiment of the present invention, wherein Figure 6 (a) is a slow speed, Figure 6 (b) is a medium speed, and Figure 6 (c) is a fast speed;

[0027] Figure 7 It is a box plot of speed variation robustness evaluation of different models under training and verification strategies of different speed combinations in a specific embodiment of the present invention;

[0028] Figure 8 The figure is a comparison chart of the prediction performance of different models in the speed combination range in a specific implementation manner of the present invention.

[0029] Fig. 9 It is a schematic diagram of the device of the present invention. DETAILED DESCRIPTION

[0030] The following examples of the invention, combined with the accompanying drawings, specifically explain the implementation plan, data processing, model construction and training, and performance verification, so as to further clarify the application field, design ideas and technical solutions of the invention.

[0031] Example 1

[0032] This embodiment relates to a gait sub-phase prediction method based on deep learning model fusion, such as Figure 1 As shown, the following steps are included:

[0033] Step S1: Through variable speed walking experiments, multi-source gait data of different individuals wearing lower limb rehabilitation exoskeleton robots at various speeds are collected, including ground contact force (GCF), surface electromyography (sEMG), and inertial measurement unit (IMU) signals. At the same time, a camera is used to record the walking process video; a unified marking event mark is used to mark the start of the experiment to accurately align the timestamps of the multi-source gait data and the walking process video.

[0034] As shown in Figure 2 (a), in this variable-speed walking experiment, the lower limb rehabilitation exoskeleton robot worn by the subjects adopted the zero-torque mode, and surface electromyography sensors, ground contact force insoles, and inertial measurement units were installed on the robot's lower limbs to synchronously collect multi-source gait data during walking. A six-lead electromyography sensor was used to record the sEMG of four muscles (8 channels in total) in both legs. The electrodes used bipolar active Ag / AgCl electrodes and were placed according to the surface electromyography standardization (SENIAM) scheme. The surface electromyography sensor had a sampling rate of 1000 Hz and monitored the electrical activity of four muscle groups: the rectus femoris, vastus medialis, vastus lateralis, and medial gastrocnemius. The IMU signal was collected by the DETA10 series micro inertial navigation system of FDISYSTEMS, which was installed at the robot's knee joint to monitor posture changes during walking in real time and record the yaw, roll, and pitch angles. The sampling rate was also 1000 Hz. To measure GCF, a commercial pressure insole RX-ES42-18 was used and fixed on the sole of the lower limb rehabilitation exoskeleton robot with a sampling rate of 100 Hz to collect GCF during walking (a total of 18 channels).

[0035] A total of 12 healthy subjects (7 males, 5 females, height 155.4-186.7cm, weight 43.9-98.7kg, age 22-28 years old) were recruited for the variable speed walking experiment. In order to reduce the impact of footwear differences on the experiment, all subjects wore flat-soled sports shoes. The experimental scheme is as follows: the subjects first walked on the treadmill at speeds of 1.0 km / h, 1.5 km / h and 2.0 km / h, and each speed lasted for 60 seconds. Then, the treadmill entered the automatic mode, and the subjects started from 1.0 km / h, gradually accelerated to 2.0 km / h, and maintained this speed for 60 seconds. After the acceleration phase, the subjects walked at a reverse speed (2.0 km / h, 1.5km / h and 1.0 km / h) for 60 seconds each to complete an experimental cycle. Each subject was required to complete 5 experimental cycles, and sufficient rest time was provided after each cycle to relieve muscle fatigue. The overall experimental scheme is shown in Figure 2 (b).

[0036] In order to ensure the time synchronization between the multi-source gait data and the video recording, the experiment adopted a unified marking event: at the beginning of the experiment, the subject tapped the treadmill surface with the toe of his right foot to generate an obvious high-voltage spike signal, and the action was accurately captured by the camera; the marking event was used as a timestamp reference to ensure that the subsequently collected multi-source gait data was accurately aligned with the walking process video.

[0037] Step S2: preprocessing the collected multi-source gait data, including filtering, denoising, and normalization, and assigning corresponding gait sub-phase labels to the preprocessed multi-source gait data based on GCF and the walking process video.

[0038] First, for the collected sEMG (8 channels in total), a 50Hz notch filter was applied to remove power line interference. Then, a 5th-order Butterworth filter was used to bandpass filter the sEMG from 25Hz to 400Hz. Next, the maximum-minimum normalization method was used for amplitude standardization; the IMU signal was low-pass filtered at 5Hz using a 4th-order Butterworth filter to remove baseline drift, and the same maximum-minimum normalization was performed; the collected GCF was low-pass filtered at 40Hz using a 4th-order Butterworth filter, and after the baseline drift was removed, the filtered and denoised GCF was upsampled to 1000Hz using the cubic spline interpolation method to ensure consistency with the sampling rate of the sEMG and IMU signals.

[0039] Next, after filtering and denoising the 18-channel GCF, it was divided into four groups according to the "forefoot", "arch", "front heel" and "heel" regions, and the average value of the GCF of each group was calculated to obtain the average values ​​of the four groups of GCF. Then, the average values ​​of the four groups of GCF were processed based on threshold binarization, and the threshold was set to 5% of the subject's body weight. The four groups of GCF were converted into ground state (value "1") or ground state (value "0"), and the binarized values ​​were spliced ​​into a four-dimensional pressure distribution vector to provide support for the assignment of gait sub-phase labels for subsequent training data sets.

[0040] Subsequently, 10% of the original videos were randomly selected from the walking process videos and converted into static images at a frequency of 20 frames per second. Based on the definition of 8 types of gait sub-phases, the static images were annotated and the corresponding gait sub-phase labels were assigned, including: Initial Contact (IC), Loading Response (LR), MidStance (MSt), Terminal Stance (TSt), Pre-swing (PSw), Initial Swing (ISw), Mid Swing (MSw) and Terminal Swing (TSw). By uniformly marking event alignment images and multi-source gait data, the gait sub-phase labels of the static images were synchronized to the multi-source gait data, and the corresponding gait sub-phase labels were assigned to each sampling point. Subsequently, the time proportion of each subject in the three gait sub-phases (ISw, MSw and TSw) in the swing phase was calculated at different speeds.

[0041] Finally, by analyzing the four-dimensional pressure distribution vector, it is preliminarily determined whether the gait is in the support phase or the swing phase. If any element in the four-dimensional pressure distribution vector is non-zero, the gait is in the support phase; if all elements are zero, the gait is in the swing phase. On this basis, according to the distribution of "1" in the four-dimensional pressure distribution vector, the label assignment of the five gait sub-phases (IC, LR, MSt, TSt and PSw) in the support phase is clarified. If the four-dimensional pressure distribution vector is , assign the label "1", indicating that it is in "initial contact"; if the four-dimensional pressure distribution vector is or , assign label "2", indicating that it is in "load response"; if the four-dimensional pressure distribution vector is , assign label "3", indicating "standing in the middle"; if the four-dimensional pressure distribution vector is or , assign the label "4", indicating "terminal standing"; if the four-dimensional pressure distribution vector is , assign the label "5" to indicate that it is in the "pre-swing" phase. At the same time, according to the time sequence and the time proportion of the three gait sub-phases in the swing phase, the gait multi-source data initially assigned the label "9" are replaced with the labels "6", "7" and "8" in sequence. Among them, the label "6" indicates that it is in the "initial swing", the label "7" indicates that it is in the "mid-swing", and the label "8" indicates that it is in the "final swing".

[0042] Step S3: Use the sliding overlapping window technique to divide the multi-source gait data into time windows of fixed length and assign a true label (T label ) and the predicted label (P label ), and set the prediction time advance (P w ) to construct a standardized supervised learning training dataset.

[0043] The sliding overlapping window technique is used to divide the multi-source gait data processed in step S2 into time windows of fixed length. The window size is set to 30 sampling points and the sliding step is 5 sampling points. The specific operation is as follows: starting from the first sampling point of the multi-source gait data, 30 sampling points are intercepted to form the first time window; then, each time the step is 5 sampling points, the time window of the same length is intercepted as the next sample until the entire sequence is covered.

[0044] Each time window is assigned two labels: the true label (T label ) and predicted labels (P label ). Among them, T label Indicates the highest frequency label of the gait sub-item in the current time window, P label is the distance P from the current time window w The time window of step size Tlabel , P w is a preset constant, indicating the amount of advance prediction time. In this way, P label Mapped with the data features of each time window to achieve P w The input of the training dataset is a 30 × N matrix, where N is the number of signal channels and the training label is the P corresponding to each time window. label , so as to realize the data characteristics of the current time window and P w Post-temporal gait subphase formation mapping.

[0045] Step S4: By hierarchically fusing convolutional neural networks (CNNs), bidirectional long short-term memory networks (Bi-LSTMs), and multi-head attention mechanisms (MHAs), a deep learning model fusion architecture is formed to construct a gait sub-phase prediction model.

[0046] First, the local features of the input data are extracted through the convolution layer, and the two-dimensional convolution kernel is used to slide along the joint time-space dimension; then, the maximum pooling layer and the global average pooling layer are connected in sequence for feature compression. The window size of the maximum pooling layer is 2×2, and the step size is set to 2;

[0047] Subsequently, the pooled output is adjusted to a three-dimensional tensor through a vector reshaping layer to adapt to the input format of the Bi-LSTM; the Bi-LSTM adopts a four-layer stacking structure, the number of hidden neurons decreases layer by layer, and the long-term and short-term dependencies are extracted through a bidirectional gating mechanism;

[0048] Finally, the output of the last layer of Bi-LSTM is input into three fully connected layers with 64 hidden neurons to generate three query vectors, key vectors and value vectors with 64 dimensions. The multi-head attention mechanism MHA with 3 attention heads is used for feature interaction modeling, and the calculation dimension of each attention head is set to 8. A batch normalization layer is set at the output of MHA to optimize the data distribution, and the normalized momentum parameter is set to 0.99. After the dimension is reduced by the global average pooling layer, it is connected to the relay fully connected layer with 64 hidden neurons, and the ReLU activation function is used for nonlinear transformation. Finally, the Softmax classifier is used to output the 8×1 gait sub-phase probability distribution. At this point, the gait sub-phase prediction model is constructed, and its overall architecture is as follows: Figure 3 shown.

[0049] Step S5: Use the Bayesian optimization algorithm to automatically optimize the model hyperparameters to explore the best hyperparameter combination; then, use the stratified K-fold cross-validation strategy to train and evaluate the model, and enable the model checkpoint mechanism to prevent overfitting.

[0050] First, a multidimensional hyperparameter space is constructed, which includes five dimensions: initial learning rate, batch size, number of two-dimensional convolution kernels, size of two-dimensional convolution kernels, and number of hidden neurons in the four-layer Bi-LSTM. The candidate set of initial learning rate is {10 -5 , 10 -4 , 10 -3 , 10 -2 , 10 -1}, the candidate set of batch size is {16, 32, 64, 128, 256}, the candidate set of two-dimensional convolution kernel number is {8, 16, 64, 128, 256}, the candidate set of two-dimensional convolution kernel size is {1×1, 3×3,5×5, 7×7}, and the candidate set of Bi-LSTM four-layer hidden neuron number is {16, 32, 64, 128, 256}.

[0051] Subsequently, the Gaussian process proxy model is initialized, the covariance matrix is ​​constructed based on the radial basis function, and the nonlinear mapping relationship between the hyperparameter combination and the model performance index is established. The initial hyperparameter combination set is obtained through Latin hypercube sampling, the model training is performed and the F1 score of the validation set is recorded as the objective function value to complete the pre-training of the proxy model. On this basis, the expected improvement criterion is used to optimize the hyperparameter combination, and the expected improvement value of the candidate hyperparameter combination is calculated according to the posterior distribution of the current proxy model. ; The expected improvement value The mathematical expression is:

[0052] (1)

[0053] in, represents the expected value, which is the integral of the probability distribution of the function value based on the uncertainty of the current model; Indicates the hyperparameter combination The objective function value under , that is, the model performance; Represents the current best hyperparameter combination; the hyperparameter combination with the largest EI value is selected for experimentation; this process is updated in each round of optimization.

[0054] Next, the Bayesian optimization process is iterated to select the expected improvement value The largest candidate hyperparameter combination is used for model training, the newly obtained performance data is added to the training set, and the hyperparameter posterior distribution of the Gaussian process proxy model is updated. The parameters of the covariance matrix are adjusted by the kernel function to improve the fitting accuracy of the proxy model to the objective function response surface. At the same time, a double termination condition is set. When the preset maximum number of iterations is reached or the improvement of the optimal objective function value for 10 consecutive iterations is less than 1%, the optimization process is terminated, the historical optimal hyperparameter combination and the corresponding model performance index are output, and the historical optimal hyperparameter combination is applied to the gait sub-phase prediction model constructed in step S4.

[0055] Finally, when conducting formal model training, the dataset was divided into 10 non-overlapping folds, and each fold was divided into training set, validation set and test set in a ratio of 8:1:1, and the sample category distribution in each fold was guaranteed to be consistent with the original dataset. The adaptive moment estimation Adam algorithm was used to optimize the model. To prevent the model from overfitting, an early stopping mechanism was adopted, that is, when the loss of the validation set failed to improve within 10 consecutive cycles, the training was terminated early; the model checkpoint mechanism was enabled during the training process, and the model with the smallest validation set loss was automatically saved at the end of each round of training, so as to ensure that the best model parameters were captured and avoid performance degradation due to overfitting; all models were trained on NVIDIA 4060ti GPU.

[0056] S6: Introducing classic evaluation metrics (including accuracy, precision, recall, F1 score, confusion matrix) and a custom metric: stable prediction time (P stable ) to comprehensively evaluate the prediction performance of the gait sub-phase prediction model.

[0057] In order to quantitatively evaluate the performance of the proposed model, this paper introduces the following classic evaluation indicators:

[0058] 1) Accuracy:

[0059] (2)

[0060] 2) Precision:

[0061] (3)

[0062] 3) Recall:

[0063] (4)

[0064] Among them, TP stands for "True Positive Class", that is, the number of positive samples correctly classified as positive by the model; TN stands for "True Negative Class", that is, the number of negative samples correctly classified as negative by the model; FP stands for "False Positive Class", that is, the number of negative samples misclassified as positive by the model (false positive); FN stands for "False Negative Class", that is, the number of positive samples misclassified as negative by the model (false negative). Accuracy represents the proportion of all correct predictions (including positive and negative) of the model in the total prediction, which measures the overall correctness of the model; Precision represents the proportion of samples predicted by the model as positive that are actually positive, emphasizing the accuracy of the model when predicting as positive; Recall represents the proportion of actual positive samples correctly predicted by the model as positive, reflecting the model's ability to capture positive samples.

[0065] 4) Fraction( -Score):

[0066] (5)

[0067] in, is a non-negative parameter used to adjust the weight of Recall relative to Precision. =1, -Score -Score means that Precision and Recall are equally important. When , more attention is paid to Recall, that is, more attention is paid to reducing missed reports. When selecting In this case, the F1 score is selected as the evaluation indicator.

[0068] 5) Confusion Matrix:

[0069] (6)

[0070] The matrix dimensions are , represents the number of categories. The rows of the matrix correspond to the actual categories, and the columns correspond to the predicted categories. The actual category is The number of samples, is predicted to be the class The actual category in the sample number is The number of samples of . The elements of the matrix express and Percentage value of diagonal elements Indicates the correct classification ratio for this category, non-diagonal elements ( ) represents the misclassification ratio.

[0071] Although P is set in the time window w However, in actual situations, the actual prediction time range of the gait sub-phase prediction model may be significantly different. Therefore, the present invention defines an evaluation index: stable prediction time (P stable ), P stable The standard definition of is as follows: if the model makes three correct predictions for the gait sub-phase switching moments in a row, the above prediction is considered to be a stable prediction. Based on this definition, in order to more accurately describe the prediction performance of the model, P is used. stable To measure the starting time of stable prediction (t s ) and the sub-phase switching time (t c ), the time interval between the two is calculated as:

[0072] (7)

[0073] S7: Statistical prediction accuracy and P of gait sub-phase prediction model under different prediction time advance conditions stable ; Visual comparison is performed by drawing confusion matrix and line graph to show the different P settings w The performance of the models under different conditions.

[0074] In order to quantify the performance of the gait sub-phase prediction model, components were gradually eliminated and replaced to obtain four benchmark models: CNN, BiLSTM, MHA, and CNN-BiLSTM, to verify the contribution of each module. Figure 4 (a) to Figure 4 (d) show a detailed analysis of the prediction results of the five models on various sub-phases using a confusion matrix. By comparing the ratios of the diagonal (TP) and the non-diagonal (FP, FN), the misclassification patterns of the models on each gait sub-phase are further revealed. The results in the figure show that the gait sub-phase prediction model proposed in the present invention is significantly better than the benchmark model in the prediction of almost all gait sub-phase categories, especially in the prediction accuracy of the IC sub-phase.

[0075] At the same time, the data with a speed of 1.0 km / h were used for training, and the P values ​​of the gait sub-phase prediction model and the baseline model from 50 ms to 500 ms were calculated respectively. w The prediction accuracy and P stable , and draw the corresponding line chart; the horizontal axis of the line chart represents P w , the vertical axis corresponds to P stable and prediction accuracy; In the chart, the evaluation results of the gait sub-phase prediction model and the benchmark model are represented by different broken lines, which is convenient for intuitive comparison between the two at different P wDifferent models and evaluation metrics are distinguished by different colors, line styles, and legends, and data points are annotated to analyze key turning points.

[0076] As shown in Figure 5(a) and Figure 5(b), the gait sub-phase prediction model has different P w The prediction accuracy and P stable Specifically, in Figure 5(a), each model performs significantly better than the baseline model. w All show some hysteresis. w Increasing from 50 ms to 500 ms resulted in a higher average P stable , ranging from 33.30 ms to 307.81 ms. Figure 5(b) shows the gait sub-phase prediction model at different P w The prediction accuracy under P w As the prediction accuracy of most models increases, the prediction accuracy of most models usually decreases. However, the gait sub-phase prediction model proposed in the present invention has the smallest decrease and is significantly better than all benchmark models in most cases, with an accuracy rate between 92.05% and 94.78%. w Set to peak at 100 ms.

[0077] S8: Conduct subject-by-subject elimination cross-validation and speed migration robustness validation experiments to evaluate the adaptability and stability of the gait sub-phase prediction model under conditions of cross-individual and speed changes, and draw box plots to visualize the experimental results.

[0078] First, a cross-validation experiment was conducted by eliminating subjects one by one. The data of 12 subjects under constant speed conditions were selected and P w The uniform setting is 150 ms. In each iteration, the data of a single subject is retained as the test set, and the data of the remaining 11 subjects constitute the training set. The loop is executed until the rotation verification of all subjects is completed;

[0079] Secondly, we perform multi-dimensional performance evaluation and calculate the precision, recall and F1 score for the verification results under various speed conditions. We use the quartile statistical method to generate a box plot, with the horizontal axis representing the evaluation indicator category and the vertical axis representing the numerical distribution of the quantitative indicator. The box body shows the 25%-75% data interval, the whiskers represent the extreme value range, and the discrete points mark abnormal data.

[0080] As shown in Figure 6 (a) to Figure 6 (c), the gait sub-phase prediction model has achieved significant cross-individual generalization ability under three speed conditions (slow speed 1.0 km / h, medium speed 1.5 km / h and fast speed 2.0 km / h), and has excellent performance in accuracy, precision, recall rate and F1 score. Under the low speed condition, the average accuracy of the gait sub-phase prediction model is 85.87% and the F1-score is 80.43%; under the medium speed condition, the accuracy is 87.08% and the F1-score is 80.81%; under the fast speed condition, the accuracy is 85.75% and the F1-score is 78.24%.

[0081] In addition, a speed migration robustness verification experiment is designed to establish different speed combination strategies for the training set and the verification set, including six typical speed migration scenarios: slow-medium speed (TS-VM), slow-fast speed (TS-VF), medium-slow speed (TM-VS), medium-fast speed (TM-VF), fast-slow speed (TF-VS), and fast-medium speed (TF-VM). Finally, a sequential verification is implemented to perform end-to-end testing for each speed combination strategy and calculate the cross-speed prediction accuracy. According to the box plot standard described above, a box plot of the speed migration robustness verification experiment is drawn.

[0082] like Figure 7 As shown in the figure, the gait sub-phase prediction model performs well in most cases. Under the TS-VM, TM-VS and TF-VM strategies, the average prediction accuracy reached 90.92%, 90.91% and 91.27%, respectively. In addition, when the difference between the training and verification gait speeds is large, the prediction accuracy of the gait sub-phase prediction model is also significantly higher than that of the baseline model. Specifically, under the TS-VF and TF-VS strategies, the gait sub-items and the model achieved accuracies of 89.04% and 89.33%, respectively, which are 1.91% and 2.14% higher than CNN-BiLSTM, 7.45% and 7.02% higher than BiLSTM, and 11.44% and 10.53% higher than CNN. These results show that the method of the present invention can stably extract gait features and make accurate predictions under different speed combinations, especially when the difference between the training and verification gait speeds is large, its advantages are more significant.

[0083] S9: Build a modular gait sub-phase prediction system architecture on the Simulink platform, integrate multi-source gait data acquisition, signal preprocessing, online reasoning and result output modules, and solidify the pre-trained and cross-validated gait sub-phase prediction model parameters into the online reasoning module.

[0084] Firstly, a modular gait sub-phase prediction system architecture is built on the Simulink platform, and the system is divided into multi-source gait data acquisition, signal preprocessing, online reasoning and result output modules. Standardized data interface protocols are used for communication between modules to ensure efficient transmission and compatibility of real-time data streams.

[0085] Secondly, the surface electromyography sensor interface and the inertial measurement unit sensor interface are integrated in the multi-source gait data acquisition module, the data sampling rate is configured to be 1000 Hz, and the input data is 14-channel biosensor data consisting of 4-channel sEMG and 3-channel IMU signals of both lower limbs.

[0086] Subsequently, the signal preprocessing module sets the filtering and denoising operations consistent with step S2, and performs dynamic range normalization, that is, the normalization coefficient is updated in real time based on the maximum and minimum values ​​of the data in the last 10 seconds. At the same time, the sliding overlapping window mechanism is deployed in the signal preprocessing module, and the window size and sliding step size are set consistent with step S3.

[0087] Finally, the pre-trained and cross-validated gait sub-phase prediction model is converted into a Simulink-compatible C code form and embedded in the online reasoning module through the MATLAB Function module; the model weight parameters are solidified and the calculation structure is optimized to improve the real-time reasoning efficiency.

[0088] S10: Real-time acquisition and storage of multi-source gait data, input of sEMG and IMU signals into the modular gait sub-phase prediction system architecture, and real-time processing of data streams using a multi-threaded buffering mechanism, execution of signal processing and online reasoning modules, and dynamic output of gait sub-phase prediction results.

[0089] Multi-source gait data is collected and stored in real time. The sEMG and IMU signals are received through the multi-source gait data acquisition module. The timestamp alignment strategy is used to eliminate the clock deviation between devices, and the data is stored in a ring buffer. Subsequently, the multi-thread buffer mechanism is started, and independent threads are assigned to perform signal preprocessing and pre-online reasoning tasks, and data interaction between threads is realized through shared memory. At the same time, the preprocessing algorithm in the signal preprocessing module is called to filter and denoise the sEMG and IMU signals in real time, and input them into the online reasoning module; the gait sub-phase prediction model performs online reasoning and prediction, and imports the prediction results into the result output module. Finally, the result output module generates gait sub-phase prediction results in real time every 5ms, realizing real-time online prediction of gait sub-phases.

[0090] In addition, in order to evaluate the actual performance of the present invention in the real-time gait sub-phase prediction task, the gait multi-source data in the gait sub-phase real-time recognition task is exported, and the gait sub-phase label assignment method in step S2 is applied to obtain the true label. The true label is compared with the gait sub-phase prediction result output by the real-time online prediction, and a step comparison diagram is drawn, such as Figure 8 The present invention achieves almost completely accurate gait sub-phase prediction without obvious deviation or fluctuation, further verifying the application potential and technical advantages of the present invention in actual dynamic environments.

[0091] Example 2

[0092] Reference Figure 1 ,Figure 2(a), Figure 3 , Fig. 9 , this embodiment relates to a gait sub-phase prediction device based on deep learning model fusion. The device includes a display interface, a memory and one or more processors. The memory stores executable code, and when the processor executes the code, it can realize real-time prediction of gait sub-phases and gait cycles. The device receives gait data in real time through Bluetooth technology, and predicts gait sub-phases through the processor, and displays the real-time prediction results through the display interface. The device combines the display interface, memory, processor and executed executable code to work together to realize the gait sub-phase prediction method based on deep learning model fusion of the present invention.

[0093] Example 3

[0094] Reference Figure 1 ,Figure 2(a), Figure 3 , this embodiment relates to a computer-readable storage medium on which a program is stored. When the program is executed by a processor, the gait sub-phase prediction method based on deep learning model fusion of the present invention is implemented.

[0095] In summary, the present invention proposes a gait sub-phase prediction method and device based on deep learning model fusion, aiming to improve the accuracy, real-time and multi-scenario dynamic adaptability of gait sub-phase prediction when a wearable lower limb rehabilitation exoskeleton robot walks. By synchronously collecting multi-source gait data and combining data preprocessing, gait sub-phase label allocation and sliding overlapping window technology, a high-quality supervised learning training data set is constructed; by hierarchical fusion of CNN, Bi-LSTM and MHA, a deep learning model fusion architecture is formed to construct a gait sub-phase prediction model; the Bayesian optimization algorithm is used to automatically optimize the model hyperparameters and explore the best hyperparameter combination; classic evaluation indicators (including accuracy, precision, recall, F1 score, confusion matrix) and custom evaluation indicators are introduced: stable prediction time (P stable), comprehensively evaluate the performance of the gait sub-phase prediction model; evaluate the adaptability and stability of the model under cross-individual and speed change conditions through cross-validation and speed migration robustness verification experiments by eliminating subjects one by one; build a modular gait sub-phase prediction system architecture on the Simulink platform, integrate multi-source gait data acquisition, signal preprocessing, online reasoning and result output modules, and solidify the model parameters that have been pre-trained and cross-validated into the online reasoning module; collect and store multi-source gait data in real time, input sEMG and IMU signals into the modular gait sub-phase prediction system architecture, use a multi-threaded buffer mechanism to process data streams in real time, execute signal processing and online reasoning modules, and dynamically output the prediction results of gait sub-phases. The method of the present invention is superior to the existing technology in terms of accuracy, precision, recall rate, F1 score and stable prediction time, greatly improves the gait sub-phase prediction accuracy and multi-scenario dynamic adaptability, and has important application value, especially in the fields of intelligent control of lower limb rehabilitation exoskeleton robots and personalized rehabilitation treatment.

[0096] The above embodiment describes an implementation of the present invention. For those skilled in the art, various adjustments, changes and improvements may be made to the present invention. All adjustments, changes and improvements made within the scope of the core principle of the present invention are considered to be within the scope of the present invention.

Claims

1. A gait sub-phase prediction method based on deep learning model fusion, characterized in that: The following steps are involved: S1: Collect multi-source gait data of different individuals wearing lower limb rehabilitation exoskeleton robots at various speeds, and use cameras to record walking process videos; use a unified marking event to mark the start of the experiment to accurately align the timestamps of multi-source gait data and walking process videos; S2: preprocessing the collected multi-source gait data, and assigning corresponding gait sub-phase labels to the preprocessed multi-source gait data based on ground contact force and walking process video; S3: Use sliding overlapping window technology to divide multi-source gait data into time windows of fixed length and assign true labels to each window T label With predicted labels P label , and set the forecast time advance P w , to construct a standardized supervised learning training data set, where T label represents the highest frequency label of the gait sub-phase in the current time window, P label Indicates the distance from the current time window P w The time window of the step size T label ; S4: By hierarchically fusing convolutional neural network (CNN), bidirectional long short-term memory network (Bi-LSTM), and multi-head attention mechanism (MHA), a deep learning model fusion architecture is formed to construct a gait sub-phase prediction model. S5: The Bayesian optimization algorithm is used to automatically optimize the model hyperparameters to explore the best hyperparameter combination; then, the stratified K-fold cross-validation strategy is used for model training and evaluation, and the model checkpoint mechanism is enabled to prevent overfitting; S6: Define stable prediction time P stable , and combined with classic indicators to comprehensively evaluate the prediction performance of the gait sub-phase prediction model; the stable prediction time P stable , which is specifically defined as follows: If the model makes three correct predictions for the gait sub-phase switching moments in a row, the above prediction is considered to be a stable prediction; based on this definition, in order to more accurately describe the prediction performance of the model, the stable prediction time is used P stable To measure the start of stable forecast t s Sub-phase switching time t c The interval between is calculated as: （2） S7: Different settings P w , and the prediction accuracy and P stable ; Perform visual comparison and display in different P w The performance difference of the following models; S8: Conduct subject-by-subject elimination cross-validation and speed migration robustness validation experiments to evaluate the adaptability and stability of the gait sub-phase prediction model under conditions of cross-individual and speed changes, and draw box plots to visualize the experimental results. S9: Build a modular gait sub-phase prediction system architecture on the Simulink platform, integrate multi-source gait data acquisition, signal preprocessing, online reasoning and result output modules, and solidify the pre-trained and cross-validated gait sub-phase prediction model parameters into the online reasoning module; S10: Collect and store multi-source gait data in real time, input sEMG and IMU signals into the modular gait sub-phase prediction system architecture, and use a multi-threaded buffer mechanism to process data streams in real time, execute signal processing and online reasoning modules, and dynamically output the prediction results of gait sub-phases.

2. The gait sub-phase prediction method based on deep learning model fusion according to claim 1, characterized in that: The experiment of using unified marking event markers described in step S1 starts to accurately align the timestamps of multi-source gait data and the walking process video. The specific process is as follows: At the beginning of the experiment, the subject tapped the treadmill surface with the toe of his right foot to generate an obvious high-voltage spike signal, and the action was accurately captured by the camera; this marked event served as a timestamp reference to ensure that the subsequently collected multi-source gait data was accurately aligned with the walking process video.

3. The gait sub-phase prediction method based on deep learning model fusion according to claim 1, characterized in that: The sliding overlapping window technique described in step S3 divides the multi-source gait data into time windows of fixed length, and each time window is assigned a corresponding true label. T label With predicted labels P label , and set the forecast time advance P w , to construct a standardized supervised learning training data set. The specific process is as follows: S31. Using the sliding overlapping window technology, the multi-source gait data processed in step S2 is divided into time windows of fixed length, the window size is set to 30 sampling points, and the sliding step is 5 sampling points; the specific operation is: starting from the first sampling point of the multi-source gait data, 30 sampling points are intercepted to form the first time window; then, each time the step is 5 sampling points, the time window of the same length is intercepted as the next sample, until the entire sequence is covered; S32. Assign two labels to each time window: the true label T label and predicted labels P label ; in, P w is a preset constant, indicating the amount of advance prediction time; in this way, P label Mapped with the data features of each time window to achieve P w The prediction of gait sub-phase after time; the input of the training data set is a 30×N matrix, where N represents the number of signal channels; the training label is the corresponding P label , thereby realizing the data characteristics of the current time window and P w The gait subphases after time form a map.

4. The gait sub-phase prediction method based on deep learning model fusion according to claim 1, characterized in that: In step S4, a deep learning model fusion architecture is formed by hierarchically fusing a convolutional neural network CNN, a bidirectional long short-term memory network Bi-LSTM, and a multi-head attention mechanism MHA to construct a gait sub-phase prediction model. The specific process is as follows: S41. Extract local features of input data through convolutional layers, and use two-dimensional convolution kernels to slide along the joint time-space dimension; then, connect the maximum pooling layer and the global average pooling layer in sequence for feature compression, the maximum pooling layer window size is 2×2, and the step size is set to 2; S42. The pooled output is adjusted to a three-dimensional tensor through a vector reshaping layer to adapt to the input format of the Bi-LSTM; The Bi-LSTM adopts a four-layer stacking structure, the number of hidden neurons decreases layer by layer, and the long-term and short-term dependencies are extracted through a bidirectional gating mechanism; S43. Input the output of the last layer of Bi-LSTM into three fully connected layers with 64 hidden neurons respectively to generate three query vectors, key vectors and value vectors with dimensions of 64; adopt the multi-head attention mechanism MHA containing 3 attention heads to model feature interaction, and the calculation dimension of each attention head is set to 8; set a batch normalization layer at the output end of MHA to optimize data distribution, and the normalized momentum parameter is set to 0.99; After dimensionality reduction by the global average pooling layer, it is connected to a relay fully connected layer containing 64 hidden neurons, and the ReLU activation function is used for nonlinear transformation; finally, the 8×1 gait sub-phase probability distribution is output through the Softmax classifier.

5. The gait sub-phase prediction method based on deep learning model fusion according to claim 1, characterized in that: The Bayesian optimization algorithm described in step S5 is used to automatically optimize the hyperparameters of the gait sub-phase prediction model to explore the best hyperparameter combination. The specific process is as follows: S51. Construct a multidimensional hyperparameter space, wherein the hyperparameter space includes five dimensions: initial learning rate, batch size, number of two-dimensional convolution kernels, size of two-dimensional convolution kernels, and number of hidden neurons in the four-layer Bi-LSTM. The candidate set of initial learning rate is {10 -5 , 10 -4 , 10 -3 , 10 -2 , 10 -1 }, the candidate set of batch size is {16, 32, 64, 128, 256}, the candidate set of two-dimensional convolution kernel number is {8, 16, 64, 128, 256}, the candidate set of two-dimensional convolution kernel size is {1×1, 3×3, 5×5, 7×7}, and the candidate set of Bi-LSTM four-layer hidden neuron number is {16, 32, 64, 128, 256}; S52. Initialize the Gaussian process proxy model, construct the covariance matrix based on the radial basis function, and establish a nonlinear mapping relationship between the hyperparameter combination and the model performance index; obtain the initial hyperparameter combination set through Latin hypercube sampling, perform model training and record the validation set F1 score as the objective function value to complete the pre-training of the proxy model; S53. Use the expected improvement criterion to optimize the hyperparameter combination and calculate the expected improvement value of the candidate hyperparameter combination based on the posterior distribution of the current proxy model ; The expected improvement value The mathematical expression is: （1）， in, [·] represents the expected value, which is the integral of the probability distribution of the function value based on the uncertainty of the current model; Indicates the hyperparameter combination The objective function value under , that is, the model performance; Represents the current best hyperparameter combination; select the hyperparameter combination with the largest EI value for experiment; this process is updated in each round of optimization; S54. Iterate the Bayesian optimization process and select the expected improvement value The largest candidate hyperparameter combination is used for model training, the newly acquired performance data is added to the training set, and the hyperparameter posterior distribution of the Gaussian process surrogate model is updated; the parameters of the covariance matrix are adjusted through the kernel function to improve the fitting accuracy of the surrogate model to the response surface of the objective function; S55. Set a double termination condition. When the preset maximum number of iterations is reached or the improvement of the optimal objective function value is less than 1% after 10 consecutive iterations, terminate the optimization process, output the historical optimal hyperparameter combination and the corresponding model performance indicators, and apply the historical optimal hyperparameter combination to the gait sub-phase prediction model constructed in step S4.

6. The gait sub-phase prediction method based on deep learning model fusion according to claim 1, characterized in that: The setting described in step S7 is different P w , and the prediction accuracy and P stable ; Perform visual comparison and display in different P w The performance difference of the following models is as follows: The training data with a speed of 1.0 km / h was used to calculate the gait sub-phase prediction model and the baseline model in the range of 50ms to 500ms. P w The prediction accuracy under the conditions and P stable , and draw the corresponding line chart; the horizontal axis of the line chart represents P w , the vertical axes correspond to P stable and prediction accuracy; in the chart, the evaluation results of the gait sub-phase prediction model and the baseline model are represented by different broken lines, which is convenient for intuitive comparison between the two in different P w Different models and evaluation metrics are distinguished by different colors, line styles, and legends, and data points are annotated to analyze key turning points.

7. The gait sub-phase prediction method based on deep learning model fusion according to claim 1, characterized in that: The subject elimination cross-validation and speed migration robustness validation experiments described in step S8 are performed one by one to evaluate the adaptability and stability of the gait sub-phase prediction model under cross-individual and speed change conditions, and box plots are drawn to visualize the experimental results. The specific process is as follows: S81. Perform a cross-validation experiment by eliminating subjects one by one, select the data of 12 subjects under constant speed conditions, and P w The time interval was uniformly set to 150 ms. In each iteration, the data of a single subject was retained as the test set, and the data of the remaining 11 subjects constituted the training set. The cycle was executed until the rotation verification of all subjects was completed. S82. Perform multi-dimensional performance evaluation, calculate precision, recall and F1 score for the verification results under each speed condition; use quartile statistics to generate box plots, with the horizontal axis representing the evaluation indicator category and the vertical axis quantifying the indicator value distribution. The box body shows the 25%-75% data interval, the box whiskers represent the extreme value range, and the discrete points mark abnormal data; S83. Design a speed migration robustness verification experiment and establish different speed combination strategies for training set and verification set, including six typical speed migration scenarios: slow-medium speed, slow-fast, medium-slow, medium-fast, fast-slow, and fast-medium speed; S84. Implement sequential verification, perform end-to-end testing for each speed combination strategy, and calculate the cross-speed prediction accuracy; According to the box plot standard described in step S82, a box plot of the speed migration robustness verification experiment is drawn.

8. The gait sub-phase prediction method based on deep learning model fusion according to claim 1, characterized in that: The modular gait sub-phase prediction system architecture described in step S9 is constructed on the Simulink platform, integrating multi-source gait data acquisition, signal preprocessing, online reasoning and result output modules, and solidifying the pre-trained and cross-validated gait sub-phase prediction model parameters into the online reasoning module. The specific process is as follows: S91. Build a modular gait sub-phase prediction system architecture on the Simulink platform, and divide the system into multi-source gait data acquisition, signal preprocessing, online reasoning and result output modules; use standardized data interface protocols to communicate between modules to ensure efficient transmission and compatibility of real-time data streams; S92. Integrate the surface electromyography sensor interface and the inertial measurement unit sensor interface in the multi-source gait data acquisition module, configure the data sampling rate to be 1000 Hz, and input the 14-channel biosensor data consisting of 4-channel sEMG and 3-channel IMU signals of both lower limbs; S93. Set the filtering and denoising operations consistent with step S2 in the signal preprocessing module, and perform dynamic range normalization, that is, update the normalization coefficient in real time based on the maximum and minimum values ​​of the data in the last 10 seconds; at the same time, deploy a sliding overlapping window mechanism in the signal preprocessing module, and the window size and sliding step size are set consistent with step S31; S94. Convert the pre-trained and cross-validated gait sub-phase prediction model into a Simulink-compatible C code format and embed it into the online reasoning module through the MATLAB Function module; solidify the model weight parameters and optimize the calculation structure to improve the real-time reasoning efficiency.

9. The gait sub-phase prediction method based on deep learning model fusion according to claim 1, characterized in that: The real-time acquisition and storage of multi-source gait data in step S10 inputs sEMG and IMU signals into the modular gait sub-phase prediction system architecture, and uses a multi-threaded buffer mechanism to process data streams in real time, executes signal processing and online reasoning modules, and dynamically outputs the prediction results of gait sub-phases. The specific process is as follows: S101. Collect and store multi-source gait data in real time, receive the original data stream of sEMG and IMU signals through the multi-source gait data acquisition module, use the timestamp alignment strategy to eliminate the clock deviation between devices, and store the data in the ring buffer; S102. Start the multi-thread buffer mechanism, assign independent threads to perform signal pre-processing and pre-online reasoning tasks, and implement data interaction between threads through shared memory; S103. Call the preprocessing algorithm in the signal preprocessing module to perform real-time filtering and denoising on the sEMG and IMU signals, and input them into the online reasoning module; The gait sub-phase prediction model performs online inference prediction and imports the prediction results into the result output module; S104. The result output module generates gait sub-phase prediction results in real time every 5 ms, realizing real-time online prediction of gait sub-phases.

Images