Decision-level multi-modal human body gait prediction method
By employing a decision-level multimodal human gait prediction method, which combines inertial measurement units and surface electromyography signals to dynamically select the optimal information source, the robustness and adaptability of gait prediction in existing technologies are addressed, achieving higher accuracy and smoother human-computer interaction.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- BEIJING UNIV OF POSTS & TELECOMM
- Filing Date
- 2026-02-05
- Publication Date
- 2026-05-15
AI Technical Summary
Existing technologies fail to adequately consider the differences in effectiveness of various sensor modalities in different gait scenarios, resulting in superficial fusion strategies that lack interpretability. In particular, they are not robust or adaptable in complex and ever-changing real rehabilitation and walking environments.
A decision-level multimodal human gait prediction method is adopted. Through decision-level modality selection, combined with inertial measurement unit and surface electromyography signals, an expert prediction model is trained and the gait pattern decision-maker is used to dynamically select the optimal information source to achieve adaptive information fusion.
It improves the accuracy of gait prediction and system robustness, and enhances adaptability in complex environments and the smoothness of human-computer interaction.
Smart Images
Figure CN122030952A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the fields of neuroscience and rehabilitation medicine, and in particular relates to a decision-level multimodal human gait prediction method. Background Technology
[0002] The number of patients with walking dysfunction caused by neuromuscular diseases such as stroke and spinal cord injury is increasing, and their need to restore their walking ability is extremely urgent. Traditional rehabilitation therapies struggle to achieve high-intensity, repetitive training, while lower limb exoskeleton robots offer a new solution for rebuilding walking function by providing assistance to the wearer's hip, knee, and other joints. The key to achieving natural and smooth interaction between the exoskeleton and the human body lies in advanced gait prediction algorithms. These algorithms need to analyze human motion information in real time and estimate the wearer's gait stage or movement intention in advance.
[0003] Current gait prediction research primarily relies on single-modal sensor data. For example, single-modal methods based on surface electromyography (EMG) signals can predict gait using the neuromuscular activation characteristics of EMG signals 40 to 100 milliseconds before movement, performing well under ideal periodic gait conditions. However, in complex non-periodic scenarios involving starting, stopping, and obstacle crossing, the same muscle activation pattern may lead to completely different trajectories due to changes in external physical constraints, making the prediction mapping uncertain. Furthermore, EMG signals are highly susceptible to noise interference such as sweat and electrode slippage, resulting in insufficient stability. Another type of single-modal method based on inertial measurement units (IMUs) can provide reliable kinematic data, but its measurement lags behind the movement itself, failing to provide advance indication of movement intention, leading to prediction delays and making it difficult to meet the needs of real-time control.
[0004] To overcome the limitations of single-modal gait, multimodal data fusion methods have emerged, aiming to combine the complementary advantages of different sensors. Existing technologies mostly employ data-level or feature-level fusion strategies, directly stitching together data from multiple modalities such as inertial measurement units, plantar pressure sensors, and sound sensors before inputting them into a unified model for processing. These methods have significant drawbacks: when the quality of a particular modal signal deteriorates significantly in a specific scenario—for example, due to disordered electromyographic signal patterns in non-periodic gait or noise generated during movement—its low-quality information directly affects the prediction results of the entire model as part of the input, introducing redundancy or even interference. Furthermore, decision-level fusion methods using fixed weights or simple voting mechanisms lack the ability to perceive the dynamic characteristics of gait and cannot adaptively adjust the fusion strategy based on the fundamental differences between periodic and non-periodic gait.
[0005] In summary, the core problem with existing technologies lies in their failure to deeply consider the differences in effectiveness of various sensing modalities under different gait scenarios. The fusion strategies employed are relatively superficial and lack interpretability, resulting in the prediction performance of multimodal systems not significantly surpassing that of single-modal methods. Especially in complex and variable real-world rehabilitation and walking environments, their robustness and adaptability face severe challenges. Designing a fusion architecture capable of dynamically selecting the optimal information source based on the inherent characteristics of gait has become a key challenge in improving the accuracy and practicality of prediction algorithms. Summary of the Invention
[0006] To address the aforementioned technical problems, this invention proposes a decision-level multimodal human gait prediction method. By selecting decision-level modes, it effectively integrates the complementary advantages of different sensors, thereby improving the accuracy of gait prediction and the robustness of the system.
[0007] To achieve the above objectives, this invention provides a decision-level multimodal human gait prediction method, comprising an offline training phase and an online prediction phase: The offline training phase performs the following operations: S1. Collect inertial measurement unit data and surface electromyography signals of the human body in various gait scenarios, and label the gait variables and periodic / non-periodic gait labels accordingly. S2. The collected inertial measurement unit data and surface electromyography signals are preprocessed and divided into sliding windows to obtain a time window sequence; S3. Based on the time window sequence, train a first expert prediction model for processing inertial measurement unit data and a second expert prediction model for processing surface electromyography signals, respectively. S4. Train a gait pattern decision-maker to determine whether the gait of the window is periodic or non-periodic based on the time window sequence. The online prediction phase performs the following operations: S5. Real-time acquisition of current inertial measurement unit data and surface electromyography signals, followed by preprocessing and sliding window division; S6. Input the current window data into the gait pattern decision-maker to obtain the pattern discrimination result of the current gait; S7. Based on the pattern discrimination result, select the output of the first expert prediction model or the second expert prediction model as the final gait prediction result.
[0008] Optionally, in S1, the acquisition of inertial measurement unit data and surface electromyography signals of the human body in various gait scenarios includes: Triaxial acceleration and angular velocity signals are collected by inertial measurement units deployed on the lower limbs or torso of the human body; Electromyographic signals are collected by surface electromyography sensors deployed in the lower limb muscle groups; The target gait variables are recorded synchronously, and each time window is labeled with a classification label indicating whether it is periodic or non-periodic.
[0009] Optionally, in S2, preprocessing and sliding window partitioning include: The inertial measurement unit data is bandpass filtered to remove noise; The surface electromyography signal is subjected to bandpass filtering, rectification, and envelope extraction. The filtered multimodal signal is time-aligned and resampled to a uniform frequency; The aligned signal is slidably segmented using a preset fixed window length and step size to generate the time window sequence.
[0010] Optionally, in S3, training the first expert prediction model and the second expert prediction model includes: The inertial measurement unit data in the time window sequence is input into a time-domain convolutional network, and the target gait variable is used as a supervision signal for training to obtain the first expert prediction model. The surface electromyography signals in the time window sequence are input into another temporal convolutional network, and the target gait variable is used as a supervision signal for training to obtain the second expert prediction model.
[0011] Optionally, in S4, training the gait pattern decision-maker includes: Extract the fusion features of the time window sequence; The fused features are input into a binary classification model for training. The binary classification model uses the periodic / non-periodic gait labels as supervision signals and outputs the probability that the window belongs to the periodic gait.
[0012] Optionally, in S5, the window length of the preprocessing and sliding window division process is consistent with the preset fixed window length used in S2.
[0013] Optionally, in S6, obtaining the pattern discrimination result of the current gait includes: The gait pattern decision-maker outputs probability values; The probability value is compared with a preset threshold. If it is greater than or equal to the threshold, it is determined to be a periodic gait; otherwise, it is determined to be a non-periodic gait.
[0014] Optionally, in S7, selecting the model output as the final gait prediction result includes: If the pattern discrimination result is periodic gait, then the output of the second expert prediction model is used as the final prediction result; If the pattern discrimination result is a non-periodic gait, then the output of the first expert prediction model is used as the final prediction result.
[0015] An electronic device, the electronic device comprising: a processor and a memory storing computer program instructions; The processor implements the decision-level multimodal human gait prediction method when executing the computer program instructions.
[0016] A computer storage medium storing computer program instructions, which, when executed by a processor, implement the decision-level multimodal human gait prediction method.
[0017] Technical advantages of this invention: This invention discloses a decision-level multimodal human gait prediction method. Compared with single-modal methods, it effectively combines the kinematic determinism of inertial measurement unit signals with the neural intent-proactive advantage of surface electromyography (EMG) signals, overcoming the inherent limitations of single sensors in complex gait scenarios. Compared with existing data-level or feature-level multimodal fusion methods, the decision-level fusion architecture adopted in this invention can adaptively select and output the prediction results of the optimal expert model for the corresponding scenario based on the real-time determined gait periodicity, thereby structurally avoiding the interference of low-quality modal signals on the overall prediction performance. This scheme improves the overall prediction accuracy and system robustness of the model for periodic steady-state gait and non-periodic transient gait through decoupling training and scenario-based selection strategies, ultimately enhancing the environmental adaptability and human-computer interaction fluency of the gait prediction algorithm in practical applications such as rehabilitation exoskeletons. Attached Figure Description
[0018] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings: Figure 1 This is a flowchart illustrating a decision-level multimodal human gait prediction method according to an embodiment of the present invention. Figure 2 This is a schematic diagram showing the placement of the EMG electrode and IMU sensor in an embodiment of the present invention. Detailed Implementation
[0019] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.
[0020] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.
[0021] like Figure 1 As shown, this embodiment provides a decision-level multimodal human gait prediction method, including an offline training phase and an online prediction phase: The offline training phase performs the following operations: S1. Collect inertial measurement unit data and surface electromyography signals of the human body in various gait scenarios, and label the gait variables and periodic / non-periodic gait labels accordingly. S2. The collected inertial measurement unit data and surface electromyography signals are preprocessed and divided into sliding windows to obtain a time window sequence; S3. Based on the time window sequence, train a first expert prediction model for processing inertial measurement unit data and a second expert prediction model for processing surface electromyography signals, respectively. S4. Train a gait pattern decision-maker to determine whether the gait of the window is periodic or non-periodic based on the time window sequence. The online prediction phase performs the following operations: S5. Real-time acquisition of current inertial measurement unit data and surface electromyography signals, followed by preprocessing and sliding window division; S6. Input the current window data into the gait pattern decision-maker to obtain the pattern discrimination result of the current gait; S7. Based on the pattern discrimination result, select the output of the first expert prediction model or the second expert prediction model as the final gait prediction result.
[0022] Furthermore, in S1, the inertial measurement unit data and surface electromyography signals collected from the human body in various gait scenarios include: Triaxial acceleration and angular velocity signals are collected by inertial measurement units deployed on the lower limbs or torso of the human body; Electromyographic signals are collected by surface electromyography sensors deployed in the lower limb muscle groups; The target gait variables are recorded synchronously, and each time window is labeled with a classification label indicating whether it is periodic or non-periodic.
[0023] Specifically, the implementation process of this embodiment includes: This embodiment includes a sensor acquisition module, an IMU acquisition submodule (for acquiring triaxial acceleration, angular velocity, and other signals output by inertial measurement units deployed on the lower limbs and / or torso), and an EMG acquisition submodule (for acquiring surface electromyography signals deployed on relevant muscle groups in the lower limbs). The sensor placement and typical obstacle scenarios are illustrated below. Figure 2 As shown, the IMU and EMG sensors are fixed to the surfaces of the lower leg, thigh, instep, and corresponding muscle groups, respectively. Steps, curbs, or obstacles are set along the walking path. Within the nth sliding time window, the IMU and EMG inputs are respectively represented as: ; ; in, L This represents the number of sampling points within the window. dI and dE , respectively, are the feature dimensions of the IMU and EMG. Target gait variables (such as joint angles, foot trajectory, or ground reaction force) are denoted as . y(t)∈Rdy This invention supports [the following] at the end of the current time window. tn Predict gait sequences at multiple future moments: ; Where {τ1,…,τ H} represents several prediction time delays.
[0024] The data preprocessing module includes a filtering submodule (processing the raw IMU and EMG signals, using bandpass filtering to remove high-frequency noise and baseline drift); a synchronization and resampling submodule (performing time alignment and resampling of multi-channel and multi-modal signals to ensure that the data from each sensor correspond on the same time axis); and a sliding window segmentation submodule (sliding the preprocessed signal sequence into a series of time window segments with a preset window length of 600ms and a step size).
[0025] Next is the model training module, which includes an expert model training submodule (using IMU data behind a sliding window). EMG data and its corresponding target output Supervised learning is performed); a time-series prediction model based on TCN (multi-scale convolutional modeling of IMU and EMG sequences to obtain an IMU expert model). EMG expert model ).
[0026] Since invoking a two-modal expert model requires considering the periodicity or aperiodicity of the current gait window, this invention trains a periodicity / aperiodicity judgment submodule. Using gait samples labeled as periodic or aperiodic, a binary classification model is trained using TCN to achieve the invocation of different expert models based on gait characteristics. When a periodic gait is determined, the EMG expert model is activated; when an aperiodic gait is determined, the IMU expert model is activated. The two expert models formally implement the mapping from the time window input to the multi-step prediction output: ; Where θ IMU θ EMG These are the parameter sets for the two expert networks, respectively.
[0027] The storage and interaction module stores the trained model parameters, raw data, and prediction results, and transmits the prediction results to the mechanical exoskeleton control system. Each module can be implemented using an embedded processor, a host computer, or a cloud server, and the modules interact with each other via a bus or network connection.
[0028] Furthermore, in S2, preprocessing and sliding window partitioning include: The inertial measurement unit data is bandpass filtered to remove noise; The surface electromyography signal is subjected to bandpass filtering, rectification, and envelope extraction. The filtered multimodal signal is time-aligned and resampled to a uniform frequency; The aligned signal is slidably segmented using a preset fixed window length and step size to generate the time window sequence.
[0029] Specifically, the implementation process of this embodiment includes: (a) Offline training phase: Step S101: Multimodal gait data acquisition Experimental data collection was conducted using IMU and EMG sensors in various scenarios, including walking on flat ground, running, starting, stopping, turning, and crossing obstacles. Synchronous recording of target gait variables and the gait type label corresponding to the time window. 1}, where Indicates periodic gait. This indicates a non-periodic gait.
[0030] Step S102: Data Preprocessing S1021: Perform noise reduction filtering (0.5–20Hz bandpass filtering) on the IMU signal to remove the gravity component or perform attitude compensation on the acceleration signal; S1022: Bandpass filtering (20–450Hz), rectification, and envelope extraction are performed on the EMG signal to obtain a smooth muscle activation curve; S1023: Time alignment and resampling of multi-channel, multi-mode signals to ensure they correspond at a uniform sampling frequency.
[0031] Furthermore, in S3, training the first expert prediction model and the second expert prediction model includes: The inertial measurement unit data in the time window sequence is input into a time-domain convolutional network, and the target gait variable is used as a supervision signal for training to obtain the first expert prediction model. The surface electromyography signals in the time window sequence are input into another temporal convolutional network, and the target gait variable is used as a supervision signal for training to obtain the second expert prediction model.
[0032] Specifically, the implementation process of this embodiment includes: Step S103: IMU and EMG Expert Model Training Will , As input, As a supervisory signal, IMU expert models and EMG expert models were trained separately. Weighted mean squared error (MSE) was used as the regression loss function. ; Where N is the number of training samples, , Two experts respectively in the n The prediction results for a single-step output within a time window, where λ is the weighting coefficient for balancing the losses of the two experts.
[0033] Furthermore, in S4, training the gait pattern decision-maker includes: Extract the fusion features of the time window sequence; The fused features are input into a binary classification model for training. The binary classification model uses the periodic / non-periodic gait labels as supervision signals and outputs the probability that the window belongs to the periodic gait.
[0034] Specifically, the implementation process of this embodiment includes: Step S104: Training the gait pattern binary classification decision setter Construct a fused feature vector for each time window z ( n The binary classification decision-maker is extracted from the statistical features of the IMU / EMG or from a shallow network. Output the probability that the current time window belongs to a periodic gait: ; in σ (·) represents the sigmoid function. Next, cross-entropy loss is used to train the decision-maker: ; Taking into account both regression accuracy and pattern discrimination accuracy, this invention employs joint loss for end-to-end optimization: ; in α is a weighting coefficient used to adjust the importance of the binary classification task in the overall optimization.
[0035] Furthermore, in S5, the preprocessing and sliding window division process maintains the same window length as the preset fixed window length used in S2.
[0036] Specifically, the implementation process of this embodiment includes: (II) Online Prediction Phase: Step S201: Real-time data acquisition and preprocessing The sensor acquisition module continuously acquires raw IMU and EMG signals; the data preprocessing module filters, synchronizes, resamples, and slices the real-time signals using a sliding window to obtain the real-time window. and .
[0037] This embodiment decouples gait pattern discrimination from mode selection at the decision level: a dedicated binary classification decision unit distinguishes between "periodic" and "non-periodic" gait patterns; and selects between EMG experts and IMU experts based on the discrimination results. This structure can fully utilize the anticipatory nature of EMG in periodic behavior and the sensitivity of IMU to abrupt changes in posture in non-periodic behavior, thereby improving overall prediction performance.
[0038] The IMU expert model is optimized for non-periodic gait, focusing more on the impact of short-term acceleration and angular velocity abrupt changes on gait status; the EMG expert model is optimized for periodic gait, focusing more on muscle activation patterns during the stride cycle; this task division avoids a single model simultaneously fitting two signal distributions with drastically different statistical characteristics.
[0039] In addition, this embodiment implements a unified sliding time window (600ms) and alignment strategy for IMU and EMG, which is beneficial for the decision-maker to compare the two types of gait patterns on the same time scale, improve the discrimination accuracy, and at the same time ensure the real-time performance and consistency of online prediction.
[0040] Finally, by introducing sensor placement and obstacle scenarios and covering various complex gait situations during the training phase, the structure of this embodiment maintains high prediction accuracy and robustness in real-world environments, making it suitable for applications such as wearable devices, rehabilitation robots, and smart prostheses.
[0041] Furthermore, in S6, the pattern discrimination result of the current gait is obtained including: The gait pattern decision-maker outputs probability values; The probability value is compared with a preset threshold. If it is greater than or equal to the threshold, it is determined to be a periodic gait; otherwise, it is determined to be a non-periodic gait.
[0042] Specifically, the implementation process of this embodiment includes: Step S202: Gait pattern determination Extract the fusion feature z(n) for the current time window and input it into the trained decision-maker. To obtain the periodic gait probability Using thresholds Make a judgment and obtain the pattern label. : ; in, This is an indicator function that takes the value 1 when the condition is true and 0 otherwise.
[0043] Furthermore, in S7, selecting the model output as the final gait prediction result includes: If the pattern discrimination result is periodic gait, then the output of the second expert prediction model is used as the final prediction result; If the pattern discrimination result is a non-periodic gait, then the output of the first expert prediction model is used as the final prediction result.
[0044] Specifically, the implementation process of this embodiment includes: Step S203: Expert Model Selection and Prediction Output like (If determined to be a periodic gait), then the EMG expert model is activated, and the output is... .like (If the gait is determined to be non-periodic), then the IMU expert model is activated, and the output is... The final prediction results are selected and integrated at the decision-making level as follows: .
[0045] An electronic device, the electronic device comprising: a processor and a memory storing computer program instructions; The processor implements the decision-level multimodal human gait prediction method when executing the computer program instructions.
[0046] A computer storage medium storing computer program instructions, which, when executed by a processor, implement the decision-level multimodal human gait prediction method.
[0047] This embodiment incorporates an IMU and constructs a decision-level switching architecture, effectively addressing the issues of missing physical information and mapping uncertainty in describing non-steady-state motion using only bioelectrical signals. From a causal logic analysis of biomechanics and signal systems, EMG signals essentially reflect the nervous system's driving intention on muscles, belonging to the "cause" of motion rather than the "result" of the motion state. In ideal periodic walking on flat ground, there is a relatively stable mapping relationship between muscle activation and limb trajectory. However, in non-periodic complex scenarios involving steps, curbs, or obstacles, the movement trajectory of the human lower limbs... It depends not only on the active contraction of muscles, but also on external physical constraints such as ground reaction force, limb inertia, and gravitational torque. The single-modal EMG method attempts to establish a mapping... However, since EMG sensors cannot sense the contact force of the external environment and the inertial state of the limb, when the external load changes abruptly, the same muscle activation pattern may correspond to completely different motion trajectories, causing this inverse dynamic mapping to become a typical pathological problem, which in turn causes the prediction results to produce severe nonlinear distortion.
[0048] This embodiment introduces an IMU acquisition submodule, utilizing its output acceleration and angular velocity signals. Directly observe the kinematic state of the limbs. According to the prediction model in this embodiment, when the decision-maker identifies non-periodic gait features (i.e., The system output tends to At this point, the model no longer relies solely on uncertain bioelectrical signals to extrapolate the trajectory, but instead uses rigid body kinematic data provided by the IMU to estimate the gait state based on Newtonian mechanics principles. Because It includes the actual acceleration and rotation information of the limbs in space, which is related to the target gait variables. There is a strong correlation between these factors (such as joint angles or foot positions) that conforms to the laws of physics. Therefore, this embodiment can correct prediction biases by utilizing the physical determinism of IMU data in obstacle scenarios where electromyographic signals cannot accurately characterize external physical constraints, ensuring that trajectory prediction conforms to the geometric constraints of human kinematics.
[0049] Furthermore, single-modal EMG methods are highly susceptible to muscle fatigue, skin sweating, or changes in electrode contact impedance, leading to time-varying drift in signal amplitude over extended periods. This non-stationarity of the signal causes a significant performance degradation in trained models after prolonged use. This embodiment employs a decision-level fusion strategy to retain physical sensor channels that are insensitive to biometrics within the model structure. When an EMG signal is deemed unreliable due to a distribution shift caused by fatigue or physical artifacts, the system can adjust the weights accordingly. Automatically add pairs The IMU sensor's measurement accuracy depends primarily on the physical characteristics of the microelectromechanical system and does not decrease with changes in the human body's physiological state. Therefore, this embodiment can utilize the long-term stability of the IMU to compensate for the time-varying defects of the EMG, thereby maintaining highly consistent prediction accuracy and system reliability during long-term wearable tasks.
[0050] This embodiment constructs an independent expert model. and This achieves isolation and decoupling of modal feature spaces at the physical level. Based on the decision-level fusion formula proposed in this embodiment... The system introduces a nonlinear gating mechanism controlled by a gait pattern binary classification decision unit. Mathematical derivation shows that when the system is in a non-periodic gait or when electromyographic signals are unreliable, the pattern label output by the decision unit... The value will be determined as 0 (or close to 0), at which point the system's predicted output degenerates into... Mathematically, this mechanism is equivalent to setting up an adaptive noise blocker, which prevents the EMG signal prediction components from exhibiting high noise or artifacts. Weighted coefficient By forcing the system to zero, the causal link between electromyographic noise and the final prediction result is logically severed, thus fundamentally ensuring the prediction security of the system under single-mode failure or disturbance conditions.
[0051] Furthermore, this embodiment uses a joint loss function. The constraints effectively resolve the optimization conflict problem of heterogeneous data in a unified model. In data-level fusion, a single model attempts to simultaneously minimize the regression error of two types of signals with vastly different properties using a set of shared parameters, often leading to the model getting trapped in local optima. That is, to adapt to the smoothness characteristics of one mode, the ability to express the high-frequency features of another mode is sacrificed. This embodiment, through decision-level task splitting, enables the parameter set to be optimized. We only need to focus on learning the kinematic mapping of acceleration and angular velocity abrupt changes to the non-periodic gait, while the parameter set... The focus is solely on learning the advance mapping of muscle activation patterns to periodic movement intentions. This divide-and-conquer strategy reduces the hypothesis space complexity of each subtask, allowing both expert models to converge to the global optimum in their respective physical domains. Finally, this embodiment utilizes the probabilities output by the decision-maker. and threshold decision logic It achieves explicit perception and adaptive switching of gait dynamic characteristics, and significantly improves environmental adaptability in unstructured obstacle scenarios while ensuring the real-time prediction of periodic walking.
[0052] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.
Claims
1. A decision-level multimodal human gait prediction method, characterized in that, Includes offline training and online prediction phases: The offline training phase performs the following operations: S1. Collect inertial measurement unit data and surface electromyography signals of the human body in various gait scenarios, and label the gait variables and periodic / non-periodic gait labels accordingly. S2. The collected inertial measurement unit data and surface electromyography signals are preprocessed and divided into sliding windows to obtain a time window sequence; S3. Based on the time window sequence, train a first expert prediction model for processing inertial measurement unit data and a second expert prediction model for processing surface electromyography signals, respectively. S4. Train a gait pattern decision-maker to determine whether the gait of the window is periodic or non-periodic based on the time window sequence. The online prediction phase performs the following operations: S5. Real-time acquisition of current inertial measurement unit data and surface electromyography signals, followed by preprocessing and sliding window division; S6. Input the current window data into the gait pattern decision-maker to obtain the pattern discrimination result of the current gait; S7. Based on the pattern discrimination result, select the output of the first expert prediction model or the second expert prediction model as the final gait prediction result.
2. The decision-level multimodal human gait prediction method as described in claim 1, characterized in that, In S1, inertial measurement unit data and surface electromyography signals of the human body are collected in various gait scenarios, including: Triaxial acceleration and angular velocity signals are collected by inertial measurement units deployed on the lower limbs or torso of the human body; Electromyographic signals are collected by surface electromyography sensors deployed in the lower limb muscle groups; The target gait variables are recorded synchronously, and each time window is labeled with a classification label indicating whether it is periodic or non-periodic.
3. The decision-level multimodal human gait prediction method as described in claim 1, characterized in that, In S2, preprocessing and sliding window partitioning include: The inertial measurement unit data is bandpass filtered to remove noise; The surface electromyography signal is subjected to bandpass filtering, rectification, and envelope extraction. The filtered multimodal signal is time-aligned and resampled to a uniform frequency; The aligned signal is slidably segmented using a preset fixed window length and step size to generate the time window sequence.
4. The decision-level multimodal human gait prediction method as described in claim 2, characterized in that, In S3, training the first expert prediction model and the second expert prediction model includes: The inertial measurement unit data in the time window sequence is input into a time-domain convolutional network, and the target gait variable is used as a supervision signal for training to obtain the first expert prediction model. The surface electromyography signals in the time window sequence are input into another temporal convolutional network, and the target gait variable is used as a supervision signal for training to obtain the second expert prediction model.
5. The decision-level multimodal human gait prediction method as described in claim 1, characterized in that, In S4, training the gait pattern decision-maker includes: Extract the fusion features of the time window sequence; The fused features are input into a binary classification model for training. The binary classification model uses the periodic / non-periodic gait labels as supervision signals and outputs the probability that the window belongs to the periodic gait.
6. The decision-level multimodal human gait prediction method as described in claim 1, characterized in that, In S5, the preprocessing and sliding window division process maintains the same window length as the preset fixed window length used in S2.
7. The decision-level multimodal human gait prediction method as described in claim 1, characterized in that, In S6, the pattern discrimination result for the current gait is obtained as follows: The gait pattern decision-maker outputs probability values; The probability value is compared with a preset threshold. If it is greater than or equal to the threshold, it is determined to be a periodic gait; otherwise, it is determined to be a non-periodic gait.
8. The decision-level multimodal human gait prediction method as described in claim 1, characterized in that, In S7, the selection of model output as the final gait prediction result includes: If the pattern discrimination result is periodic gait, then the output of the second expert prediction model is used as the final prediction result; If the pattern discrimination result is a non-periodic gait, then the output of the first expert prediction model is used as the final prediction result.
9. An electronic device, characterized in that, The electronic device includes: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the decision-level multimodal human gait prediction method as described in any one of claims 1-8.
10. A computer storage medium, characterized in that, The computer storage medium stores computer program instructions, which, when executed by a processor, implement the decision-level multimodal human gait prediction method as described in any one of claims 1-8.