Human gait prediction method, device, electronic device and storage medium

Abstract

The present invention provides a method, device, electronic device and storage medium for predicting human gait, and relates to the field of deep learning technology. The present invention can obtain the current surface electromyographic signal of the lower limbs of the human body, perform electromyographic feature learning and gait pattern prediction in multiple dimensions on the current signal window corresponding to the current surface electromyographic signal, and determine the target gait prediction result according to the current prediction result and the adjacent prediction results corresponding to multiple adjacent signal windows stored in advance. In this way, through multi-dimensional electromyographic feature learning, the potential mapping relationship between the surface electromyographic signal and the gait pattern can be fully explored, and by combining the adjacent prediction results corresponding to multiple adjacent signal windows, the stability and reliability of the prediction result can be improved. Therefore, the present invention can achieve high-accuracy gait prediction, thereby improving the real-time performance of gait recognition and realizing the timely switching of the power-assisting strategy of the exoskeleton mechanism.

Description

Technical Field

[0001] The present invention relates to the field of deep learning technology, and in particular to a human gait prediction method, device, electronic device and storage medium. Background Art

[0002] Based on the continuous progress of medical robots and exoskeleton technology, walking-assist exoskeletons have gradually become a research hotspot, and have achieved continuous technological progress and application expansion. Considering the current situation in the field of medical rehabilitation, the development potential and demand for lower limb-assist exoskeletons (a type of walking-assist exoskeleton) mainly lie in the weak human-computer interaction capabilities of rehabilitation-assist machines. Using lower limb exoskeleton robots to assist patients in rehabilitation training is more efficient than traditional manual-assisted rehabilitation treatments. However, when walking in complex environments at high speeds and for long periods of time, the robot simply performs the assistance task and relies on the patient to manually switch the assistance mode, which can easily lead to inaccurate assistance timing or even incorrect assistance modes, resulting in secondary injuries to the patient.

[0003] In the past, exoskeletons mainly achieved human-machine interaction through mechanical control and manual manipulation. Limited by mechanical performance and human manual operation ability, this interaction method could not achieve a more intelligent and natural interaction effect. With the continuous development of sensor technology, robot control and artificial intelligence, the human-machine interaction methods of exoskeletons are constantly enriched, gradually making the user's control of exoskeletons intelligent, real-time and flexible.

[0004] In recent years, high-precision gait recognition based on motion physiological signals has gradually been applied to the mode control process of exoskeletons. However, there are currently two technical challenges: 1. The gait recognition effect is poor in complex terrain; 2. The lack of advance recognition of gait leads to untimely switching of the exoskeleton mechanism's assistance strategy. Summary of the invention

[0005] The purpose of the present invention is to provide a human gait prediction method, device, electronic device and storage medium to achieve high-accuracy gait prediction, thereby improving the real-time performance of gait recognition and achieving timely switching of exoskeleton mechanism assistance strategies.

[0006] In a first aspect, an embodiment of the present invention provides a method for predicting human gait, comprising:

[0007] Acquire a current surface electromyographic signal of a human lower limb, wherein the current surface electromyographic signal includes a two-dimensional time series signal corresponding to multiple sampling time points of multiple channels, and each channel corresponds to a signal collection position of a human lower limb;

[0008] Performing electromyographic feature learning and gait pattern prediction in multiple dimensions on the current signal window corresponding to the current surface electromyographic signal to obtain a current prediction result; wherein the multiple dimensions include multiple dimensions in the time domain, the space domain and the frequency domain;

[0009] The target gait prediction result is determined according to the current prediction result and adjacent prediction results corresponding to multiple adjacent signal windows stored in advance; wherein the multiple adjacent signal windows are multiple signal windows closest to the current signal window.

[0010] Furthermore, the current signal window corresponding to the current surface electromyographic signal is subjected to electromyographic feature learning and gait pattern prediction in multiple dimensions to obtain the current prediction result, including:

[0011] The current prediction result is determined according to the current signal window and the trained gait prediction model based on multidimensional feature learning; wherein, the gait prediction model based on multidimensional feature learning is composed of a spatial convolutional neural network module, a time domain / frequency domain convolutional neural network module, a bidirectional long short-term memory network module and a fully connected module, the spatial convolutional neural network module is used to learn spatial domain information, the time domain / frequency domain convolutional neural network module is used to learn time domain information of time series information, and to learn frequency domain information of spectrum information.

[0012] Furthermore, the spatial convolutional neural network module includes a first spatial CNN module and a second spatial CNN module, the time domain / frequency domain convolutional neural network module includes a first time domain / frequency domain CNN module, a second time domain / frequency domain CNN module, a third time domain / frequency domain CNN module and a fourth time domain / frequency domain CNN module, and the bidirectional long short-term memory network module includes a first BiLSTM module, a second BiLSTM module, a third BiLSTM module and a fourth BiLSTM module; the gait prediction model based on multidimensional feature learning includes a space-time branch, a time domain branch, a space-frequency branch, a frequency domain branch and a fully connected module, and the space-time branch includes a first sequentially connected A spatial CNN module, a first time domain / frequency domain CNN module and a first BiLSTM module, the time domain branch includes a second time domain / frequency domain CNN module and a second BiLSTM module connected in sequence, the space-frequency branch includes a second spatial CNN module, a third time domain / frequency domain CNN module and a third BiLSTM module connected in sequence, the frequency domain branch includes a fourth time domain / frequency domain CNN module and a fourth BiLSTM module connected in sequence, and the output end of the first BiLSTM module, the output end of the second BiLSTM module, the output end of the third BiLSTM module and the output end of the fourth BiLSTM module are all connected to the input end of the fully connected module;

[0013] According to the current signal window and the trained gait prediction model based on multi-dimensional feature learning, the current prediction result is determined, including:

[0014] The current signal window is input into the space-time branch and the time domain branch respectively, and the current signal window is Fourier transformed and input into the space-frequency branch and the frequency domain branch respectively to obtain the current prediction result output by the fully connected module in the gait prediction model based on multidimensional feature learning.

[0015] Furthermore, the spatial convolutional neural network module includes a first convolutional layer, a first batch normalization layer and a dimension shuffling layer; in the spatial convolutional neural network module, the input data is subjected to the convolution operation of the first convolutional layer and the normalization processing of the first batch normalization layer, and then subjected to nonlinear transformation through a rectified linear unit activation function, and finally subjected to dimension rearrangement through the dimension shuffling layer and then output;

[0016] The time domain / frequency domain convolutional neural network module includes a second convolutional layer, a second batch normalization layer and three convolutional units connected in sequence, and the convolutional unit includes a third convolutional layer, a third batch normalization layer and a maximum pooling layer connected in sequence; in the time domain / frequency domain convolutional neural network module, the input data is processed by the convolution operation of the second convolutional layer and the normalization of the second batch normalization layer, and then output after being processed by the three convolutional units;

[0017] The bidirectional long short-term memory network module includes a bidirectional long short-term memory network, a flattening layer, and a first fully connected layer connected in sequence; in the bidirectional long short-term memory network module, input data is processed by the bidirectional long short-term memory network and flattened by the flattening layer, and then nonlinearly transformed by the first fully connected layer and the rectified linear unit activation function before being output;

[0018] The fully connected module includes a second fully connected layer and an output layer connected in sequence; in the fully connected module, the input data is linearly transformed by the second fully connected layer, normalized by a softmax function, and outputted through the output layer.

[0019] Further, the target gait prediction result includes a current predicted gait pattern; determining the target gait prediction result according to the current prediction result and adjacent prediction results corresponding to a plurality of pre-stored adjacent signal windows includes:

[0020] Integrate the current prediction result and adjacent prediction results corresponding to multiple adjacent signal windows in chronological order to obtain the current time series prediction result;

[0021] The current time series prediction result is input into the trained adaptive voting model to obtain the current gait classification result output by the adaptive voting model; wherein the adaptive voting model is used to perform gait classification through a fully connected layer;

[0022] The gait pattern with the largest probability value in the current gait classification result is determined as the current predicted gait pattern.

[0023] Furthermore, the target gait prediction result also includes predicting a gait switching mode; after determining the gait mode with the largest probability value in the current gait classification result as the current predicted gait mode, the human gait prediction method also includes:

[0024] Obtain the most recently predicted N+1 historical predicted gait patterns and sort them in order of prediction time from recent to far; where N is a preset positive integer;

[0025] Based on each historical predicted gait pattern, determine whether the current predicted gait pattern meets the preset gait switching requirements; wherein the gait switching requirements include that the current predicted gait pattern is the same as the previous N historical predicted gait patterns, and the current predicted gait pattern is different from the N+1th historical predicted gait pattern;

[0026] If it meets the requirements, the gait switching pattern from the N+1th historical predicted gait pattern to the current predicted gait pattern is determined as the predicted gait switching pattern.

[0027] Furthermore, the current surface electromyographic signal of the lower limbs of the human body is obtained, including:

[0028] The current surface electromyographic signal is acquired by setting a plurality of patch electrodes on a single lower limb of the human body; wherein, the surface electromyographic signal of one channel is obtained by performing a differential operation on the signals of two adjacent patch electrodes on the same muscle.

[0029] In a second aspect, an embodiment of the present invention further provides a human gait prediction device, comprising:

[0030] An acquisition module is used to acquire the current surface electromyographic signal of the lower limbs of the human body, the current surface electromyographic signal includes a two-dimensional time series signal corresponding to multiple sampling time points of multiple channels, and each channel corresponds to a signal acquisition position of the lower limbs of the human body;

[0031] A prediction module is used to perform electromyographic feature learning and gait pattern prediction in multiple dimensions on the current signal window corresponding to the current surface electromyographic signal to obtain a current prediction result; wherein the multiple dimensions include multiple dimensions in the time domain, the space domain and the frequency domain;

[0032] A determination module is used to determine a target gait prediction result based on a current prediction result and adjacent prediction results corresponding to a plurality of pre-stored adjacent signal windows; wherein the plurality of adjacent signal windows are a plurality of signal windows closest to the current signal window.

[0033] In a third aspect, an embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the computer program, the human gait prediction method of the first aspect is implemented.

[0034] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the human gait prediction method of the first aspect is executed.

[0035] The human gait prediction method, device, electronic device and storage medium provided by the embodiment of the present invention can obtain the current surface electromyographic signal of the lower limb of the human body, and the current surface electromyographic signal includes a two-dimensional time series signal corresponding to multiple sampling time points of multiple channels, and each channel corresponds to a signal collection position of the lower limb of the human body; then the current signal window corresponding to the current surface electromyographic signal is subjected to electromyographic feature learning and gait pattern prediction in multiple dimensions to obtain the current prediction result; wherein the multiple dimensions include multiple in the time domain, the space domain and the frequency domain; and then according to the current prediction result and the adjacent prediction results corresponding to the multiple adjacent signal windows stored in advance, the target gait prediction result is determined; wherein the multiple adjacent signal windows are the multiple signal windows closest to the current signal window. In this way, through multi-dimensional electromyographic feature learning, the potential mapping relationship between the surface electromyographic signal and the gait pattern can be fully explored, and by combining the adjacent prediction results corresponding to the multiple adjacent signal windows, the stability and reliability of the prediction result can be improved. Therefore, the human gait prediction method, device, electronic device and storage medium provided by the embodiment of the present invention can be applied to achieve high-accuracy gait prediction, thereby improving the real-time performance of gait recognition and realizing the timely switching of the power-assisting strategy of the exoskeleton mechanism. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0037] Figure 1 A schematic diagram of a flow chart of a human gait prediction method provided by an embodiment of the present invention;

[0038] Figure 2 A schematic diagram of the structure of a gait prediction model based on multidimensional feature learning provided by an embodiment of the present invention;

[0039] Figure 3 A schematic diagram of the structure of a human gait prediction device provided by an embodiment of the present invention;

[0040] Figure 4 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The technical solution of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] In human-computer interaction with exoskeletons, by collecting and analyzing physical information of movement (acceleration, inertia, plantar pressure, body center of mass position, etc.) and electrophysiological information (electroencephalogram signals, electromyography signals), etc., multiple gait intentions can be recognized, allowing users to flexibly control the switching of exoskeleton power assistance modes. In addition to improving the body stability and support of patients with motor dysfunction, walking-assistance exoskeletons also need to take into account the diverse wearing scenarios and the various gait requirements of users wearing exoskeletons. Therefore, the number of gaits that an exoskeleton can achieve is an important indicator for evaluating exoskeleton performance.

[0043] Based on this, the embodiments of the present invention provide a human gait prediction method, device, electronic device and storage medium, which adopt gait prediction technology based on multi-dimensional feature learning of surface electromyography signals, and can predict gait intentions for various gait patterns and gait switching patterns in continuous gait tasks.

[0044] To facilitate understanding of this embodiment, a human gait prediction method disclosed in an embodiment of the present invention is first introduced in detail.

[0045] The embodiment of the present invention provides a method for predicting human gait. The method can be performed by an electronic device with data processing capability. The electronic device can be, but is not limited to, a data processing device such as a processor of a lower limb exoskeleton robot. Figure 1 The flowchart of a method for predicting human gait is shown in FIG. 1 , and the method for predicting human gait mainly includes the following steps S110 to S130:

[0046] Step S110, obtaining the current surface electromyographic signal of the lower limb of the human body, the current surface electromyographic signal includes a two-dimensional time series signal corresponding to multiple sampling time points of multiple channels, and each channel corresponds to a signal collection position of the lower limb of the human body.

[0047] The above-mentioned channel refers to the signal acquisition channel. Multiple patch electrodes can be set on the unilateral lower limb of the human body, and the surface electromyography (sEMG) can be collected in real time through these patch electrodes. By performing differential operations on the signals of two adjacent patch electrodes on the same muscle, a surface electromyography signal of one channel can be obtained. Electromyography conditioning circuits can also be used to perform noise suppression, filtering, amplification and lifting processing on the collected weak surface electromyography signals. Among them, the patch electrodes can be set on the left lower limb or on the right lower limb, and this embodiment does not limit this. The number of patch electrodes and the sampling frequency of sEMG signals can be set according to actual needs, and two electrode patches are placed on each muscle. For example, 16 patch electrodes are used, and the sampling frequency is 1200 times per second, so that a two-dimensional time series signal corresponding to 1200 sampling time points (referred to as sampling points) of 8 channels can be obtained per second, that is, a two-dimensional time series sEMG signal window with a size of 8×1200.

[0048] Based on this, in some possible embodiments, the above step S110 can be implemented in the following way: by setting multiple patch electrodes on one side of the lower limb of the human body, the current surface electromyography signal is collected; wherein, by performing differential operation on the signals of two adjacent patch electrodes on the same muscle, a surface electromyography signal of one channel is obtained.

[0049] Step S120, performing electromyographic feature learning and gait pattern prediction in multiple dimensions on the current signal window corresponding to the current surface electromyographic signal to obtain a current prediction result; wherein the multiple dimensions include multiple ones in the time domain, space domain and frequency domain.

[0050] Deep neural networks can be used to learn multi-dimensional electromyographic features and predict gait patterns. Deep neural networks can combine multi-dimensional information such as the signal's time domain, spatial domain, and frequency domain for learning, and finally connect them through a layer of fully connected network.

[0051] Based on this, in some possible embodiments, the above step S120 can be implemented in the following manner: determine the current prediction result according to the current signal window and the trained gait prediction model based on multidimensional feature learning; wherein the gait prediction model based on multidimensional feature learning is composed of a spatial convolutional neural network module, a time domain / frequency domain convolutional neural network module, a bidirectional long short-term memory network module and a fully connected module, the spatial convolutional neural network module is used to learn spatial domain information, that is, to learn the mapping information between the spatial features corresponding to the surface electromyography signal and the gait pattern, the time domain / frequency domain convolutional neural network module is used to learn the time domain information of the time series information, that is, to learn the mapping information between the time series features corresponding to the surface electromyography signal and the gait pattern, and to learn the frequency domain information of the spectrum information, that is, to learn the mapping information between the frequency domain features corresponding to the surface electromyography signal and the gait pattern. For the time domain / frequency domain convolutional neural network module: when the input data is a time domain signal, the time domain / frequency domain convolutional neural network module can be called a time domain convolutional neural network module; when the input data is a frequency domain signal, the time domain / frequency domain convolutional neural network module can be called a frequency domain convolutional neural network module.

[0052] In the above-mentioned gait prediction model based on multidimensional feature learning, the spatial convolutional neural network module, the time domain / frequency domain convolutional neural network module and the bidirectional long short-term memory network module can be combined according to actual needs, and then connected with the fully connected module to output the prediction results through the fully connected module.

[0053] For ease of understanding, the following is an exemplary introduction to each module in the gait prediction model based on multidimensional feature learning:

[0054] 1. Spatial convolutional neural network module, also known as spatial CNN (Convolutional Neural Networks) module

[0055] The spatial convolutional neural network module may include a first convolutional layer, a first batch normalization layer (batch normalization layer, i.e. BatchNorm, abbreviated as BN) and a dimension shuffling layer (i.e. Permute); in the spatial convolutional neural network module, the input data is subjected to the convolution operation of the first convolutional layer and the normalization processing of the first batch normalization layer, and then subjected to a nonlinear transformation through a rectified linear unit activation function (LeakyReLU), and finally output after dimension rearrangement through the dimension shuffling layer.

[0056] 2. Time domain / frequency domain convolutional neural network module, also known as time domain / frequency domain CNN module

[0057] The time domain / frequency domain convolutional neural network module may include a second convolutional layer, a second batch normalization layer and three convolutional units connected in sequence, and the convolutional unit includes a third convolutional layer, a third batch normalization layer and a maximum pooling layer connected in sequence; in the time domain / frequency domain convolutional neural network module, the input data is processed by the convolution operation of the second convolutional layer and the normalization processing of the second batch normalization layer, and then output after being processed by the three convolutional units; wherein, in each convolutional unit, the input data is processed by the third batch normalization layer after the convolution operation of the third convolutional layer, and is activated using LeakyReLU, and then is subjected to maximum pooling through the maximum pooling layer.

[0058] 3. Bidirectional long short-term memory network module, also known as BiLSTM (Bi-directional Long Short-Term Memory) module

[0059] The bidirectional long short-term memory network module may include a bidirectional long short-term memory network (BiLSTM), a flattening layer and a first fully connected layer connected in sequence; in the bidirectional long short-term memory network module, the input data is processed by the bidirectional long short-term memory network and flattened by the flattening layer, and then output after nonlinear transformation through the first fully connected layer and the rectified linear unit activation function.

[0060] 4. Fully connected module

[0061] The fully connected module may include a second fully connected layer and an output layer connected in sequence; in the fully connected module, the input data is linearly transformed by the second fully connected layer, normalized by the Softmax function, and outputted through the output layer.

[0062] It should be noted that the embodiment of the present invention does not limit the specific structure of each module in the above-mentioned gait prediction model based on multi-dimensional feature learning. In other embodiments, other structures may also be used.

[0063] The embodiment of the present invention also provides a specific structure of a gait prediction model based on multidimensional feature learning. The gait prediction model based on multidimensional feature learning includes four branches, which involve two spatial convolutional neural network modules, four time domain / frequency domain convolutional neural network modules, and four bidirectional long short-term memory network modules, as follows:

[0064] The spatial convolutional neural network module includes a first spatial CNN module and a second spatial CNN module, the time domain / frequency domain convolutional neural network module includes a first time domain / frequency domain CNN module, a second time domain / frequency domain CNN module, a third time domain / frequency domain CNN module and a fourth time domain / frequency domain CNN module, and the bidirectional long short-term memory network module includes a first BiLSTM module, a second BiLSTM module, a third BiLSTM module and a fourth BiLSTM module; on this basis, the gait prediction model based on multidimensional feature learning includes a space-time branch, a time domain branch, a space-frequency branch, a frequency domain branch and a fully connected module, and the space-time branch includes a first sequentially connected A spatial CNN module, a first time domain / frequency domain CNN module and a first BiLSTM module, the time domain branch includes a second time domain / frequency domain CNN module and a second BiLSTM module connected in sequence, the space-frequency branch includes a second spatial CNN module, a third time domain / frequency domain CNN module and a third BiLSTM module connected in sequence, the frequency domain branch includes a fourth time domain / frequency domain CNN module and a fourth BiLSTM module connected in sequence, and the output end of the first BiLSTM module, the output end of the second BiLSTM module, the output end of the third BiLSTM module and the output end of the fourth BiLSTM module are all connected to the input end of the fully connected module.

[0065] Based on the above-mentioned gait prediction model structure based on multidimensional feature learning, determining the current prediction result according to the current signal window and the trained gait prediction model based on multidimensional feature learning can include: inputting the current signal window into the space-time branch and the time domain branch respectively, and performing Fourier transform on the current signal window and inputting them into the space-frequency branch and the frequency domain branch respectively, to obtain the current prediction result output by the fully connected module in the gait prediction model based on multidimensional feature learning.

[0066] It should be noted that the above-mentioned gait prediction model based on multidimensional feature learning may include only some of the above-mentioned four branches, such as only the space-time branch, or only the space-time branch and the space-frequency branch, etc., or may include more branches, which is not limited in this embodiment.

[0067] Step S130, determining a target gait prediction result according to the current prediction result and adjacent prediction results corresponding to a plurality of pre-stored adjacent signal windows; wherein the plurality of adjacent signal windows are a plurality of signal windows closest to the current signal window.

[0068] Each time a prediction result is obtained through a gait prediction model based on multidimensional feature learning, the prediction result will be stored. The currently obtained prediction result is called the current prediction result, and the prediction result stored before the current moment is called the historical prediction result. Only the preset number of historical prediction results closest to the current moment can be stored, so that multiple adjacent prediction results and the current prediction result in the historical prediction results can be used to capture the trend of myoelectric changes and improve the stability and reliability of the prediction results. Among them, the adjacent prediction results are the historical prediction results used to determine the target gait prediction results. The preset number and the number of adjacent signal windows (or adjacent prediction results) can be set according to actual needs, and there is no limitation here. For example, the preset number is 20, and the number of adjacent signal windows is 4, that is, the target gait prediction result is obtained by integrating the prediction results of 5 consecutive signal windows.

[0069] In some possible embodiments, the target gait prediction result includes the current predicted gait pattern; based on this, step S130 may include: integrating the current prediction result and the adjacent prediction results corresponding to multiple adjacent signal windows in chronological order to obtain the current time series prediction result; inputting the current time series prediction result into the trained adaptive voting model to obtain the current gait classification result output by the adaptive voting model; wherein the adaptive voting model is used to perform gait classification through a fully connected layer; and determining the gait pattern with the largest probability value in the current gait classification result as the current predicted gait pattern. The current predicted gait pattern may be one of the following: standing, walking, going up stairs, going down stairs, going uphill, going downhill, crossing obstacles, retreating, and striding.

[0070] Furthermore, the above-mentioned target gait prediction result also includes a predicted gait switching mode, which can assist the switching of the assistant mode of the lower limb exoskeleton robot. Each time the predicted gait pattern is obtained, the predicted gait pattern can be stored. The currently obtained predicted gait pattern is called the current predicted gait pattern, and the predicted gait pattern stored before the current moment is called the historical predicted gait pattern. The historical predicted gait pattern and the current predicted gait pattern can be used to determine whether there is a gait switch, thereby obtaining the predicted gait switching pattern. Based on this, after determining the gait pattern with the largest probability value in the current gait classification result as the current predicted gait pattern, the above-mentioned human gait prediction method also includes: obtaining the N+1 historical predicted gait patterns obtained by the most recent prediction, and sorting them in order of prediction time from near to far; wherein N is a preset positive integer; based on each historical predicted gait pattern, judging whether the current predicted gait pattern meets the preset gait switching requirements; wherein the gait switching requirements include that the current predicted gait pattern is the same as the previous N historical predicted gait patterns, and the current predicted gait pattern is different from the N+1th historical predicted gait pattern; if so, determining the gait switching pattern from the N+1th historical predicted gait pattern to the current predicted gait pattern as the predicted gait switching pattern; if not, confirming that no gait switch is predicted.

[0071] The above N can be set according to actual needs and is not limited here. For example, N is 5, that is, the predicted gait patterns are the same for 5 consecutive times, and they are all different from the predicted gait patterns before these 5 times, then it is confirmed that the gait switch is predicted. The gait switching mode can include: standing → crossing obstacles, crossing obstacles → walking, walking → climbing stairs, climbing stairs → walking, walking → downhill, downhill → walking, standing → walking, walking → uphill, uphill → walking, walking → down stairs, down stairs → walking, walking → crossing obstacles, crossing obstacles → standing, standing → backing up, standing → striding.

[0072] The human gait prediction method provided by the embodiment of the present invention can obtain the current surface electromyographic signal of the human lower limb, and the current surface electromyographic signal includes a two-dimensional time series signal corresponding to multiple sampling time points of multiple channels, and each channel corresponds to a signal collection position of the human lower limb; then the current signal window corresponding to the current surface electromyographic signal is subjected to electromyographic feature learning and gait pattern prediction in multiple dimensions to obtain the current prediction result; wherein the multiple dimensions include multiple in the time domain, the space domain and the frequency domain; and then according to the current prediction result and the adjacent prediction results corresponding to the multiple adjacent signal windows stored in advance, the target gait prediction result is determined; wherein the multiple adjacent signal windows are the multiple signal windows closest to the current signal window. In this way, through multi-dimensional electromyographic feature learning, the potential mapping relationship between the surface electromyographic signal and the gait pattern can be fully explored, and by combining the adjacent prediction results corresponding to the multiple adjacent signal windows, the stability and reliability of the prediction result can be improved. Therefore, the human gait prediction method provided by the embodiment of the present invention can achieve high-accuracy gait prediction, thereby improving the real-time performance of gait recognition and realizing the timely switching of the exoskeleton mechanism power-assisting strategy.

[0073] For ease of understanding, the above-mentioned human gait prediction method is exemplarily introduced below.

[0074] The embodiment of the present invention proposes a multi-dimensional feature learning algorithm for lower limb gait prediction: Deep-STF (Spatial-Temporal-Frequency). The algorithm uses a deep neural network (i.e., the above-mentioned gait prediction model based on multi-dimensional feature learning) to predict the surface electromyography signal of unilateral lower limb. The deep neural network combines the multi-dimensional information of the surface electromyography signal such as the time domain, spatial domain, and frequency domain for learning, and finally connects them through a layer of fully connected network. The structure of the deep neural network (i.e., the structure of the gait prediction model based on multi-dimensional feature learning) is as follows: Figure 2 shown.

[0075] The embodiment of the present invention also proposes an adaptive voting strategy, which connects the prediction results of five consecutive time series windows (i.e., signal windows), uses a fully connected layer to automatically learn the voting weight, and realizes the learning of adaptive voting coefficients, thereby automatically capturing the trend of myoelectric changes. This strategy can be combined with any deep network.

[0076] Refer to the following Figure 2 A detailed introduction to the deep neural network structure of Deep-STF:

[0077] like Figure 2As shown in the figure, the Deep-STF model (i.e., a gait prediction model based on multidimensional feature learning) contains four modules, involving four branches. The input signal is a two-dimensional time series sEMG signal window (i.e., time domain sEMG signal) of size 8×1200 (i.e., 8 channels×1200 sampling points). The signal window is transposed and input into the space-time branch and time domain branch of the Deep-STF model. The time dimension of the transposed signal window is Fourier transformed to obtain the frequency domain sEMG signal, and the frequency domain sEMG signal is input into the space-frequency branch and frequency domain branch of the Deep-STF model. Each branch consists of four modules of the Deep-STF model. The following is an introduction to each module:

[0078] (1) In the spatial CNN module (A), after a layer of convolution operation (such as A 1 The number of output channels is increased to 8, and the convolution kernel size is ((1,8),8). Subsequently, the batch normalization layer (i.e., BatchNorm) is used for normalization to reduce the internal covariate shift during training, and the LeakyReLU activation function is used for nonlinear transformation. The processed feature map is then re-arranged by Permute, transposing its channel dimension and spatial dimension.

[0079] (2) In the time domain / frequency domain CNN module (B), a convolution layer (such as B 1 Conv on the left) and a BN layer (such as B 1 The BatchNorm on the left in the figure shows that the convolution kernel size is ((3,1),8). Then, after passing through three layers of convolution units, the convolution kernel size is ((3,1),16), and the number of channels is expanded to 16 and remains unchanged. In the convolution unit, in each convolution layer (such as B 1 Conv on the right side of the middle) and then pass through the BN layer (such as B 1 BatchNorm on the right in the figure) and use LeakyReLU for activation, followed by maximum pooling (as in B 1 Max-Pool in , the pooling layer parameters are (1,4).

[0080] (3) In the BiLSTM module (C), the input data first passes through a bidirectional long short-term memory network (such as C 1 The BiLSTM in the example above is processed to obtain the output tensor x, where the input feature dimension is 128, the hidden state dimension is 64, and the number of stacking layers is 3. Then, through torch.reshape (such as C 1 Flatten in ) flattens the output tensor x to adapt it to the input requirements of the fully connected layer. Finally, through the fully connected layer (such as C 1The FC1) and LeakyReLU activation functions in the module perform nonlinear transformation on the flattened features to obtain the output of the module. The parameters of the fully connected layer are (2048, 64).

[0081] (4) In the fully connected module (D), the input data passes through a fully connected layer (such as D 1 The data dimension is changed from 256 to 9, and the output is normalized by the Softmax function to obtain the final output of each gait probability. 1 Output) output in .

[0082] The four branches combine modules A, B, and C and finally merge in D. The following is a detailed introduction to each branch:

[0083] (1) Space-time branch: A 2D time-series sEMG signal window with a size of 1200 × 8 is first input into the spatial CNN module (A 1 ) and then input into the time domain / frequency domain CNN module (B 1 ), and then input into the BiLSTM module (C 1 ). 1 The module is flattened at the end, and the output dimension is 1×64.

[0084] (2) Time domain branch: A two-dimensional time series sEMG signal window with a size of 1200 × 8 is first input into the time domain / frequency domain CNN module (B 2 ) and then input into the BiLSTM module (C 2 ), the dimension of the output result is 1×64.

[0085] (3) Space-frequency branch: The time dimension of the signal window is Fourier transformed. After the transformation, the signal dimension remains 1200×8 and is first input into the spatial CNN module (A 2 ) and then input into the time domain / frequency domain CNN module (B 3 ), and then input into the BiLSTM module (C 3 ). 3 The module is flattened at the end, and the output dimension is also 1×64.

[0086] (4) Frequency domain branch: The time dimension of the signal window is Fourier transformed. The transformed signal dimension is 1200×8 and is first input into the time domain / frequency domain CNN module (B 4 ) and then input into the BiLSTM module (C 4 ), the dimension of the output result is 1×64.

[0087] Finally, the outputs of the four branches are combined into a one-dimensional vector through the fully connected module D to form a 1×256 overall output. After passing through the fully connected layer and normalized using the Softmax layer, the output size is reduced to the final output dimension (1,9) of the Deep-STF model.

[0088] The following is an introduction to the adaptive voting strategy:

[0089] The adaptive voting strategy integrates the prediction results of five consecutive time series windows, that is, the input signal is (1,45), and shrinks 45 to 9 again through a fully connected layer (i.e., the adaptive voting layer), corresponding to the classification of nine gait types, thereby capturing the changing trend of gait characteristics in five consecutive signal windows.

[0090] The following is an exemplary introduction to the training methods of the above Deep-STF model and adaptive voting model:

[0091] The collected historical surface electromyography data was divided into training set, validation set and test set according to the ratio of 7:1:2. In the training of Deep-STF model, the training was stopped early according to the change of the accuracy of the validation set. The specific method is: when the accuracy of the validation set is not improved for 20 consecutive rounds of training, the first round parameters of the 20 rounds are taken as the final model parameters. In the training of the adaptive voting layer (i.e., the adaptive voting model), the validation set is used as the training data, and the number of training rounds is 5000.

[0092] The key technical points of the embodiments of the present invention are: 1. Only surface electromyography signals are used, and multimodal signal fusion is not involved, which can reduce the complexity of the system. 2. Multi-dimensional feature learning is performed on surface electromyography signals, and the signals are analyzed from the time domain, space domain and frequency domain to make full use of the signals. 3. An adaptive voting strategy is used to improve the stability and reliability of the prediction results.

[0093] The embodiment of the present invention has the following beneficial effects: it can classify 9 gait modes (standing, walking, going up stairs, going down stairs, going uphill, going downhill, crossing obstacles, backing up, and striding) and 15 gait switching (standing → crossing obstacles, crossing obstacles → walking, walking → going up stairs, going up stairs → walking, walking → going downhill, going downhill → walking, standing → walking, walking → going uphill, going uphill → walking, walking → going down stairs, going down stairs → walking, walking → crossing obstacles, crossing obstacles → standing, standing → backing up, and standing → striding). The Deep-STF model was verified in 12 subjects; the results showed that the prediction time for gait switching reached 31ms-371ms, and the prediction accuracy reached 93.22%-96.60%. A comparative verification was also carried out on another four deep neural networks, namely artificial neural network (ANN), convolutional neural network (CNN), long short term memory network (LSTM) and CNN-LSTM hybrid deep network; the results showed that the prediction accuracy of the method of this embodiment was higher than that of these four deep neural networks, and the adaptive voting strategy proposed in the embodiment of the present invention showed a significant improvement in accuracy in the Deep-STF model and the above four deep neural networks.

[0094] Corresponding to the above-mentioned human gait prediction method, the embodiment of the present invention further provides a human gait prediction device. Figure 3 A schematic structural diagram of a human gait prediction device is shown, the human gait prediction device comprising:

[0095] An acquisition module 301 is used to acquire a current surface electromyographic signal of a human lower limb, wherein the current surface electromyographic signal includes a two-dimensional time series signal corresponding to a plurality of sampling time points of a plurality of channels, and each channel corresponds to a signal acquisition position of a human lower limb;

[0096] Prediction module 302, used for performing electromyographic feature learning and gait pattern prediction in multiple dimensions on the current signal window corresponding to the current surface electromyographic signal, to obtain a current prediction result; wherein the multiple dimensions include multiple dimensions in time domain, space domain and frequency domain;

[0097] The determination module 303 is used to determine the target gait prediction result according to the current prediction result and the adjacent prediction results corresponding to the plurality of pre-stored adjacent signal windows; wherein the plurality of adjacent signal windows are the plurality of signal windows closest to the current signal window.

[0098] The human gait prediction device provided by the embodiment of the present invention can obtain the current surface electromyographic signal of the human lower limb, and the current surface electromyographic signal includes a two-dimensional time series signal corresponding to multiple sampling time points of multiple channels, and each channel corresponds to a signal collection position of the human lower limb; then the current signal window corresponding to the current surface electromyographic signal is subjected to multiple-dimensional electromyographic feature learning and gait pattern prediction to obtain the current prediction result; wherein the multiple dimensions include multiple in the time domain, the space domain and the frequency domain; and then according to the current prediction result and the adjacent prediction results corresponding to the multiple adjacent signal windows stored in advance, the target gait prediction result is determined; wherein the multiple adjacent signal windows are the multiple signal windows closest to the current signal window. In this way, through multi-dimensional electromyographic feature learning, the potential mapping relationship between the surface electromyographic signal and the gait pattern can be fully explored, and by combining the adjacent prediction results corresponding to the multiple adjacent signal windows, the stability and reliability of the prediction result can be improved. Therefore, the human gait prediction device provided by the embodiment of the present invention can achieve high-accuracy gait prediction, thereby improving the real-time performance of gait recognition and realizing the timely switching of the power-assisting strategy of the exoskeleton mechanism.

[0099] Furthermore, the acquisition module 301 is specifically used to collect the current surface electromyography signal by setting multiple patch electrodes on one side of the lower limb of the human body; wherein, the surface electromyography signal of one channel is obtained by performing differential operation on the signals of two adjacent patch electrodes on the same muscle.

[0100] Furthermore, the above-mentioned prediction module 302 is specifically used to: determine the current prediction result according to the current signal window and the trained gait prediction model based on multidimensional feature learning; wherein, the gait prediction model based on multidimensional feature learning is composed of a spatial convolutional neural network module, a time domain / frequency domain convolutional neural network module, a bidirectional long short-term memory network module and a fully connected module, the spatial convolutional neural network module is used to learn spatial domain information, the time domain / frequency domain convolutional neural network module is used to learn time domain information of time series information, and to learn frequency domain information of spectral information.

[0101] Furthermore, the spatial convolutional neural network module includes a first spatial CNN module and a second spatial CNN module, the time domain / frequency domain convolutional neural network module includes a first time domain / frequency domain CNN module, a second time domain / frequency domain CNN module, a third time domain / frequency domain CNN module and a fourth time domain / frequency domain CNN module, and the bidirectional long short-term memory network module includes a first BiLSTM module, a second BiLSTM module, a third BiLSTM module and a fourth BiLSTM module; the gait prediction model based on multidimensional feature learning includes a space-time branch, a time domain branch, a space-frequency branch, a frequency domain branch and a fully connected module, the space-time branch includes a first spatial CNN module, a first time domain / frequency domain CNN module and a first BiLSTM module connected in sequence, and the time domain branch includes a second time domain / frequency domain CNN module connected in sequence. The prediction module 302 is further used to: input the current signal window to the space-time branch and the time domain branch respectively, and perform Fourier transform on the current signal window and input them to the space-frequency branch and the frequency domain branch respectively, so as to obtain the current prediction result output by the fully connected module in the gait prediction model based on multidimensional feature learning.

[0102] Furthermore, the spatial convolutional neural network module includes a first convolutional layer, a first batch normalization layer and a dimension shuffling layer; in the spatial convolutional neural network module, the input data is subjected to the convolution operation of the first convolutional layer and the normalization processing of the first batch normalization layer, and then subjected to nonlinear transformation through a rectified linear unit activation function, and finally subjected to dimension rearrangement through the dimension shuffling layer and then output;

[0103] The time domain / frequency domain convolutional neural network module includes a second convolutional layer, a second batch normalization layer and three convolutional units connected in sequence, and the convolutional unit includes a third convolutional layer, a third batch normalization layer and a maximum pooling layer connected in sequence; in the time domain / frequency domain convolutional neural network module, the input data is processed by the convolution operation of the second convolutional layer and the normalization of the second batch normalization layer, and then output after being processed by the three convolutional units;

[0104] The bidirectional long short-term memory network module includes a bidirectional long short-term memory network, a flattening layer, and a first fully connected layer connected in sequence; in the bidirectional long short-term memory network module, input data is processed by the bidirectional long short-term memory network and flattened by the flattening layer, and then nonlinearly transformed by the first fully connected layer and the rectified linear unit activation function before being output;

[0105] The fully connected module includes a second fully connected layer and an output layer connected in sequence; in the fully connected module, the input data is linearly transformed by the second fully connected layer, normalized by a softmax function, and outputted through the output layer.

[0106] Furthermore, the above-mentioned target gait prediction result includes the current predicted gait pattern; the above-mentioned determination module 303 is specifically used to: integrate the current prediction result and the adjacent prediction results corresponding to multiple adjacent signal windows in chronological order to obtain the current time series prediction result; input the current time series prediction result into the trained adaptive voting model to obtain the current gait classification result output by the adaptive voting model; wherein the adaptive voting model is used to perform gait classification through a fully connected layer; and determine the gait pattern with the largest probability value in the current gait classification result as the current predicted gait pattern.

[0107] Furthermore, the target gait prediction result also includes a predicted gait switching pattern; the determination module 303 is also used to: obtain the most recently predicted N+1 historical predicted gait patterns, and sort them in order of prediction time from near to far; wherein N is a preset positive integer; based on each historical predicted gait pattern, determine whether the current predicted gait pattern meets the preset gait switching requirements; wherein the gait switching requirements include that the current predicted gait pattern is the same as the first N historical predicted gait patterns, and the current predicted gait pattern is different from the N+1th historical predicted gait pattern; if so, determine the gait switching pattern from the N+1th historical predicted gait pattern to the current predicted gait pattern as the predicted gait switching pattern.

[0108] The human gait prediction device provided in this embodiment has the same implementation principle and technical effects as those of the aforementioned human gait prediction method embodiment. For the sake of brief description, for matters not mentioned in the human gait prediction device embodiment, reference may be made to the corresponding contents in the aforementioned human gait prediction method embodiment.

[0109] like Figure 4 As shown, an electronic device 400 provided by an embodiment of the present invention includes: a processor 401, a memory 402 and a bus, the memory 402 stores a computer program that can be run on the processor 401, when the electronic device 400 is running, the processor 401 and the memory 402 communicate through the bus, and the processor 401 executes the computer program to implement the above-mentioned human gait prediction method.

[0110] Specifically, the memory 402 and the processor 401 can be general-purpose memories and processors, which are not specifically limited here.

[0111] The embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the human gait prediction method in the previous method embodiment is executed. The computer-readable storage medium includes: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a RAM, a magnetic disk, or an optical disk, etc., which can store program codes.

[0112] The term "and / or" herein is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set consisting of A, B, and C.

[0113] In all examples shown and described herein, any specific values ​​should be interpreted as merely exemplary and not as limiting, and thus other examples of the exemplary embodiments may have different values.

[0114] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0115] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0116] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0117] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0118] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A human gait prediction method, characterized in that: include: Acquire a current surface electromyographic signal of a human lower limb, wherein the current surface electromyographic signal includes a two-dimensional time series signal corresponding to a plurality of sampling time points of a plurality of channels, and each of the channels corresponds to a signal collection position of the human lower limb; Performing electromyographic feature learning and gait pattern prediction in multiple dimensions on the current signal window corresponding to the current surface electromyographic signal to obtain a current prediction result; wherein the multiple dimensions include multiple dimensions in the time domain, the space domain, and the frequency domain; Determine a target gait prediction result according to the current prediction result and adjacent prediction results corresponding to a plurality of pre-stored adjacent signal windows; wherein the plurality of adjacent signal windows are a plurality of signal windows closest to the current signal window; The target gait prediction result includes a current predicted gait pattern; and determining the target gait prediction result according to the current prediction result and adjacent prediction results corresponding to a plurality of pre-stored adjacent signal windows includes: Integrate the current prediction result and adjacent prediction results corresponding to the multiple adjacent signal windows in chronological order to obtain a current time series prediction result; Inputting the current time series prediction result into the trained adaptive voting model to obtain the current gait classification result output by the adaptive voting model; wherein the adaptive voting model is used to perform gait classification through a fully connected layer; The gait pattern with the largest probability value in the current gait classification result is determined as the current predicted gait pattern.

2. The human gait prediction method according to claim 1, characterized in that: The performing electromyographic feature learning and gait pattern prediction in multiple dimensions on the current signal window corresponding to the current surface electromyographic signal to obtain the current prediction result includes: The current prediction result is determined according to the current signal window and the trained gait prediction model based on multidimensional feature learning; wherein the gait prediction model based on multidimensional feature learning is composed of a spatial convolutional neural network module, a time domain / frequency domain convolutional neural network module, a bidirectional long short-term memory network module and a fully connected module, the spatial convolutional neural network module is used to learn spatial domain information, the time domain / frequency domain convolutional neural network module is used to learn time domain information of time series information, and to learn frequency domain information of spectrum information.

3. The human gait prediction method according to claim 2, characterized in that: The spatial convolutional neural network module includes a first spatial CNN module and a second spatial CNN module, the time domain / frequency domain convolutional neural network module includes a first time domain / frequency domain CNN module, a second time domain / frequency domain CNN module, a third time domain / frequency domain CNN module and a fourth time domain / frequency domain CNN module, the bidirectional long short-term memory network module includes a first BiLSTM module, a second BiLSTM module, a third BiLSTM module and a fourth BiLSTM module; the gait prediction model based on multidimensional feature learning includes a space-time branch, a time domain branch, a space-frequency branch, a frequency domain branch and the fully connected module, the space-time branch includes the first spatial CNN module, the first time domain CNN module and the first time domain CNN module connected in sequence. The time domain branch includes the second time domain / frequency domain CNN module and the second BiLSTM module connected in sequence, the space-frequency branch includes the second space CNN module, the third time domain / frequency domain CNN module and the third BiLSTM module connected in sequence, the frequency domain branch includes the fourth time domain / frequency domain CNN module and the fourth BiLSTM module connected in sequence, and the output end of the first BiLSTM module, the output end of the second BiLSTM module, the output end of the third BiLSTM module and the output end of the fourth BiLSTM module are all connected to the input end of the fully connected module; Determining the current prediction result according to the current signal window and the trained gait prediction model based on multi-dimensional feature learning includes: The current signal window is input into the space-time branch and the time domain branch respectively, and the current signal window is Fourier transformed and then input into the space-frequency branch and the frequency domain branch respectively to obtain the current prediction result output by the fully connected module in the gait prediction model based on multidimensional feature learning.

4. The human gait prediction method according to claim 2, characterized in that: The spatial convolutional neural network module includes a first convolutional layer, a first batch normalization layer and a dimension shuffling layer; in the spatial convolutional neural network module, after the input data passes through the convolution operation of the first convolutional layer and the normalization processing of the first batch normalization layer, it is nonlinearly transformed through a rectified linear unit activation function, and finally, it is output after dimension rearrangement through the dimension shuffling layer; The time domain / frequency domain convolutional neural network module includes a second convolutional layer, a second batch normalization layer and three convolutional units connected in sequence, and the convolutional unit includes a third convolutional layer, a third batch normalization layer and a maximum pooling layer connected in sequence; in the time domain / frequency domain convolutional neural network module, the input data is processed by the convolution operation of the second convolutional layer and the normalization processing of the second batch normalization layer, and then output after being processed by the three convolutional units; The bidirectional long short-term memory network module comprises a bidirectional long short-term memory network, a flattening layer and a first fully connected layer connected in sequence; in the bidirectional long short-term memory network module, after the input data is processed by the bidirectional long short-term memory network and flattened by the flattening layer, it is nonlinearly transformed by the first fully connected layer and the rectified linear unit activation function and then output; The fully connected module includes a second fully connected layer and an output layer connected in sequence; in the fully connected module, the input data is linearly transformed by the second fully connected layer, normalized by the Softmax function, and outputted through the output layer.

5. The human gait prediction method according to claim 1, characterized in that: The target gait prediction result also includes a predicted gait switching mode; after determining the gait mode with the largest probability value in the current gait classification result as the current predicted gait mode, the human gait prediction method also includes: Obtain the most recently predicted N+1 historical predicted gait patterns, and sort them in order of prediction time from recent to far; wherein N is a preset positive integer; Based on each of the historical predicted gait patterns, determining whether the current predicted gait pattern meets a preset gait switching requirement; wherein the gait switching requirement includes that the current predicted gait pattern is the same as the first N historical predicted gait patterns, and the current predicted gait pattern is different from the N+1th historical predicted gait pattern; If it meets the requirements, the gait switching pattern from the N+1th historical predicted gait pattern to the current predicted gait pattern is determined as the predicted gait switching pattern.

6. The human gait prediction method according to any one of claims 1 to 5, characterized in that: The step of obtaining the current surface electromyographic signal of the lower limbs of the human body comprises: The current surface electromyographic signal is collected by setting a plurality of patch electrodes on a single lower limb of the human body; wherein, a surface electromyographic signal of one channel is obtained by performing a differential operation on the signals of two adjacent patch electrodes on the same muscle.

7. A human gait prediction device, characterized in that: include: An acquisition module, used for acquiring a current surface electromyographic signal of a human lower limb, wherein the current surface electromyographic signal comprises a two-dimensional time series signal corresponding to a plurality of sampling time points of a plurality of channels, and each of the channels corresponds to a signal acquisition position of the human lower limb; A prediction module, used for performing electromyographic feature learning and gait pattern prediction in multiple dimensions on the current signal window corresponding to the current surface electromyographic signal to obtain a current prediction result; wherein the multiple dimensions include multiple ones in the time domain, space domain and frequency domain; A determination module, configured to determine a target gait prediction result according to the current prediction result and adjacent prediction results corresponding to a plurality of pre-stored adjacent signal windows; wherein the plurality of adjacent signal windows are a plurality of signal windows closest to the current signal window; The target gait prediction result includes the current predicted gait pattern; the determination module is specifically used to: integrate the current prediction result and the adjacent prediction results corresponding to the multiple adjacent signal windows in chronological order to obtain the current time series prediction result; input the current time series prediction result into the trained adaptive voting model to obtain the current gait classification result output by the adaptive voting model; wherein the adaptive voting model is used to perform gait classification through a fully connected layer; and determine the gait pattern with the largest probability value in the current gait classification result as the current predicted gait pattern.

8. An electronic device comprising a memory and a processor, wherein the memory stores a computer program that can be run on the processor, characterized in that: When the processor executes the computer program, the human gait prediction method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the human gait prediction method according to any one of claims 1 to 6 is executed.

Images