Gait prediction model acquisition method, gait prediction method, gait prediction device and walking aid equipment

Abstract

The invention relates to the technical field of machine learning, and discloses a gait prediction model acquisition method, a gait prediction method, a gait prediction device and walking aid equipment, the gait prediction model acquisition method comprises the following steps: acquiring a surface electromyogram signal of a lower limb of a human body; the gait prediction model acquisition method comprises the following steps: performing data enhancement on an initial training sample in a closed environment data set according to an out-of-distribution data enhancement mode to obtain an enhanced training sample; wherein the initial training sample is a training sample formed by collected surface electromyogram signals of preset collection points of lower limbs of a human body under a plurality of preset gait types; inputting the target training sample into an evidence convolutional neural network for training to obtain a gait prediction model; according to the method, the training sample is subjected to out-of-distribution enhancement by adopting an out-of-distribution data enhancement mode during model training, and the existing convolutional neural network is improved, so that the accuracy of gait prediction in the open environment is realized.

Description

Technical Field

[0001] The present application relates to the field of machine learning technology, and in particular to a gait prediction model acquisition method, a gait prediction method, a device, and a walking aid. Background Art

[0002] The existing gait prediction technology based on electromyographic signals usually only focuses on a few predefined gait types and cannot adapt to undefined gait types. Therefore, walking aids that use existing gait prediction technology often continue to output erroneous prediction results when encountering undefined gait types, which in turn puts people using the walking aids at risk of falling. Summary of the invention

[0003] In view of this, the embodiments of the present application provide a gait prediction model acquisition method, a gait prediction method, an apparatus and a walking aid device, which can effectively predict gait in an open environment.

[0004] In a first aspect, an embodiment of the present application provides a method for obtaining a gait prediction model, comprising: Performing data enhancement on the initial training samples in the closed environment data set according to the out-of-distribution data enhancement method to obtain enhanced training samples; wherein the initial training samples are training samples formed by the surface electromyography signals of preset collection points of the lower limbs of the human body under multiple preset gait types; The target training samples are input into the evidence convolutional neural network for training to obtain a gait prediction model; wherein the target training samples include the initial training samples and the enhanced training samples.

[0005] In some embodiments, the data enhancement of the initial training samples in the closed environment dataset according to the out-of-distribution data enhancement method to obtain enhanced training samples includes: Before each iterative training, a preset number of the initial training samples are randomly selected from the closed environment data set; The initial training samples are processed by linear interpolation to obtain the corresponding enhanced training samples.

[0006] In some embodiments, the training samples in the enhanced training samples are obtained by a preset linear interpolation formula; Wherein, the preset linear interpolation formula is: ; in, and are two different initial training samples; For Beta( ) distribution to obtain the interpolation ratio, is the default value.

[0007] In some embodiments, inputting the target training sample into the evidence convolutional neural network for training to obtain a gait prediction model includes: The target training sample is input into the evidence convolutional neural network, and the evidence convolutional neural network is trained according to a loss function to obtain the gait prediction model; the loss function is used to indicate the deviation between the prediction probability and the expected prediction result.

[0008] In some embodiments, the evidence convolutional neural network includes four consecutive convolution modules, a feature processing module and an output module; The step of inputting the target training sample into the evidence convolutional neural network and training the evidence convolutional neural network according to a loss function to obtain the gait prediction model includes: In each iterative training process, the target training sample is preprocessed and then sequentially input into each convolution module to extract key features; Inputting the key features into the feature processing module to map the key features to a preset dimensional space and generate preset dimensional features; Inputting the preset dimensional features into an output module to obtain a predicted probability of the preset gait type; According to the predicted probability, a loss between the predicted probability and the true value is calculated in combination with a preset loss function; The gradient is calculated according to the loss, and the parameters of the evidence convolutional neural network are updated through forward propagation and back propagation according to the gradient until the updated evidence convolutional neural network meets the preset cutoff condition, thereby obtaining the gait prediction model.

[0009] In some embodiments, inputting the preset dimensional features into an output module to obtain the predicted probability of the preset gait type includes: Mapping the preset dimensional features to scores of a plurality of the preset gait types; Processing the score with an activation function to obtain a non-negative evidence vector, and parameterizing the Dirichlet distribution probability according to the evidence vector to obtain evidence parameters corresponding to each training sample in the current iterative training; Calculate and obtain total evidence according to the evidence parameter corresponding to each of the training samples; The predicted probability corresponding to the current training sample is determined according to the evidence parameter corresponding to each training sample and the total evidence.

[0010] In some embodiments, the loss function is:

[0011] In the formula, KL is the divergence term, which is used to measure the Dirichlet distribution Uniform Dirichlet distribution The difference between A predicted probability vector representing other gait types other than the preset gait type; is the i-th training sample; is the weight of the KL divergence regularization term; is the actual label of the jth gait type of the i-th training sample; is the predicted probability of the jth gait type for the i-th training sample; K is the total number of preset gait types; for Expected value; for Expected value; is a measure of the difference between two probability distributions; are the training parameters of the gait prediction model; is the actual label of the one-hot encoding of the i-th training sample; is the gait prediction model input under the condition of given gait prediction model parameters θ The quality of the predicted probability.

[0012] In a second aspect, an embodiment of the present application provides a gait prediction method, comprising: Acquire surface electromyographic signals of preset collection points of human lower limbs; The surface electromyography signal is input into the gait prediction model obtained by the above method to obtain a target gait prediction result.

[0013] In a third aspect, an embodiment of the present application provides a gait prediction model acquisition device, comprising: A data enhancement module is used to perform data enhancement on the initial training samples in the closed environment data set according to the out-of-distribution data enhancement method to obtain enhanced training samples; wherein the initial training samples are training samples formed by the surface electromyography signals of preset collection points of the lower limbs of the human body under multiple preset gait types; The model training module is used to input the target training samples into the evidence convolutional neural network for training to obtain a gait prediction model; wherein the target training samples include the initial training samples and the enhanced training samples.

[0014] In a fourth aspect, an embodiment of the present application provides a walking aid device, which includes a processor and a memory, wherein the memory stores a computer program, and the processor is used to execute the computer program to implement the above-mentioned gait prediction method.

[0015] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium storing a computer program, which, when executed on a processor, implements the above-mentioned gait prediction model acquisition method; or implements the above-mentioned gait prediction method.

[0016] The embodiments of the present application have the following beneficial effects: the present application performs out-of-distribution enhancement on training samples during model training by using out-of-distribution data enhancement, and improves the loss function, thereby achieving the accuracy of gait prediction in an open environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 A schematic diagram of a gait prediction model scenario in a closed environment paradigm in the prior art is shown; Figure 2 A first flow chart of the method for acquiring a gait prediction model according to an embodiment of the present application is shown; Figure 3a A schematic diagram of a surface electromyography signal acquisition experimental paradigm according to an embodiment of the present application is shown; Figure 3b A schematic diagram of obtaining gait type through a gait prediction model in an embodiment of the present application is shown; Figure 4 A second flow chart of the method for acquiring a gait prediction model according to an embodiment of the present application is shown; Figure 5 A third flow chart of the method for acquiring a gait prediction model according to an embodiment of the present application is shown; Figure 6 A fourth flow chart of the method for acquiring a gait prediction model according to an embodiment of the present application is shown; Figure 7 A comparison chart of experimental results of gait prediction of the present application and gait prediction of the prior art is shown; Figure 8 A schematic diagram of a process flow of a gait prediction method according to an embodiment of the present application is shown; Fig. 9 A structural schematic diagram of a gait prediction model acquisition device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0019] The technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments.

[0020] The components of the embodiments of the present application generally described and shown in the drawings herein may be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0021] Hereinafter, the terms "including", "having" and their cognates that can be used in various embodiments of the present application are intended only to indicate specific features, numbers, steps, operations, elements, components or a combination of the foregoing items, and should not be understood as first excluding the existence of one or more other features, numbers, steps, operations, elements, components or a combination of the foregoing items or increasing the possibility of one or more features, numbers, steps, operations, elements, components or a combination of the foregoing items. In addition, the terms "first", "second", "third" and the like are only used to distinguish descriptions and cannot be understood as indicating or implying relative importance.

[0022] Unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meanings as those generally understood by those skilled in the art to which the various embodiments of the present application belong. The terms (such as those defined in generally used dictionaries) will be interpreted as having the same meanings as the contextual meanings in the relevant technical field and will not be interpreted as having idealized meanings or overly formal meanings unless clearly defined in the various embodiments of the present application.

[0023] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0024] Gait generally refers to movement patterns such as standing, walking on flat ground, climbing stairs, and going uphill. Accurately identifying gait in advance is crucial to helping lower limb exoskeletons provide effective walking assistance. At present, surface electromyography (sEMG) signals have shown great potential in intention recognition and human-computer interaction. Combining sEMG signals and deep learning models has become an effective solution for gait pattern recognition. However, existing gait prediction technologies based on surface electromyography signals only focus on closed environment scenarios with predefined and limited sets of motion patterns. This limits the adaptability to ambiguous or unexpected motion patterns encountered in real-world deployments, and also brings significant risks, which may lead to misunderstanding of user intentions and dangerous situations. For example Figure 1 Taking the example shown in the figure, if only the gait types of standing, walking on flat ground, and climbing stairs are predefined during model training, but the movement type of going down stairs is not predefined, then in actual application, the prediction model will continue to output wrong results when encountering this situation, which may cause the walking exoskeleton to provide inappropriate assistance based on the wrong intention decoding, thereby leading to the risk of falling on the stairs.

[0025] It can be understood that the gait prediction in the prior art is based on the closed world assumption, that is, it does not consider and provide the model's response plan when facing unknown classes, that is, it does not involve the analysis of rejection ability. Based on this, the present application proposes a gait prediction model acquisition method, a gait prediction method, a device and a walking aid, by performing data enhancement on the training samples according to a preset out-of-distribution data enhancement method during the training of the Evidential Convolutional Neural Network (ECNN) to generate out-of-distribution data, and incorporating it into the training process, thereby improving the recognition ability and uncertainty estimation of the Evidential Convolutional Neural Network for unknown data; in addition, the present application adopts a robust Evidential Convolutional Neural Network, which can generate less evidence on wrong predictions, by increasing the ignorance of the wrong category while maintaining confidence in the correct category, and increasing the robustness of the training.

[0026] The method for obtaining the gait prediction model is described below in conjunction with some specific embodiments.

[0027] Figure 2 A schematic flow chart of a method for obtaining a gait prediction model according to an embodiment of the present application is shown. Exemplarily, the method for obtaining a gait prediction model comprises the following steps: Step S100, performing data enhancement on the initial training samples in the closed environment data set according to the out-of-distribution data enhancement method to obtain enhanced training samples.

[0028] The initial training samples are training samples formed by surface electromyographic signals of preset collection points of the lower limbs of the human body under multiple preset gait types.

[0029] The initial training samples in this embodiment are collected based on the closed world paradigm, and can stand, walk on flat ground, cross obstacles, go up stairs and downhill surface electromyographic signals. When the evidence convolutional neural network is trained by the collected initial training samples, a preset number of training samples are randomly generated in each iterative training, so that the selected training samples can be ensured to come from different gait types, and then the training samples of this iteration are enhanced according to the preset out-of-distribution data enhancement method to obtain out-of-distribution data (enhanced training samples), that is, to obtain other gait types other than the preset gait types under the closed time paradigm, that is, to make the training samples more diversified, and obtain other gait data between any two gait types, thereby improving the recognition ability and uncertainty estimation of the evidence convolutional neural network for unknown data. The evidence convolutional neural network in this embodiment is obtained by a convolutional neural network set evidence deep learning network, and its evidence convolutional network includes four consecutive convolution modules, a feature processing module and an output module. The evidence convolutional neural network in this embodiment is obtained by improving the convolutional neural network, and the improvements include but are not limited to improvements in the layer structure, improvements in the loss function, improvements in the activation function, etc., which can improve the robustness of the convolutional neural network and make the evidence convolutional neural network generate less evidence for incorrect predictions, by increasing the ignorance of the wrong category while maintaining confidence in the correct category.

[0030] In this embodiment, when obtaining the initial training samples, several gait types are first preset, such as Figure 3a As shown, this embodiment is carried out in a laboratory environment simulating real-world conditions, using a scene containing stairs, platforms and ramps, and setting up some artificially placed obstacles. The training uses a forward path to simulate a closed environment scene, and a backward path to simulate an open environment scene. The closed environment dataset is used for model training, verification and testing, including five types of movements: standing (ST), crossing obstacles (SO), walking on flat ground (LW), climbing stairs (SA) and descending a slope (RD). It can be understood that Figure 3a The above are merely exemplary, and the preset gait types in the closed world paradigm may not only be limited to standing, crossing obstacles, walking on flat ground, climbing stairs and descending slopes, but may also include other gait types.

[0031] When training the evidence convolutional neural network, the initial training samples used are Figure 3aThe surface electromyographic signals of the subject's legs and walking-related muscles were collected in the manner and position shown. The signal collection complies with the standard for collecting surface electromyographic signals in SENIAM (Surface ElectroMyoGraphy for the Non-Invasive Assessment of Muscles). Bipolar active Ag-AgCl electrodes were used for placement. The data was recorded at a sampling rate of 1200 Hz. In order to prevent power line interference with the collected signals, a 50 Hz notch filter was applied in this embodiment to reduce power line interference.

[0032] Among them, the collected surface electromyographic signals include the biceps femoris of the right leg ( Figure 3a The muscle part of middle 1), semitendinosus ( Figure 3a The muscle part of middle 2), gastrocnemius lateralis ( Figure 3a The muscle part of middle 3), medial gastrocnemius ( Figure 3a The muscle part of the middle 4), the vastus lateralis ( Figure 3a The muscle part of middle 5), rectus femoris ( Figure 3a The muscle part of middle 6), vastus medialis ( Figure 3a 7) and the tibialis anterior ( Figure 3a When collecting the surface electromyographic signals of muscles, bipolar active Ag-AgCl electrodes are placed at the corresponding positions of the human body for collection, such as Figure 3a Located within the oval box of the mid-leg.

[0033] In some embodiments, Figure 4 As shown, step S100 includes: Step S110: Before each iterative training, a preset number of initial training samples are randomly selected from the closed environment data set. Each initial training sample includes sample data and sample labels; the sample data refers to the specific content of the sample, that is, the feature vector; and the sample label refers to the gait type that can be output based on the sample data.

[0034] In this step, the collected closed environment data set will be divided into training set, validation set and test set according to a certain ratio before training the evidence convolutional neural network. In each iterative training, a preset number of initial training samples are randomly selected from the training set, which can largely ensure that the training samples used in this iterative training include samples of different gait types, so that more out-of-distribution data can be obtained after data enhancement of the training samples by linear interpolation.

[0035] Step S120: Process the initial training samples by linear interpolation to obtain corresponding enhanced training samples.

[0036] Exemplarily, the training samples in the enhanced training samples are obtained by a preset linear interpolation formula; Among them, the preset linear interpolation formula is: ; in, and are two different initial training samples; For Beta( ) distribution to obtain the interpolation ratio, is the default value.

[0037] It can be understood that the sample data and sample labels in each initial training sample are calculated by the above preset linear interpolation, where: Can be set to 10.

[0038] In this example, Set it to 10, which makes The concentration is around 0.5, which means that the generated enhanced training samples can more evenly combine samples of two different gait types, thereby making the training samples (initial training samples + enhanced training samples) more diverse, helping the evidence convolutional neural network of this embodiment to better understand the boundaries between gait types. In addition, since the newly generated sample labels are obtained by random interpolation, these training samples are essentially high-uncertainty training samples. This embodiment can improve the generalization ability of the evidence convolutional neural network through this random interpolation method. By introducing training samples that are different from the original training distribution during the training process, the convolutional neural network can learn a wider range of features, thereby enhancing the recognition ability of samples outside the distribution.

[0039] Step S200: input the target training sample into the evidential convolutional neural network for training to obtain a gait prediction model.

[0040] The target training samples include initial training samples and enhanced training samples.

[0041] The evidence convolutional neural network used in this embodiment includes four consecutive convolution modules, a feature processing module and an output module.

[0042] like Figure 3b As shown, the evidence convolutional neural network of this embodiment includes four consecutive convolution modules, each of which includes a convolution layer, a batch normalization layer, a ReLU activation function layer, a Dropout layer and a maximum pooling layer.

[0043] Among them, the first convolution layer is equipped with 8 output channels, and the subsequent three convolution layers are equipped with 16 output channels. The four convolution layers all use a convolution kernel size of 1×3. Each convolution layer is connected to a batch normalization layer, a ReLU activation function layer, a Dropout layer, and a maximum pooling layer; among them, the batch normalization layer is used to standardize the input of each convolution layer to accelerate the training process; the ReLU activation function layer is used to introduce nonlinear data to enhance the expression ability of the convolutional neural network of the present embodiment; the Dropout layer is used to randomly discard some neurons to prevent overfitting; the maximum pooling layer uses a 1×4 pooling window to reduce the feature dimension of the training sample and extract key features.

[0044] The feature processing module includes a flatten layer, a fully connected layer, and a ReLU activation and Dropout layer. The flatten layer is used to convert the output data of the convolution layer into a one-dimensional vector feature after the last convolution layer and pooling; the fully connected layer is used to map the features input from the flatten layer to a 128-dimensional space; the ReLU activation and Dropout layers are used to further enhance the nonlinear expression and regularization of the features.

[0045] The output module includes an evidence vector layer and a Dirichlet distribution layer. The evidence vector layer is used to map the 128-dimensional features obtained from the feature processing module to the scores of the preset five gait types, and then the scores are processed by the activation function ReLU to obtain a non-negative evidence vector. The Dirichlet distribution layer is used to use the mathematical modeling of evidence deep learning to convert the evidence vector into a Dirichlet distribution probability, and based on this, calculate the predicted probability of each gait type corresponding to the current training sample.

[0046] In some embodiments, the target training sample is input into the evidence convolutional neural network, and the evidence convolutional neural network is trained according to a loss function to obtain a gait prediction model, wherein the loss function is used to indicate the deviation between the predicted probability and the expected prediction result.

[0047] In some embodiments, Figure 5 As shown, the target training sample is input into the evidence convolutional neural network, and the evidence convolutional neural network is trained according to the loss function to obtain a gait prediction model, including steps S210 to S250: Step S210: During each iterative training process, the target training sample is preprocessed and then input into each convolution module in sequence to extract key features.

[0048] Before training the evidence convolutional neural network of this embodiment, the weights of each module are first initialized, that is, by traversing all modules and / or layers in the gait prediction model, for each convolutional layer (nn.Conv2d) and linear layer (nn.Linear), the weights are initialized using Kaiming normal distribution (kaiming_normal_). If there is a bias term in the layer, the data of the bias term is initialized to zero. This allows the evidence convolutional neural network to converge better and improve the training effect and training stability.

[0049] In this step, combine Figure 3b Preprocess the training samples used in the current iterative training. Since the collected surface electromyographic signals are filtered once during the collection, the 50HZ notch filter used for filtering is mainly to reduce power line interference. When the evidence convolutional neural network is subsequently trained, the training samples used are all those that have passed the preliminary filtering. During training, the training samples are preprocessed by first filtering them with an 8th-order Butterworth filter (with a cutoff frequency of 20-500 Hz). After filtering, all negative values ​​are rectified by taking their absolute values, and then the sEMG data of all channels are normalized using the mean and standard deviation of each channel.

[0050] In this step, after preprocessing the training samples, the training samples are input into the convolution layer using a sliding window method, where the window size of the sliding window is 1000 milliseconds, the step length is 50 milliseconds, and the prediction time domain is 50 milliseconds (i.e., the evidence convolutional neural network is used to predict the gait state within the next 50 milliseconds).

[0051] After the preprocessed training samples are input into the first convolution module, the second convolution module, the third convolution module and the fourth convolution module in sequence, the key features are obtained after the training samples are processed by each convolution module, where the key features are high-dimensional features related to the predicted gait.

[0052] Step S220, inputting the key features into a feature processing module to map the key features to a preset dimensional space and generate preset dimensional features.

[0053] In this step, the key features obtained in step S210 are sequentially input into the flattening layer, the fully connected layer, the ReLU activation layer, and the Dropout layer, and a 128-dimensional spatial feature is obtained after being processed in each layer.

[0054] Step S230: input the preset dimensional features into the output module to obtain the predicted probability of the preset gait type.

[0055] For example, Figure 6As shown, step S230 includes: Step S231, mapping the preset dimensional features to scores of a plurality of preset gait types.

[0056] Step S232, the scores are processed by an activation function to obtain a non-negative evidence vector, and the Dirichlet distribution probability is parameterized according to the evidence vector to obtain the evidence parameter corresponding to each training sample in the current iterative training.

[0057] Step S233, calculating the total evidence according to the evidence parameters corresponding to each training sample.

[0058] Step S234, determining the prediction probability corresponding to the current training sample according to the evidence parameters and the total evidence corresponding to each training sample.

[0059] Exemplarily, after the preset dimensional features are input into the output module, the output module first maps the preset dimensional information into multiple numerical values, which are scores of each preset gait type, and inputs the mapped scores of multiple preset gait types into the evidence vector layer, and after being processed by the activation function ReLU of the evidence vector layer, a non-negative evidence vector is obtained. In this embodiment, by using the activation function ReLU instead of the Softmax in the prior art, direct generation of the evidence vector can be achieved.

[0060] The Dirichlet distribution probability is parameterized according to the evidence vector to obtain the evidence parameters corresponding to each training sample in the current iterative training. Then the total evidence is calculated based on the evidence parameters corresponding to each training sample; finally, the prediction probability corresponding to the current training sample is determined based on the evidence parameters corresponding to each training sample and the total evidence.

[0061] Exemplarily, the evidence vector of each training sample is obtained through the above steps After that, It is based on the training samples ( ) obtained, ( ) represents the i-th training sample, where is a one-hot encoded vector representing the true gait type.

[0062] Among them, the formula for parameterizing the Dirichlet distribution probability according to the evidence vector to obtain the evidence parameter corresponding to each training sample in the current iterative training is: .

[0063] The total evidence is calculated as: .

[0064] The formula for calculating the predicted probability is: = .

[0065] In the above formula, is the evidence vector of the i-th training sample; is the evidence parameter corresponding to the i-th training sample; S is the total evidence; is the predicted probability corresponding to the i-th training sample.

[0066] Step S240, calculating the loss between the predicted probability and the true value based on the predicted probability and a preset loss function.

[0067] Among them, the loss function is:

[0068] In the formula, KL is the divergence term, which is used to measure the Dirichlet distribution Uniform Dirichlet distribution The difference between A predicted probability vector representing other gait types other than the preset gait type; is the i-th training sample; is the weight of the KL divergence regularization term; is the actual label of the jth gait type of the i-th training sample; is the predicted probability of the jth gait type for the i-th training sample; K is the total number of preset gait types; for Expected value; for Expected value; is a measure of the difference between two probability distributions; are the training parameters of the gait prediction model; is the actual label of the one-hot encoding of the i-th training sample; is the gait prediction model input under the condition of given gait prediction model parameters θ The predicted probability mass of ; where λ=0.01.

[0069] This loss function takes into account unknown gait types (i.e., gait types other than the true gait type j), which allows the evidence convolutional neural network to produce less evidence on incorrect predictions by increasing ignorance of the wrong types while maintaining confidence in the correct types.

[0070] In order to control the weight of the KL divergence term in the loss function, It is used as a key regularization parameter. In traditional ECNN, λ=min(1.0,t / 10) is an annealing coefficient that increases with the increase of training round t and remains at 1 after t reaches 10. By starting with a smaller λ and gradually increasing it, ECNN can first focus on learning the main task (minimizing prediction error) and then gradually introduce regularization to manage uncertainty.

[0071] In this embodiment, because out-of-distribution data is added through out-of-distribution data enhancement technology, that is, a large number of high-uncertainty training samples are introduced when training the evidence convolutional neural network, in order to ensure more robust training, the choice of λ in this embodiment is fixed to 0.01. A smaller λ value helps the model remain stable when processing out-of-distribution data, allowing the evidence convolutional neural network to handle a certain degree of uncertainty without significantly changing the learned feature representation. That is, this embodiment can effectively improve the performance of the evidence convolutional neural network in predicting lower limb gait in an open environment in this way, while improving the recognition ability of unknown data and the accuracy of uncertainty estimation.

[0072] Step S250, calculate the gradient according to the loss, and update the parameters of the evidence convolutional neural network through forward propagation and back propagation according to the gradient until the updated evidence convolutional neural network meets the preset cutoff condition, thereby obtaining a gait prediction model.

[0073] The parameters of the convolutional neural network modified according to the gradient include but are not limited to batch size (i.e. the number of training samples used in each iterative training, learning rate, activation function, regularization parameter, etc.).

[0074] During each training iteration, the evidence convolutional neural network forward propagates and backward propagates through the training samples, and uses the validation set in the closed environment dataset to evaluate the performance of the pre-trained gait prediction model and monitor overfitting. After the model is trained, the test set in the closed environment dataset can also be used to test whether the gait prediction model is accurate for gait prediction in the closed environment paradigm.

[0075] In some embodiments, when training the gait prediction model, an early stopping method is used to balance the comprehensiveness and efficiency of the training, and the stopping condition is that the performance of the validation set does not improve within 20 consecutive cycles, or the training is stopped when the maximum number of training cycles is reached. In addition, the Adam optimizer is used in this embodiment, the weight decay value is set to 0.0004, the exponential decay rate of the first-order moment estimate of Adam is 0.5, and the exponential decay rate of the second-order moment estimate is 0.99.

[0076] The training of the gait prediction model in this embodiment enables the gait prediction model to effectively extract key features from the input training samples, and at the same time, the robustness and uncertainty estimation capabilities of unknown data can be enhanced through the out-of-distribution data augmentation method and the loss function used in model training. The setting of the sliding window and the prediction time domain ensures that the model can capture the dynamic characteristics of time series data, while the configuration of the optimizer and the early stopping strategy help the model converge to the optimal solution while avoiding overfitting.

[0077] In addition, in order to verify the prediction effect of the gait prediction model in this embodiment for the open environment, this embodiment collects the surface electromyography signals of the gait in the open environment as the open environment data set. The collection conditions of the surface electromyography signals in the open environment are the same as those of the closed environment data set, except that Figure 3a As shown, this embodiment uses a forward path to simulate a closed environment scene, and a backward path to simulate an open environment scene, that is, the data collected by the forward path is used as a closed environment data set, and the data collected by the backward path is an open environment data set. The open environment data set is only used to test the performance of the gait prediction model of this embodiment in predicting open environment movement. The gait types in the open environment include known categories (including but not limited to standing, walking on flat ground, crossing obstacles, going up stairs and downhill, etc.) and unknown categories (going up slopes, going down stairs, etc.).

[0078] Figure 7 The experimental results in an open environment are shown. Figure 7 It can be seen from the experimental results shown that the gait prediction model in this embodiment has achieved substantial improvements in prediction accuracy and failure detection capabilities (including misclassified samples of known types and samples of unknown types); specifically, in an open environment, the gait prediction model of this embodiment achieved a prediction accuracy rate (ACC) of 79.53%, which is 1.27% higher than the pure CNN (convolutional neural network model) in the prior art (p<0.05), and its prediction accuracy has also been improved compared to the ECNN using the CNN loss function. More importantly, in terms of fault detection, the gait prediction model of this embodiment reduces the AURC (area under the confidence risk curve) to 217.72‰, which is 26.66% lower than the CNN baseline (from 296.88‰ to 217.73‰, p≪ 0.05). This substantial improvement fully reflects the outstanding effectiveness of the present invention in improving model reliability. It should be noted that Figure 7The CNN row represents a traditional convolutional neural network, and CNN+OODAug* represents the data enhancement technology of this embodiment adopted on the basis of the traditional convolutional neural network; the ECNN* row represents a convolutional neural network that only adopts the model structure of this embodiment but does not adopt the loss function in this embodiment, and only adopts the loss function in CNN, and does not adopt the data enhancement technology of this embodiment; ECNN*+ OODAug* represents the model structure of this embodiment but does not adopt the loss function in this embodiment, and adopts the data enhancement technology of this embodiment; the gait prediction model* row represents the model structure of this embodiment and adopts the loss function in this embodiment but does not adopt the data enhancement technology of this embodiment, and the gait prediction model*+ OODAug* represents the model structure of this embodiment and adopts the loss function in this embodiment and adopts the data enhancement technology of this embodiment.

[0079] Further analysis of the ablation experimental results reveals the contribution of the out-of-distribution data augmentation technology combined with the model in the gait prediction model of this embodiment. Among them, the introduction of the out-of-distribution data augmentation technology (OODAug) brings about comprehensive performance improvement. For CNN+OODAug (out-of-distribution data augmentation technology), it increases the AUROC of fault detection from 70.42% to 78.08%, while reducing the FPR95 (false positive rate 95%) from 82.07% to 69.05%, and the prediction accuracy is also increased from 78.26% to 79.23%. For ECNN+OODAug (out-of-distribution data augmentation technology), it increases the AUROC of fault detection from 70.08% to 79.55%, while reducing the FPR95 (false positive rate 95%) from 78.88% to 63.64%, and the prediction accuracy is also increased from 77.05% to 77.98%. For the gait prediction model + OODAug (out-of-distribution data enhancement technology) of this embodiment, where the gait prediction model * represents the experimental data of the gait prediction model of this embodiment without the out-of-distribution data enhancement technology, and the last line represents the experimental data of the gait prediction model of this embodiment with the out-of-distribution data enhancement technology, it can be concluded that the AUROC of fault detection with the out-of-distribution data enhancement technology has been improved, the FPR95 has been reduced, and the prediction accuracy has been improved. These data clearly show that the out-of-distribution data enhancement technology (OODAug) plays a key role in enhancing the robustness of the model to unknown data. On the other hand, the learning network that adopts the model structure of this embodiment and the loss function of this embodiment also shows significant advantages over the learning network that only adopts the model structure of this embodiment, reducing the AURC from 290.13‰ to 264.56‰, and at the same time increasing the prediction accuracy from 77.05% to 78.87%. This improvement highlights the superiority of the learning network that adopts the model structure of this embodiment and the loss function of this embodiment in balancing prediction accuracy and uncertainty estimation. In general, these results strongly demonstrate the significant technical effect of the gait prediction model of this embodiment in improving the accuracy, reliability and adaptability of the sEMG-based lower limb gait prediction system, especially in the face of open environments and unknown situations, laying a solid foundation for the promotion of this technology in practical applications.

[0080] This application uses out-of-distribution data augmentation to perform out-of-distribution augmentation on training samples during model training, and improves the loss function to achieve the accuracy of gait prediction in an open environment.

[0081] Figure 8 A schematic flow chart of a gait prediction method according to an embodiment of the present application is shown. Exemplarily, the gait prediction method includes: Step S10, acquiring surface electromyographic signals of preset collection points of the lower limbs of the human body.

[0082] In this embodiment, in order to predict gait, it is necessary to collect surface electromyographic signals of muscles of the human leg related to walking. The method of collecting surface electromyographic signals is the same as that in the above embodiment, which will not be repeated here. Step S20: input the surface electromyography signal into the gait prediction model obtained by the gait prediction model acquisition method to obtain a target gait prediction result.

[0083] The gait prediction model acquisition method used in the gait prediction method of the present application performs out-of-distribution enhancement on training samples during model training by adopting out-of-distribution data enhancement, and improves the loss function, thereby achieving the accuracy of gait prediction in an open environment.

[0084] Fig. 9 A schematic diagram of the structure of a gait prediction model acquisition device according to an embodiment of the present application is shown. Exemplarily, the gait prediction model acquisition device includes: The data enhancement module 100 is used to perform data enhancement on the initial training samples in the closed environment data set according to the out-of-distribution data enhancement method to obtain enhanced training samples. The initial training samples are training samples formed by surface electromyographic signals of preset collection points of the lower limbs of the human body under multiple preset gait types.

[0085] The model training module 200 is used to input the target training sample into the evidence convolutional neural network for training to obtain a gait prediction model.

[0086] The target training samples include initial training samples and enhanced training samples.

[0087] It can be understood that the device of this embodiment corresponds to the gait prediction method of the above embodiment, and the options in the above embodiment are also applicable to this embodiment, so they will not be described repeatedly here.

[0088] The present application also provides a walking aid device, which exemplarily includes a processor and a memory, wherein the memory stores a computer program, and the processor runs the computer program to enable the walking aid device to perform the functions of each module in the above-mentioned gait prediction method or the above-mentioned gait prediction device. The walking aid device is a walking aid rehabilitation device, which decodes the user's intention and effectively helps the user improve and enhance walking ability.

[0089] Among them, the processor can be an integrated circuit chip with signal processing capabilities. The processor can be a general-purpose processor, including a central processing unit (CPU), a graphics processing unit (GPU), a network processor (NP), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or at least one of other programmable logic devices, discrete gates or transistor logic devices, and discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc., which can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of the present application.

[0090] The memory may be, but is not limited to, a random access memory (RAM), a read only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electric erasable programmable read-only memory (EEPROM), etc. The memory is used to store a computer program, and the processor may execute the computer program accordingly after receiving an execution instruction.

[0091] The present application also provides a computer-readable storage medium for storing the computer program used in the above-mentioned walking aid device. For example, the computer-readable storage medium may include but is not limited to: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0092] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can also be implemented in other ways. The device embodiments described above are merely schematic. For example, the flowcharts and structure diagrams in the accompanying drawings show the possible architecture, functions and operations of the devices, methods and computer program products according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in an alternative implementation, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the structure diagram and / or the flow diagram, and the combination of boxes in the structure diagram and / or the flow diagram, can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0093] In addition, the functional modules or units in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0094] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a smart phone, a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application.

[0095] The above description is only a specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application.

Claims

1. A method for obtaining a gait prediction model, characterized in that: include: Performing data enhancement on the initial training samples in the closed environment data set according to the out-of-distribution data enhancement method to obtain enhanced training samples; wherein the initial training samples are training samples formed by surface electromyographic signals of preset collection points of human lower limbs under multiple preset gait types; The target training samples are input into the evidence convolutional neural network for training to obtain a gait prediction model; wherein the target training samples include the initial training samples and the enhanced training samples.

2. The method for acquiring a gait prediction model according to claim 1, characterized in that: The method of performing data enhancement on the initial training samples in the closed environment data set according to the out-of-distribution data enhancement method to obtain enhanced training samples includes: Before each iterative training, a preset number of the initial training samples are randomly selected from the closed environment data set; The initial training samples are processed by linear interpolation to obtain the corresponding enhanced training samples.

3. The gait prediction model acquisition method according to claim 2, characterized in that: The training samples in the enhanced training samples are obtained by a preset linear interpolation formula; Wherein, the preset linear interpolation formula is: ; in, and are two different initial training samples; For Beta( ) distribution to obtain the interpolation ratio, is the default value.

4. The method for acquiring a gait prediction model according to claim 1, characterized in that: The step of inputting the target training sample into the evidence convolutional neural network for training to obtain a gait prediction model includes: The target training sample is input into the evidence convolutional neural network, and the evidence convolutional neural network is trained according to a loss function to obtain the gait prediction model; the loss function is used to indicate the deviation between the prediction probability and the expected prediction result.

5. The method for acquiring a gait prediction model according to claim 4, characterized in that: The evidence convolutional neural network includes four consecutive convolution modules, a feature processing module and an output module; The step of inputting the target training sample into the evidence convolutional neural network and training the evidence convolutional neural network according to a loss function to obtain the gait prediction model includes: In each iterative training process, the target training sample is preprocessed and then sequentially input into each convolution module to extract key features; Inputting the key features into the feature processing module to map the key features to a preset dimensional space and generate preset dimensional features; Inputting the preset dimensional features into an output module to obtain a predicted probability of the preset gait type; According to the predicted probability, a loss between the predicted probability and the true value is calculated in combination with a preset loss function; The gradient is calculated according to the loss, and the parameters of the evidence convolutional neural network are updated through forward propagation and back propagation according to the gradient until the updated evidence convolutional neural network meets the preset cutoff condition, thereby obtaining the gait prediction model.

6. The method for acquiring a gait prediction model according to claim 5, characterized in that: The step of inputting the preset dimensional features into an output module to obtain a predicted probability of the preset gait type includes: Mapping the preset dimensional features to scores of a plurality of the preset gait types; The scores are processed by an activation function to obtain a non-negative evidence vector, and the Dirichlet distribution probability is parameterized according to the evidence vector to obtain the evidence parameter corresponding to each training sample in the current iterative training; Calculate and obtain total evidence according to the evidence parameter corresponding to each of the training samples; The predicted probability corresponding to the current training sample is determined according to the evidence parameter corresponding to each training sample and the total evidence.

7. The method for acquiring a gait prediction model according to claim 6, characterized in that: The loss function is: In the formula, KL is the divergence term, which is used to measure the Dirichlet distribution Uniform Dirichlet distribution The difference between A predicted probability vector representing other gait types other than the preset gait type; is the i-th training sample; is the weight of the KL divergence regularization term; is the actual label of the jth gait type of the i-th training sample; is the predicted probability of the jth gait type for the i-th training sample; K is the total number of preset gait types; for Expected value; for Expected value; is a measure of the difference between two probability distributions; are the training parameters of the gait prediction model; is the actual label of the one-hot encoding of the i-th training sample; is the gait prediction model input under the condition of given gait prediction model parameters θ The quality of the predicted probability.

8. A gait prediction method, characterized in that: include: Acquire surface electromyographic signals of preset collection points of human lower limbs; The surface electromyography signal is input into a gait prediction model obtained by the method according to any one of claims 1 to 7 to obtain a target gait prediction result.

9. A gait prediction model acquisition device, characterized in that: include: A data enhancement module is used to perform data enhancement on the initial training samples in the closed environment data set according to the out-of-distribution data enhancement method to obtain enhanced training samples; wherein the initial training samples are training samples formed by surface electromyographic signals of preset collection points of the lower limbs of the human body under multiple preset gait types; The model training module is used to input the target training samples into the evidence convolutional neural network for training to obtain a gait prediction model; wherein the target training samples include the initial training samples and the enhanced training samples.

10. A walking aid, characterized in that: The walking aid device comprises a processor and a memory, wherein the memory stores a computer program, and the processor is configured to execute the computer program to implement the gait prediction method according to claim 8.

Images