Gait phase prediction method and system based on sequential sequence and LSTM-CNN

Abstract

The invention provides a gait phase prediction method and system based on a sequential sequence and an LSTM-CNN, and relates to the technical field of gait prediction. Comprising the steps of obtaining gait time sequence data of a to-be-predicted person, and performing preprocessing; inputting the preprocessed gait time sequence data into an LSTM-CNN network model, extracting local features of the gait time sequence data by using a convolutional layer, and inputting the extracted local features into an LSTM layer to capture time dependence to obtain a prediction result of a gait cycle percentage; the LSTM-CNN is obtained through pre-training, and pre-training data comprises gait time sequence data of a plurality of gait cycles and actual gait cycle percentage tags corresponding to the gait time sequence data of the plurality of gait cycles. According to the method, the gait phase and the gait cycle percentage of the old and weak people are predicted through the long-short-term memory network and the convolutional neural network, the percentage of each stage can be accurately predicted in real time, and the prediction of the gait cycle is more accurate.

Description

Technical Field

[0001] The present invention belongs to the technical field of gait prediction, and in particular relates to a gait phase prediction method and system based on time series and LSTM-CNN. Background Art

[0002] With the increase in the elderly population, the importance of gait analysis is becoming increasingly apparent. Accurate prediction of the gait cycle is not only important for gait training and rehabilitation, but also can provide data support for personalized health management. Although a variety of gait analysis systems have appeared on the market, these systems still have several limitations in practical applications.

[0003] First, existing systems perform poorly in adapting to the different age groups and physical conditions of the elderly population. The elderly and young people have significant differences in gait and physiological characteristics, and existing gait analysis systems often fail to distinguish these key differences.

[0004] The limitations of traditional gait analysis methods cannot be ignored. These methods usually rely on laboratory equipment, such as force plates and computerized walking track systems, to collect gait data. These devices are not only inconvenient and expensive to use, but may not provide real-time monitoring and feedback in practical applications. In particular, traditional inertial measurement unit devices have limitations in measuring key parameters such as step width and step length, which are crucial for gait cycle prediction.

[0005] Adaptive frequency oscillator (AFO) is a gait cycle prediction method commonly used in gait assist devices, which mainly relies on traditional feature extraction techniques and usually only predicts the stand phase and swing phase. Although AFO synchronizes with the user's gait cycle by adjusting its internal parameters and uses sensor feedback to detect gait events such as heel strike (HS) and take-off (TO), it requires a certain transition time to resynchronize in the face of sudden gait changes, which can lead to prediction delays and inaccuracies.

[0006] Furthermore, traditional methods fail to fully utilize complex gait data, thus limiting their predictive power and adaptability. Summary of the invention

[0007] In order to overcome the shortcomings of the above-mentioned prior art, the present invention provides a gait phase prediction method and system based on time series and LSTM-CNN, which uses an inertial measurement unit installed on the foot to obtain a time series to collect gait data, and predicts the gait phase and gait cycle percentage of the elderly and frail people through a long short-term memory network and a convolutional neural network. It can predict the gait phase in real time and accurately, and also innovatively predict the percentage of each stage, making the prediction of the gait cycle more accurate.

[0008] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions:

[0009] The first aspect of the present invention provides a gait phase prediction method based on time series and LSTM-CNN.

[0010] The gait phase prediction method based on time series and LSTM-CNN includes the following steps:

[0011] Obtain the gait time series data of the person to be predicted and perform preprocessing;

[0012] The preprocessed gait timing data is input into the LSTM-CNN network model, and the local features of the gait timing data are extracted using the convolutional layer. The extracted local features are then input into the LSTM layer to capture the time dependency, and the prediction results of the gait cycle percentage are obtained.

[0013] The LSTM-CNN is obtained through pre-training, wherein the pre-training data includes gait timing data of multiple gait cycles and actual gait cycle percentage labels corresponding to the gait timing data of multiple gait cycles.

[0014] The second aspect of the present invention provides a gait phase prediction system based on time series and LSTM-CNN.

[0015] The gait phase prediction system based on time series and LSTM-CNN includes:

[0016] The data acquisition module is configured to: acquire the gait time series data of the person to be predicted and perform preprocessing;

[0017] The gait cycle percentage prediction module is configured to: input the preprocessed gait time series data into the LSTM-CNN network model, extract the local features of the gait time series data using the convolution layer, and input the extracted local features into the LSTM layer to capture the time dependency, so as to obtain the prediction result of the gait cycle percentage;

[0018] The LSTM-CNN is obtained through pre-training, wherein the pre-training data includes gait timing data of multiple gait cycles and actual gait cycle percentage labels corresponding to the gait timing data of multiple gait cycles.

[0019] The third aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps in the gait phase prediction method based on time series and LSTM-CNN as described in the first aspect of the present invention.

[0020] The fourth aspect of the present invention provides an electronic device, comprising a memory, a processor, and a program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the gait phase prediction method based on time series and LSTM-CNN as described in the first aspect of the present invention are implemented.

[0021] One or more of the above technical solutions have the following beneficial effects:

[0022] The present invention provides a gait phase prediction method and system based on time series and LSTM-CNN, which collects gait data through an inertial measurement unit (IMU) installed on the foot, and uses a combination of long short-term memory network and convolutional neural network for analysis and prediction. Different from the traditional AFO method, this method can not only predict the gait phase in real time and accurately, but also innovatively predict the percentage of each stage, making the prediction of the gait cycle more accurate.

[0023] The present invention uses an end-to-end LSTM-CNN model to learn features directly from raw gait data, avoiding information loss in the traditional feature selection process and achieving real-time data processing without a transition period, thereby providing a faster prediction response. In addition, the model can capture complex patterns in gait data, has high adaptability and generalization capabilities, and meets the needs of gait training and rehabilitation under different environments and physiological conditions. Through this innovative method, the present invention improves the accuracy of gait analysis and provides effective support for gait health monitoring.

[0024] In the gait cycle percentage prediction model of the present invention, we have adopted an innovative optimization strategy. We first adopted an innovative data preprocessing method to enhance the generalization ability of the model. Specifically, we select data from multiple consecutive moments from the data window, and determine the actual gait cycle percentage corresponding to the data of the next moment of the last moment in these multiple consecutive moments as the training label. In the process of forming the input vector, one of the innovations of the present invention is to randomly shuffle the data of multiple consecutive moments and the training labels and then combine them. This randomization method performs data enhancement, which helps the model learn more generalized feature representations, thereby improving the prediction performance of the model under different data distributions. Furthermore, these randomized input vectors are input into the LSTM-CNN model for training, and the resulting model shows significant advantages in prediction accuracy.

[0025] Secondly, in the model training phase, in order to solve the common gradient vanishing problem in deep network training, we introduced residual connection technology, which allows the signal in the network to bypass one or more layers and pass directly to the subsequent layers, effectively enhancing the learning and generalization capabilities of the model. Specifically, we strategically integrate the residual connection between the LSTM layer and the convolution layer, and form a residual connection by adding the local feature vector output by the convolution layer to the output of the LSTM layer. This design not only reduces the loss of information in the network, but also speeds up the training speed and improves the convergence of the model.

[0026] In addition, we applied layer normalization technology in the model to normalize the output of the LSTM layer and the convolution layer to ensure the consistency of the data input distribution of different batches. Layer normalization helps to reduce "internal covariate shift", stabilize the learning process, and speed up the convergence of the model.

[0027] In order to balance the computing speed and model accuracy, we also adopted a mixed precision training method. During the forward propagation of the model, we use half-precision floating point numbers to store and calculate intermediate results to reduce memory usage and speed up calculations. In the back propagation and parameter update phases, we switch back to single precision to maintain sufficient numerical stability and model accuracy. This mixed precision strategy enables the model to converge quickly during training while maintaining the accuracy of the prediction results. Through this innovative data preprocessing and training method, the present invention can effectively improve the model's ability to predict the gait cycle, providing a new technical solution for gait analysis and related application fields.

[0028] In the model evaluation phase, we adopted a new loss function. Since the LSTM layer considers past historical standards, evaluates the current unknown pattern through self-adjustment, and makes future predictions in advance, this patent adopts an enhanced loss function, namely the improved loss function (ILF), and uses statistical evaluation parameters to verify the obtained ILF-LSTM prediction value. Compared with traditional loss functions such as MAE, ILF can better evaluate the prediction performance of the model.

[0029] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0031] Figure 1 This is a flow chart of the method of the first embodiment.

[0032] Figure 2 This is a model structure diagram of the first embodiment. DETAILED DESCRIPTION

[0033] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0034] It should be noted that the terms used herein are for describing specific embodiments only and are not intended to be limiting of exemplary embodiments according to the present invention.

[0035] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0036] Embodiment 1

[0037] The gait cycle percentage refers to the proportion of a specific gait phase in a continuous gait cycle. The gait phase prediction method based on time series and LSTM-CNN disclosed in this embodiment can be used to predict the gait cycle percentage of elderly frail people. The gait data is collected by using an inertial measurement unit installed on the foot, and the long short-term memory network and convolutional neural network are combined for data analysis to achieve accurate prediction, thereby providing a new means for the gait health of the elderly, avoiding the dependence of traditional gait analysis on specific environments and equipment, and improving the accessibility and practicality of monitoring.

[0038] During implementation, the gait data of the elderly is first obtained under natural walking conditions through IMU devices, and preprocessed to ensure the accuracy and reliability of the data, including synchronizing sensor readings, excluding data points in the static period, and removing outliers. Then, the gait data of consecutive moments are extracted from the data window and paired with the actual gait cycle percentage of the next moment as labels. These data and labels are randomly shuffled to enhance the data and help the model capture a wider range of feature representations. Subsequently, these randomized input vectors are used to train the LSTM-CNN model, which consists of an input layer, a bidirectional LSTM layer, a convolutional layer, and a fully connected layer. It can effectively capture long-term dependencies and local features in the time series, thereby improving the prediction accuracy of the gait cycle percentage.

[0039] In order to improve the performance of the model, a series of optimization techniques are adopted: data enhancement is used to improve the robustness of the model, residual connections and layer normalization are used to accelerate training and information flow, and the added residual connections allow the signal in the network to bypass one or more layers and pass directly to the subsequent layers, effectively solving the gradient vanishing problem in deep network training, thereby enhancing the learning ability and generalization ability of the model. In the specific implementation of the model, the residual connection is strategically integrated between the LSTM layer and the convolution layer. Specifically, the local feature vector output by the convolution layer is added to the output of the LSTM layer before being input into the LSTM layer to form a residual connection. In a deep network, information may encounter obstacles when flowing through each layer, resulting in reduced training efficiency. The residual connection bypasses the calculation of some layers by directly connecting the input and output, reducing the loss of information in the network, thereby speeding up the training speed and improving the convergence of the model.

[0040] Layer normalization is applied to each layer of LSTM and CNN in this model. Specifically, layer normalization ensures the consistency of data input distribution of different batches by normalizing the output of the layer, that is, calculating the mean and variance of the activation output of each sample, and scaling and translating it. This operation helps to reduce "internal covariate shift", that is, the problem of changing input distribution at each layer of the network. In the LSTM-CNN model of the present invention, layer normalization is particularly applied to the output of the LSTM layer and the convolution layer to stabilize the learning process and speed up convergence.

[0041] We also use a mixed precision training method that combines single-precision and half-precision floating-point numbers for calculation. In the model training process of the present invention, mixed precision training is used to balance the calculation speed and model accuracy. For example, in the forward propagation process of the model, half-precision floating-point numbers are used to store and calculate intermediate results to reduce memory usage and speed up calculations; in the backward propagation and parameter update phase, it is switched back to single precision to maintain sufficient numerical stability and model accuracy. This strategy enables the model to converge quickly during training while maintaining the accuracy of the prediction results.

[0042] In addition, a learning rate scheduler is used to gradually reduce the learning rate during training to improve convergence. At the same time, the Dropout technique is implemented to prevent overfitting and enhance the generalization ability of the model.

[0043] In the present invention, in order to improve the accuracy and robustness of gait cycle percentage prediction, an ensemble learning method is used to integrate multiple LSTM-CNN models. The key to this method is how to effectively integrate the prediction results of each model, thereby improving the generalization ability and prediction stability of the overall model.

[0044] First, we trained several LSTM-CNN models independently, each using the same network architecture but with different initial weights and random partitioning of the training data to ensure diversity between models. During training, we used early stopping to prevent overfitting, a strategy that dynamically decides when to stop training by monitoring performance on a validation set. Subsequently, these independently trained models were evaluated on the validation set to select the model with the best performance. The performance evaluation not only considered the model's prediction accuracy, but also its stability and generalization ability on different validation sets.

[0045] In the model integration stage, a weighted average method is used to merge the predictions of each model. Unlike simple arithmetic average, weighted average takes into account the performance indicators of each model and gives greater weight to models with higher accuracy. The weight distribution is based on the performance of the model on the validation set, and the statistical method Beta distribution is used to dynamically adjust the weight of each model so that the integrated model can minimize the error overall.

[0046] In addition, to further improve the performance of the ensemble model, we also introduced a meta-learner. This learner uses the output of each base model as a feature, where the base model is the selected multiple LSTM-CNN models. By learning the relationship between these features and the true label, the final prediction result is optimized. This step is equivalent to secondary learning based on the model ensemble, which further improves the accuracy of the prediction.

[0047] Finally, we built an integrated gait cycle percentage prediction system. This system integrates the advantages of multiple LSTM-CNN models and significantly improves the ability to monitor and intervene in the gait health of the elderly through the dual optimization of weighted average and meta-learner.

[0048] like Figure 1 As shown, the gait phase prediction method based on time series and LSTM-CNN includes the following steps:

[0049] Obtain the gait time series data of the person to be predicted and perform preprocessing;

[0050] The preprocessed gait timing data is input into the LSTM-CNN network model, and the local features of the gait timing data are extracted using the convolutional layer. The extracted local features are then input into the LSTM layer to capture the time dependency, and the prediction results of the gait cycle percentage are obtained.

[0051] The LSTM-CNN is obtained through pre-training, wherein the pre-training data includes gait timing data of multiple gait cycles and actual gait cycle percentage labels corresponding to the gait timing data of multiple gait cycles after randomization.

[0052] The technical solution of the embodiment of the present invention has the ability to efficiently process gait data, realize real-time monitoring and accurately predict gait cycles, is suitable for the elderly, and has wide applicability and can adapt to different users and environments, providing strong technical support for gait training and rehabilitation.

[0053] In some embodiments, the gait timing data specifically includes three acceleration components and three angular velocity components of the foot in a gait cycle, and the gait timing data is obtained by an inertial measurement unit installed on the foot of the person.

[0054] In some embodiments, the gait time series data of the person to be predicted is preprocessed, specifically including:

[0055] Exclude stationary period data points;

[0056] Apply statistical rules to eliminate outliers;

[0057] Calculate the gait cycle percentage and synchronize the gait timing data with the gait cycle percentage.

[0058] In some embodiments, the pre-training process of the LSTM-CNN is specifically as follows:

[0059] Constructing gait time series data of multiple gait cycles in the pre-training data into a data window;

[0060] Select data of multiple consecutive moments from the data window, and use the actual gait cycle percentage corresponding to the next moment data of the last moment in the multiple consecutive moments as a training label;

[0061] The data and training labels of multiple consecutive moments are randomly shuffled and combined to form an input vector;

[0062] The input vector is fed into the LSTM-CNN, and the features are forward propagated through the convolutional layer and the LSTM layer to obtain the predicted gait cycle percentage;

[0063] The improved loss function ILF between the predicted gait cycle percentage and the training label is minimized to complete the pre-training process of LSTM-CNN.

[0064] In some embodiments, the LSTM-CNN network model adopts a mixed precision training method, using half-precision floating point numbers to store and calculate intermediate results during the forward propagation process of the model, and using single-precision floating point numbers in the back propagation and parameter update stages.

[0065] In some embodiments, the method further includes constructing an LSTM-CNN network model, wherein the LSTM-CNN network model specifically includes an input layer, a convolutional layer, an LSTM layer, and a fully connected output layer, wherein:

[0066] The input layer is used to receive gait time series data;

[0067] The convolutional layer is used to extract local features of gait time series data;

[0068] The LSTM layer is used to capture the temporal dependency of local features;

[0069] The output layer is used to predict the percentage of gait cycle;

[0070] A residual connection is integrated between the LSTM layer and the convolution layer, and the residual connection is formed by adding a local feature vector output by the convolution layer to an output of the LSTM layer.

[0071] In some embodiments, it also includes integrating multiple LSTM-CNN models using an integrated learning method, specifically including:

[0072] Independently train multiple LSTM-CNN models and screen them;

[0073] The selected multiple LSTM-CNN models are integrated using a weighted average method, wherein the weight of each model is dynamically adjusted using Beta distribution based on the performance of each model on the validation set;

[0074] A meta-learner is introduced, which takes the output of each base model as a feature and optimizes the final prediction result by learning the relationship between the output features of the base model and the true label.

[0075] In some embodiments, the LSTM-CNN network model applies layer normalization technology to normalize the outputs of the LSTM layer and the convolution layer.

[0076] The method of this embodiment is particularly suitable for elderly frail people who use foot-mounted IMUs to collect gait data. The specific explanation is as follows:

[0077] S101: Acquire gait data; construct a training set based on the gait data collected by the IMU, wherein the training set includes: gait data with known gait cycle percentage labels.

[0078] S102: Performing routine preprocessing on the gait data of the training set, including synchronizing sensor readings with gait cycle percentages, excluding static period data points, and applying statistical rules to remove outliers.

[0079] S103: Perform special preprocessing on the input data, extract gait data of consecutive moments from the data window, and pair it with the actual gait cycle percentage of the next moment as a label. Randomly shuffle these data and labels to perform data enhancement to help the model capture a wider range of feature representations.

[0080] S104: Construct the LSTM-CNN model architecture, including input layer, convolution layer, bidirectional LSTM layer and fully connected output layer. The input layer is designed to accept multi-dimensional feature vectors from IMU, the convolution layer is used to extract local features of gait data, thereby enhancing the model's sensitivity to the details of gait changes, the bidirectional LSTM layer is used to capture temporal dependencies, and the output layer is used to predict the percentage of gait cycles. When designing the model architecture, a variety of methods including residual connections, layer normalization and Dropout technology are used to enhance the generalization ability of the model and prevent overfitting.

[0081] At the same time, the ensemble learning method is used to integrate multiple LSTM-CNN models to improve the generalization ability of the model. By combining the advantages of CNN and LSTM, the accuracy of prediction is improved.

[0082] S105: Use ILF to improve the loss function and use the Adam optimization algorithm to train the CNN-LSTM model. Use the 10-fold cross-validation method to evaluate the model performance and ensure the accuracy and generalization ability of the model.

[0083] Furthermore, the S101: obtaining gait data is collected by an IMU installed on the user's foot. The IMU can record key parameters of the gait process, such as step length, gait time, speed and foot angle.

[0084] Furthermore, the S102: performing conventional preprocessing on the gait data of the training set, including calculating the gait cycle percentage and applying the 3σ rule to exclude outliers, so as to improve the data quality.

[0085] Let X = {x1, x2, x3, ..., x N} is the gait data sequence collected from the IMU, where N is the total number of data points. This embodiment organizes these data points into a data window for network input and analysis. In particular, we focus on the data from time point t to t+50, which are used to construct the vector of input data:

[0086] X (t) =x t , x t+1 , ..., x t+50

[0087] In this data window, at each time point x iContains six sensor measurements, namely three acceleration components and three angular velocity components. Therefore, the data window X (t) is converted into a 300-dimensional data vector, represented as:

[0088]

[0089] The subscripts 1 to 6 here represent the six sensor measurements at each time point xi.

[0090] Furthermore, in S103: in order to enhance the generalization ability of the model, we randomly shuffle the gait data of 50 consecutive time points in the data window.

[0091] Define the index Π of the random permutation:

[0092] Π={t+π1,t+π2,...,t+π 51}

[0093] where π i is a randomly ordered sequence of integers satisfying π1, π2, ..., π 51 It is a permutation of 0, 1, ..., 50.

[0094] Reorganize the data window X′ according to the randomly permuted index π:

[0095]

[0096] Convert the shuffled data window X′ into a 300-dimensional data vector v′:

[0097]

[0098] For each time point t, we use the gait cycle percentage at time t+51 as the label y, combined with the 300-dimensional data vector v′ to form an input vector:

[0099] z = [v′, y]

[0100] Furthermore, in the S104: when designing the LSTM architecture, the input layer of the LSTM-CNN model receives the input vector z and forward propagates through the convolution layer and the LSTM unit. The convolution layer is used to extract local features, and its output can be expressed as:

[0101] C=f(W c *v′+b c )

[0102] Among them, C is the output of the convolutional layer, f is the activation function, and W c is the convolution kernel, b c is the bias term, and * indicates the convolution operation.

[0103] The hidden state h of the LSTM unit l and cell state c l Updated by the following formula:

[0104] 1. Forget gate: [f l =σ(W f ·[h l-1 , z]+b f )]

[0105] 2. Input gate: [i l =σ(W i ·[h l-1 , z]+b i )]

[0106] 3. Cell status update:

[0107] It can be expressed as: y = f(h L )

[0108] Where f is the activation function of the output layer; W f is the weight matrix of the forget gate; b f is the bias vector of the forget gate; h l-1 is the hidden state of the previous time step; W i is the weight matrix of the input gate; b i is the bias vector of the input gate; is the candidate cell state; h is the hidden state; h L is the hidden state of the last LSTM layer.

[0109] Furthermore, in S105: during the model training process, an Improved loss function (ILF) loss function and an Adam optimizer are used, and the learning rate is dynamically adjusted through a StepLR learning rate scheduler to improve the model training efficiency.

[0110] The training goal of the model is to minimize the ILF loss function between the predicted value y^ and the actual gait cycle percentage y:

[0111]

[0112] The loss function L is minimized through the gradient descent algorithm and the Adam optimizer, and the network parameters are updated:

[0113]

[0114] Where η is the learning rate, is the loss function over the weights W l The partial derivative of .

[0115] The embodiment of the present invention uses an inertial measurement unit device installed on the foot to collect gait data of elderly individuals under natural walking conditions to form a training set containing rich gait information. The training set data is preprocessed to ensure the accuracy of the data and the reliability of subsequent analysis. A deep neural network architecture is designed and implemented. The network is trained through deep learning technology, and the trained optimal model is applied to new gait data using appropriate loss functions and optimization algorithms. That is, the gait data of the object to be predicted is input into the trained optimal LSTM-CNN model to obtain the gait state prediction result, realize the real-time prediction of the gait phase percentage, and provide a scientific basis for the evaluation of the gait state.

[0116] It should be understood that the gait cycle percentage refers to the proportion of a specific gait phase in a continuous gait cycle. For example, a complete gait cycle starts from the heel strike of one foot and ends with the next heel strike of the same foot. The gait cycle percentage indicates the relative time position of a specific event in this cycle.

[0117] It should be understood that IMU sensors, including three-axis accelerometers and three-axis gyroscopes, are used to measure linear acceleration and angular velocity, respectively. These sensors can capture tiny changes in human motion during walking and provide accurate data support for gait analysis.

[0118] It should be understood that layer normalization is a technique in deep learning that is used to adjust and scale the activation output to have a fixed mean and variance. This helps reduce the so-called "internal covariate shift", accelerates model convergence, and improves the model's robustness to changes in the input data distribution.

[0119] It should be understood that the ILF loss function refers to the instance loss function where the output f(x) generated by the LSTM model and the actual target y are used to calculate each data point. The calculated error is processed through a nonlinear transformation to produce a weight for adjusting the loss. In the forward propagation phase of the LSTM unit, the preprocessed input is passed through the hidden state and the output is finally obtained. After obtaining this output, the model evaluates the difference between the predicted value and the true value. At this point, the ILF loss function is applied, and only after the error is calculated, the backpropagation phase begins, which updates the matrix weights and cell states in the LSTM. The ILF loss function effectively minimizes the error and promotes fast convergence based on the backpropagation process adopted.

[0120] It should be understood that the dropout technique is a regularization method that prevents the network from overfitting the training data by randomly "discarding" some neurons in the network (i.e. setting their outputs to zero) during the training process, thereby improving the generalization ability of the model.

[0121] It should be understood that the Adam optimizer is an adaptive learning rate optimization algorithm that combines the advantages of momentum estimation and adaptive learning rate adjustment, and can dynamically adjust the learning rate of each parameter according to the first-order moment estimation and second-order moment estimation of the gradient.

[0122] It should be understood that the StepLR learning rate scheduler is a learning rate adjustment strategy that reduces the learning rate according to a predetermined number of steps or epochs, so that the learning rate gradually decreases as the training progresses, which helps the model to make more detailed parameter adjustments in the later stages of training and avoid oscillations.

[0123] It should be understood that 10-fold cross validation is a statistical analysis method used to evaluate and improve the predictive performance of the model. The data set is divided into 10 parts, 9 parts are used as training sets in turn, and the remaining 1 part is used as a test set. Ten such training and testing are performed, and the average value of the model performance index is finally taken to evaluate the stability and reliability of the model.

[0124] This embodiment uses a foot-mounted IMU to accurately collect gait data of elderly individuals, and then conducts in-depth analysis of the percentage of each phase of the gait cycle. The gait cycle percentage is a key indicator that reflects the current stage of the gait cycle and is of great significance for gait training and rehabilitation. The method of the present invention focuses on predicting the current and future states of the gait cycle, rather than predicting each gait parameter separately.

[0125] Gait variability, such as changes in the gait cycle, is an important indicator of the stability of an individual's gait. By analyzing the data collected by the IMU, the system of the present invention can evaluate the consistency and stability of the gait cycle, thereby providing a more accurate assessment of the gait cycle percentage. This analysis method is particularly important for elderly individuals because their gait may be more variable and more susceptible to various factors.

[0126] The LSTM-CNN model of the embodiment of the present invention can predict the percentage of each phase of the gait cycle by learning the patterns in the IMU data, thereby realizing real-time monitoring of gait stability. This method can not only identify the current state of gait, but also predict short-term changes in gait, providing a scientific basis for gait training.

[0127] Applications:

[0128] The method of the embodiment of the present invention has a wide range of applications and is suitable for individuals of different ages and physiological states, and particularly shows significant application potential in the following fields:

[0129] Elderly health management: For the elderly, this method can provide personalized gait monitoring and fall risk assessment to help prevent fall accidents and improve the quality of life.

[0130] Rehabilitation medicine: During rehabilitation training, this method can monitor the patient's gait recovery in real time and provide data support for the formulation and adjustment of rehabilitation plans.

[0131] Sports biomechanics: In sports training and performance analysis, this method can assist in analyzing the gait characteristics of athletes and optimizing training programs.

[0132] Medical assistive devices: Integrated into smart devices, this method can provide real-time gait feedback to assist individuals with limited mobility in daily activities.

[0133] This embodiment has a real-time monitoring function, can instantly analyze the gait data collected by the IMU, and predict the gait cycle percentage of an individual. Through the application of the present invention, a personalized gait training plan can be provided for the elderly, and through real-time monitoring and feedback adjustment, it can help them improve gait stability and promote gait rehabilitation. In addition, the present invention can also provide auxiliary tools for clinicians and rehabilitation professionals to help them better understand the gait status of elderly patients and formulate more effective treatment and rehabilitation strategies.

[0134] Embodiment 2

[0135] This embodiment discloses a gait phase prediction system based on time series and LSTM-CNN.

[0136] The gait phase prediction system based on time series and LSTM-CNN includes:

[0137] The data acquisition module is configured to: acquire the gait time series data of the person to be predicted and perform preprocessing;

[0138] The gait cycle percentage prediction module is configured to: input the preprocessed gait time series data into the LSTM-CNN network model, extract the local features of the gait time series data using the convolution layer, and input the extracted local features into the LSTM layer to capture the time dependency, so as to obtain the prediction result of the gait cycle percentage;

[0139] The LSTM-CNN is obtained through pre-training, wherein the pre-training data includes gait timing data of multiple gait cycles and actual gait cycle percentage labels corresponding to the gait timing data of multiple gait cycles after randomization.

[0140] This embodiment provides a gait phase percentage prediction system based on time series and LSTM-CNN, including the following modules:

[0141] The data acquisition module is configured to: acquire gait data from the IMU; and construct a training set based on the IMU gait data, wherein the training set includes: gait data with known gait cycle percentage labels.

[0142] The data preprocessing module is configured to preprocess the gait data of the training set, including synchronizing sensor readings with gait cycle percentages, excluding static period data points, and applying statistical rules to remove outliers.

[0143] Furthermore, it also includes:

[0144] The network architecture design module is configured as: input layer, convolutional layer, bidirectional LSTM layer and fully connected output layer.

[0145] The input layer is designed to accept the multi-dimensional feature vector from the IMU, the convolutional layer is used to extract the local features of the gait data and enhance the model’s sensitivity to the details of gait changes, the bidirectional LSTM layer is used to capture the temporal dependency of the gait data, and the output layer is used to predict the percentage of the gait cycle.

[0146] When designing the model architecture, we use a variety of methods to prevent overfitting to enhance the generalization ability of the model. At the same time, we introduce residual connections to help information flow more efficiently in the network, thereby accelerating training and improving model performance.

[0147] The model training module is configured to: use the improved loss function and optimization algorithm to train the LSTM-CNN model, and use the 10-fold cross-validation method to evaluate the model performance to ensure the accuracy and generalization ability of the model. Through the collaborative work of the above modules, the method can achieve accurate gait phase prediction and provide effective support for gait analysis and rehabilitation.

[0148] The gait prediction module is configured to: obtain new gait data and use the trained optimal LSTM-CNN model to perform real-time prediction of gait cycle percentage.

[0149] Furthermore, the data acquisition module collects gait data of elderly individuals under natural walking conditions through an IMU device, and the IMU device can record key parameters such as step length, gait time, speed and foot angle.

[0150] Furthermore, the data preprocessing module improves data quality by calculating the gait cycle percentage and applying the 3σ rule, laying a solid foundation for subsequent deep learning analysis.

[0151] Furthermore, the network architecture design module incorporates residual connection technology to alleviate the gradient vanishing problem in deep networks and enhance the learning ability of the model. This design effectively improves training efficiency and model convergence by directly connecting the output of the LSTM layer and the convolution layer.

[0152] In addition, by introducing layer normalization technology, the generalization ability of the model is enhanced, while the internal covariate shift phenomenon in the training process is reduced, which speeds up the convergence of the model.

[0153] The model training module adopts the ILF loss function, which is optimized for gait prediction. It combines the Adam optimizer and the StepLR learning rate scheduler to improve the training efficiency and the convergence performance of the model.

[0154] The gait prediction module uses the trained LSTM-CNN model to achieve real-time prediction of new gait data, providing a scientific basis for gait training and fall risk assessment, and enhancing the model's instant feedback capability in practical applications.

[0155] Furthermore, the gait prediction module can use the LSTM-CNN model after ensemble learning to perform real-time prediction of new gait data, providing a scientific basis for gait training and fall risk assessment.

[0156] Embodiment 3

[0157] The purpose of this embodiment is to provide a computer-readable storage medium.

[0158] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the gait phase prediction method based on time series and LSTM-CNN as described in Example 1 of the present disclosure.

[0159] Embodiment 4

[0160] The purpose of this embodiment is to provide an electronic device.

[0161] An electronic device includes a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, the steps in the gait phase prediction method based on time series and LSTM-CNN as described in Example 1 of the present disclosure are implemented.

[0162] The steps involved in the apparatuses of the above embodiments 2, 3 and 4 correspond to the method embodiment 1, and the specific implementation methods can refer to the relevant description part of embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood to include any medium that can store, encode or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.

[0163] Those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computer device, or alternatively, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0164] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A gait phase prediction method based on time series and LSTM-CNN, characterized in that: The following steps are involved: Obtain the gait time series data of the person to be predicted and perform preprocessing; The preprocessed gait timing data is input into the LSTM-CNN network model, and the local features of the gait timing data are extracted using the convolutional layer. The extracted local features are then input into the LSTM layer to capture the time dependency, and the prediction results of the gait cycle percentage are obtained. The LSTM-CNN is obtained through pre-training, wherein the pre-training data includes gait timing data of multiple gait cycles and actual gait cycle percentage labels corresponding to the gait timing data of multiple gait cycles.

2. The gait phase prediction method based on time series and LSTM-CNN as claimed in claim 1, characterized in that: The pre-training process of the LSTM-CNN is specifically as follows: Constructing gait time series data of multiple gait cycles in the pre-training data into a data window; Select data of multiple consecutive moments from the data window, and use the actual gait cycle percentage corresponding to the next moment data of the last moment in the multiple consecutive moments as a training label; The data and training labels of multiple consecutive moments are randomly shuffled and combined to form an input vector; The input vector is fed into the LSTM-CNN, and the features are forward propagated through the convolutional layer and the LSTM layer to obtain the predicted gait cycle percentage; The improved loss function ILF between the predicted gait cycle percentage and the training label is minimized to complete the pre-training process of LSTM-CNN.

3. The gait phase prediction method based on time series and LSTM-CNN as claimed in claim 1, characterized in that: The LSTM-CNN network model adopts a mixed precision training method, using half-precision floating point numbers to store and calculate intermediate results during the forward propagation process of the model, and using single-precision floating point numbers in the back propagation and parameter update stages.

4. The gait phase prediction method based on time series and LSTM-CNN as claimed in claim 1, characterized in that: It also includes building an LSTM-CNN network model, wherein the LSTM-CNN network model specifically includes an input layer, a convolutional layer, an LSTM layer, and a fully connected output layer, wherein: The input layer is used to receive gait timing data; The convolutional layer is used to extract local features of gait time series data; The LSTM layer is used to capture the temporal dependency of local features; The output layer is used to predict the percentage of gait cycle; A residual connection is integrated between the LSTM layer and the convolution layer, and the residual connection is formed by adding a local feature vector output by the convolution layer to an output of the LSTM layer.

5. The gait phase prediction method based on time series and LSTM-CNN as claimed in claim 1, characterized in that: It also includes the use of ensemble learning methods to integrate multiple LSTM-CNN models, including: Independently train multiple LSTM-CNN models and screen them; The selected multiple LSTM-CNN models are integrated using a weighted average method, wherein the weight of each model is dynamically adjusted using Beta distribution based on the performance of each model on the validation set; A meta-learner is introduced, which takes the output of each base model as a feature and optimizes the final prediction result by learning the relationship between the output features of the base model and the true label.

6. The gait phase prediction method based on time series and LSTM-CNN as claimed in claim 1, characterized in that: The LSTM-CNN network model applies layer normalization technology to normalize the outputs of the LSTM layer and the convolution layer.

7. The gait phase prediction method based on time series and LSTM-CNN as claimed in claim 1, characterized in that: The gait timing data specifically includes three acceleration components and three angular velocity components of the foot in the gait cycle, and the gait timing data is obtained by an inertial measurement unit installed on the foot of the person; or, It also includes preprocessing the gait time series data of the person to be predicted, including: Exclude stationary period data points; Apply statistical rules to eliminate outliers; Calculate the gait cycle percentage and synchronize the gait timing data with the gait cycle percentage.

8. Gait phase prediction system based on time series and LSTM-CNN, characterized by: include: The data acquisition module is configured to: acquire the gait time series data of the person to be predicted and perform preprocessing; The gait cycle percentage prediction module is configured to: input the preprocessed gait time series data into the LSTM-CNN network model, extract the local features of the gait time series data using the convolution layer, and input the extracted local features into the LSTM layer to capture the time dependency, so as to obtain the prediction result of the gait cycle percentage; The LSTM-CNN is obtained through pre-training, wherein the pre-training data includes gait timing data of multiple gait cycles and actual gait cycle percentage labels corresponding to the gait timing data of multiple gait cycles.

9. A computer-readable storage medium having a program stored thereon, characterized in that: When the program is executed by a processor, the steps in the gait phase prediction method based on time series and LSTM-CNN are implemented as described in any one of claims 1 to 7.

10. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps in the gait phase prediction method based on time series and LSTM-CNN are implemented as described in any one of claims 1 to 7.

Images