CNN-BiLSTM-Attention abnormal gait detection method based on TBE optimization

Through TBE optimization of the CNN-BiLSTM-Attention abnormal gait detection method, using deep learning and attention mechanism, the problem of abnormal gait detection relies on experience and subjective judgment in the existing technology is solved, and the gait with high accuracy and sensitivity is achieved Identify, adapt to multiple gait types.

CN118940151APending Publication Date: 2024-11-12HARBIN UNIV OF SCI & TECH
View PDF 0 Cites 4 Cited by

Patent Information

Application Number
CN202411001575.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-25
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

The existing abnormal gait detection methods rely on experience and subjective judgment, resulting in misdiagnosis and lack of fast, objective and accurate detection methods.

Method used

A method for abnormal gait detection based on TBE is designed, using sole pressure information, inertial sensor acceleration and angular velocity data, and using deep learning technology and attention mechanism to build a device for collecting gait data and optimize the model. Hyperparameters, the introduction of Squeeze-and-Excitation and Temporal Attention mechanisms can reduce the risk of overfitting and improve feature extraction and classification accuracy.

Benefits of technology

It significantly improves the accuracy and sensitivity of abnormal gait recognition, adapts to a variety of abnormal gait types, and can easily expand and identify new gait types, providing an objective diagnostic basis.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure BDA0004961715380000031
    Figure BDA0004961715380000031
  • Figure BDA0004961715380000041
    Figure BDA0004961715380000041
  • Figure BDA0004961715380000042
    Figure BDA0004961715380000042
Patent Text Reader

Abstract

The invention discloses a CNN-BiLSTM-Attention abnormal gait detection method based on TBE (Tunnel Bit Element) optimization. According to the method, a Squeeze-and-Excitation (SE) attention mechanism is introduced into a CNN (Convolutional Neural Network), and a time attention mechanism is applied in a BiLSTM (Bidirectional Long Short Term Memory), so that key gait features can be effectively extracted and strengthened. Gait data are collected by using a plantar pressure sensor and an IMU inertial sensor. A ReliefF algorithm is adopted to select key features, and hyper-parameters are optimized through a TPE algorithm. After the model is trained, the CNN is used for extracting spatial features, the BiLSTM is used for extracting time sequence features, the features are fused through an attention mechanism, and finally classification prediction is performed through a softmax activation function. The whole model is reasonable in design, the attention mechanism enhances the capturing capability of important information, and overfitting is prevented. The method is suitable for abnormal gait detection in clinical and daily health monitoring, and an efficient and accurate technical means is provided for disease diagnosis and health assessment.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence, and in particular relates to an abnormal gait detection method based on TBE optimized CNN-BiLSTM-Attention. Background Art

[0002] In recent years, with the development of computer and sensor technology, biometric recognition technology has become increasingly popular. As one of the biometric features, gait analysis can obtain important implicit information such as identity and health status. Neurodegenerative diseases can cause gait changes, and the direct manifestation is abnormal gait, and the abnormal manifestations caused by different diseases are different. Typical abnormal gait is instructive for clinical diagnosis. In clinical practice, doctors make diagnoses based on the patient's gait characteristics. However, this method relies heavily on experience and cooperation, and subjective judgment can lead to misdiagnosis. Therefore, it is particularly important to develop a fast, objective and accurate abnormal gait detection method.

[0003] Nowadays, with the rapid development of artificial intelligence, deep learning technology has gradually shown great potential in the field of abnormal gait detection. By building and training deep neural network models, such as CNN, LSTM and other network structures, and introducing attention mechanisms to enhance the model's feature learning ability, key information can be effectively extracted from complex sensor data. This information can not only help accurately identify different types of abnormal gait, but also provide an objective basis for early intervention and personalized treatment. Summary of the invention

[0004] To solve the above problems, the present invention takes artificial intelligence-assisted diagnosis as a guide, uses plantar pressure information, inertial sensor acceleration and angular velocity data as information, applies deep learning technology to the field of abnormal gait detection, and designs an abnormal gait detection method based on TBE optimized CNN-BiLSTM-Attention.

[0005] To achieve the above object, the present invention provides a method for abnormal gait detection based on TBE optimized CNN-BiLSTM-Attention, comprising the following steps:

[0006] S1: Build a device for collecting gait data, including a flexible film pressure sensor, an inertial sensor, a Bluetooth data receiving module, and a host computer;

[0007] S2: Place the pressure sensor on the sole of the foot and the inertial sensor on the calf to collect information on healthy gait and simulate four types of gait: Parkinson's disease, hemiplegia, and diabetes in a laboratory environment;

[0008] S3: De-noise the collected data to make a data set and divide it, select features using the ReliefF algorithm, and convert the data into a preset form;

[0009] S4: Use the TPE algorithm to optimize the model hyperparameters and select the optimal hyperparameter values, including:

[0010] S4.1: The improved CNN-BiLSTM model is trained using the best hyperparameter configuration optimized by the TPE algorithm to extract the spatial and temporal features of gait data. The CNN part is responsible for capturing the spatial information of gait data.

[0011] S4.2: The BiLSTM part processes the time series data and performs feature fusion based on the comprehensive consideration of the feature information extracted from the two parts;

[0012] S5: In the CNN stage, the Squeeze-and-Excitation attention mechanism is introduced to enhance the model's attention to important features in the gait data. A Global Average Pooling layer is added after each convolution, and a fully connected layer is added after the pooling layer to reshape the dimension so that the features can be correctly input into the BiLSTM layer.

[0013] S6: Temporal Attention mechanism is introduced in the BiLSTM layer to strengthen the model's attention to each time step of gait data, helping the model to better capture long-term dependencies and dynamic characteristics of gait motion;

[0014] S7: Add a Dropout layer after Temporal Attention to reduce the risk of overfitting of the model. Specifically:

[0015] S7.1: The output layer uses the cross entropy loss function and the softmax function to predict the gait category of the input data and achieve the ultimate goal of abnormal gait detection;

[0016] S7.2: The output dimension of the network model is set to 4.

[0017] Compared with the prior art, the technical solution adopted by the present invention has the following advantages:

[0018] The present invention effectively improves the ability to extract key information from complex data by utilizing the spatial feature recognition capability of CNN and the time series processing advantage of BiLSTM. The addition of the attention mechanism significantly improves the accuracy and sensitivity of gait classification and recognition. At the same time, through the hyperparameter adjustment optimized by the TBE algorithm and the structural design of the model, the present invention is not only suitable for several known abnormal gait types, but also can easily expand the recognition of new or uncommon abnormal gait types. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0020] Figure 1 It is a diagram of the implementation plan;

[0021] Figure 2 This is the principle diagram of the ReliefF algorithm;

[0022] Figure 3 It is the flow chart of TPE algorithm;

[0023] Figure 4 CNN-BiLSTM model diagram for improved attention mechanism;

[0024] Figure 5 It is the SE attention mechanism module framework;

[0025] Figure 6 Framework for the temporal attention module. DETAILED DESCRIPTION

[0026] In order to fully explain the purpose, technical implementation scheme and corresponding advantages of the present invention, the present invention will be clearly and completely described below through the specific examples in the accompanying drawings. It should be understood that these descriptions are only exemplary and are not intended to limit the scope of protection of the present invention. The structures, proportions and dimensions shown in the accompanying drawings are only used to cooperate with the specification to help professionals familiar with the technical field to understand and read, and are not intended to impose technical limitations on the implementation methods of the present invention. Any form of structural modification, proportional adjustment or size change, as long as it does not affect the effects and purposes that can be achieved by the present invention, should be regarded as still within the technical scope disclosed by the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work belong to the scope of protection of the present invention.

[0027] like Figure 1 As shown, a specific embodiment of the present invention comprises the following steps:

[0028] S1: A device for collecting gait data, including a flexible film pressure sensor, an inertial sensor, a Bluetooth data receiving module, and a host computer;

[0029] S1.1: Inertial sensors are placed on both sides of the calves, of which the accelerometer is responsible for collecting the wearer's acceleration information and the gyroscope is used to collect the wearer's angular velocity information. The pressure sensor is placed on the sole of the foot to collect the sole pressure data of the wearer when walking.

[0030] S1.2: Both the inertial sensor and the plantar pressure sensor are charged through the USB interface, and there is a power supply battery inside the module. The acquisition and transmission module of the plantar pressure sensor is placed near the calf without affecting its wearing position. After the inertial sensor and pressure sensor collect information, the data information is transmitted to the receiving communication module via Bluetooth, and the data is saved, pre-processed, and gait detected in the host computer.

[0031] S2: De-noise the collected data to make a data set and divide it into training set, validation set and test set according to 6:2:2. Use the ReliefF algorithm to select features and convert the data into a preset format. Figure 2 shown.

[0032] S2.1: When the ReliefF algorithm processes multi-classification tasks, it randomly selects a sample R from the training samples each time, then finds the k nearest neighbor samples of R from the sample set of the same type as R, finds the k nearest neighbor samples from each sample set of different classes of R, and then updates the weight of each feature. The formula is as follows:

[0033]

[0034] Among them, W(A) represents the weight of attribute A, diff(A,R,H j ) is to calculate the sample R and its similar sample H j The difference in attribute A, M j (C) indicates class P(class(R)) is the prior probability of the class of sample R, and P(C) is the prior probability of class C.

[0035] S3: Use the TPE algorithm to optimize the model hyperparameters and select the optimal hyperparameter values. The TPE algorithm flow chart is as follows: Figure 3 As shown. The TPE algorithm is a non-standard Bayesian optimization algorithm based on a tree structure. The computational efficiency of the TPE algorithm is usually higher than other Bayesian algorithms and is more suitable for deep learning. Specifically, it includes:

[0036] S3.1: According to Bayes' theorem:

[0037]

[0038] Among them, p(y|x) is the posterior probability, that is, the conditional probability of target y given parameter x, p(x|y) is the likelihood probability, that is, the probability of parameter x given target y, p(y) is the prior probability, that is, the probability of target y, and p(x) is the evidence factor, which serves as a normalizing constant.

[0039] S3.2: The TPE algorithm performs a partitioning on p(x|y):

[0040]

[0041] Among them, l(x) is called a good distribution, which is when y is less than the threshold y * g(x) is called the poor distribution, which is when y is greater than the threshold y * For the selection of the threshold, a parameter needs to be set, and the value in the embodiment of the present invention is 0.2.

[0042] S4: The preprocessed data is input into the CNN-BiLSTM network model with improved attention mechanism. The CNN-BiLSTM model with improved attention mechanism is shown in the figure below. Figure 4 As shown, specifically including:

[0043] S4.1: The CNN part of the model first receives processed input data and extracts key spatial features from it using multiple convolutional layers and pooling layers. After each convolutional layer, not only the ReLU activation function is applied to increase the nonlinear processing capability, but also batch normalization is used to speed up the training process and improve the stability of the model.

[0044] S4.2: BiLSTM is used to further analyze the temporal features to effectively detect abnormal gait;

[0045] S5: Add a Squeeze-and-Excitation (SE) module after each convolution of CNN. This module compresses the spatial dimension and channel dimension of the feature map and then learns the importance of different channels. The framework of the SE attention mechanism module is as follows: Figure 5 As shown, specifically including:

[0046] S5.1: First, compression is performed. In this step, the goal is to aggregate the spatial information (i.e., width and height) of each channel into a single value in order to obtain the global information of the channel. This is usually achieved through global average pooling, which calculates the average value of all elements on each channel. The formula is as follows:

[0047]

[0048] Where T is the length of the time series, x c (t) is the value of channel c at time t.

[0049] S5.2: Perform excitation to calculate the importance score of the channel. The formula is as follows:

[0050] S C =σ(W2δ(W1z c))C

[0051] Where S C is the weight of the channel, which determines the importance of the features of the corresponding channel in subsequent processing. W1 and W2 represent the weight matrices of the first and second fully connected layers respectively. c It is the global information of channel C obtained after the Squeeze operation. δ represents the ReLU activation function, and σ represents the Sigmoid activation function.

[0052] S5.3: Scale, adjust each channel of the original sensor data according to the weight learned for each channel. The formula is as follows:

[0053]

[0054] in is the scaled feature, X C Represents the eigenvalue of the Cth channel input.

[0055] S6: Introduce the Temporal Attention mechanism in the BiLSTM layer to strengthen the model's attention to each time step of gait data, helping the model to better capture long-term dependencies and dynamic features of gait motion. The framework of the temporal attention module is as follows: Figure 6 As shown, specifically including:

[0056] S6.1: Temporal Pattern Attention (TPA) is used as the temporal attention mechanism, which can significantly improve the model performance of processing time series data. The attention weight α of each time step t The calculation formula is as follows:

[0057]

[0058] Among them, e t is a fully connected layer with an activation function, which is used to evaluate the importance of the data point at time step t to the current task. Figure 6 In (h t-w ,h t-w+1 ,...,h t tV t ) represents the hidden state of the time series data at different time steps. The hidden state at a specific time point t is used as a reference point and compared with the historical state to calculate the attention weight. According to the calculated attention weight, V t It represents the weighted sum of the hidden states of each historical state, which reflects the impact of the past state on the current state. Finally, a fully connected layer is used to generate the final output.

[0059] S7: The superiority of the present invention is demonstrated by comparing it with other existing models such as Transformer, and an ablation experiment is performed to obtain the best network model. The present invention uses accuracy, precision, recall and F1-score to evaluate the test results. In order to evaluate the performance of the model in an intuitive and comprehensive way, the confusion matrix is ​​used to display the classification results of each type of data. The accuracy, precision, recall and F1-score are defined as follows:

[0060]

[0061]

[0062] Among them, TN represents the number of samples predicted as positive samples; TN represents the number of samples predicted as negative samples; FP represents the number of samples predicted as positive samples; FN represents the number of samples predicted as negative samples; TP+FP represents the total number of samples predicted as positive samples; TP+FN represents the total number of positive samples.

[0063] The above description of the disclosed embodiments is not intended to limit the present invention, and many modifications and changes can be made to the common technology in the field without creative work based on the concept of the present invention. Therefore, all technical solutions that can be obtained by technicians in the technical field through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the existing technology should be included in the protection scope of the present invention.

Claims

1. A method for abnormal gait detection based on TBE optimized CNN-BiLSTM-Attention, It is characterized in that The specific steps include: S1: Place the plantar pressure sensor in the shoe to collect gait pressure data when the human body is walking, build a data set, place the IMU inertial sensor on the calf, and collect data by simulating abnormal gait such as Parkinson's disease, hemiplegia, and diabetic foot; S2: Reduce the noise of the data set and divide the data set, use the ReliefF algorithm to select features, and convert the data into a preset form; S3: Using the training set after feature selection, the hyperparameters of the CNN-BiLSTM model with improved attention mechanism are optimized through the TBE algorithm, and finally the model is trained and evaluated using the optimal hyperparameters; S4: The model extracts spatial features of data through CNN, extracts temporal features using LSTM, and performs feature fusion; S5: Add the attention mechanism Squeeze-and-Excitation (SE) to CNN, add a fully connected layer after the output of the pooling layer and reshape the dimension to provide the correct input shape for the subsequent BiLSTM layer; S6: Temporal Attention is added to BiLSTM to strengthen the BiLSTM model's attention to each time step, helping the model better capture long-term dependencies in sequence data; S7: A Dropout layer is added after the temporal attention mechanism to reduce the risk of overfitting. The final output layer uses a softmax activation function to predict the category of the input data.

2. According to claim 1, the abnormal gait detection method based on TBE optimized CNN-BiLSTM-Attention, It is characterized in that In the step 1, the plantar pressure sensor uses an array distributed flexible film pressure sensor, and the inertial sensor includes an accelerometer and a gyroscope, which are responsible for collecting acceleration and angular velocity information respectively.

3. According to claim 1, the abnormal gait detection method based on TBE optimized CNN-BiLSTM-Attention, It is characterized in that In the step 2, the ReliefF function converts the label data into a classification format by setting the nearest neighbor parameter K and selecting the number of features Nf, and uses the ReliefF function to calculate the importance weight of the feature, and selects the top N most important features based on the weight.

4. According to claim 1, the abnormal gait detection method based on TBE optimized CNN-BiLSTM-Attention, It is characterized in that In step 3, Hyperopt is used to implement Bayesian optimization through TPE.

5. According to claim 1, the abnormal gait detection method based on TBE optimized CNN-BiLSTM-Attention, It is characterized in that The fused improved CNN-BiLSTM-Attention network model consists of the following: the input layer accepts data with a shape of (19,400,1); the convolution kernel size is (4,1); the SE attention mechanism learns the importance weight of each feature channel through global average pooling and two connection layers; the reshape layer adjusts the output to a shape suitable for the BiLSTM layer input (1,64); the temporal attention layer is applied to the BiLSTM output to enhance the attention to the key time steps; the Dropout layer is added after the fully connected layer and the BiLSTM layer; the output layer uses a fully connected layer with a softmax activation function for multi-category classification tasks.

Citation Information

Cited By

  • Tourism demand prediction method based on two-stage feature selection and Time2Vec

    CN119692557A

  • CNN-BiLSTM-Attention building structure health monitoring missing data restoration method based on AO optimization

    CN120372157A

  • Building structure health monitoring missing data repair method based on CNN-BiLSTM-attention optimized

    CN120372157B

  • Aero-engine anomaly detection reference model construction method based on self-attention CNN-BisLSTM

    CN120724246A