Self-powered plantar pressure monitoring and gait analysis system integrated with convolutional neural network
By combining liquid metal/PDMS porous sponge structure with CNN/LSTM model, the problems of insufficient sensitivity and poor environmental adaptability of existing gait analysis systems are solved, realizing high-precision gait recognition and monitoring, which is suitable for daily health monitoring and exercise management.
Patent Information
- Application Number
- CN202511473472.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-15
- Publication Date
- 2026-01-23
AI Technical Summary
Existing gait analysis systems are bulky, have limited application range, and interfere with the movement of the subject. Traditional sensors are significantly affected by ambient humidity, are brittle, rely on external power supplies, and have complex wiring and short battery life. They have failed to effectively solve the coupling error between the nonlinear compression characteristics of flexible sensors under dynamic loads and the electrical signal output.
A triboelectric nanogenerator with a liquid metal/PDMS porous sponge structure is constructed by combining a convolutional neural network (CNN) and a long short-term memory network (LSTM) model to build a multi-physics coupling model of "pressure-inertia-thickness". The flexible insole-type sensor module monitors the plantar pressure signal in real time, and the temperature compensation unit eliminates the influence of environmental temperature changes, so as to achieve high-precision acquisition and analysis of the signal.
It achieves highly sensitive and environmentally adaptable gait analysis, accurately identifying eight gait patterns including normal gait and limping, with an accuracy rate of 98.7%. The measurement error is small within the range of -10℃ to 60℃, and environmental noise interference is eliminated, improving the practicality and accuracy of the system.
Smart Images
Figure CN121370132A_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application belongs to the field of gait analysis, and particularly relates to a self-powered plantar pressure monitoring and gait analysis system integrated with a convolutional neural network. BACKGROUND
[0002] Foot diseases, osteoarthrosis, nervous system diseases, and diabetes and other health problems are increasing, and these diseases are often accompanied by abnormal changes in gait. Gait abnormalities are not only an early signal of many diseases, but also have a significant impact on an individual's daily life and quality of life. Therefore, developing a system that can accurately and real-time monitor gait has important significance for assessing human health status and early diagnosis of diseases.
[0003] Existing gait analysis systems have some limitations, such as large volume, limited use, and interference with the movement of the measured person, which limits their application in real life. In order to overcome these limitations, researchers have begun to explore new methods based on flexible electronics and nano-energy technology to develop a wearable gait detection system that is both soft and comfortable and adaptable to the environment.
[0004] The emergence of triboelectric nanogenerator (TENG) technology provides new possibilities for this topic. As a newly developed self-powered device, triboelectric nanogenerator (TENG) does not require additional energy supply and has received increasing attention since its invention by Wang Zhonglin et al. in 2012. It can convert various mechanical signals into electrical signals through the coupling of contact electrification and electrostatic induction, providing a new energy solution for wearable devices. Combined with the function of pressure sensors, TENG can not only serve as an energy collector, but also directly as a pressure detection sensor, achieving the dual functions of energy collection and signal detection. Traditional triboelectric nanogenerators have certain limitations in achieving low density and high porosity. Similarly, although traditional foam-based sensors have excellent low density, high porosity, and electrical conductivity, they also face challenges in achieving ultra-low detection limits and wide linear detection ranges. The significant change in contact area / contact points of conductive foams under small strain enables them to respond sensitively to subtle stimuli. At the same time, the high modulus of the foam matrix and the mutual extrusion of the foam skeleton enable the sensor to have a relatively high triggering sensing distance, making it suitable for the preparation of high-performance strain sensors.
[0005] The application of gait detection sensors in the field of footwear has important significance for sports health monitoring, rehabilitation evaluation and human-computer interaction. The existing technology has the following shortcomings: (1) the traditional metal electrode sensor has high rigidity (Shore hardness > 80A), poor wearing comfort, and is easy to cause foot pressure sores after long-term use; (2) the sensitivity of the capacitive sensor is significantly affected by the environmental humidity (error rate > 15%), and cannot meet the stability requirements of daily shoe wearing scenes; (3) the piezoelectric ceramic device has high brittleness (fracture elongation < 5%), and is easy to cause mechanical failure under gait impact; (4) most sensors rely on external power supply, and have the problems of complex wiring and short endurance.
[0006] In Chinese patent CN2025114330842, a gait sensing unit of a triboelectric nanogenerator based on liquid metal / PDMS porous sponge and a preparation method thereof are disclosed. The method uses white sugar templates to construct a three-dimensional interconnected porous structure, realizes the uniform dispersion of LM in the PDMS matrix, and solves the technical problem that traditional TENG cannot simultaneously have low density and high sensitivity. The preparation method includes steps such as mixing and stirring, template impregnation, solidification molding and template dissolution, and can batch produce flexible sensors with thousands of cycle stabilities, which are suitable for wearable health monitoring devices.
[0007] In addition, the existing technology relies only on single pressure signal analysis for traditional foot pressure monitoring system. At the same time, the coupling error between the nonlinear compression characteristics of LM / PDMS composite material under dynamic load (such as the change of contact area caused by porosity change) and the electrical signal output is not considered, resulting in differences in sensing sensitivity and feature distortion in different pressure intervals. Furthermore, the nonlinear influence of insole or sensor thickness dynamic change on foot pressure sensing data is not considered in the existing technology, resulting in coupling error between material compression deformation and electrical signal output. SUMMARY
[0008] In view of the existing technical problems in the prior art, the present application proposes a gait sensing unit of a triboelectric nanogenerator based on liquid metal / PDMS porous sponge and a preparation method thereof, aiming to solve the coupling error between the nonlinear compression characteristics of LM / PDMS composite material under dynamic load (such as the change of contact area caused by porosity change) and the electrical signal output. The present application constructs a "pressure-inertia" dual-mode fusion analysis framework, uses the local receptive field mechanism of CNN model to capture the micro-fluctuation characteristics of voltage signal, and combines the bidirectional LSTM network to model the spatiotemporal correlation of foot pressure transmission, so as to solve the problem of insufficient gait feature extraction accuracy caused by nonlinear coupling between material compression deformation and electrical response.
[0009] To achieve the above object, the application provides a self-powered plantar pressure monitoring and gait analysis system based on integrated convolutional neural network, comprising: a flexible insole sensor module, a signal acquisition unit and a CNN analysis module; the flexible insole sensor module is electrically connected with the signal acquisition unit, and the signal acquisition unit is connected with the CNN analysis module through a wireless transmission mode;
[0010] The flexible insole sensor module comprises a gait sensing unit; the gait sensing unit is formed of a porous sponge structure composed of liquid metal and polydimethylsiloxane; the porous sponge structure has a three-dimensionally interconnected pore network; the liquid metal is a gallium-based alloy, which is uniformly distributed in a polydimethylsiloxane matrix in the form of a micro-nano scale dispersed phase;
[0011] The signal acquisition unit is used for receiving a plantar dynamic pressure signal collected by a sensor array, and comprises an adaptive filtering module, which uses a least mean square algorithm (LMS) to filter out motion noise points and environmental electromagnetic interference in real time, and a filtering bandwidth can be dynamically adjusted according to signal frequency characteristics;
[0012] The CNN analysis module comprises a multi-modal fusion feature extraction unit, a one-dimensional convolutional layer, a batch normalization layer, an activation function layer, a pooling layer, an attention mechanism layer, a bidirectional LSTM network and a fully connected layer, which are used for end-to-end feature extraction and classification of original voltage signals and inertial sensor data; the multi-modal fusion feature extraction unit is used for integrating plantar pressure signals and inertial sensor data, and realizes cross-modal information fusion by using a feature splicing and weight adaptive allocation mechanism; the bidirectional LSTM network comprises a forward LSTM layer and a backward LSTM layer, each layer containing 64 hidden units, which are used for capturing the spatio-temporal correlation features of gait signals; the attention mechanism layer is composed of a channel attention module and a spatial attention module in cascade, the channel attention realizes adaptive allocation of feature weights through a Squeeze-and-Excitation network structure, and the spatial attention extracts spatial correlation information of a feature map through a 7x1 convolution kernel and realizes gait state recognition.
[0013] In a specific embodiment, the CNN analysis module further comprises a thickness deformation data acquisition unit, the thickness deformation data acquisition unit uses a micro optical fiber grating sensor array to collect the thickness change of the insole in real time through the wavelength shift of light; and the multi-modal fusion feature extraction unit is further used for integrating thickness deformation data, and realizes cross-modal information fusion by using a feature splicing and dynamic weight allocation mechanism.
[0014] In a specific embodiment, the flexible insole sensor module further comprises a temperature compensation unit, the temperature compensation unit uses a nickel-chromium alloy thin film resistor in parallel with the sensing unit, and a Wheatstone bridge circuit is used to eliminate the pressure detection error caused by the change of environmental temperature from-20 DEG C to 60 DEG C.
[0015] In a specific embodiment, the CNN analysis module adopts a three-layer one-dimensional convolution structure, the convolution kernel sizes are 3x1, 5x1 and 7x1 respectively, the activation function adopts ReLU, and the optimizer is adaptive moment estimation (Adam).
[0016] In a specific embodiment, the workflow of the system includes:
[0017] (1) Training phase: the plantar pressure signal is divided into a training set and a test set in a ratio of 7:3, after the input CNN model is processed by convolution, normalization and pooling, the six types of gait prediction results are output through the full connection layer;
[0018] (2) Test phase: the model inferences the new sample to output the predicted category, and compares it with the real label to realize accuracy evaluation.
[0019] In a specific embodiment, the thickness of the nickel-chromium alloy thin film resistor of the temperature compensation unit is 50-200 nm, the sheet resistance value is controlled to be 100-500 Ω / □, and the temperature compensation unit is deposited on a polyimide flexible substrate through a magnetron sputtering process.
[0020] In a specific embodiment, the step factor of the LMS algorithm of the adaptive filtering module is set to 0.01-0.1, the iteration number is greater than or equal to 50 times, the convergence error is less than or equal to 0.001 mV, and the adaptive filtering module supports dynamic filtering of signals in a frequency band of 1-500 Hz.
[0021] In a specific embodiment, the micro fiber grating sensor array of the thickness deformation data acquisition unit includes three sensing points, which are respectively arranged in the forefoot, arch and heel regions, and synchronous acquisition is realized by using wavelength division multiplexing technology, and the sampling frequency is consistent with the pressure signal (≥1 kHz).
[0022] In a specific embodiment, the multi-modal fusion feature extraction unit adopts a three-step processing procedure of "time series alignment-feature dimension expansion-weight dynamic allocation" for the thickness deformation data, wherein the thickness features are mapped in dimension (from 128 dimensions to 64 dimensions) through a 1x1 convolution kernel, then spliced with the pressure / inertial features, and the thickness feature weight is adaptively adjusted to 0.12-0.28 through an attention mechanism layer.
[0023] In a specific embodiment, the compression ratio of the channel attention mechanism is set to 16, the excitation function adopts Sigmoid, and the number of convolution kernels of the spatial attention mechanism is 64, forming a serial cascade structure with the channel attention mechanism.
[0024] Compared with the prior art, the present application has the following remarkable beneficial effects:
[0025] 1. The signal acquisition unit integrates the LMS adaptive filtering algorithm, the dynamic filtering bandwidth can be adjusted in real time according to the signal characteristics, the noise suppression ratio is improved by 40dB, the motion artifacts and electromagnetic interference are effectively eliminated, and the collection accuracy of the plantar pressure signal is ensured;
[0026] 2. The CNN analysis module fuses multi-modal feature extraction and attention mechanism, through a three-layer one-dimensional convolution structure and a dynamic weight distribution mechanism, the gait recognition accuracy reaches 98.7%, which is improved by 15.3 percentage points compared with the traditional machine learning method, and 8 typical gait modes such as normal gait, limp and foot inversion can be accurately identified;
[0027] 3. The "pressure-thickness-inertia" multi-physical field coupling model is innovatively constructed, and the temperature compensation unit is combined, so that the measurement error is small in the environment temperature range of-10℃ to 60℃, the technical bottleneck of the flexible sensor temperature drift sensitivity is solved, and the environmental adaptability of the system is improved. BRIEF DESCRIPTION OF DRAWINGS
[0028] Figure 1 It is a gait recognition model and data analysis diagram in an embodiment of the application;
[0029] Figure 2 It is a structure schematic view of the neural network classifier for gait recognition in an embodiment of the application. DETAILED DESCRIPTION
[0030] The embodiments of the present patent are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference signs represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary and are only used to explain the present patent, and cannot be understood as a limitation of the present patent.
[0031] Embodiment one
[0032] In the first embodiment of the application, a self-powered plantar pressure monitoring and gait analysis system integrating a convolutional neural network is provided, comprising: a flexible insole sensor module, a signal acquisition unit and a CNN analysis module; the flexible insole sensor module is electrically connected with the signal acquisition unit, and the signal acquisition unit is connected with the CNN analysis module through a wireless transmission mode;
[0033] The flexible insole sensor module comprises a gait sensing unit; the gait sensing unit is formed of a porous sponge structure composed of liquid metal and polydimethylsiloxane; the porous sponge structure has a three-dimensionally interconnected pore network; the liquid metal is a gallium-based alloy, which is uniformly distributed in a polydimethylsiloxane matrix in the form of a micro-nano scale dispersed phase;
[0034] The signal acquisition unit is used for receiving the plantar dynamic pressure signals collected by the sensor array, and contains an adaptive filtering module, which uses a least mean square algorithm (LMS) to filter out motion noise and environmental electromagnetic interference in real time, and the filtering bandwidth can be dynamically adjusted according to the signal frequency characteristics;
[0035] The CNN analysis module contains a multi-modal fusion feature extraction unit, a one-dimensional convolution layer, a batch normalization layer, an activation function layer, a pooling layer, an attention mechanism layer, a bidirectional LSTM network and a full connection layer, which are used for end-to-end feature extraction and classification of original voltage signals and inertial sensor data; the multi-modal fusion feature extraction unit is used for integrating plantar pressure signals and inertial sensor data, and realizes cross-modal information fusion by using a feature splicing and weight adaptive allocation mechanism; the bidirectional LSTM network contains a forward LSTM layer and a backward LSTM layer, each layer containing 64 hidden units, which are used for capturing the spatio-temporal correlation features of gait signals; the attention mechanism layer is composed of a channel attention module and a spatial attention module in cascade, the channel attention realizes feature weight adaptive allocation through a Squeeze-and-Excitation network structure, and the spatial attention extracts spatial correlation information of the feature map through a 7x1 convolution kernel, and realizes gait state recognition.
[0036] The prior art does not consider the nonlinear influence of dynamic changes in the thickness of insoles or sensors on plantar pressure sensing data, resulting in coupling errors between material compression deformation and electrical signal output. The present application introduces a thickness deformation monitoring dimension, establishes a "pressure-thickness-inertia" multi-physical field coupling model, and solves the feature distortion problem caused by the difference in compression characteristics of LM / PDMS composite materials in traditional single pressure signal analysis.
[0037] Specifically, in the present embodiment, the CNN analysis module further contains a thickness deformation data acquisition unit, which uses a micro fiber Bragg grating sensor array to collect the thickness change of the insole in real time through the wavelength shift of light (sensitivity 15pm / μm), the detection range is 0.1-5mm, and the resolution is ≤10μm; the multi-modal fusion feature extraction unit is also used for integrating thickness deformation data, and realizes cross-modal information fusion by using a feature splicing and dynamic weight allocation mechanism (the thickness feature weight 0.12-0.28 is adaptively adjusted through the attention mechanism layer). The present embodiment introduces a thickness deformation monitoring dimension, establishes a "pressure-thickness-inertia" multi-physical field coupling model, and solves the feature distortion problem caused by the difference in compression characteristics of LM / PDMS composite materials in traditional single pressure signal analysis.
[0038] Further, the flexible insole sensor module further comprises a temperature compensation unit, which adopts a parallel structure of a nichrome thin film resistor and a sensing unit, and eliminates the pressure detection error caused by the change of environmental temperature from -20°C to 60°C through a Wheatstone bridge circuit.
[0039] In the embodiment, the CNN analysis module adopts a three-layer one-dimensional convolution structure, the convolution kernel sizes are 3x1, 5x1 and 7x1 respectively, the activation function adopts ReLU, and the optimizer is adaptive moment estimation (Adam).
[0040] The workflow of the embodiment includes:
[0041] (1) Training phase: the plantar pressure signal is divided into a training set and a test set in a ratio of 7:3, after the input CNN model is processed through convolution, normalization and pooling, the six types of gait prediction results are output through a full connection layer;
[0042] (2) Test phase: the model inferences the new sample to output the predicted category, and compares it with the real label to realize accuracy evaluation.
[0043] Further, the thickness of the nichrome thin film resistor of the temperature compensation unit is 50-200 nm, the sheet resistance value is controlled to be 100-500 Ω / □, and the nichrome thin film resistor is deposited on the polyimide flexible substrate through a magnetron sputtering process.
[0044] In the embodiment, the step factor of the LMS algorithm of the adaptive filtering module is set to 0.01-0.1, the iteration number is greater than or equal to 50 times, the convergence error is less than or equal to 0.001 mV, and the adaptive filtering module supports dynamic filtering of signals in a frequency band of 1-500 Hz.
[0045] In the embodiment, the micro fiber grating sensor array of the thickness deformation data acquisition unit comprises three sensing points, which are respectively arranged in the forefoot, arch and heel regions, and synchronous acquisition is realized through wavelength division multiplexing technology, and the sampling frequency is consistent with the pressure signal (≥1 kHz).
[0046] Further, the multi-modal fusion feature extraction unit adopts a three-step processing procedure of "time sequence alignment-feature dimension expansion-weight dynamic allocation" for the thickness deformation data, wherein the thickness features are mapped in dimension (from 128 dimensions to 64 dimensions) through a 1x1 convolution kernel, then spliced with the pressure / inertial features, and the thickness feature weight is adaptively adjusted to 0.12-0.28 through an attention mechanism layer.
[0047] In the embodiment, the compression ratio of the channel attention mechanism is set to 16, the excitation function adopts Sigmoid, and the number of convolution kernels of the spatial attention mechanism is 64, and the channel attention mechanism and the spatial attention mechanism form a serial cascade structure.
[0048] AsFigure 1 As shown in Figure a, the system provided in this embodiment is used to collect plantar pressure signals to characterize the wearer's gait characteristics, movement state, and balance ability. The system consists of five PLMFT (liquid metal / PDMS sponge-based triboelectric nanosensor) gait sensing units, employing a wireless sensor module design. The overall process is as follows: Figure 1 As shown in b, the five-channel sensor system is distributed across the forefoot region (channel 1), heel region (channels 2 and 3), and lateral foot edge region (channels 4 and 5), enabling real-time monitoring of walking, running, jumping, and other movement states. Figure 1 d).
[0049] In practical applications, the central processing unit transmits plantar pressure data to a smartphone via Bluetooth, enabling wireless sensing and data recording. Figure 1 c demonstrates a dynamic gait anomaly monitoring process incorporating wireless sensing capabilities. By monitoring the signal output of each channel in real time, the system can not only accurately reflect the user's movement status but also possess the ability to detect abnormal gait (e.g., ...). Figure 1 (as shown in e).
[0050] Figure 1 f shows the output signal distribution of each channel in three typical gait states: supination, pronation, and normal running. In the supination state, the output signals of channels 1 and 2 are significantly higher than those of channels 3, 4, and 5; in the normal running state, the output signals of each channel tend to be consistent; in the pronation state, the output signals of channels 3, 4, and 5 are higher than those of channels 1 and 2. By comparing the maximum output values of each channel, the differences between the three gait states can be intuitively distinguished.
[0051] The following section introduces neural network classifiers used for gait recognition.
[0052] Gait motion during walking includes multi-dimensional information such as gait speed, plantar contact pressure, PLMFT sensor triggering sequence, individual gait patterns, and signal repeatability. To improve the efficiency and accuracy of gait recognition, this embodiment uses a convolutional neural network (CNN) deep learning algorithm to analyze plantar pressure data, which can be effectively applied to daily personal health monitoring and exercise management scenarios.
[0053] Figure 2aThe overall framework of the gait recognition system is shown. This embodiment collects 1000 sets of labeled plantar pressure data, establishes a multi-class label sample database according to the basic parameters of gait analysis, and divides the samples into six categories: walking, varus, valgus, falling, running and jumping. In the training stage, the plantar pressure signal is subjected to deep feature extraction by the convolution feature extraction module, and in the classification stage, the corresponding category is output by the full connection layer to achieve accurate recognition; the adaptive moment estimation (Adam) optimizer is used to update the model parameters to improve the training efficiency and stability. The data processing flow is divided into training and testing: the training set and the test set are divided in the ratio of 7:3, the plantar pressure signal is input into the CNN model in the training stage, and after processing by one-dimensional convolution, batch normalization, activation function, pooling and adaptive pooling, etc., the six categories of gait prediction results are output by the full connection layer; in the test stage, the new sample is inferred and compared with the true label to evaluate the performance.
[0054] To evaluate the superiority of the convolutional neural network (CNN) in gait voltage signal recognition, this embodiment compares it with Bayesian neural network (BNN), support vector machine (SVM), long short-term memory network (LSTM) and random forest (RF). The gait signal is essentially time series data of dynamic changes in plantar voltage, with characteristics of high dimensionality, local mutation and global dependence. The experimental results show that Figure 2 b), the recognition accuracy of CNN is significantly better than that of traditional methods, and its advantages come from three core mechanisms: traditional machine learning methods (such as SVM / RF) need to rely on PCA dimensionality reduction preprocessing, which easily loses the local time series information of the original voltage signal, and BNN has limited expression ability for high-dimensional data feature space, which easily leads to overfitting and decreased generalization ability; although the time series model LSTM can capture long-range dependencies, it responds slowly to voltage signal transient mutation characteristics (such as pressure peak gradient change); CNN directly processes the original signal by cascading Conv1d layers, accurately captures voltage transient patterns through small-scale convolution kernels, and abstracts the spatiotemporal characteristics of plantar pressure transmission through multi-level pooling. Its local receptive field mechanism naturally fits the essence of voltage signal short-time correlation, and establishes an end-to-end mapping from micro voltage fluctuations to macro gait categories, providing an effective solution for time series pressure signal analysis.
[0055] Figure 2 The confusion matrix in c shows that the recognition accuracy of the six gait states is 99.51%; Figure 2 The t-SNE visualization result in d shows that the same data points are closely clustered and the different data points are clearly separated, verifying that the model has strong discrimination ability and consistent prediction performance; the accuracy and loss curves of the training and verification process are shown in Figure 2 e-f; Figure 2 g shows the receiver operating characteristic (ROC) curve of the model, and the AUC value of each classification is not less than 0.999, indicating that the overall classification ability of the gait state is excellent.
[0056] The results of this study fully verify the feasibility and effectiveness of the plantar pressure detection and gait analysis system based on the CNN deep learning algorithm in the gait recognition task. In the future, with the continuous collection of more wearer data and the continuous optimization of the model, the system is expected to have greater application potential in gait abnormality detection, motor function evaluation, and disease prediction.
[0057] The above describes the preferred embodiments of the present application in detail. It should be understood that those skilled in the art can make many modifications and changes without creative labor according to the concept of the present application. Therefore, any technical solutions obtained by logical analysis, reasoning or limited experiments based on the existing technology according to the concept of the present application shall be within the protection scope determined by the claims.
Claims
1. An integrated convolutional neural network based self-powered plantar pressure monitoring and gait analysis system, characterized in that, The system comprises a flexible insole sensor module, a signal acquisition unit and a CNN analysis module; the flexible insole sensor module is electrically connected with the signal acquisition unit, and the signal acquisition unit is connected with the CNN analysis module through a wireless transmission mode; The flexible insole sensor module comprises a gait sensing unit; the gait sensing unit is formed of a porous sponge structure of a liquid metal and polydimethylsiloxane; the porous sponge structure has a three-dimensional interconnected pore network; the liquid metal is a gallium-based alloy, which is uniformly distributed in a polydimethylsiloxane matrix in the form of a micro-nano scale dispersed phase; The signal acquisition unit is used for receiving plantar dynamic pressure signals collected by a sensor array, and comprises an adaptive filtering module, which uses a least mean square algorithm (LMS) to filter out motion noise points and environmental electromagnetic interference in real time, and the filtering bandwidth can be dynamically adjusted according to the signal frequency characteristics; The CNN analysis module comprises a multi-modal fusion feature extraction unit, a one-dimensional convolution layer, a batch normalization layer, an activation function layer, a pooling layer, an attention mechanism layer, a bidirectional LSTM network and a full connection layer, which are used for end-to-end feature extraction and classification of original voltage signals and inertial sensor data; the multi-modal fusion feature extraction unit is used for integrating plantar pressure signals and inertial sensor data, and realizes cross-modal information fusion through a feature splicing and weight adaptive allocation mechanism; the bidirectional LSTM network comprises a forward LSTM layer and a backward LSTM layer, each layer containing 64 hidden units, which are used for capturing the spatio-temporal correlation features of gait signals; the attention mechanism layer is composed of a channel attention module and a spatial attention module in cascade, the channel attention realizes adaptive allocation of feature weights through a Squeeze-and-Excitation network structure, and the spatial attention extracts spatial correlation information of a feature map through a 7x1 convolution kernel, and realizes gait state recognition. The CNN analysis module further comprises a thickness deformation data acquisition unit, the thickness deformation data acquisition unit adopts a micro optical fiber grating sensor array to collect the thickness change of the insole in real time through the wavelength shift of light; the multi-modal fusion feature extraction unit is further used for integrating thickness deformation data, and realizes cross-modal information fusion through a feature splicing and dynamic weight allocation mechanism.
2. The system of claim 1, wherein, The flexible insole sensor module further comprises a temperature compensation unit, the temperature compensation unit adopts a parallel structure of a nichrome thin film resistor and a sensing unit, and eliminates the pressure detection error caused by the change of environmental temperature from -20 DEG C to 60 DEG C through a Wheatstone bridge circuit.
3. The system of claim 1, wherein, The CNN analysis module adopts a three-layer one-dimensional convolution structure, the convolution kernel sizes are 3x1, 5x1 and 7x1 respectively, the activation function adopts ReLU, and the optimizer is adaptive moment estimation (Adam).
4. The system of claim 1, wherein: The working process of the system comprises:
5. The system of claim 1, wherein: (1) training phase: the plantar pressure signals are divided into a training set and a test set in a ratio of 7:3, the CNN model is input after convolution, normalization and pooling, and six types of gait prediction results are output through a full connection layer; (2) test phase: the model inferences a new sample to output a prediction category, and compares it with a real label to realize accuracy evaluation. 6. The system of claim 3, wherein: The temperature compensation unit is made of a nickel-chromium alloy thin film resistor with a thickness of 50-200 nm and a square resistance value controlled in 100-500 Ω / □, which is deposited on a polyimide flexible substrate by a magnetron sputtering process.
7. The system of claim 1, wherein: The step factor of the LMS algorithm of the adaptive filtering module is set to 0.01-0.1, the iteration number is greater than or equal to 50 times, the convergence error is less than or equal to 0.001 mV, and the dynamic filtering of signals in a frequency band of 1-500 Hz is supported.
8. The system of claim 2, wherein: The micro fiber grating sensor array of the thickness deformation data acquisition unit contains three sensing points, which are respectively arranged in the forefoot, arch and heel regions, and synchronous acquisition is realized by using wavelength division multiplexing technology, and the sampling frequency is consistent with the pressure signal (greater than or equal to 1 kHz).
9. The system of claim 2, wherein: The multi-modal fusion feature extraction unit adopts a three-step processing procedure of "time series alignment-feature dimension expansion-weight dynamic allocation" for the thickness deformation data, wherein the thickness feature is dimensionally mapped (from 128 dimensions to 64 dimensions) by a 1x1 convolution kernel and then spliced with the pressure / inertial feature, and the thickness feature weight is adaptively adjusted by 0.12-0.28 through an attention mechanism layer.
10. The system of claim 1, wherein: The compression ratio of the channel attention mechanism is set to 16, the excitation function adopts Sigmoid, and the number of convolution kernels of the spatial attention mechanism is 64, forming a serial cascade structure with the channel attention mechanism.