Gait data acquisition monitoring system, machine learning algorithm and three-dimensional visualization method

Abstract

The invention relates to gait data acquisition and monitoring, in particular to a gait data acquisition and monitoring system, a machine learning algorithm and a three-dimensional visualization method. The gait data acquisition monitoring system comprises a sensor acquisition layer used for acquiring gait data; the machine learning layer can classify and predict the gait data; the data monitoring layer is used for carrying out two-dimensional visualization and three-dimensional visualization on the gait data; and the network transmission layer is used for connecting the sensor acquisition layer, the machine learning layer and the data monitoring layer. The wearable gait data acquisition and visual monitoring system based on multi-sensor fusion and the machine learning algorithm thereof provided by the invention have multiple beneficial effects that multi-modal, high-precision and omnibearing acquisition of gait information is realized by integrating three types of sensors of plantar pressure, limb movement posture and hand inertia; and the data integrity is obviously improved.

Description

Technical Field

[0001] The present invention relates to gait data collection and monitoring, and in particular to a gait data collection and monitoring system, a machine learning algorithm and a three-dimensional visualization method. Background Art

[0002] With the increasing incidence of movement disorders and the accelerating aging of the global population, sports health monitoring has become an important research area in preventive and rehabilitation medicine. Gait, the most fundamental human movement pattern, is the coordinated movement of various body parts during walking, conveying a wealth of physiological and pathological information. Characteristics of gait, such as its periodicity, symmetry, and coordination, can effectively reflect the function of the human musculoskeletal system, skeletal joints, and overall health, and therefore holds significant importance in sports health monitoring.

[0003] Gait information typically encompasses three dimensions: kinematics, dynamics, and biomechanics. With the development of inertial measurement units (IMUs), flexible pressure sensors, and electromyographic (EMG) signal acquisition technologies, the acquisition of multidimensional gait data has become possible. IMUs can capture motion trajectory and posture changes, flexible pressure sensors monitor changes in plantar pressure distribution, and EMG signals reflect the neuromuscular mechanisms and muscle coordination strategies underlying gait control. The combination of these sensor technologies offers broad application prospects for gait monitoring in areas such as medical rehabilitation, disease diagnosis, and sports training.

[0004] Although gait monitoring technology has made significant progress, it still faces many challenges in practical applications. In terms of data acquisition, inertial sensors have the advantage of portability, but they suffer from cumulative errors and long-term drift, which affect the accuracy and stability of the data. Traditional gait acquisition relies heavily on optical motion capture systems and force platforms in laboratory environments. These devices are expensive, complex to operate, and subject to environmental constraints, making them difficult to meet the needs of daily health monitoring. Although optical systems are highly accurate, they are sensitive to lighting conditions and occlusion, and their high cost limits their widespread adoption.

[0005] Therefore, given the shortcomings of existing wearable gait monitoring technologies in terms of data integrity, acquisition accuracy, and widespread adoption, there is an urgent need to develop a wearable gait monitoring system based on multi-sensor fusion. This system should be able to collect multimodal gait data in real time, implement data filtering and task identification and classification on an embedded platform, and present two-dimensional monitoring results and three-dimensional motion collaborative analysis through data visualization technology. This will improve the practicality and accuracy of gait monitoring and promote its application in health management and rehabilitation medicine. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to overcome the deficiencies in the prior art and provide a gait data acquisition and monitoring system, a machine learning algorithm and a three-dimensional visualization method.

[0007] The present invention is achieved through the following technical solutions:

[0008] A gait data collection and monitoring system, the system comprising:

[0009] Sensor collection layer, used to collect gait data;

[0010] A machine learning layer capable of classifying and predicting the gait data;

[0011] A data monitoring layer, configured to perform two-dimensional and three-dimensional visualization of the gait data;

[0012] The network transmission layer is used to connect the sensor acquisition layer, the machine learning layer and the data monitoring layer.

[0013] Preferably, the sensor acquisition layer includes a plantar pressure signal acquisition sensor, a limb movement posture signal acquisition sensor, and a hand inertia signal acquisition sensor.

[0014] A machine learning algorithm is deployed in the machine learning layer of the gait data collection and monitoring system described above, and includes the following steps:

[0015] S1, data collection;

[0016] S2, filtering and normalizing the data;

[0017] S3, performing segmentation processing on the data;

[0018] S4, performing feature extraction on the data;

[0019] S5, train the SVM classifier based on the features extracted in S4;

[0020] S6, deploying the feature extraction algorithm and the SVM classifier in the machine learning layer.

[0021] Preferably, step S1 includes the following steps: collecting gait data of multiple actions, wherein the actions include walking straight, turning left, turning right, going upstairs, going downstairs, standing up and sitting down.

[0022] Preferably, step S2 includes the following steps:

[0023] The data collected by S1 is filtered using a Kalman filter;

[0024] The filtered data is normalized by the minimum and maximum normalization method. The normalization formula is:

[0025] .

[0026] Preferably, step S3 includes the following steps:

[0027] Based on the sliding window method, gait data segmentation is performed on the data processed in step S2.

[0028] Preferably, step S4 includes the following steps:

[0029] Extract time domain signal features and frequency domain signal features;

[0030] The PCA method is used to perform combined optimization on the time domain signal features and the frequency domain signal features to obtain 7 main features.

[0031] Preferably, step S5 includes the following steps:

[0032] Divide the training set and test set according to the main features obtained by S4;

[0033] The SVM classifier is trained using the training set and the test set.

[0034] A three-dimensional visualization method is provided. The three-dimensional visualization method is used for the data monitoring layer in the gait data acquisition and monitoring system described above. The three-dimensional visualization method comprises the following steps:

[0035] The quaternion method is used for attitude calculation, and the first-order Runge-Kutta method is used to solve the quaternion:

[0036]

[0037] in:

[0038]

[0039]

[0040] Substituting into the quaternion formula we get:

[0041]

[0042] Among them: The initial values ​​are 1, 0, 0, 0 respectively. , , The body coordinate axis axis, axis, axis angular velocity, is the sampling period;

[0043] According to the quaternion formula, the Euler angle can be obtained:

[0044]

[0045] in, 、 、 They are roll angle, pitch angle, and yaw angle respectively;

[0046] The quaternion results are respectively solved by the posture solving method for the multiple sensor raw data received in real time, and are input into the control blueprint of each part of the limb of the host computer, so that the robot can display the movement posture in real time.

[0047] The beneficial effects of the present invention are as follows: a multi-sensor fusion wearable gait data acquisition and visualization monitoring system and its machine learning algorithm provided by the present invention have multiple beneficial effects: by integrating three types of sensors, namely plantar pressure, limb movement posture and hand inertia, multimodal, high-precision and all-round acquisition of gait information is achieved, significantly improving data integrity; embedded packaging and wireless communication technology are adopted to ensure that the system has good real-time, portability and low power consumption performance; Kalman filtering, normalization, sliding window segmentation and PCA+SVM algorithm are combined to achieve efficient feature extraction and accurate gait classification prediction; two-dimensional graphics and three-dimensional virtual simulation methods based on quaternion posture solution are introduced to construct a real-time dynamic display of human motion model, enhancing visualization effect and user interaction experience; at the same time, the system has good modularity and scalability, and can flexibly adapt to different application scenarios, breaking through the technical bottlenecks of existing wearable gait monitoring systems in terms of difficult signal fusion, low precision and weak visualization capabilities, and providing an efficient, reliable and intelligent solution for rehabilitation medicine, smart elderly care and sports health monitoring and other fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 This is a schematic diagram of the overall structure of a multi-sensor fusion wearable gait data acquisition and visualization monitoring system provided by the present invention;

[0049] Figure 2 This is a structural diagram of the sensor acquisition layer provided by the present invention;

[0050] Figure 3 This is a timing diagram of I2C data reception involved in the present invention;

[0051] Figure 4 This is a flow chart of the gait experiment involved in the present invention;

[0052] Figure 5 Schematic diagram of SVM classification involved in the present invention;

[0053] Figure 6 This is a two-dimensional and three-dimensional visualization host computer interface diagram involved in the present invention. DETAILED DESCRIPTION

[0054] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is further described in detail below with reference to the accompanying drawings and the best embodiment. Based on the embodiments of the invention, all other embodiments obtained by those skilled in the art without making any creative work shall fall within the scope of protection of the invention.

[0055] In the description of the invention, it should be noted that the terms "center", "up", "down", "left", "right", "vertical", "horizontal", "inside", "outside", etc., indicating directions or positional relationships, are based on the directions or positional relationships shown in the accompanying drawings. They are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operated in a specific direction. Therefore, they cannot be understood as limiting the invention.

[0056] Furthermore, it should be noted that, in the description of the present invention, unless otherwise expressly specified or limited, the terms "installed," "disposed," and "connected" should be understood in a broad sense. For example, they may refer to fixed connections, detachable connections, or integral connections; they may refer to direct connections, indirect connections through an intermediate medium, or internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on the specific circumstances.

[0057] The present invention provides a multi-sensor fusion wearable gait data acquisition and visualization monitoring system, which includes a sensor acquisition layer, a network transmission layer, a machine learning layer and a data monitoring layer.

[0058] The sensor acquisition layer is used to collect gait data. The sensor acquisition layer specifically includes a foot pressure signal acquisition sensor, a limb movement posture signal acquisition sensor, and a hand inertia signal acquisition sensor.

[0059] The plantar pressure signal acquisition sensor uses an FSR402 thin-film pressure sensor made of flexible material. Several thin-film pressure sensors are placed at key points on the sole of the foot and fixed to the insole to create a plantar pressure acquisition insole. The main control unit is an STM32F103C8T6. Its working circuit converts the plantar pressure signal into an electrical signal. Using AD acquisition, the analog electrical signal is converted into a digital signal that is input to the main control unit. DMA transmission reduces the main control unit's processing time and corresponding delays, ensuring real-time data transmission.

[0060] The limb motion and posture signal acquisition sensor uses a six-axis inertial sensor to capture inertial signals, including a three-axis gyroscope and a three-axis accelerometer. The inertial sensor is integrated with the STM32F103C8T6 peripheral circuit to collect three-dimensional linear acceleration and three-dimensional angular velocity values ​​from a single node. These values ​​are then transmitted to a relay node via a wireless transceiver module for data integration, ultimately detecting the current motion state.

[0061] The hand inertial signal acquisition sensor uses an inertial sensor to obtain information about subtle hand movements. Its main control unit is STM32F103C8T6. The acquisition node communicates with the main control unit in a rotating manner through the simulated I2C protocol to monitor finger movements.

[0062] The limb movement posture signal acquisition and hand inertia signal acquisition sensors both use the I2C protocol to communicate with the microcontroller. A start signal is set before data transmission, a stop signal is set at the end of transmission, and a response signal is set after receiving data.

[0063] The network transmission layer connects the sensor acquisition layer to the machine learning layer and the data monitoring layer via 2.4GHz wireless transmission. Data and command acquisition and transmission are achieved through 2.4GHz wireless transmission, utilizing the STM32 series processors as signal transmission relay nodes. Specifically, an STM32F103C8T6 is deployed as a transmitting node at the transmitting end of each subsystem in the sensor acquisition layer (one for each foot, one for plantar pressure signal acquisition sensors, one for limb motion and posture signal acquisition sensors, and one for each hand, for a total of five). An STM32F103ZET6 is deployed as a receiving node at the receiving end of the data monitoring layer. The machine learning layer is connected to the data monitoring layer via a wired connection to ensure stable reception of sensor data and perform relevant calculations. To ensure real-time data transmission and reception, data transmission is performed in turns using DMA.

[0064] Furthermore, the network transport layer includes three operating modes: power-down, standby, and transceiver. In power-down mode, all network transport layer functions are disabled to minimize current consumption and save the last transmitted data to registers. In standby mode, the clock buffer operates and waits for instructions, ensuring fast startup while reducing the average system current consumption. Upon receiving a transceiver instruction, the system switches to transceiver mode and receives and sends data according to the instruction.

[0065] The machine learning layer deploys machine learning algorithm into embedded platform, adopts 32-bit single-chip microcomputer STM32F103ZET6 as microprocessor unit, the microprocessor frequency is 72MHz, and 512-kB embedded flash memory is built-in, 4kB of which is used for storing intermediate motion data for feature extraction, online fast calculation of time-frequency domain features and nonlinear functions by using embedded mathematical functions in DSP library, classification and prediction of gait task, and transmission of results to data monitoring layer upper computer platform through network transmission layer.

[0066] Further, the machine learning layer deploying machine learning algorithm into embedded platform specifically comprises the following steps:

[0067] Step one, based on daily activities, design experiments to collect sensor data, and the experiment is specifically divided into four parts:

[0068] (1) Walking. The subject first stands for 10 seconds, and then walks straight for 10 meters in a fixed direction after hearing the instruction, and then turns around to return to the original place;

[0069] (2) Going up and down the stairs. The subject goes down 20 steps and goes up to return to the original place;

[0070] (3) Sit and stand. The subject sits on a chair, and after hearing the instruction, repeats standing up and sitting down 5 times;

[0071] (4) Turning around. The subject turns around the chair, first left for 5 turns, and then right for 5 turns.

[0072] The four experiments can be divided into seven kinds of gait movements, including straight walking, left turning, right turning, going upstairs, going downstairs, standing up, and sitting down, and each experiment is repeated 10 times.

[0073] Step two, filtering and normalizing the collected data. In order to eliminate the influence of noise data, Kalman filter is used to filter the original data; in order to eliminate the numerical scale imbalance problem caused by the difference of measurement units between different features, minimum maximum normalization method is used to normalize the data obtained by filtering, and the normalization formula is:

[0074]

[0075] Step three, the time series data obtained by the above processing is segmented, and the sliding window based method is used for gait data segmentation, with 1 second as the time interval (including a complete gait cycle), and the overlap rate between gait data frames is set to 50%.

[0076] Step 4: Perform feature extraction on the data after segmentation in step 3. First, extract 6 time domain signal features (mean, peak, peak-to-peak value, standard deviation, energy, and skewness) and 5 frequency domain signal features (mean frequency, peak frequency, center of gravity frequency, frequency standard deviation, and frequency root mean square). Then, use the PCA method to combine and optimize the above features to achieve the purpose of dimensionality reduction. The specific steps of PCA processing are as follows:

[0077] (1) Write the obtained data into the following matrix form:

[0078]

[0079] (2) Standardize the original data, with the number of sample acquisitions as rows and the signal features as columns, and write the standardized features obtained from the training set data into a matrix In the form of:

[0080]

[0081] (3) Using the standardized data to calculate the correlation matrix:

[0082]

[0083] (4) Solve the characteristic equation:

[0084]

[0085] in, is the unit matrix, and the m characteristic roots are .

[0086] (5) Solve for the unit eigenvector corresponding to the feature, and we get:

[0087]

[0088] (6) Principal component analysis expression:

[0089]

[0090] The selection criteria for selecting 80% comprehensive index contribution rate as the main component:

[0091]

[0092] For each gait task, the data is cut into time windows, and the features of the three signals in each part are extracted separately. The calculated eigenvalues ​​are arranged from large to small to obtain the first 7 features. The eigenvectors corresponding to the eigenvalues ​​are calculated, and then the corresponding 7 principal components are obtained.

[0093] Step 5: Based on the seven features selected by the PCA analysis above, the training set and the test set are divided and the first seven features are used to train the SVM classifier. Plantar pressure, limb movement, and hand inertia represent the three subsystems, and two-class classification and seven-class classification are performed respectively. The data of the three subsystems are then fused two by two and the same operation is performed. Finally, the three subsystems are fused to classify, identify, and predict all seven gait movements.

[0094] In step six, the trained SVM classifier and feature extraction algorithm are deployed on the STM32F103ZET6, the main control chip in the machine learning layer. The mathematical functions embedded in its DSP library are used to rapidly calculate time-frequency domain features and nonlinear functions, accelerating the motion state classification process. The calculated features and classification prediction results are transmitted to the data monitoring layer via a wired connection to the network transmission layer for visualization.

[0095] The data monitoring layer includes two-dimensional visualization and three-dimensional visualization. It receives the collected data through the network transmission layer and draws two-dimensional line graphs and distribution graphs. The included three-dimensional human body model receives data in real time, performs posture calculation and realizes motion capture.

[0096] The two-dimensional visualization includes a serial port channel, which can receive data sent by multiple acquisition nodes, perform base-to-base conversion, and store the data. It can receive and display the acquired data in real time on the host computer, display the data features and classification prediction results calculated by the machine learning layer in real time, and draw data fluctuation line graphs and pressure distribution graphs.

[0097] 3D visualization uses quaternion method for attitude solution, and the first-order Runge-Kutta method is used to solve the quaternion:

[0098]

[0099] in:

[0100]

[0101]

[0102] Substituting into the quaternion formula we get:

[0103]

[0104] Among them: The initial values ​​are 1, 0, 0, 0 respectively. , , The body coordinate axis axis, axis, axis angular velocity, is the sampling period.

[0105] According to the quaternion formula, the Euler angle can be obtained:

[0106]

[0107] in, 、 、 They are roll angle, pitch angle, and yaw angle respectively.

[0108] The quaternion results are respectively solved by the posture solving method for the multiple sensor raw data received in real time, and are input into the control blueprint of each part of the limb of the host computer, so that the robot can display the movement posture in real time.

[0109] The host computer display interface consists of serial port settings, a limb motion signal display unit, a hand inertia signal display unit, a foot pressure signal unit, and a 3D human body model unit. Data reception can be configured in different receiving modes based on the device's transmission rate. The host computer displays collected data in real time and plots the corresponding data waveforms based on the selected nodes. The 3D human body model captures motion based on limb motion node signals and displays the classification eigenvalues ​​of the embedded SVM classifier and the current gait motion recognition and prediction results in real time. This provides a better visual monitoring platform for daily gait activity collection.

[0110] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A gait data collection and monitoring system, characterized in that: include: Sensor collection layer, used to collect gait data; A machine learning layer capable of classifying and predicting the gait data; A data monitoring layer, configured to perform two-dimensional and three-dimensional visualization of the gait data; The network transmission layer is used to connect the sensor acquisition layer, the machine learning layer and the data monitoring layer.

2. A gait data collection and monitoring system according to claim 1, characterized in that: The sensor acquisition layer includes a foot pressure signal acquisition sensor, a limb movement posture signal acquisition sensor, and a hand inertia signal acquisition sensor.

3. A machine learning algorithm, characterized in that The machine learning algorithm is deployed in the machine learning layer of the gait data collection and monitoring system according to any one of claims 1 to 2, and the machine learning algorithm includes the following steps: S1, data collection; S2, filtering and normalizing the data; S3, performing segmentation processing on the data; S4, performing feature extraction on the data; S5, train the SVM classifier based on the features extracted in S4; S6, deploying the feature extraction algorithm and the SVM classifier in the machine learning layer.

4. The machine learning algorithm according to claim 3, wherein: Step S1 includes the following steps: collecting gait data of multiple actions, wherein the actions include walking straight, turning left, turning right, going upstairs, going downstairs, standing up and sitting down.

5. The machine learning algorithm according to claim 3, wherein: Step S2 includes the following steps: The data collected by S1 is filtered using a Kalman filter; The filtered data is normalized by the minimum and maximum normalization method. The normalization formula is: 。 6. The machine learning algorithm according to claim 3, wherein: Step S3 includes the following steps: Based on the sliding window method, gait data segmentation is performed on the data processed in step S2.

7. The machine learning algorithm according to claim 3, wherein: Step S4 includes the following steps: Extract time domain signal features and frequency domain signal features; The PCA method is used to perform combined optimization on the time domain signal features and the frequency domain signal features to obtain 7 main features.

8. The machine learning algorithm according to claim 7, wherein: Step S5 includes the following steps: Divide the training set and test set according to the main features obtained by S4; The SVM classifier is trained using the training set and the test set.

9. A three-dimensional visualization method, characterized in that: The three-dimensional visualization method is used for the data monitoring layer in the gait data acquisition and monitoring system according to any one of claims 1 to 2, and the three-dimensional visualization method comprises the following steps: The quaternion method is used for attitude calculation, and the first-order Runge-Kutta method is used to solve the quaternion: in: Substituting into the quaternion formula we get: Among them: The initial values ​​are 1, 0, 0, 0 respectively. , , The body coordinate axis axis, axis, axis angular velocity, is the sampling period; The Euler angle can be obtained according to the quaternion formula: in, 、 、 They are roll angle, pitch angle, and yaw angle respectively; The quaternion results are respectively solved by the posture solving method for the multiple sensor raw data received in real time, and are input into the control blueprint of each part of the limb of the host computer, so that the robot can display the movement posture in real time.

Images