Bipedal gait recognition system and method based on multi-sensor fusion

Abstract

The present invention relates to the field of human body motion posture recognition, and in particular to a bipedal gait recognition system and method based on multi-sensor fusion. According to the system, a corresponding gait deep learning LSTM neural network model is obtained by training, and the deep learning LSTM neural network model is stored in a computer. Motion information or data of a human body collected by using the bipedal gait recognition system of the present application is transmitted into the trained deep learning LSTM neural network model, and the current motion state of a user is recognized. The system of the present application collects human body motion information by means of direct contact with a human body, so that the system can conveniently and accurately collect motion information, is easy to carry, is suitable for most motion scenarios and has wide development prospects.

Description

A multi-sensor fusion bipedal gait recognition system and method Technical Field

[0001] The present invention relates to the field of human motion posture recognition, and in particular to a multi-sensor fusion bipedal gait recognition system. Background Art

[0002] In recent years, with the continuous development and improvement of MEMS sensor technology, especially its miniaturization and functional diversification, MEMS sensors have been widely used in many fields, such as identity recognition, medical rehabilitation, sports training, fatigue detection, and gait recognition. Using a single type of sensor cannot fully demonstrate the motion characteristics of human gait. Compared with computer image recognition, signal recognition based on multi-sensor fusion can more flexibly and accurately capture information during movement, and has significant advantages such as low cost, low power consumption, portability, and comfort.

[0003] However, multi-sensor acquisition equipment has poor portability and requires the subject to hold the device. The acquisition process is easily interfered by wires, and the synchronization and real-time performance of data acquisition are not high. In addition, the subsequent data processing and feature extraction make it difficult to more accurately analyze and identify the human body's movement status.

[0004] Therefore, designing a wearable multi-sensor fusion gait recognition system will help to better identify human posture and study motion status. Technical issues

[0005] The embodiments of the present invention provide a multi-sensor fusion bipedal gait recognition system and method, which at least solve the problems of poor portability of existing multi-sensor acquisition equipment and low synchronization and real-time performance of data acquisition. Technical Solutions

[0006] According to one embodiment of the present invention, a multi-sensor fusion bipedal gait recognition system is provided, comprising:

[0007] The sensor is used to collect human motion information in real time by contacting the human foot. The human motion information includes the state information of the human foot sole and the posture information of the human body during movement;

[0008] The main control module is used to obtain motion information, synchronize and pre-process the motion information, and obtain motion data;

[0009] The host computer is used to obtain motion data and perform motion recognition on the motion data through a deep learning LSTM neural network model to determine the current state of human motion;

[0010] Sensors include:

[0011] The plantar pressure sensor module is used to obtain the status information of the plantar pressure of the human body, including the plantar pressure of the left foot and the plantar pressure of the right foot;

[0012] The nine-axis sensor module is used to obtain human posture information during movement.

[0013] Furthermore, the multi-sensor fusion bipedal gait recognition system also includes:

[0014] The Bluetooth module is used to transmit data. The main control module sends the motion data to the host computer through the Bluetooth module.

[0015] Furthermore, the host computer includes:

[0016] Normalization module, used to normalize the motion data obtained by the host computer;

[0017] The filter preprocessing module is used to perform filter preprocessing on the normalized motion data.

[0018] Furthermore, the deep learning LSTM neural network model specifically includes:

[0019] Collect the original motion information for preprocessing, and use the preprocessed motion data as a data set;

[0020] 80% of the data in the dataset is used as the training set and 20% of the data is used as the test set;

[0021] Build an LSTM neural network model, where the shape of the LSTM neural network model input data is set to 34, the number of hidden neurons is set to 64, the number of LSTM unit layers is 2, and each training is fed to the LSTM neural network model with a sample size of 20 units, the number of iterations is set to 30, and the learning rate is set to 0.001.

[0022] A multi-sensor fusion bipedal gait recognition method, the method comprising the following steps:

[0023] Collect human motion information in real time by contacting the human foot. The human motion information includes the status information of the human foot sole and the posture information during the human movement;

[0024] Acquire motion information, and perform data synchronization and preprocessing on the motion information to obtain motion data;

[0025] Obtain motion data and perform motion recognition on the motion data through a deep learning LSTM neural network model to determine the current state of human motion;

[0026] Real-time collection of human motion information, including the status information of the human foot and the posture information during human motion, specifically including:

[0027] Plantar pressure sensors are arranged at different positions on the soles of human feet in the form of an array, and status information of the human soles is obtained through the plantar pressure sensors. The status information of the human soles includes plantar pressure information, and the pressure information includes plantar pressure of the left foot and plantar pressure of the right foot. The plantar pressure sensors are provided with a Wheatstone bridge and an operational amplifier circuit.

[0028] The precise voltage value is obtained through the Wheatstone bridge and operational amplifier circuit. The pressure information corresponds to the voltage value, and the obtained voltage value is transmitted to the main control module.

[0029] A nine-axis sensor module is set at the ankle joint of the human leg to capture the posture information of the human body during movement. The posture information during human movement includes the movement posture of the human leg, and the data is transmitted to the main control module through the I2C communication protocol.

[0030] Furthermore, the accurate voltage value is obtained through the Wheatstone bridge and operational amplifier circuit. The pressure information corresponds to the voltage value. After the obtained voltage value is transmitted to the main control module, the following steps are included:

[0031] The analog-to-digital conversion is performed through the 12-bit ADC inside the main control module to obtain the analog voltage value;

[0032] Convert the analog voltage value into an actual digital value, including the digital value of the left foot plantar pressure and the digital value of the right foot plantar pressure;

[0033] The main control module synchronously summarizes the digital quantities of the left foot plantar pressure and the right foot plantar pressure.

[0034] Furthermore, motion data is acquired and motion recognition is performed on the motion data using a deep learning LSTM neural network model to determine the current state of human motion, specifically including:

[0035] The main control module transmits the aggregated digital quantity to the host computer;

[0036] The host computer saves and processes the received digital quantity, including normalization and filtering preprocessing;

[0037] The processed data is identified and matched with features through a deep learning LSTM neural network model, thereby identifying the motion status information of the current athlete and determining the current state of human motion.

[0038] Furthermore, the deep learning LSTM neural network model specifically includes:

[0039] Collect the original motion information for preprocessing, and use the preprocessed motion data as a data set;

[0040] 80% of the data in the dataset is used as the training set and 20% of the data is used as the test set;

[0041] Build an LSTM neural network model, where the shape of the LSTM neural network model input data is set to 34, the number of hidden neurons is set to 64, the number of LSTM unit layers is 2, and each training is fed to the LSTM neural network model with a sample size of 20 units, the number of iterations is set to 30, and the learning rate is set to 0.001.

[0042] A computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in any of the multi-sensor fusion bipedal gait recognition methods described above.

[0043] A terminal device comprises: a processor, a memory and a communication bus; the memory stores a computer-readable program that can be executed by the processor;

[0044] The communication bus realizes the connection and communication between the processor and the memory;

[0045] When the processor executes the computer-readable program, the processor implements the steps of any one of the above-mentioned methods for multi-sensor fusion bipedal gait recognition. Beneficial effects

[0046] The multi-sensor fusion bipedal gait recognition system and method in the embodiment of the present invention is based on the training of the corresponding gait deep learning LSTM neural network model, and the deep learning LSTM neural network model is saved to the computer. The bipedal gait recognition system of the present application is used to collect the motion information of the human body or the data is transferred into the trained deep learning LSTM neural network model to identify the current motion state of the user. The system of the present application collects human motion information by direct contact with the human body. The collected motion information is convenient and accurate, easy to carry, applicable to most sports scenes, and has broad development prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0048] FIG1 is a schematic diagram of a multi-sensor fusion bipedal gait recognition system according to the present invention;

[0049] FIG2 is a schematic diagram of a preferred principle of a multi-sensor fusion bipedal gait recognition system according to the present invention;

[0050] FIG3 is a diagram of a Wheatstone bridge and differential amplifier circuit designed according to the present invention;

[0051] FIG4 is a flow chart of tasks established after transplanting a real-time operating system according to the present invention;

[0052] FIG5 is a confusion matrix of the motion recognition results of the present invention;

[0053] FIG6 is a flow chart of a bipedal gait recognition method based on multi-sensor fusion according to the present invention;

[0054] FIG7 is a diagram of a terminal device according to the present invention.

[0055] Reference numerals:

[0056] 100 - sensor 100 , 101 - plantar pressure sensor module 101 , 102 - nine-axis sensor module 102 , 200 - main control module 200 , 300 - host computer 300 , 400 - Bluetooth module 400 , 500 - power management module 500 . Modes for Carrying Out the Invention

[0057] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0058] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0059] Example 1

[0060] According to one embodiment of the present invention, a multi-sensor fusion bipedal gait recognition system is provided, as shown in Figures 1 and 2, including:

[0061] The sensor 100 is used to collect human motion information in real time by contacting the human foot. The human motion information includes the state information of the human foot sole and the posture information of the human body during movement;

[0062] The main control module 200 is used to obtain motion information, synchronize and pre-process the motion information, and obtain motion data;

[0063] The host computer 300 is used to obtain motion data and perform motion recognition on the motion data through a deep learning LSTM neural network model to determine the current state of human motion;

[0064] The sensor 100 includes:

[0065] The plantar pressure sensor module 101 is used to obtain the status information of the plantar pressure of the human body, including the plantar pressure of the left foot and the plantar pressure of the right foot;

[0066] The nine-axis sensor module 102 is used to obtain posture information of the human body during movement.

[0067] The corresponding gait deep learning LSTM neural network model is trained and saved to a computer. The bipedal gait recognition system of the present application collects human motion information or data into the trained deep learning LSTM neural network model to identify the user's current motion state. The system of the present application collects human motion information by direct contact with the human body. The motion information is collected conveniently and accurately, is easy to carry, is applicable to most sports scenes, and has broad development prospects.

[0068] Specifically, the system of the present application includes a plantar pressure sensor module 101, a nine-axis sensor module 102, a main control module 200, a Bluetooth module 400, a power management module 500 and a host computer 300.

[0069] First, a plantar pressure sensor module 101 and a nine-axis sensor module 102 are placed on the sole of the foot to obtain information about the plantar pressure and the body's posture during exercise. The nine-axis sensor 100 is an MPU9250, which features a 3-axis gyroscope, 3-axis accelerometer, and 3-axis magnetometer. The plantar pressure sensor is a flexible pressure sensor.

[0070] The plantar pressure sensor module 101 collects pressure changes on the sole of the foot and generates corresponding voltage changes through a Wheatstone bridge circuit and a differential operational amplifier circuit. The schematic diagram is shown in Figure 3. Resistors R1, R10, R4, and RX form the bridge circuit; resistors R3, R5, R6, R7, and operational amplifier U1 form the differential circuit; and R8, R9, RP1, and operational amplifier U2 form the amplifier circuit. RP1 is an adjustable resistor that adjusts the voltage amplification factor. R1, R10, and R4 are standard resistors, and RX is the plantar pressure sensor. When pressure is applied to the sole of the foot, the resistance value of RX changes, which is converted into a voltage change through the bridge circuit. The voltage across the two ends then passes through the differential circuit to reset the actual voltage change from 0. The collected voltage is then amplified by the amplifier circuit and finally sent to the STM32 via the ADC for reading. The specific plantar pressure value can be read by the built-in 12-bit ADC of the main control module 200. The nine-axis sensor module 102 communicates with the STM32 main control module 200 through the I2C protocol to obtain the values ​​of the 3-axis gyroscope, 3-axis acceleration and 3-axis magnetometer.

[0071] The main control module 200 is an STM32 chip of model STM32F103C8T6, which is based on the Cortex-M3 core architecture and has a 32-bit ARM core-based built-in 64KB bytes of programmable flash memory (FLASH) and 20KB bytes of RAM.

[0072] Bluetooth module 400 uses the Huicheng HC-04 Bluetooth module as the wireless communication module. Main control module 200 aggregates the left foot's overall data and transmits it to the right foot's STM32 via Bluetooth module 400. The right foot's STM32 synchronizes the data between the two feet using the FREERTOS real-time operating system and transmits it to host computer 300 via Bluetooth module 400. The task flow chart established by this operating system is shown in Figures 2 and 4.

[0073] The power management module 500 is powered by a 3.7V lithium battery and has an internal chip IP5306. The IP5306 is a multifunctional power management chip that mainly has a boost function, lithium battery charging protection, and a lithium battery remaining power display function.

[0074] The host computer 300 includes:

[0075] A normalization module, used for normalizing the motion data acquired by the host computer 300;

[0076] The filter preprocessing module is used to perform filter preprocessing on the normalized motion data.

[0077] After the motion data is transmitted to the host computer 300 via the Bluetooth module 400, the transmitted data is directly saved, normalized, and filtered for preprocessing via Python.

[0078] The data that has been normalized and filtered is directly fed into the deep learning LSTM neural network model trained by deep learning for action recognition.

[0079] The design method of the deep learning LSTM neural network model is:

[0080] Raw data was collected and preprocessed, and the preprocessed motion data was used as the dataset. 80% of the data was used as the training set, and 20% as the test set. An LSTM neural network model was constructed. The input data shape of the LSTM neural network model was set to 34, the number of hidden neurons was set to 64, the number of LSTM unit layers was set to 2, and each training session fed the neural network a sample size of 20 units, the number of iterations was set to 30, and the learning rate was set to 0.001.

[0081] Example 2

[0082] According to another embodiment of the present invention, a multi-sensor fusion bipedal gait recognition method is provided, as shown in Figures 2 and 6, including:

[0083] S100: collecting human motion information in real time by contacting the human foot, the human motion information including state information of the human foot sole and posture information during the human motion process;

[0084] S200: Acquire motion information, and perform data synchronization and preprocessing on the motion information to obtain motion data;

[0085] S300: Acquire motion data and perform motion recognition on the motion data using a deep learning LSTM neural network model to determine the current state of human motion;

[0086] Real-time collection of human motion information, including the status information of the human foot and the posture information during human motion, specifically including:

[0087] Plantar pressure sensors 100 are arranged in an array at different locations on the sole of a human foot. The plantar pressure sensors 100 are used to obtain status information of the human sole. The status information of the human sole includes plantar pressure information, and the pressure information includes plantar pressure of the left foot and plantar pressure of the right foot. The plantar pressure sensors 100 are provided with a Wheatstone bridge and an operational amplifier circuit.

[0088] The precise voltage value is obtained through the Wheatstone bridge and operational amplifier circuit. The pressure information corresponds to the voltage value, and the obtained voltage value is transmitted to the main control module 200;

[0089] A nine-axis sensor module 102 is set at the ankle joint of the human leg. The nine-axis sensor 102 is set to capture the posture information of the human body during movement. The posture information during human body movement includes the movement posture of the human leg, and data is transmitted with the main control module 200 through the I2C communication protocol.

[0090] The present invention integrates a bipedal acquisition system of a plantar pressure sensor 100 and a nine-axis inertial sensor 100, and adds a real-time operating system (FreeRTOS) to improve the synchronization and real-time performance of the entire acquisition system. The motion data of a normal person walking in place, walking on flat ground, going up and down stairs, standing and sitting are collected, and then the collected data is preprocessed, and the LSTM neural network model is trained using a deep learning method to obtain a corresponding gait model, and the model is saved to a computer. After the motion data of the human body is collected by the bipedal acquisition system of the present application and transmitted to the trained LSTM neural network model through the Bluetooth module 400, the user's current motion state can be identified. The overall recognition rate of this system can reach 98%, and the single motion recognition rate can reach more than 96%. This system is small in size, low in cost, easy to carry, suitable for most sports scenes, and has broad development prospects.

[0091] In this embodiment, motion data is acquired and motion recognition is performed on the motion data using a deep learning LSTM neural network model to determine the current state of human motion. Specifically, the following are included:

[0092] The main control module 200 transmits the summarized digital quantity to the host computer 300;

[0093] The host computer 300 stores and processes the received digital quantity, wherein the processing includes normalization and filtering preprocessing;

[0094] The processed data is identified and matched with features through a deep learning LSTM neural network model, thereby identifying the motion status information of the current athlete and determining the current state of human motion.

[0095] The design method of the deep learning LSTM neural network model is:

[0096] Raw data was collected and preprocessed, and the preprocessed motion data was used as the dataset. 80% of the data was used as the training set, and 20% as the test set. An LSTM neural network model was constructed. The input data shape of the LSTM neural network model was set to 34, the number of hidden neurons was set to 64, the number of LSTM unit layers was set to 2, and each training session fed the neural network a sample size of 20 units, the number of iterations was set to 30, and the learning rate was set to 0.001.

[0097] The present application is described in detail below through specific embodiments:

[0098] Step 1: Design the plantar pressure sensor 100 acquisition circuit and the nine-axis sensor 100 acquisition circuit to collect human motion information in real time by contacting the human foot:

[0099] The plantar pressure sensor 100 is arranged in an array at different positions on the sole of the human foot to obtain plantar pressure information. A Wheatstone bridge and an operational amplifier circuit are designed in the plantar pressure sensor 100 to obtain an accurate voltage value. The output pin is connected to the ADC input port of the STM32 (main control module 200). The nine-axis sensor 100 is set at the ankle joint to capture the human leg movement posture. Data is transmitted with the STM32 through the I2C communication protocol, and the movement data is sent to the STM32.

[0100] The plantar pressure sensor module 101 collects the pressure changes on the plantar surface and obtains the corresponding voltage changes through the Wheatstone bridge circuit and the differential operation circuit. The circuit schematic is shown in FIG3 .

[0101] The specific plantar pressure value can be read by the built-in 12-bit ADC of the main control module 200. The nine-axis sensor module 102 communicates with the STM32 main control module 200 through the I2C protocol to obtain the values ​​of the 3-axis gyroscope, 3-axis acceleration and 3-axis magnetometer, see Figure 2.

[0102] Step 2: Build a data acquisition program to transfer motion information or motion data to the STM32 for data synchronization and preprocessing:

[0103] The serial port baud rate of the STM32 is set to 115200, with 1 stop bit and no parity bit. After collecting the plantar pressure value, the analog-to-digital conversion is performed through the internal 12-bit ADC of the STM32 to convert the analog voltage value into an actual digital quantity. The I2C communication interface in the STM32 is configured to use fast mode and a 7-bit address length. FreeRTOS (real-time operating system) is ported for information synchronization between STM32 in this application system. The sampling frequency of both feet is 20 Hz / s.

[0104] The STM32 aggregates the data of the left foot and transmits it to the right foot STM32 via the Bluetooth module 400. The right foot STM32 synchronizes the data between the two feet via the FreeRTOS system and sends it to the host computer 300 via the Bluetooth module 400. The task flow chart established by the operating system is shown in Figures 2 and 4.

[0105] Step 3: Data transmission and motion recognition on host computer 300:

[0106] After STM32 synchronizes the bipedal data, it transmits the data to the computer (host computer 300) in real time through the serial port via the Bluetooth module 400. The computer directly saves, normalizes, and pre-processes the received data through Python.

[0107] The processed data is then identified and matched using a previously trained deep learning model (a deep learning LSTM neural network model) to determine the current athlete's motion status. The design method for the deep learning LSTM neural network model is described above and will not be further elaborated here.

[0108] In order to verify the effect of the present invention, the inventors also conducted the following test examples:

[0109] The experiment involved eight healthy subjects (four men and four women) with no motor impairments, a weight range of 65 ± 10 kg, and a height range of 170 ± 15 cm. Subjects were informed of the specific experimental items and requirements beforehand. The experimental items primarily covered six common movements: static movements including standing and sitting; and dynamic movements including walking in place, climbing stairs, descending stairs, and walking on flat ground. The data acquisition equipment used was the wearable data acquisition system designed by this invention. The confusion matrix for recognition is shown in Figure 5. The confusion matrix shows that all 116 test set samples for walking in place, walking on flat ground, and descending stairs were recognized, with a recognition accuracy of 100%. Two samples of the stair climbing movement were misidentified as descending stairs, with a recognition accuracy of 98.28%. Four samples of the standing movement were misidentified as walking in place, with a recognition accuracy of 96.55%. Three samples of the sitting movement were misidentified as climbing stairs, with a recognition accuracy of 97.41%. The overall recognition accuracy of the model is 98.71%, and the recognition rate of a single movement reaches over 96%.

[0110] The present invention provides a multi-sensor fusion gait recognition system design, which integrates the plantar pressure sensor 100 and the inertial sensor 100 of both feet while being easy to wear. Compared with traditional methods, it has higher recognition accuracy, low cost, good portability, and adopts a real-time operating system to provide sampling frequency and synchronization. It has broad application prospects in identity recognition, medical rehabilitation, sports training, fatigue detection, gait recognition, etc.

[0111] Example 3

[0112] Based on the above-mentioned method for detecting mirror seeing in a telescope system, this embodiment provides a computer-readable storage medium. The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the method for detecting mirror seeing in a telescope system as in the above-mentioned embodiment.

[0113] Example 4

[0114] A terminal device includes: a processor, a memory, and a communication bus; the memory stores a computer-readable program executable by the processor; the communication bus enables connection and communication between the processor and the memory; and when the processor executes the computer-readable program, the steps of the mirror seeing detection method in the telescope system are implemented.

[0115] Based on the mirror seeing detection method in the above-mentioned telescope system, the present application provides a terminal device, as shown in Figure 7, which includes at least one processor (processor) 20; a display screen 21; and a memory (memory) 22, and may also include a communication interface (Communications Interface) 23 and a bus 24. Among them, the processor 20, the display screen 21, the memory 22 and the communication interface 23 can communicate with each other through the bus 24. The display screen 21 is configured to display a preset user guide interface in the initial setting mode. The communication interface 23 can transmit information. The processor 20 can call the logic instructions in the memory 22 to execute the method in the above-mentioned embodiment.

[0116] In addition, the logic instructions in the memory 22 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0117] The memory 22, as a computer-readable storage medium, can be configured to store software programs or computer-executable programs, such as program instructions or modules corresponding to the methods in the embodiments of the present disclosure. The processor 20 executes the software programs, instructions, or modules stored in the memory 22 to perform functional applications and data processing, thereby implementing the methods in the above embodiments.

[0118] The memory 22 may include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function; the data storage area may store data created based on the use of the terminal device. In addition, the memory 22 may include high-speed random access memory and non-volatile memory. For example, various media that can store program code, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, may also be transient storage media.

[0119] In addition, the specific process of loading and executing the multiple instructions in the storage medium and the processor in the terminal device has been described in detail in the above method and will not be described here one by one.

[0120] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A multi-sensor fusion bipedal gait recognition system, characterized in that: include: The sensor is used to collect human motion information in real time by contacting with human feet, wherein the human motion information includes state information of the sole of the human foot and posture information of the human body during motion; A main control module, used for acquiring the motion information, and performing data synchronization and preprocessing on the motion information to obtain the motion data; The host computer is used to obtain the motion data and perform motion recognition on the motion data through a deep learning LSTM neural network model to determine the current state of human body motion; The sensor comprises: A plantar pressure sensor module, used to obtain the state information of the plantar pressure of a human body, wherein the plantar pressure of a human body includes the plantar pressure of a left foot and the plantar pressure of a right foot; The nine-axis sensor module is used to obtain the posture information of the human body during movement.

2. The multi-sensor fusion bipedal gait recognition system according to claim 1, characterized in that: The multi-sensor fusion bipedal gait recognition system also includes: The Bluetooth module is used for transmitting data. The main control module sends the motion data to the host computer via the Bluetooth module.

3. The multi-sensor fusion bipedal gait recognition system according to claim 2, characterized in that: The host computer comprises: A normalization module, used for normalizing the motion data acquired by the host computer; The filtering preprocessing module is used for performing filtering preprocessing on the normalized motion data.

4. The multi-sensor fusion bipedal gait recognition system according to claim 1, characterized in that: The deep learning LSTM neural network model specifically includes: Collecting original motion information for preprocessing, and using the preprocessed motion data as a data set; 80% of the data in the data set is used as a training set, and 20% of the data is used as a test set; The LSTM neural network model is built, wherein the shape of the input data of the LSTM neural network model is set to 34, the number of hidden neurons is set to 64, the number of layers of LSTM units is 2, 20 unit size samples are delivered to the LSTM neural network model each time training, the number of iterations is set to 30, and the learning rate is set to 0.

001.

5. A multi-sensor fusion bipedal gait recognition method, characterized in that: The method comprises the following steps: Collecting human motion information in real time by contacting the human foot, the human motion information including the state information of the human foot sole and the posture information of the human body during the motion process; Acquiring the motion information, and performing data synchronization and preprocessing on the motion information to obtain the motion data; Acquire the motion data, and perform motion recognition on the motion data through a deep learning LSTM neural network model to determine the current state of human body motion; The real-time collection of human motion information, including the state information of the sole of the human body and the posture information of the human body during the motion process, specifically includes: Plantar pressure sensors are arranged at different positions of the sole of a human foot in the form of an array, and state information of the sole of a human foot is obtained through the plantar pressure sensors, wherein the state information of the sole of a human foot includes plantar pressure information, and the pressure information includes plantar pressure of a left foot and plantar pressure of a right foot, wherein the plantar pressure sensor is provided with a Wheatstone bridge and an operational amplifier circuit; The accurate voltage value is obtained by the Wheatstone bridge and the operational amplifier circuit, the pressure information corresponds to the voltage value, and the obtained voltage value is transmitted to the main control module; A nine-axis sensor module is arranged at the ankle joint of the human leg, and the posture information of the human body during movement is captured by the nine-axis sensor. The posture information during the human body movement includes the movement posture of the human leg, and data is transmitted with the main control module through the I2C communication protocol.

6. The method for bipedal gait recognition based on multi-sensor fusion according to claim 5, characterized in that: The method of obtaining an accurate voltage value through the Wheatstone bridge and the operational amplifier circuit, wherein the pressure information corresponds to the voltage value, and transmitting the obtained voltage value to the main control module comprises: The analog-to-digital conversion is performed by the 12-bit ADC inside the main control module to obtain an analog voltage value; Converting the analog voltage value into an actual digital value, including a digital value of the left foot plantar pressure and a digital value of the right foot plantar pressure; The main control module synchronously summarizes the digital quantity of the left foot plantar pressure and the digital quantity of the right foot plantar pressure.

7. The multi-sensor fusion bipedal gait recognition method according to claim 6, characterized in that: The acquiring of the motion data and performing motion recognition on the motion data through a deep learning LSTM neural network model to determine the current state of human motion specifically includes: The main control module transmits the aggregated digital quantity to the host computer; The host computer stores and processes the received digital quantity, wherein the processing includes normalization and filtering preprocessing; The processed data is identified and matched with features through the deep learning LSTM neural network model, so as to identify the motion status information of the current athlete and judge the current state of human motion.

8. The multi-sensor fusion bipedal gait recognition method according to claim 7, characterized in that: The deep learning LSTM neural network model specifically includes: Collecting original motion information for preprocessing, and using the preprocessed motion data as a data set; 80% of the data in the data set is used as a training set, and 20% of the data is used as a test set; The LSTM neural network model is built, wherein the shape of the input data of the LSTM neural network model is set to 34, the number of hidden neurons is set to 64, the number of layers of LSTM units is 2, 20 unit size samples are delivered to the LSTM neural network model each time training, the number of iterations is set to 30, and the learning rate is set to 0.

001.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the multi-sensor fusion bipedal gait recognition method as described in any one of claims 5-8.

10. A terminal device, characterized in that: include: Processor, memory and communication bus; The memory stores a computer-readable program executable by the processor; The communication bus realizes the connection and communication between the processor and the memory; When the processor executes the computer-readable program, the steps in the multi-sensor fusion bipedal gait recognition method according to any one of claims 5 to 8 are implemented.

Images