A human posture recognition method

Abstract

The present invention provides a method for human posture recognition, comprising: acquiring posture data uploaded by a wearable device; performing preset classification on the posture data to obtain non-periodic posture data and periodic posture data; decomposing the non-periodic posture data to obtain a first time series feature matrix; performing spatial feature extraction on the periodic posture data to obtain a spatial feature matrix, performing signal decomposition on the periodic posture data to obtain a second time series feature matrix, and fusing the spatial feature matrix with the second time series feature matrix to obtain a fused feature matrix; classifying the time series feature matrix to obtain a first posture feature; inputting the fused feature matrix into multiple classifiers for classification respectively to obtain multiple classification results, and voting on the multiple classification results to obtain a second posture feature; performing judgment based on the first posture feature and the second posture feature to obtain a posture recognition result; and improving the accuracy of posture recognition.

Description

Technical Field

[0001] The present invention relates to the technical field of posture recognition, and in particular to a method for human posture recognition. Background Art

[0002] With the development of technologies such as the Internet of Things and pattern recognition, wearable devices are becoming increasingly popular in the market. Wearable devices can be used to further monitor the human body's condition. The rapid development of wearable sensor technologies such as AIOT devices, bracelets, and watches has changed the monitoring of human life.

[0003] Thanks to the continuous development of microelectronics technology, wearable sensors have a wide range of real-world applications, such as human activity recognition (HAR). These applications include security surveillance systems, healthcare monitoring, sports assistance, interactive 3D games, and smart homes. In security, HAR systems are used to detect uncertain events, including fights, falls, and strikes, and to implement safety measures to address violent activities in the surrounding area. In healthcare monitoring, patients' heart rate, body movement, brain activity, and other important health data can be analyzed to help them detect normal or abnormal patient behavior. In smart homes, remote care is provided, such as child daycare and elderly activity monitoring. Researchers are currently paying close attention to the situation of elderly people living alone, especially fall detection. Once a fall occurs, it can have serious consequences, and timely detection and sending of alarm information are particularly important.

[0004] Currently, most wearable devices on the market are VR devices, bracelets, and belts that are mounted on the head, wrist, or waist. Research on foot-based wearable devices for human behavior recognition is lacking. Foot data, unlike other body parts, can provide higher resolution for sensitive lower limb movements.

[0005] Human behavior recognition is a type of pattern recognition that analyzes and processes information collected by sensors to ultimately classify human movements. However, the difficulty lies in the fact that human movement is a complex process. Unlike rigid bodies like airplanes and cars, human movements are complex and vary from person to person, making human behavior recognition a significant challenge. Summary of the Invention

[0006] The present invention provides a method for human body posture recognition, the purpose of which is to improve the accuracy of posture recognition.

[0007] In order to achieve the above object, the present invention provides a human body posture recognition method, comprising:

[0008] Step 1: Obtain the posture data uploaded by the wearable device;

[0009] Step 2: Preset and classify the posture data to obtain non-periodic posture data and periodic posture data, where the non-periodic posture data includes first acceleration block signal data and first angular velocity block signal data, and the periodic posture data includes second acceleration block signal data, second angular velocity block signal data, acceleration sheet signal data, and angular velocity sheet signal data;

[0010] Step 3: Decompose the first acceleration block signal data and the first angular velocity block signal data to obtain a first time series feature matrix;

[0011] Step 4: Input the acceleration piece signal data and the angular velocity piece signal data into the first residual network and the second residual network respectively to extract spatial features to obtain a spatial feature matrix, perform signal decomposition on the second acceleration block signal data and the second angular velocity block signal data to obtain a second temporal feature matrix, and fuse the spatial feature matrix with the second temporal feature matrix to obtain a fused feature matrix;

[0012] Step 5: classify the time series feature matrix to obtain the first posture feature;

[0013] Step 6: Input the fused feature matrix into multiple classifiers for classification to obtain multiple classification results, and refer to the first posture feature to vote on the multiple classification results to obtain the second posture feature;

[0014] Step 7: judging the posture data by combining the first posture feature and the second posture feature to obtain a posture recognition result.

[0015] More specifically, step 1 includes:

[0016] The device terminal collects the acceleration and angular velocity of the object under test when it completes the action at a sampling rate of 150Hz to obtain posture data.

[0017] More specifically, step 3 includes:

[0018] performing discrete wavelet transform on the first acceleration block signal data and the first angular velocity block signal data;

[0019] Performing multi-level data decomposition on the first acceleration block signal data and the first angular velocity block signal data after discrete wavelet transformation to obtain a plurality of signal detail coefficients;

[0020] Reconstructing the first acceleration block signal data and the first angular velocity block signal data after discrete wavelet transformation using a plurality of signal detail coefficients to obtain first reconstructed signal data;

[0021] The first reconstructed signal data is fused with the data block to obtain a first data block matrix, and the time series characteristics of the first data block matrix are calculated to obtain a first time series characteristic matrix.

[0022] More specifically, step 4 includes:

[0023] performing discrete wavelet transform on the second acceleration block signal data and the second angular velocity block signal data;

[0024] Performing multi-level data decomposition on the second acceleration block signal data and the second angular velocity block signal data after discrete wavelet transformation to obtain multiple signal detail coefficients;

[0025] Reconstructing the second acceleration block signal data and the second angular velocity block signal data after discrete wavelet transformation using a plurality of signal detail coefficients to obtain second reconstructed signal data;

[0026] Fusing the second reconstructed signal data with the data block to obtain a second data block matrix, and calculating the time series characteristics of the second data block matrix to obtain a second time series characteristic matrix;

[0027] Based on the multi-mode fusion characteristics of three-axis acceleration and three-axis angular velocity, the acceleration plate signal data and the angular velocity plate signal data are fused at the data level to obtain the acceleration matrix and angular velocity matrix;

[0028] Inputting the acceleration matrix into the first residual network to perform spatial feature extraction, thereby obtaining a first spatial feature extraction result;

[0029] The angular velocity matrix is ​​input into the second residual network for spatial feature extraction to obtain a second spatial feature extraction result, and the second spatial feature extraction result and the first spatial feature extraction result are compressed using principal component analysis to obtain a spatial feature matrix;

[0030] The spatial feature matrix is ​​fused with the second temporal feature matrix to obtain a fused feature matrix.

[0031] Furthermore, the first residual network and the second residual network both include: a first weight layer, a second weight layer and an output layer, the input end of the first weight layer is connected to the input end of the output layer, the output end of the first weight layer is connected to the input end of the second weight layer, the output end of the second weight layer is connected to the input end of the output layer, the input end of the first weight layer in the first residual network serves as the input end of the first residual network, the input end of the first weight layer in the second residual network serves as the input end of the second residual network, the output end of the output layer in the first residual network serves as the output end of the first residual network, and the output end of the output layer in the second residual network serves as the output end of the second residual network.

[0032] Specifically, wearable devices are worn on the feet and include:

[0033] The sensor module is used to collect the acceleration and angular velocity of the object when it completes the action to obtain posture data;

[0034] Power supply module, used to ensure long-term reliable operation of equipment terminals;

[0035] The processing module is used to read the posture data collected by the sensor and send it to the host computer;

[0036] The input end of the sensor module is electrically connected to the input end of the processing module, and the output end of the power module is electrically connected to the input end of the processing module.

[0037] Furthermore, the processing module uses a NodeMCU chip, which is equipped with an ESP8266-WIFI chip to achieve wireless communication between the processing module and the host computer, and upload the posture data read by the processing module to the host computer.

[0038] Further, the power module includes a transmitting submodule, a receiving submodule, and a switch;

[0039] The first end of the transmitting submodule is connected to the input end of the processor, the second end of the transmitting submodule is connected to the second end of the switch, the first end of the switch is connected to the output end of the receiving submodule, and the third end of the switch is connected to the input end of the processing module.

[0040] Furthermore, the sensor module uses a three-axis acceleration + three-axis gyroscope sensor, and the chip model is MPU6050-GY521.

[0041] The above solution of the present invention has the following beneficial effects:

[0042] The present invention obtains posture data uploaded by a device terminal; pre-classifies the posture data to obtain non-periodic posture data and periodic posture data; then decomposes the data to obtain a first time series feature matrix; extracts spatial features from the periodic posture data to obtain a spatial feature matrix, and fuses the spatial feature matrix with the time series feature matrix to obtain a fused feature matrix; classifies the time series feature matrix to obtain a first posture feature; classifies the fused feature matrix to obtain multiple classification results, and votes with reference to the first posture feature to obtain a second posture feature; judges the posture data in combination with the first posture feature and the second posture feature to obtain a posture recognition result; compared with the existing technology, the periodic posture data and non-periodic posture data are obtained by pre-classification, and the time series features and spatial features of the periodic posture data and the non-periodic posture data are extracted respectively, and then the time series features and spatial features are fused and classified to obtain a posture recognition result, thereby improving the accuracy of human posture recognition.

[0043] Other beneficial effects of the present invention will be described in detail in the subsequent specific implementation section. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 Schematic diagram of a flow chart of an embodiment of the present invention;

[0045] Figure 2 A flowchart of an embodiment of the present invention;

[0046] Figure 3 Schematic diagram of a module of a device terminal according to an embodiment of the present invention;

[0047] Figure 4 is a circuit schematic diagram of a device terminal in an embodiment of the present invention;

[0048] Figure 5 This is a flowchart of preset classification and feature extraction in an embodiment of the present invention;

[0049] Figure 6 This is a statistical chart of the recognition accuracy of different postures according to an embodiment of the present invention;

[0050] Figure 7 A comparison diagram of gesture recognition after and without preset classification according to an embodiment of the present invention;

[0051] Figure 8 A comparison diagram of gesture recognition using weighted random forest and random forest in an embodiment of the present invention;

[0052] Figure 9 This is a comparison diagram of the embodiment of the present invention using acceleration and angular velocity fusion and directly using acceleration or angular velocity for gesture recognition. DETAILED DESCRIPTION

[0053] To make the technical problems, technical solutions, and advantages to be solved by the present invention more clear, the following is a detailed description with reference to the accompanying drawings and specific embodiments. It is obvious that the embodiments described are only some of the embodiments of the present invention, not all of them. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0054] In the description of the present invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings and are intended solely to facilitate and simplify the description of the present invention. They are not intended to indicate or imply that the devices or components referred to must have, be constructed, or operate in a specific orientation, and therefore should not be construed as limitations on the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0055] In the description of the present invention, it should be noted that, unless otherwise expressly specified or limited, the terms "mounted," "connected," and "connected" should be understood broadly. For example, they may refer to a locking connection, a detachable connection, or an integral connection; they may refer to a mechanical connection or an electrical connection; they may refer to a direct connection or an indirect connection through an intermediate medium; and they may refer to internal communication between two components. Those skilled in the art will understand the specific meanings of the above terms in the present invention based on the specific circumstances.

[0056] In addition, the technical features involved in the different embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0057] The present invention aims at solving the existing problems and provides a method for human body posture recognition.

[0058] like Figure 1 As shown, an embodiment of the present invention provides a human body posture recognition method, comprising:

[0059] Step 1: Obtain the posture data uploaded by the wearable device;

[0060] Step 2: Preset and classify the posture data to obtain non-periodic posture data and periodic posture data, where the non-periodic posture data includes first acceleration block signal data and first angular velocity block signal data, and the periodic posture data includes second acceleration block signal data, second angular velocity block signal data, acceleration sheet signal data, and angular velocity sheet signal data;

[0061] Step 3: Decompose the first acceleration block signal data and the first angular velocity block signal data to obtain a first time series feature matrix;

[0062] Step 4: Input the acceleration piece signal data and the angular velocity piece signal data into the first residual network and the second residual network respectively to extract spatial features to obtain a spatial feature matrix, perform signal decomposition on the second acceleration block signal data and the second angular velocity block signal data to obtain a second temporal feature matrix, and fuse the spatial feature matrix with the second temporal feature matrix to obtain a fused feature matrix;

[0063] Step 5: classify the time series feature matrix to obtain the first posture feature;

[0064] Step 6: Input the fused feature matrix into multiple classifiers for classification to obtain multiple classification results, and refer to the first posture feature to vote on the multiple classification results to obtain the second posture feature;

[0065] Step 7: judging the posture data by combining the first posture feature and the second posture feature to obtain a posture recognition result.

[0066] In an embodiment of the present invention, the classified postures include six types: static posture, walking up stairs, walking on flat ground, running, jumping and wrestling. Among them, static posture, jumping and wrestling are set as non-periodic postures, among which jumping and wrestling only have single actions and do not have the repeatability of single actions. Static posture does not have a single action, only a smooth straight line without ups and downs; walking up stairs, running and walking up stairs are set as periodic postures, which have both single actions and the repeatability of single actions. Therefore, non-periodic motion contains the temporal characteristics of a single action, and periodic motion contains both the temporal characteristics of a single action and the spatial characteristics of multiple actions. Spatial characteristics, as unique characteristics of periodic motion, can help identify periodic postures.

[0067] The static posture is the easiest to distinguish among the six postures. Compared with other postures, the data features of the static posture are stable from the time series diagram of acceleration and angular velocity, and the values ​​oscillate around 0. The acceleration and angular velocity do not change much. The static posture exists in many scenarios in the classification and the number of postures is large, so there is data imbalance in the classification.

[0068] The two postures of jumping and wrestling are non-periodic postures and are single-time actions. The jumping and wrestling mentioned in the embodiments of the present invention refer to single actions, and the actions have a large random occurrence.

[0069] From the acceleration and angular velocity of the jumping motion, it can be seen that different jumping motions have obvious randomness, and the force and intensity of each jumping motion are different. At the same time, it can be seen that the jumping motion occurs in a short period of time. Generally, the amplitude of the acceleration of a jumping motion in the X and Y axes does not exceed ±1g, but in the Z axis direction, the jumping motion will reach +2g. Because jumping is a positive upward motion, a huge upward acceleration is required in the early stage of the jump to achieve the purpose of jumping upward, which will result in an acceleration of +2g in the positive direction of the Z axis. Since the X and Z axes have different forms of jumping, there are forward and horizontal jumps. Therefore, the amplitudes of the X and Z axes are not generalized, and the changes in angular velocity have no obvious characteristics.

[0070] Like jumping, falls are non-periodic motions with significant randomness. Each fall is different, resulting in a different direction. However, each fall is brief, typically lasting around 0.5 seconds. The angular velocity of a fall has a large amplitude on the Y-axis. Because a fall involves falling, the fall occurs over a 90-degree angle along the Y-axis.

[0071] Stair climbing, ground walking, and running share certain similarities and differences. They are all cyclical movements, repeating the same action over a period of time. Therefore, as can be seen from the timing diagrams of acceleration and angular velocity, the time interval between a single movement and the next within the same movement is similar.

[0072] Stair climbing is a cyclical motion. The cadence of stair climbing and ground walking is similar, approximately 80 steps per minute, equivalent to a single movement interval of approximately 0.7s-0.8s. The intensity of the movements is also similar for walking and stair climbing. Unlike running, the cadence is equivalent to the distance between single movements and is a spatial characteristic of the movement. Therefore, spatial characteristics are helpful in distinguishing cyclical motions. The acceleration amplitude of stair climbing and walking is generally ±1g (maximum amplitude is ±2g). Therefore, in cyclical motion, the foot data of stair climbing is similar to that of walking. Considering that the posture of stair climbing is generally upward or downward (ascending and descending stairs), while the walking posture has two sub-movements on the z-axis: raising and lowering the leg, but the overall z-axis of walking is unchanged, there is a certain difference between walking and stair climbing on the z-axis.

[0073] Walking is a gentle, cyclical motion. Both acceleration and angular velocity have a certain period. Different people have different walking speeds and postures, resulting in variations in walking data among the people in the experiment. However, most cycle times are within 0.6s-0.75s, as shown in the time domain graph of each person's walking. The amplitudes of acceleration and angular velocity are not large.

[0074] The acceleration and angular velocity of running show that although running is also a cyclical motion, the periodic distances of its peaks and troughs are significantly different from those of jumping, walking, and stair climbing. Running is an intense sport, characterized by large amplitude changes in a short period of time, and the cycle time is shorter. Different people ran at different speeds in the experiment, so each person's cycle time was also different, but the cycle time for the same person was similar, with most people falling within the range of 0.2s-0.35s. The cycle time for walking and stair climbing is 0.7s-0.8s, so running has a certain degree of differentiation in cycle time. There are also differences in the amplitudes of acceleration and angular velocity. Because running is a more intense sport, the peak amplitude of acceleration generally reaches ±2g, and the peak amplitude of angular velocity generally reaches ±250 degrees / s.

[0075] Specifically, if Figure 2 As shown, the first residual network and the second residual network both include: a first weight layer, a second weight layer and an output layer, the input end of the first weight layer is connected to the input end of the output layer, the output end of the first weight layer is connected to the input end of the second weight layer, the output end of the second weight layer is connected to the input end of the output layer, the input end of the first weight layer in the first residual network serves as the input end of the first residual network, the input end of the first weight layer in the second residual network serves as the input end of the second residual network, the output end of the output layer in the first residual network serves as the output end of the first residual network, and the output end of the output layer in the second residual network serves as the output end of the second residual network.

[0076] Specifically, if Figure 3 As shown, the wearable device is worn on the foot, and the wearable device includes:

[0077] The sensor module is used to collect the acceleration and angular velocity of the object when it completes the action to obtain posture data;

[0078] Power supply module, used to ensure long-term reliable operation of equipment terminals;

[0079] The processing module is used to read the posture data collected by the sensor and send it to the host computer;

[0080] The input end of the sensor module is electrically connected to the input end of the processing module, and the output end of the power module is electrically connected to the input end of the processing module.

[0081] Specifically, if Figure 4 As shown, the processing module uses a NodeMCU chip, which is equipped with an ESP8266-WIFI chip to achieve wireless communication between the processing module and the host computer, and upload the posture data read by the processing module to the host computer.

[0082] In this embodiment of the present invention, the processing module utilizes the open-source IoT platform NodeMCU. NodeMCU is equipped with an ESP8266-WIFI chip and is one of the most stable and user-friendly development boards developed early on based on the ESP8266 secondary packaging. The NodeMCU secondary packaging consists of both hardware and software firmware. In terms of hardware, NodeMCU builds upon the ESP8266 by adding some peripheral USB serial port circuitry. In terms of software, NodeMCU incorporates the lightweight scripting language Lua, supporting an event-driven API and Arduino IDE development. NodeMCU can enable WiFi access through simple interface calls, build servers, and implement IoT by controlling I / O ports, PWM, timers, and more. The NodeMCU used in this embodiment of the present invention can receive sensor data through Arduino programming and transmit it to a host computer program. Designed specifically for mobile devices, wearable electronics, and IoT applications, NodeMCU can also be configured through AT commands to communicate with the microcontroller's serial port and transmit data via WiFi.

[0083] Specifically, the power module adopts a wireless charging module; the wireless charging module is divided into two modules, a transmitting submodule and a receiving submodule. The receiving submodule can output a stable voltage of 3.7V through an electromagnetic coil to power the device terminal or charge the lithium battery; the transmitting submodule of the embodiment of the present invention uses a 3.7V lithium battery with a capacity of 1200 mAh, which can power the device terminal for about 24-36 hours and can be repeatedly charged and reused.

[0084] The first end of the 3.7V lithium battery is connected to the GND pin of the NodeMCU chip, the second end of the 3.7V lithium battery is connected to the second end of the switch, the first end of the switch is connected to the output end of the receiving submodule, and the third end of the switch is connected to the Vin pin of the NodeMCU chip.

[0085] The receiving submodule specifically includes: XKT-R2 chip, resistor R1, resistor R2, resistor R3, resistor R4, resistor R5, resistor R7, capacitor C1, capacitor C2, capacitor C3, switch tube Q1, light-emitting diode D1, diode D2, diode D3 and inductor H1. The first pin of the XKT-R2 chip is connected to the anode of the light-emitting diode D1, and the cathode of the light-emitting diode D1 is grounded. The second pin of the XKT-R2 chip is respectively connected to the gate of the switch tube Q1 and the first end of the resistor R4. The second end of the resistor R4 is respectively connected to the source of the switch tube Q1, one end of the resistor R3, capacitor C1, and the cathode of the diode D2. The drain of the switch tube Q1 is respectively connected to the first end of the switch, the cathode of the diode D2, and the drain of the switch tube Q1. One end of the resistor R1 is connected, the other end of the capacitor C1 is grounded, the anode of the diode D2 is respectively connected to the anode of the diode D3, one end of the inductor H1, and one end of the capacitor C3, the cathode of the diode D3 is connected to one end of the resistor R7, the other end of the resistor R7 is respectively connected to the first end of the capacitor C2, one end of the resistor R5, and the fifth pin of the XKT-R2 chip, the other end of the inductor H1 is respectively connected to the other end of the capacitor C3, the other end of the capacitor C2, the other end of the resistor R5, and the fourth pin of the XKT-R2 chip and grounded, the third pin of the XKT-R2 chip is respectively connected to the other end of the resistor R3, one end of the resistor R2, and the other end of the resistor R1, and the other end of the resistor R2 is grounded.

[0086] Specifically, the sensor module uses a three-axis acceleration + three-axis gyroscope sensor, the chip model is MPU6050-GY521. It integrates a three-axis MEMS gyroscope and a three-axis MEMS accelerometer. It has a three-axis angular velocity sensor (gyroscope) with a full sensing range of ±250, ±500, ±1000, and ±2000° / sec, a three-axis accelerometer with ±2g, ±4g, ±8g, and ±16g, and an expandable Digital Motion Processor (DMP). Due to its ease of use, it is widely used in sports, drones, AR / VR games, and robotics. The NodeMCU connects to the host computer through an Arduino program when the NodeMCU development board is powered on, establishing a TCP connection. The MPU6050-GY-521 reads 150 samples per second, and the NodeMCU uses a 3-second sampling window, with 450 samples in each window. Every time 450*6 data points are collected, they are transmitted to the host computer; the first pin and the second pin of the MPU6050-GY-521 chip are connected to the 3.3V pin and the GND pin of the NodeMCU chip, the SCL pin of the MPU6050-GY-521 chip is connected to the D1 pin of the NodeMCU chip, and the SDA pin of the MPU6050-GY-521 chip is connected to the D2 pin of the NodeMCU chip.

[0087] The device terminal in the embodiment of the present invention can be a smart shoe or other wearable foot device, which is 20cm*10cm*2cm in size, has a 150hz sampling rate to receive acceleration and angular velocity, has the function of transmitting data to the host computer via WiFi, has an independent power supply to provide long-term independent power supply, and can be powered by wireless charging when the device is idle; the purpose is to collect the angular velocity and acceleration data generated by the foot in real time and for a long time, and transmit the data to the host computer. The device terminal provides the hardware foundation for the host computer to realize posture recognition.

[0088] Specifically, step 1 includes:

[0089] The device terminal collects the acceleration and angular velocity of the object under test when it completes the action at a sampling rate of 150Hz to obtain posture data.

[0090] Specifically, the acceleration is stored in the memory at the subscript address 0X3B, the angular velocity is stored in the memory at the subscript address 0X43, and the 6 bytes read from the subscript address 0X3B are A xL ,A xH ,A yL ,A yH ,A zL ,A zH , read 6 bytes at address 0X43, which are G xL ,G xH ,G yL ,G yH ,G zL ,G zH .

[0091] a x =((A xH <<8)|A xL ) / 32768*16g (g is the acceleration due to gravity, which can be 9.8m / s2)

[0092] a y =((A yH <<8)|A yL ) / 32768*16g (g is the acceleration due to gravity, which can be 9.8m / s2)

[0093] a z =((A zH <<8)|A zL ) / 32768*16g (g is the acceleration due to gravity, which can be 9.8m / s2)

[0094] g x =((G xH <<8)|G xL ) / 32768

[0095] g y =((G yH <<8)|G YL ) / 32768

[0096] g z =((G ZH <<8)|G ZL ) / 32768

[0097] Among them a x ,a y ,a z ,g x ,g y ,g z All figures are rounded to five decimal places.

[0098] MPU6050-GY-521 can continuously collect angular velocity and acceleration at a sampling rate of 150HZ, and define the acceleration collected in three directions as a x ,a y ,a z , and the angular velocity in three directions is defined as g x ,g y ,g z After powering on, the wearable device was left to stand for 2 seconds. To eliminate data fluctuations and initialization during device sampling, 450 sampling points were generated during the 3 seconds of standing still, each containing 6 data points. The average value was calculated using the following formula:

[0099]

[0100]

[0101] Where x, y, and z represent three directions, and M is 3*150 sampling points.

[0102] With the rapid development of MEMS microelectronic devices, sensor performance is constantly improving. However, the measurement data has inherent errors and measurement errors, which can affect the subsequent recognition and classification. Therefore, data preprocessing is very important in the process.

[0103] In an embodiment of the present invention, the posture data needs to be preprocessed before the preset classification is performed. The preprocessing includes: data denoising and data normalization. The error values ​​and deviation values ​​in the collected data, and the noise in the data will affect the robustness of the behavior classification model. Commonly used filtering noise reduction methods such as Butterworth filtering, moving average filtering, median filtering, etc., effectively reduce the noise problem in the measurement; the embodiment of the present invention uses a Butterworth filter to denoise the data. The characteristic of the Butterworth filter is that the frequency response curve within the passband is maximally flat; Normalization is a way to simplify calculations. The feature data is expressed in the same measurement unit and a common scale or range is used. Normalization gives the data equal weight, so the comparison and aggregation between attributes are easier, the convergence conditions of the data are better, and the efficiency of data analysis is improved.

[0104] The six postures in the embodiment of the present invention can be divided into two categories: non-periodic: jumping, falling, sitting; and periodic: walking, running, and walking up stairs. Based on the above posture types and distinctions, and the differences in posture data, the normalized posture data is preset and classified by a preset classifier. The preset classification algorithm is as follows: Figure 5 As shown in the figure, the preset classification is helpful for the later data segmentation, extraction of spatial features, and classification accuracy; for the three-axis acceleration and three-axis angular velocity, a sliding window is used, and the window length is 3000 sampling points, and the initial position of the window is p 初始 =0, the posture data can be divided into non-periodic posture data and periodic posture data based on the preset classification algorithm.

[0105] The embodiment of the present invention also needs to segment the non-periodic gesture data and the periodic gesture data, and extract single actions from the non-periodic gestures.

[0106] In the embodiments of the present invention, the time series segments of a single action are collectively referred to as block signals, and the time series segments of a collection of multiple actions are collectively referred to as slice signals. For periodic postures, both the single action is intercepted to obtain the action timing characteristics, and the time series collection of the single action is intercepted to extract the spatial characteristics of the posture. For periodic postures, posture data is stored, referred to as ACC slice signal data and GYRO slice signal data. The slice signal is cut through the peak to form 10 ACC block signal data and GYRO block signal data with a length of 200. For jumps and falls in non-periodic movements, the 240 sampling points before the peak and the 260 sampling points after the peak are intercepted to form ACC block signal data and GYRO block signal data. If the peak number threshold is 0, the posture is judged as sitting still, and the 3000-byte slice signal is divided into 5 ACC block signal data and GYRO block signal data.

[0107] Specifically, if Figure 5As shown, step 3 includes:

[0108] performing discrete wavelet transform on the first acceleration block signal data and the first angular velocity block signal data;

[0109] Performing multi-level data decomposition on the first acceleration block signal data and the first angular velocity block signal data after discrete wavelet transformation to obtain a plurality of signal detail coefficients;

[0110] Reconstructing the first acceleration block signal data and the first angular velocity block signal data after discrete wavelet transformation using a plurality of signal detail coefficients to obtain first reconstructed signal data;

[0111] The first reconstructed signal data is fused with the data block to obtain a first data block matrix, and the time series characteristics of the first data block matrix are calculated to obtain a first time series characteristic matrix.

[0112] Specifically, step 4 includes:

[0113] performing discrete wavelet transform on the second acceleration block signal data and the second angular velocity block signal data;

[0114] Performing multi-level data decomposition on the second acceleration block signal data and the second angular velocity block signal data after discrete wavelet transformation to obtain multiple signal detail coefficients;

[0115] Reconstructing the second acceleration block signal data and the second angular velocity block signal data after discrete wavelet transformation using a plurality of signal detail coefficients to obtain second reconstructed signal data;

[0116] Fusing the second reconstructed signal data with the data block to obtain a second data block matrix, and calculating the time series characteristics of the second data block matrix to obtain a second time series characteristic matrix;

[0117] Based on the multi-mode fusion characteristics of three-axis acceleration and three-axis angular velocity, the acceleration plate signal data and the angular velocity plate signal data are fused at the data level to obtain the acceleration matrix and angular velocity matrix;

[0118] Inputting the acceleration matrix into the first residual network to perform spatial feature extraction, thereby obtaining a first spatial feature extraction result;

[0119] The angular velocity matrix is ​​input into the second residual network for spatial feature extraction to obtain a second spatial feature extraction result, and the second spatial feature extraction result and the first spatial feature extraction result are compressed using principal component analysis to obtain a spatial feature matrix;

[0120] The spatial feature matrix is ​​fused with the second temporal feature matrix to obtain a fused feature matrix.

[0121] In an embodiment of the present invention, the first time series feature matrix and the second time series feature matrix are both composed of mean, median, variance, standard deviation, maximum value, minimum value, correlation coefficient, zero crossing point, signal amplitude, signal vector amplitude, signal mobility, signal complexity, fractal dimension, power spectrum entropy, Kolmogorov entropy and Shannon entropy.

[0122] Specifically, the embodiment of the present invention determines the periodicity of the motion posture and cuts the data based on the preset classification of the number of peaks, and the detailed implementation is as follows:

[0123] By acquiring the denoised and normalized ACC and GYRO signals at a starting position P with a length of 3000 samples, we determine whether the motion within the 3000 sample length (within 20 seconds) is periodic. For non-periodic motion, if the number of peaks is zero, the 20 seconds are considered static. The 3000 sample points are divided into six blocks of 500 samples each, and the P value is updated. If the number of peaks is one, it is considered a fall or jump. The 240 sample points before and 260 sample points after the peak are intercepted, and the P value is set to 800 sample points after the peak. For periodic motion, the 140 sample points before and 160 sample points after each peak are intercepted to form a block signal. The distances before and after each peak are calculated. The block signal of N = 20 peaks is intercepted, and the distances between peaks are also calculated. If the distance between peaks differs significantly from the previous peak distance, the P slice signal is updated to the current value, and the slice signal is intercepted.

[0124] The embodiment of the present invention is based on the multi-mode fusion of acceleration and angular velocity, and the spatiotemporal feature extraction based on the residual network, and is implemented in detail as follows:

[0125] The three-axis acceleration signal ACC data is combined into an acceleration matrix, and the three-axis angular velocity signal GYRO data is combined into an angular velocity matrix. The residual network model contains two weight layers. The input acceleration and angular velocity matrices are passed through the two weight layers for spatial feature extraction. Simultaneously, the input ACC and GYRO matrices are directly connected to the output layer using jumpers. By obtaining the features passed through the weight layers and the residual values ​​of the ACC and GYRO matrices, the residual value feedback is used to control the residual network model and extract more spatial morphological features. The number of extracted spatial features is relatively large, so principal component analysis (PCA) is used to compress and reduce the spatial features, generating a fusion feature matrix F_space: F_fusion = {α·F_sequence,β·F_space}, where α and β are empirical parameters.

[0126] Since more features mean more feature information, having too many features increases computational complexity, leading to information redundancy, which may obliterate the internal laws of things and cause some classifiers to have poor classification and recognition effects on a large number of features. Therefore, in an embodiment of the present invention, reducing information redundancy through feature dimensionality reduction is a method to reduce complexity. The main purpose of feature dimensionality reduction is to extract a new feature subset from the original feature space. The new feature subset has fewer dimensions than the original feature space, but the recognition effect is the same.

[0127] There are two types of feature matrices in this application: time series feature matrix and space series feature matrix.

[0128] For the dimensionality reduction of time series feature evidence, the time series evidence is analyzed. The time series statistical features meet the following conditions: 1. The feature ranks in the top 30% in terms of feature importance among all the features in the time series statistical features; 2. The feature is selected by three different feature selection algorithms at the same time: univariate feature selection algorithm (UFS), recursive feature elimination algorithm (RFE), and Pearson correlation coefficient selection algorithm (PCCs).

[0129] For the feature dimensionality reduction of spatial features, principal component analysis (PCA) is mainly used to reduce the dimensionality of the original features, and then the feature subset selection algorithm is used to select the new feature space.

[0130] The embodiment of the present invention proposes a weighted random forest to address data imbalance in non-periodic postures. The specific implementation is as follows:

[0131] Given a non-periodic posture data set D containing n samples, the number of static posture samples in the non-periodic posture data set D is much greater than that of wrestling and jumping. The data set is divided into a training set and a test set.

[0132] N sub-training sets are extracted from the training set, and those that are not sampled will form out-of-bag samples (Out-Of-Bag).

[0133] Undersample N subsets to form a data-balanced training set, so that the number of samples of the three categories in the training set is the same.

[0134] In the training set with N balanced samples, the training generates a CART decision tree {h1(x),h2(x),...,h N (x)}

[0135] Perform performance tests on each CART decision set on the OOB test set and obtain the F1-score on each OOB test set:

[0136]

[0137] The weight of each decision tree

[0138] The results of each decision tree are weighted and voted, and finally each decision tree set forms a weighted random forest, which is expressed as follows:

[0139]

[0140] I(*) indicates that when the decision tree classification is correct, the output is 1, and when it is wrong, the output is 0.

[0141] The embodiment of the present invention implements the multi-classifier fusion decision for periodic motion as follows:

[0142] Training support vector machine SVM, linear discriminant function: here It is a kernel function. The Gaussian radial basis kernel is used as the kernel function of SVM. An integrated classifier is constructed based on several decision trees. Random forest classifier and Bayesian classifier are trained.

[0143] For the fusion of spatial features and temporal features generated by periodic motion, the results produced by the three classifiers are voted, and the result samples produced by the temporal features of the block signal data in the slice signal data set are referred to, and finally the posture features corresponding to the periodic posture data are generated.

[0144] The embodiments of the present invention demonstrate the technical effects through specific experiments: Experimental conditions and data details:

[0145] The experimental population consisted of 10 people, 6 men and 4 women, aged 18-27. During the experiment, the wearable device was placed on the toe of the left foot. Each person also sat quietly for 5 minutes, walked for 5 minutes, ran for 5 minutes, walked up the stairs for 5 minutes, jumped for 5 minutes, and simulated a fall 10 times. The data collected during the test was divided into a sampling window of 200 sampling points and labeled with posture. The experimental results are shown in the figure below. Figure 5 As shown, from Figure 6 It can be seen from the figure that the method provided by the present invention has a recognition rate of 98.8%, 89.0%, 91.0%, 96.0%, 96.5% and 94.2% for static posture, jumping, falling, walking on flat ground, walking up stairs and running respectively. Figure 7 It can be seen that the comparison of the accuracy of each posture recognition using the preset classification and not using the preset classifier shows that the preset classifier is of great help in the recognition of periodic motion. Figure 8 It can be seen that due to the excessive number of sitting samples, not using weighted random forest classification will lead to overfitting of sitting posture recognition, resulting in most samples being misidentified as sitting. Using weighted random forest, the recognition rate of wrestling and jumping is greatly improved. Figure 9It can be seen that multi-mode fusion combines the features of acceleration and angular velocity, the features of angular velocity alone, and the features of acceleration alone. The comparison shows that multi-mode fusion of acceleration and angular velocity improves recognition.

[0146] From the above, it can be concluded that the embodiment of the present invention obtains the posture data uploaded by the device terminal; pre-classifies the posture data to obtain non-periodic posture data and periodic posture data; then decomposes the data to obtain a first time series feature matrix; extracts spatial features from the periodic posture data to obtain a spatial feature matrix, and fuses the spatial feature matrix with the time series feature matrix to obtain a fused feature matrix; classifies the time series feature matrix to obtain a first posture feature; classifies the fused feature matrix to obtain multiple classification results, and votes with reference to the first posture feature to obtain a second posture feature; judges the posture data in combination with the first posture feature and the second posture feature to obtain a posture recognition result; compared with the existing technology, the preset classification method is used to obtain periodic posture data and non-periodic posture data, and the time series features and spatial features of the periodic posture data and non-periodic posture data are extracted respectively, and then the time series features and spatial features are feature-fused and classified to obtain a posture recognition result, thereby improving the accuracy of human posture recognition.

[0147] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A human body posture recognition method, characterized in that: include: Step 1: Obtain the posture data uploaded by the wearable device; Step 2: Preset classification of the posture data to obtain non-periodic posture data and periodic posture data, wherein the non-periodic posture data includes first acceleration block signal data and first angular velocity block signal data, and the periodic posture data includes second acceleration block signal data, second angular velocity block signal data, acceleration slice signal data, and angular velocity slice signal data, wherein the block signal is a time sequence segment of a single action, and the slice signal is a time sequence segment of a collection of multiple actions; Step 3, decomposing the first acceleration block signal data and the first angular velocity block signal data to obtain a first time series feature matrix, including: performing discrete wavelet transform on the first acceleration block signal data and the first angular velocity block signal data; Performing multi-level data decomposition on the first acceleration block signal data and the first angular velocity block signal data after discrete wavelet transformation to obtain a plurality of signal detail coefficients; Reconstructing the first acceleration block signal data and the first angular velocity block signal data after discrete wavelet transformation using the plurality of signal detail coefficients to obtain first reconstructed signal data; Fusing the first reconstructed signal data with the data block to obtain a first data block matrix, and calculating the time series characteristics of the first data block matrix to obtain a first time series characteristic matrix; Step 4: Input the acceleration piece signal data and the angular velocity piece signal data into the first residual network and the second residual network respectively to extract spatial features to obtain a spatial feature matrix, decompose the second acceleration block signal data and the second angular velocity block signal data to obtain a second temporal feature matrix, and fuse the spatial feature matrix with the second temporal feature matrix to obtain a fused feature matrix; Step 5: classify the first time series feature matrix to obtain a first posture feature; Step 6: Input the fused feature matrix into multiple classifiers for classification to obtain multiple classification results, and refer to the first posture feature to vote on the multiple classification results to obtain a second posture feature; Step 7: judge the posture data in combination with the first posture feature and the second posture feature to obtain a posture recognition result.

2. The human body posture recognition method according to claim 1, wherein: The step 1 comprises: The wearable device collects acceleration and angular velocity of the measured object when completing the action at a sampling rate of 150 Hz to obtain posture data.

3. The human body posture recognition method according to claim 1, wherein The step 4 comprises: performing discrete wavelet transform on the second acceleration block signal data and the second angular velocity block signal data; Performing multi-level data decomposition on the second acceleration block signal data and the second angular velocity block signal data after discrete wavelet transformation to obtain multiple signal detail coefficients; Reconstructing the second acceleration block signal data and the second angular velocity block signal data after discrete wavelet transformation using the plurality of signal detail coefficients to obtain second reconstructed signal data; Fusing the second reconstructed signal data with the data block to obtain a second data block matrix, and calculating the time series characteristics of the second data block matrix to obtain a second time series characteristic matrix; Based on the multi-mode fusion characteristics of three-axis acceleration and three-axis angular velocity, the acceleration piece signal data and the angular velocity piece signal data are fused at the data level to obtain an acceleration matrix and an angular velocity matrix; Inputting the acceleration matrix into a first residual network to perform spatial feature extraction to obtain a first spatial feature extraction result; Inputting the angular velocity matrix into a second residual network to perform spatial feature extraction to obtain a second spatial feature extraction result, and compressing the second spatial feature extraction result and the first spatial feature extraction result using principal component analysis to obtain a spatial feature matrix; The spatial feature matrix and the second temporal feature matrix are fused to obtain a fused feature matrix.

4. The human body posture recognition method according to claim 3, wherein: The first residual network and the second residual network both include: a first weight layer, a second weight layer and an output layer, the input end of the first weight layer is connected to the input end of the output layer, the output end of the first weight layer is connected to the input end of the second weight layer, the output end of the second weight layer is connected to the input end of the output layer, the input end of the first weight layer in the first residual network serves as the input end of the first residual network, the input end of the first weight layer in the second residual network serves as the input end of the second residual network, the output end of the output layer in the first residual network serves as the output end of the first residual network, and the output end of the output layer in the second residual network serves as the output end of the second residual network.

5. The human body posture recognition method according to claim 2, wherein: The wearable device is worn on the foot, and the wearable device includes: The sensor module is used to collect the acceleration and angular velocity of the object when it completes the action to obtain posture data; A power module, used to ensure long-term reliable operation of the wearable device; The processing module is used to read the posture data collected by the sensor and send it to the host computer; The input end of the sensor module is electrically connected to the input end of the processing module, and the output end of the power module is electrically connected to the input end of the processing module.

6. The human body posture recognition method according to claim 5, characterized in that: The processing module adopts a NodeMCU chip, which is equipped with an ESP8266-WIFI chip to realize wireless communication between the processing module and the host computer, and upload the posture data read by the processing module to the host computer.

7. The human body posture recognition method according to claim 6, characterized in that: The power module includes a transmitting submodule, a receiving submodule and a switch; The first end of the transmitting submodule is connected to the input end of the processing module, the second end of the transmitting submodule is connected to the second end of the switch, the first end of the switch is connected to the output end of the receiving submodule, and the third end of the switch is connected to the input end of the processing module.

8. The human body posture recognition method according to claim 5, characterized in that: The sensor module uses a three-axis acceleration + three-axis gyroscope sensor, and the chip model is MPU6050-GY521.

Images