High-risk action real-time prediction method and system based on multi-inertia sensing

Abstract

The invention provides a high-risk action real-time prediction method and system based on multi-inertia sensing, and relates to the technical field of behavior prediction.The method comprises the steps that angular velocity, acceleration and magnetic field information data are obtained, and posture information of limbs is obtained through an extended Kalman data fusion algorithm; acquiring length information of the limb, and calculating to obtain a three-dimensional relative position of a key part for prediction in combination with posture information of the limb; calculating a human body balance factor through the obtained three-dimensional relative position, and carrying out unbalance judgment; performing drop judgment through the obtained three-dimensional relative position in combination with a speed component and a displacement component; when it is judged that the human body is unbalanced and is in a falling state at the same time, real-time early warning is conducted. According to the invention, the accuracy of high-risk action recognition is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of behavior prediction, and in particular relates to a method and system for real-time prediction of high-risk actions based on multi-inertial sensors. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] As consumer-grade MEMS inertial measurement units (IMUs) are widely used in daily life, wearable human posture capture devices based on IMU technology have emerged. IMUs are used and developed in human posture monitoring due to their small size, low cost, and few scene restrictions, especially in early warning of high-risk human behaviors (such as falls).

[0004] When it comes to the selection of IMUs for real-time warning of high-risk behaviors, there can be one or more IMUs; the methods used include motion feature analysis, deep learning-based modeling, etc.

[0005] Using an IMU sensor is a common solution. It has the advantages of easy application and low cost, but it also has obvious disadvantages. First, the behavioral characteristics obtained by a sensor cannot completely distinguish between high-risk behaviors and daily behaviors (such as sitting, running, going up and down stairs, etc.), which can easily lead to misidentification or missed identification in the prediction process. Secondly, using a sensor can only obtain posture information. The speed obtained by integrating the acceleration cannot be effectively applied due to drift problems.

[0006] The solution of using multiple sensors, or comparing sensor data from multiple limbs and selecting a limb position with obvious characteristics to place the sensor, or using deep learning methods to fuse the features of data collected by multiple sensors, also has the following problems: the former is actually equivalent to selecting a sensor with obvious placement characteristics, while the latter requires collecting different types of data, training and adjusting models, which is time-consuming and labor-intensive, and the training of a complete model requires a large amount of computation, making it difficult to implement real-time recognition functions on embedded devices.

[0007] At the same time, the traditional high-risk behavior instant prediction algorithm uses waist sensors to obtain human posture information and identify high-risk behaviors. It cannot distinguish between more intense daily behaviors and falling movements, and often adopts a compromise approach, that is, in slow daily activities, it guarantees a certain recognition rate. In slightly more intense daily activities, the proportion of false recognition is high, or the proportion of missed recognition is high, making it difficult to ensure application in actual scenarios. Summary of the invention

[0008] In order to overcome the shortcomings of the above-mentioned prior art, the present invention provides a method and system for instant prediction of high-risk actions based on multi-inertial sensing, which adopts multiple inertial sensors to obtain more limb information, such as relative speed, displacement, etc., to improve the accuracy of high-risk action recognition. At the same time, it is also oriented to the embedded device side to realize real-time behavior monitoring and high-risk instant prediction functions.

[0009] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solutions: A first aspect of the present invention provides a method for real-time prediction of high-risk actions based on multi-inertial sensing.

[0010] A high-risk action instant prediction method based on multi-inertial sensing, comprising: Acquire angular velocity, acceleration and magnetic field information data, and obtain limb posture information through extended Kalman data fusion algorithm; Obtain the length information of the limbs, and combine it with the posture information of the limbs to calculate the three-dimensional relative position of the key parts used for prediction; By obtaining the three-dimensional relative position, the human body balance factor is calculated to determine the imbalance; The fall is judged by combining the velocity component and the displacement component through the obtained three-dimensional relative position; When it is determined that the human body is out of balance and in a falling state at the same time, an immediate warning is issued.

[0011] A second aspect of the present invention provides a high-risk action instant prediction system based on multi-inertial sensing.

[0012] A high-risk action instant prediction system based on multi-inertial sensing, comprising: The posture information acquisition module is configured to: acquire angular velocity, acceleration and magnetic field information data, and obtain the posture information of the limbs through an extended Kalman data fusion algorithm; The three-dimensional position calculation module is configured to: obtain the length information of the limbs, and calculate the three-dimensional relative position of the key parts used for prediction in combination with the posture information of the limbs; The imbalance judgment module is configured to: calculate the human body balance factor through the obtained three-dimensional relative position and make imbalance judgment; The fall judgment module is configured to: perform fall judgment by combining the velocity component and the displacement component through the obtained three-dimensional relative position; The early warning module is configured to issue an immediate early warning when it is determined that the human body is in an unbalanced and falling state.

[0013] The third aspect of the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in a method as described in the first aspect of the present invention are implemented.

[0014] A fourth aspect of the present invention provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps in a method as described in the first aspect of the present invention.

[0015] A fifth aspect of the present invention provides a computer program product comprising instructions, which, when running on a computer, enables the computer program to implement the steps in a method as described in the first aspect of the present invention when executed by a processor.

[0016] One or more of the above technical solutions have the following beneficial effects: The present invention adopts multiple inertial sensors, combined with lower limb posture and limb chain structure, to obtain more high-precision limb information, such as relative speed, displacement, etc., providing a reliable data source for balance conditions and falling process conditions, improving the accuracy of high-risk action recognition, and reducing the misrecognition rate.

[0017] The present invention integrates the movement characteristics of multiple parts by calculating the judgment method of the human body balance factor, and can have obvious distinction in more occasions (such as sitting down, sitting and falling), thereby improving the recognition accuracy.

[0018] The present invention integrates the human body balance information, the speed of the falling process, and the height information from the ground. In the real-time prediction process, it can effectively distinguish various forms of falls from daily behaviors and provide preparation time for activating the human body protection mechanism.

[0019] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0021] Figure 1 This is a flow chart of a method for real-time prediction of high-risk actions based on multi-inertial sensors according to Embodiment 1 of the present invention. DETAILED DESCRIPTION

[0022] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present invention. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present invention belongs.

[0023] It should be noted that the terms used herein are for describing specific embodiments only and are not intended to be limiting of exemplary embodiments according to the present invention.

[0024] In the absence of conflict, the embodiments of the present invention and the features of the embodiments may be combined with each other.

[0025] The overall idea proposed by the present invention is to use multiple inertial sensors to monitor different motion characteristic data of limbs, integrate the postures of different parts and the chain structure of human joints, calculate the three-dimensional relative position information, and finally realize the process of predicting high-risk behaviors through the balance information, speed information, and position information of the human body.

[0026] Embodiment 1 This embodiment discloses a method for instant prediction of high-risk actions based on multi-inertial sensing, which adopts multiple inertial sensors, combines the lower limb posture and limb chain structure to obtain more high-precision limb information, and calculates the human body balance factor, integrates the human body balance information, and combines the velocity component and the displacement component to make a fall judgment, thereby improving the accuracy of high-risk action recognition and reducing the misrecognition rate.

[0027] In order to explain this embodiment more clearly, Figure 1 As shown in the figure, the implementation process of real-time prediction of high-risk actions based on multi-inertial sensing can be specifically described as follows: Step 1: Obtain angular velocity, acceleration and magnetic field information data, and obtain limb posture information through the extended Kalman data fusion algorithm.

[0028] In this embodiment, angular velocity, acceleration and magnetic field information data are obtained, specifically, 5 inertial sensors are used to monitor the angular velocity, acceleration and magnetic field information and other motion characteristic data of different parts of the limbs.

[0029] Among them, each inertial sensor specifically includes a three-axis accelerometer, a three-axis gyroscope and a magnetometer, and an inertial sensor is placed on the waist, left thigh, right thigh, left calf and right calf respectively.

[0030] In this embodiment, the posture information of the limbs is obtained by using the extended Kalman data fusion algorithm, specifically: Prediction part: ; ; Kalman gain: ; Updated part: ; ; in, yes The attitude quaternion at the moment, the superscripts (- / +) represent the prior and posterior values ​​respectively, are process noise and measurement noise matrices respectively, yes The state matrix at time, is the state covariance matrix, yes The measurement matrix at time, yes The measured value at the moment, is the identity matrix.

[0031] Euler angles can be expressed as: ; in, are the Euler angles of rotation around the X, Y, and Z axes, respectively. is the real part of the attitude quaternion, Represent the three imaginary parts of the attitude quaternion, forming .

[0032] The posture information of the limbs is obtained specifically as the three-axis posture, which can be expressed as: ; in, i The limb parts are numbered, 1-5 represents waist, left thigh, right thigh, left calf, and right calf respectively.

[0033] Step 2: Obtain the length information of the limbs, and combine it with the posture information of the limbs to calculate the three-dimensional relative position of the key parts used for prediction.

[0034] In this embodiment, the length information of the limbs is obtained, specifically: the thigh length is obtained , calf length .

[0035] In this embodiment, the key parts are specifically: left and right knees and ankles.

[0036] In this embodiment, relative displacement is calculated, that is, the posture information of the limbs is used, combined with the thigh and calf length information, to calculate the three-dimensional relative position information of the left and right knees and ankles. Currently, accurate relative displacement cannot be obtained through a single sensor, and accurate values ​​of relative displacement of human body parts can be obtained through multiple inertial sensors.

[0037] The specific process is as follows: Calculate the left knee With right knee Relative position: ; Calculate the left ankle position: ; Calculate the right ankle position: ; in, Respectively represent the rotation matrix of the body parts relative to the waist, Contains a three-dimensional representation of the length of the calf and leg.

[0038] Step 3: Calculate the human body balance factor through the obtained three-dimensional relative position and make imbalance judgment.

[0039] Compared with daily behavior, when the human body performs high-risk actions, the limbs will gradually lose balance. When the human body is in an unbalanced state, if there is no external support, it will fall. Therefore, imbalance judgment can be used as a prerequisite for detecting the falling process.

[0040] There are many methods for detecting human imbalance. In this embodiment, the human balance factor is calculated by the obtained three-dimensional relative position. Specifically, the calculation can be performed by the following three-dimensional relative position information: The human body balance factor is calculated by the horizontal position of the center of the two feet relative to the waist, which is specifically expressed as follows: ; in, is the projection of the centers of the two ankles on the horizontal plane, When the human body is in a balanced state, the balance factor is closer to 1, and when the human body is in an unbalanced state, the balance factor is closer to 0.

[0041] Step 4: Perform a fall judgment by combining the obtained three-dimensional relative position with the velocity component and the displacement component.

[0042] The falling process of a human body is mainly judged by the vertical falling speed, and the severity of the fall is judged by the distance of the buttocks relative to the ground.

[0043] When the velocity component of the center of the two feet relative to the waist in the vertical direction is greater than a certain value, it can be determined that the human body is in a falling state. Combined with the relative distance of the waist, the possible severity of the falling state can be determined.

[0044] In this embodiment, the fall is judged by the velocity component and the displacement component, which can be specifically expressed as: ; in, are the velocity component and displacement component of the center of the two feet relative to the waist in the vertical direction, are velocity and displacement thresholds respectively.

[0045] In comparison, due to the accuracy of the calculated values, the balance factor is used to obtain the balance degree of the human body, and it is possible to distinguish between daily dynamic behaviors (such as walking, running, jumping, going up and down stairs, sitting down, etc.) and the falling process; then the speed is used to determine whether there is a falling trend, and to distinguish those unstable behavior states that are supported by external objects (such as sitting, lying, etc.); and then combined with the relative distance between the ankle and the waist, the urgency of the need to activate protection is determined. However, the current fall process prediction algorithm based on a single inertial sensor cannot accurately obtain the relative speed and displacement, and its application will bring some data inaccuracies (such as data drift, etc.), causing some overlaps in behavioral characteristics, which are difficult to accurately distinguish and judge. These problems are solved in this patent method.

[0046] Step 5: When it is determined that the human body is out of balance and in a falling state, an immediate warning is issued.

[0047] Embodiment 2 The purpose of this embodiment is to provide a high-risk action instant prediction system based on multi-inertial sensing, including: The posture information acquisition module is configured to: acquire angular velocity, acceleration and magnetic field information data, and obtain the posture information of the limbs through an extended Kalman data fusion algorithm; The three-dimensional position calculation module is configured to: obtain the length information of the limbs, and use the posture information of the limbs to calculate the three-dimensional relative position of the key parts used for prediction; The imbalance judgment module is configured to: calculate the human body balance factor through the obtained three-dimensional relative position and make imbalance judgment; The fall judgment module is configured to: perform fall judgment by combining the velocity component and the displacement component through the obtained three-dimensional relative position; The early warning module is configured to issue an immediate early warning when it is determined that the human body is in an unbalanced and falling state.

[0048] Embodiment 3 The purpose of this embodiment is to provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above method when executing the program.

[0049] Embodiment 4 The purpose of this embodiment is to provide a computer-readable storage medium.

[0050] A computer-readable storage medium stores a computer program, which executes the steps of the above method when executed by a processor.

[0051] Embodiment 5 The purpose of this embodiment is to provide a computer program product containing instructions, which, when running on a computer, enables the computer to execute the methods and functions involved in any of the above embodiments. The steps involved in the apparatus of the above embodiment correspond to the method embodiment 1, and the specific implementation method can refer to the relevant description part of embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood to include any medium that can store, encode or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.

[0052] Those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computer device, or alternatively, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0053] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A high-risk action instant prediction method based on multi-inertial sensing, characterized in that: include: Acquire angular velocity, acceleration and magnetic field information data, and obtain limb posture information through extended Kalman data fusion algorithm; Obtain the length information of the limbs, and combine it with the posture information of the limbs to calculate the three-dimensional relative position of the key parts used for prediction; By obtaining the three-dimensional relative position, the human body balance factor is calculated to determine the imbalance; The fall is judged by combining the velocity component and the displacement component through the obtained three-dimensional relative position; When it is determined that the human body is out of balance and in a falling state at the same time, an immediate warning is issued.

2. The high-risk action instant prediction method based on multi-inertial sensing as claimed in claim 1, characterized in that: Specifically, multiple inertial sensors are used to obtain angular velocity, acceleration, and magnetic field information data of different parts of the limbs; Among them, each inertial sensor specifically includes a three-axis accelerometer, a three-axis gyroscope and a magnetometer, and an inertial sensor is placed on the waist, left thigh, right thigh, left calf and right calf respectively.

3. The high-risk action instant prediction method based on multi-inertial sensing as claimed in claim 1, characterized in that: The posture information of the limbs is obtained by extending the Kalman data fusion algorithm, specifically: Prediction part: Kalman gain: Updated part: in, yes The attitude quaternion at the moment, the superscript - or + indicates the prior and a posteriori values ​​respectively, are process noise and measurement noise matrices respectively, yes The state matrix at time, is the state covariance matrix, yes The measurement matrix at time, yes The measured value at the moment, is the identity matrix; The posture information of the limbs is obtained specifically as the three-axis posture, which can be expressed by the Euler angle as follows: in, i The limb parts are numbered, 1-5 represents waist, left thigh, right thigh, left calf, and right calf respectively.

4. The high-risk action instant prediction method based on multi-inertial sensing as claimed in claim 1, characterized in that: The length information of the limbs is obtained, and combined with the posture information of the limbs, the three-dimensional relative position of the key parts used for prediction is calculated. The specific process is as follows: Calculate the left knee With right knee Relative position: Calculate the left ankle position: Calculate the right ankle position: in, Respectively represent the rotation matrix of the body parts relative to the waist, Contains a three-dimensional representation of the length of the calf and leg.

5. The high-risk action instant prediction method based on multi-inertial sensing as claimed in claim 1, characterized in that: The method of calculating the human body balance factor and making imbalance judgment by using the obtained three-dimensional relative position specifically includes: The human body balance factor is calculated by the horizontal position of the center of the two feet relative to the waist. The specific expression is as follows: in, is the projection model of the two ankle centers on the horizontal plane, They are the length of the thigh and calf respectively; when the human body is in a balanced state, the balance factor is closer to 1, and when the person is in an unbalanced state, the balance factor is closer to 0.

6. The high-risk action instant prediction method based on multi-inertial sensing as claimed in claim 1, characterized in that: The three-dimensional relative position obtained is combined with the velocity component and the displacement component to perform a fall judgment, which specifically includes: The fall is judged by the velocity component and the displacement component, which can be specifically expressed as: in, are the velocity component and displacement component of the center of the two feet relative to the waist in the vertical direction, are velocity and displacement thresholds respectively.

7. A high-risk action instant prediction system based on multi-inertial sensing, characterized in that: include: The posture information acquisition module is configured to: acquire angular velocity, acceleration and magnetic field information data, and obtain the posture information of the limbs through an extended Kalman data fusion algorithm; The three-dimensional position calculation module is configured to: obtain the length information of the limbs, and calculate the three-dimensional relative position of the key parts used for prediction in combination with the posture information of the limbs; The imbalance judgment module is configured to: calculate the human body balance factor through the obtained three-dimensional relative position and make imbalance judgment; The fall judgment module is configured to: perform fall judgment by combining the velocity component and the displacement component through the obtained three-dimensional relative position; The early warning module is configured to issue an immediate early warning when it is determined that the human body is in an unbalanced and falling state.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the method described in any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 6 are performed.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images