A fall detection method and system based on inertial sensor network

Abstract

The present invention discloses a fall detection method and system based on an inertial sensor network, and relates to the field of edge computing technology. The method comprises the following steps: using an inertial sensor network to collect motion data; using a filtering method to perform sensor posture estimation on the motion data; using a forward kinematics method to calculate the sensor posture estimation result to obtain a human skeleton posture data sequence; using the human skeleton posture data sequence to train an LSTM model; inputting the data to be tested into the trained LSTM model to perform fall detection. The present invention uses multiple inertial sensors to obtain motion data by fusing inertial sensor network data, and obtains a human skeleton posture data sequence based on a filtering method and a forward kinematics method, and uses the human skeleton posture data sequence to train an LSTM model to perform fall detection. Compared with the fall detection of a single-point inertial sensor, the detection accuracy is improved and the occurrence of false detection is greatly reduced.

Description

Technical Field

[0001] The present invention relates to the field of edge computing technology, and in particular to a fall detection method and system based on an inertial sensor network. Background Art

[0002] The statements in this section merely provide background information related to the present invention and do not necessarily constitute prior art.

[0003] Due to their poor physical fitness and weak balance ability, the elderly are very likely to be injured if they cannot get help and treatment in time when they fall, and in serious cases, their lives may be in danger. Therefore, being able to accurately and quickly detect the occurrence of falls and give an alarm in time is an important means to deal with the threat of falls for the elderly.

[0004] As technology develops, inertial sensors are gradually coming into the sight of researchers. Researchers are beginning to use information from inertial sensors to enhance the results of three-dimensional detection. Information such as the position and direction of limbs is obtained through inertial sensors. By applying this information to the human kinematics model, the position of the target person's joints in three-dimensional space can be obtained.

[0005] However, the inventors found that a single-point inertial sensor can provide less information, and using multiple inertial sensors can obtain more feature information. The human body posture is a complex continuous state during the action process. The existing algorithm does not process information deeply enough and cannot perform a comprehensive analysis of the complete human skeleton posture data. The obtained detection model has poor robustness, and therefore the accuracy of the detection results cannot be guaranteed. Summary of the invention

[0006] In view of the shortcomings of the prior art, the purpose of the present invention is to provide a fall detection method and system based on an inertial sensor network, which utilizes multiple inertial sensors to obtain human skeleton posture data by fusing inertial sensor network data, and uses the skeleton posture data to perform fall detection. Compared with fall detection by a single-point inertial sensor, the detection accuracy is improved and the occurrence of false detection is greatly reduced.

[0007] In order to achieve the above object, the present invention is implemented through the following technical solutions:

[0008] A first aspect of the present invention provides a fall detection method based on an inertial sensor network, comprising the following steps:

[0009] Collect motion data using an inertial sensor network;

[0010] Use filtering methods to estimate sensor pose from motion data;

[0011] The forward kinematics method is used to calculate the sensor posture estimation results to obtain the human skeleton posture data sequence;

[0012] Use human skeleton posture data sequence to train LSTM model;

[0013] The data to be tested is input into the trained LSTM model for fall detection.

[0014] Furthermore, motion data is collected through an inertial motion capture kit, which includes a 9-axis inertial sensor unit and a wireless communication module.

[0015] Furthermore, the specific steps of using the filtering method to estimate the sensor posture of the motion data are as follows:

[0016] Perform initial attitude calibration on the target;

[0017] For the sampling data of each inertial sensor, the sensor coordinate system attitude is estimated by using the improved Kalman filtering method.

[0018] Furthermore, the specific steps for initial posture calibration of the target are:

[0019] The target stands still with both hands raised horizontally and palms facing downwards;

[0020] The initial attitude quaternion of the sensor coordinate system is estimated using the inertial sensor readings, and the relative rotation quaternion between each inertial sensor coordinate system and the corresponding limb coordinate system is recorded.

[0021] Furthermore, for each inertial sensor sampling data, the specific steps of estimating the sensor coordinate system attitude by the improved Kalman filtering method are as follows:

[0022] Predict the quaternion of the sensor coordinate system attitude at time t;

[0023] Calculate the sensor coordinate system attitude observation at time t using optimization method;

[0024] The predicted sensor coordinate system attitude quaternion and the sensor coordinate system attitude observation are fused to obtain the final coordinate system attitude quaternion at time t.

[0025] Furthermore, the specific steps of calculating the sensor coordinate system attitude observation at time t using the optimization method are:

[0026] Construct optimization function;

[0027] The gradient descent method is used to solve the optimization function with the sensor coordinate system attitude quaternion at time t as the initial point to obtain the sensor coordinate system attitude observation at time t.

[0028] Furthermore, the specific steps of using the forward kinematics method to calculate the sensor posture estimation results to obtain the human skeleton posture data sequence are as follows:

[0029] According to the DH convention, the measured human body parameters are used to calculate the coordinates of each joint using the forward kinematics method to obtain the joint position;

[0030] The pose estimation results and joint positions are combined to obtain a complete human skeleton pose data sequence.

[0031] A second aspect of the present invention provides a fall detection system based on an inertial sensor network, comprising:

[0032] a data acquisition module configured to collect motion data using an inertial sensor network;

[0033] a posture estimation module configured to perform sensor posture estimation on the motion data using a filtering method;

[0034] A human skeleton posture data calculation module is configured to calculate the sensor posture estimation result using a forward kinematics method to obtain a human skeleton posture data sequence;

[0035] A model training module, configured to train an LSTM model using a human skeleton posture data sequence;

[0036] The fall detection module is configured to input the test data into the trained LSTM model to perform fall detection.

[0037] A third aspect of the present invention provides a medium having a program stored thereon, which, when executed by a processor, implements the steps in the fall detection method based on an inertial sensor network as described in the first aspect of the present invention.

[0038] A fourth aspect of the present invention provides a device, comprising a memory, a processor, and a program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps in the fall detection method based on an inertial sensor network as described in the first aspect of the present invention are implemented.

[0039] One or more of the above technical solutions have the following beneficial effects:

[0040] The present invention obtains motion data by fusing inertial sensor network data, obtains human skeleton posture data sequence based on filtering method and forward kinematics method, and uses human skeleton posture data sequence to train LSTM model to perform fall detection. Compared with the fall detection of single-point inertial sensor, the detection accuracy is improved and the occurrence of false detection is greatly reduced. The present invention uses LSTM neural network to classify fall events with skeleton posture data sequence, thereby improving the robustness of fall detection.

[0041] Advantages of additional aspects of the present invention will be given in part in the following description, and in part will become obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings in the specification, which constitute a part of the present invention, are used to provide a further understanding of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute improper limitations on the present invention.

[0043] Figure 1 This is a flow chart of calculating human skeleton posture data in a fall detection method based on an inertial sensor network in Example 1 of the present invention;

[0044] Figure 2 This is a schematic diagram of the wearing position of the sensor in the first embodiment of the present invention;

[0045] Figure 3 This is a structural diagram of the LSTM neural network model in Example 1 of the present invention. DETAILED DESCRIPTION

[0046] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0047] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "include" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or their combinations;

[0048] Embodiment 1:

[0049] Embodiment 1 of the present invention provides a fall detection method based on an inertial sensor network, comprising the following steps:

[0050] S1: Collect motion data using an inertial sensor network.

[0051] The motion data is collected by an inertial motion capture kit, which includes a 9-axis inertial sensor unit and a wireless communication module. In this embodiment, 15 inertial motion capture kits are used, such as Figure 2As shown, the inertial sensor units are worn on the corresponding body parts of the target person, and the specific positions are: 1. Waist, 2. Chest, 3. Head, 4. Left forearm, 5. Left forearm, 6. Left hand, 7. Right forearm, 8. Right forearm, 9. Right hand, 10. Left thigh, 11. Left calf, 12. Left foot, 13. Right thigh, 14. Right calf, 15. Right foot. The motion data of daily behaviors such as falling, walking, jogging, going up and down stairs, standing, sitting, etc. are collected respectively, and different labels are used for marking according to different behaviors.

[0052] S2: Use filtering methods to estimate the sensor posture of motion data.

[0053] S2.1: Initially calibrate the target’s attitude;

[0054] S2.1.1: The target stands still with both hands raised horizontally and palms facing downward.

[0055] S2.1.2: Estimate the initial attitude quaternion q0 of the sensor coordinate system using the inertial sensor readings.

[0056] S2.2: For the sampling data of the i-th inertial sensor at sampling time t, the sampling data includes the three-axis angular velocity collected by the gyroscope, the three-axis acceleration collected by the accelerometer, and the magnetic field strength signal collected by the magnetometer. Estimate the sensor coordinate system attitude through the improved Kalman filtering method.

[0057] S2.2.1: Predict the quaternion q of the sensor coordinate system attitude at time t t :

[0058] q t =A t q t-1 +n t

[0059] Among them, q t-1 The quaternion representing the sensor coordinate system attitude at time t-1, q t Represents the sensor coordinate system attitude quaternion at time t, A t represents the system transfer matrix at time t, n t represents the system transfer white noise at time t.

[0060] S2.2.2: Calculate the sensor coordinate system attitude observation z at time t using the optimization method t :

[0061] (1) Construct optimization function J:

[0062]

[0063]

[0064] Among them, a t represents the accelerometer reading at time t, m t represents the reading of the magnetometer at time t, a represents the acceleration vector in the reference coordinate system, and m represents the magnetic field intensity vector in the reference coordinate system.

[0065] (2) Using the gradient descent method, the sensor coordinate system attitude quaternion q at time t is obtained. t Solve the optimization function J as the initial point to obtain the sensor coordinate system attitude observation z at time t t .

[0066] S2.2.3: Fusion predicted sensor coordinate system attitude quaternion q t and the sensor coordinate system attitude observation z t , get the final coordinate system attitude quaternion q′ at time t t :

[0067] z t =H t q t +v t

[0068] Among them, H t represents the system observation matrix at time t, v t represents the system observation white noise at time t.

[0069] S3: Use the forward kinematics method to calculate the sensor posture estimation results to obtain the human skeleton posture data sequence.

[0070] S3.1: According to the DH convention, the measured human body parameters are used to calculate the coordinates of each joint using the forward kinematics method to obtain the joint position;

[0071] The pose estimation results and joint positions are combined to obtain a complete human skeleton pose data sequence. The complete human skeleton pose data sequence includes the position and pose of each joint of the human body, so it can fully reflect the state of the human body at a certain moment.

[0072] S4: Use the human skeleton posture data sequence to train the LSTM model. The LSTM model is as follows: Figure 2 shown.

[0073] S5: Input the test data into the trained LSTM model to perform fall detection.

[0074] Using the collected fall and ADL data segments with labels to train the LSTM model, it is possible to classify the motion data into falls and ADL. The data to be tested is input into the trained LSTM model, and the fall or ADL behavior results are output.

[0075] Embodiment 2:

[0076] Embodiment 2 of the present invention provides a fall detection system based on an inertial sensor network, comprising:

[0077] a data acquisition module configured to collect motion data using an inertial sensor network;

[0078] a posture estimation module configured to perform sensor posture estimation on the motion data using a filtering method;

[0079] A human skeleton posture data calculation module is configured to calculate the sensor posture estimation result using a forward kinematics method to obtain a human skeleton posture data sequence;

[0080] A model training module, configured to train an LSTM model using a human skeleton posture data sequence;

[0081] The fall detection module is configured to input the test data into the trained LSTM model to perform fall detection.

[0082] Embodiment three:

[0083] Embodiment 3 of the present invention provides a medium on which a program is stored. When the program is executed by a processor, the steps of the fall detection method based on an inertial sensor network as described in Embodiment 1 of the present invention are implemented. The steps are:

[0084] Collect skeleton pose data sequence samples;

[0085] Using inertial sensor network to estimate sensor attitude of skeleton attitude data sequence samples by filtering method;

[0086] The human skeleton posture data is obtained by calculating the sensor posture estimation results through the forward kinematics method;

[0087] Use human skeleton posture data to train the LSTM model;

[0088] The data to be tested is input into the trained LSTM model for fall detection.

[0089] The detailed steps are the same as those of the fall detection method based on the inertial sensor network provided in Example 1, and will not be repeated here.

[0090] Embodiment 4:

[0091] Embodiment 4 of the present invention provides a device, including a memory, a processor, and a program stored in the memory and executable on the processor. When the processor executes the program, the steps in the fall detection method based on the inertial sensor network as described in Embodiment 1 of the present invention are implemented, and the steps are:

[0092] Collect skeleton pose data sequence samples;

[0093] Using inertial sensor network to estimate sensor attitude of skeleton attitude data sequence samples by filtering method;

[0094] The human skeleton posture data is obtained by calculating the sensor posture estimation results through the forward kinematics method;

[0095] Use human skeleton posture data to train the LSTM model;

[0096] The data to be tested is input into the trained LSTM model for fall detection.

[0097] The detailed steps are the same as those of the fall detection method based on the inertial sensor network provided in Example 1, and will not be repeated here.

[0098] The steps involved in the above embodiments 2, 3 and 4 correspond to the method embodiment 1, and the specific implementation methods can refer to the relevant description part of embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood to include any medium that can store, encode or carry an instruction set for execution by a processor and enable the processor to execute any method in the present invention.

[0099] Those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computer device, or alternatively, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. The present invention is not limited to any specific combination of hardware and software.

[0100] Although the above describes the specific implementation mode of the present invention in conjunction with the accompanying drawings, it is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art on the basis of the technical solution of the present invention without creative work are still within the scope of protection of the present invention.

Claims

1. A fall detection method based on an inertial sensor network, characterized in that: The following steps are involved: Collect motion data using an inertial sensor network; Use filtering methods to estimate sensor pose from motion data; The forward kinematics method is used to calculate the sensor posture estimation results to obtain the human skeleton posture data sequence; Use human skeleton posture data sequence to train LSTM model; Input the data to be tested into the trained LSTM model for fall detection; Among them, motion data is collected through an inertial motion capture kit, which includes a 9-axis inertial sensor unit and a wireless communication module. Multiple inertial motion capture kits are used, and inertial sensor units are worn on the corresponding body parts of the target person to collect motion data of daily behaviors such as falling, walking, jogging, going up and down stairs, standing, and sitting, and different labels are used for marking according to different behaviors. Among them, the LSTM model is trained using the collected fall and ADL data fragments with labels, which can classify the motion data into falls and ADLs; the data to be tested is input into the trained LSTM model to output the fall or ADL behavior results.

2. The fall detection method based on an inertial sensor network as claimed in claim 1, characterized in that: The specific steps of using filtering method to estimate the sensor posture of motion data are: Perform initial attitude calibration on the target; For the sampling data of each inertial sensor, the sensor coordinate system attitude is estimated by using the improved Kalman filtering method.

3. The fall detection method based on inertial sensor network as claimed in claim 2, characterized in that: The specific steps for initial posture calibration of the target are: The target stands still with both hands raised horizontally and palms facing downwards; The initial attitude quaternion of the sensor coordinate system is estimated using the inertial sensor readings, and the relative rotation quaternion between each inertial sensor coordinate system and the corresponding limb coordinate system is recorded.

4. The fall detection method based on inertial sensor network as claimed in claim 2, characterized in that: For the sampled data of each inertial sensor, the specific steps of estimating the sensor coordinate system attitude by the improved Kalman filtering method are as follows: Predict the quaternion of the sensor coordinate system attitude at time t; Calculate the sensor coordinate system attitude observation at time t using optimization method; The predicted sensor coordinate system attitude quaternion and the sensor coordinate system attitude observation are fused to obtain the final coordinate system attitude quaternion at time t.

5. The fall detection method based on inertial sensor network as claimed in claim 4, characterized in that: The specific steps of calculating the attitude observation of the sensor coordinate system at time t using the optimization method are: Construct optimization function; The gradient descent method is used to solve the optimization function with the sensor coordinate system attitude quaternion at time t as the initial point to obtain the sensor coordinate system attitude observation at time t.

6. The fall detection method based on inertial sensor network as claimed in claim 1, characterized in that: The specific steps of using the forward kinematics method to calculate the sensor posture estimation results to obtain the human skeleton posture data sequence are: According to the DH convention, the measured human body parameters are used to calculate the coordinates of each joint using the forward kinematics method to obtain the joint position; The pose estimation results and joint positions are combined to obtain a complete human skeleton pose data sequence.

7. A fall detection system based on an inertial sensor network, characterized in that: include: a data acquisition module configured to collect motion data using an inertial sensor network; a posture estimation module configured to perform sensor posture estimation on the motion data using a filtering method; A human skeleton posture data calculation module is configured to calculate the sensor posture estimation result using a forward kinematics method to obtain a human skeleton posture data sequence; A model training module, configured to train an LSTM model using a human skeleton posture data sequence; The fall detection module is configured to input the test data into the trained LSTM model to perform fall detection; Among them, motion data is collected through an inertial motion capture kit, which includes a 9-axis inertial sensor unit and a wireless communication module. Multiple inertial motion capture kits are used, and inertial sensor units are worn on the corresponding body parts of the target person to collect motion data of daily behaviors such as falling, walking, jogging, going up and down stairs, standing, and sitting, and different labels are used for marking according to different behaviors. Among them, the LSTM model is trained using the collected fall and ADL data fragments with labels, which can classify the motion data into falls and ADLs; the data to be tested is input into the trained LSTM model to output the fall or ADL behavior results.

8. A computer-readable storage medium, characterized in that: A plurality of instructions are stored therein, and the instructions are suitable for being loaded by a processor of a terminal device and executing the fall detection method based on an inertial sensor network described in any one of claims 1 to 6.

9. A terminal device, characterized in that: The invention comprises a processor and a computer-readable storage medium, wherein the processor is used to implement various instructions; and the computer-readable storage medium is used to store multiple instructions, wherein the instructions are suitable for being loaded by the processor and executing the fall detection method based on an inertial sensor network as described in any one of claims 1 to 6.

Images