An elderly fall detection and alarm system based on multi-modal sensors

Abstract

The present application relates to the technical field of fall detection, and relates to an old person fall detection and alarm system based on a multi-modal sensor.The present application collects the height of the center of gravity from the ground in real time, combines the allowable height deviation of the detection object, analyzes whether the center of gravity is moving downward, calculates the azimuth of the center of gravity displacement based on the displacement vector of the center of gravity in the reference rectangular coordinate system, constructs a real-time center of gravity feature parameter set, obtains the azimuth of the hands and the distance from the hands to the center of gravity within a set time period, combines the center of gravity feature parameter set and the historical normal behavior data set of the detection object, determines whether the current state is reasonable, if not, analyzes the overall offset consistency based on the azimuth of the center of gravity offset and the azimuth of the hands, combines the time sequence of the height of the center of gravity from the ground and the height of the hands from the ground to determine whether a fall has occurred, reduces the false alarm and missed alarm probability, improves the recognition accuracy of the detection system, and guarantees the personal safety of the old people.

Description

Technical Field

[0001] This invention relates to the field of fall detection technology, and specifically to a fall detection and alarm system for the elderly based on a multimodal sensor. Background Technology

[0002] As the population ages, the safety of the elderly at home and while traveling has become a major social concern. Falls are a common accident in the daily lives of the elderly, and if they are not detected in time, they will seriously threaten their lives and health, and impose a heavy care and medical burden on families and society. Therefore, fall detection systems are essential to ensure the safety of the elderly.

[0003] However, existing technologies have the following problems: 1. Existing technologies mostly determine falls by detecting the height of a person's center of gravity and acceleration in combination with fixed thresholds, without taking into account the differences in height and daily habits of the elderly. This results in a lack of specificity in fall detection, frequent false alarms, and difficulty in meeting the requirements of accuracy and timeliness in fall detection for the elderly.

[0004] 2. Existing technologies mostly focus on the overall center of gravity for fall detection, but do not consider combining hand features for collaborative analysis. During a fall, the abnormal shift of the center of gravity is often highly correlated with the emergency reaction of the hands, which has important reference value. If the hand features are ignored and the change of the center of gravity is analyzed alone, it is difficult to distinguish whether the downward shift of the center of gravity is an abnormal state caused by a fall or a reasonable state of daily behavior. Summary of the Invention

[0005] This invention aims to address the shortcomings of existing technologies by providing a fall detection and alarm system for the elderly based on multimodal sensors. By focusing on analyzing detection data from the hands and a smart waist belt, the system comprehensively detects falls, thereby improving the accuracy of fall detection.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: a fall detection and alarm system for the elderly based on multimodal sensors, comprising a center of gravity detection module, a hand state judgment module, a comprehensive analysis module, and a fall determination module. The connection relationships between the modules are as follows: the center of gravity detection module is connected to the hand state judgment module, and the comprehensive analysis module is connected to both the hand state judgment module and the fall determination module.

[0007] The center of gravity detection module collects the height of the center of gravity above the ground in real time to analyze whether the center of gravity has shifted downward. When downward shift of the center of gravity is detected, the module calculates the azimuth angle of the center of gravity offset based on the displacement vector of the current center of gravity in the reference three-dimensional coordinate system and constructs a real-time center of gravity feature parameter set.

[0008] The hand state judgment module takes the azimuth angle of both hands and the offset distance of both hands from the center of gravity within a set time period starting from the reference time point. Combined with the real-time center of gravity feature parameter set and the historical normal behavior dataset of the detected object, it determines whether the current state is reasonable.

[0009] The comprehensive analysis module, when it detects that the current state is unreasonable, analyzes the overall offset consistency judgment result based on the center of gravity offset azimuth angle and the offset azimuth angle of both hands.

[0010] The fall detection module analyzes whether a fall has occurred based on a time sequence of the center of gravity height above the ground and the height of both hands above the ground, combined with the result of offset consistency. When a fall is detected, a corresponding alarm is issued.

[0011] Compared with the prior art, the present invention has the following beneficial effects: (1) The present invention analyzes whether the center of gravity has shifted downward by extracting the height of the center of gravity in real time and combining it with the allowable height deviation of the detection object. It ensures early identification of the risk of falling from the time dimension, solves the adaptability defect of the traditional fixed threshold, and makes the judgment result more in line with the actual physical movement state of the elderly.

[0012] (2) By calculating the azimuth angle of the center of gravity offset, the present invention constructs a real-time set of center of gravity feature parameters, realizes the quantification of the direction of the center of gravity movement of the detected object and the comprehensive and real-time perception of the body movement state, avoids the one-sidedness of information from monitoring a single part, and improves the accuracy of the judgment results.

[0013] (3) By combining the historical normal behavior dataset of the detection object, the present invention determines whether the state of the center of gravity shift is reasonable, thereby eliminating the center of gravity shift caused by normal daily behavior, locking the unreasonable state of abnormal center of gravity shift for further analysis, reducing the probability of normal behavior being misjudged as abnormal, and improving the recognition accuracy of the detection system.

[0014] (4) This invention analyzes the overall offset consistency judgment results based on the center of gravity offset azimuth angle and the offset azimuth angle of both hands. After initially judging the state to be unreasonable, the offset consistency analysis further filters out abnormal movement states with fall characteristics, excludes random abnormal limb movements that are not fall-related, and reduces invalid analysis of subsequent fall judgments.

[0015] (5) This invention comprehensively analyzes whether a fall has occurred by combining the time sequence of the center of gravity height and the height of the hands from the ground with the determination result of the offset consistency. When a fall is determined, a corresponding alarm is issued, which breaks through the limitations of a single height threshold or a single movement feature determination, making the fall determination more in line with the actual limb movement pattern of a fall, effectively reducing the probability of misjudgment and missed reporting, and ensuring the personal safety of the elderly. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of the system module connections of the present invention.

[0018] Figure 2 This is a schematic diagram illustrating the specific steps involved in determining whether the current state is reasonable in this invention.

[0019] Figure 3 This is a schematic diagram illustrating the specific steps involved in the comprehensive analysis of whether a fall has occurred in this invention. Detailed Implementation

[0020] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the invention. Furthermore, it should be understood that, for ease of description, the dimensions of the various parts shown in the drawings are not drawn to actual scale.

[0021] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the invention or its application or use. Techniques, methods, and apparatus known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and apparatus should be considered part of the specification.

[0022] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0023] This invention collects real-time data on the height of the center of gravity from the ground, combines this data with the allowable height deviation of the detected object, and analyzes whether the center of gravity has shifted downwards. Based on the displacement vector of the center of gravity in the reference rectangular coordinate system, it calculates the azimuth angle of the center of gravity offset and constructs a real-time center of gravity feature parameter set. It obtains the azimuth angles of the hands' offset and the offset distance of the hands from the center of gravity within a set time period. Combining the center of gravity feature parameter set with the historical normal behavior dataset of the detected object, it determines whether the current state is reasonable. If unreasonable, it analyzes the overall offset consistency based on the azimuth angles of the center of gravity offset and the azimuth angles of the hands' offset, and determines whether a fall has occurred based on the time sequence of the height of the center of gravity from the ground and the height of the hands' ground. This reduces the probability of false alarms and missed alarms, improves the recognition accuracy of the detection system, and protects the personal safety of the elderly.

[0024] Please see Figure 1 As shown, this invention provides a fall detection and alarm system for the elderly based on multimodal sensors, including a center of gravity detection module, a hand state judgment module, a comprehensive analysis module, and a fall determination module. The connections between the modules are as follows: the center of gravity detection module is connected to the hand state judgment module, and the comprehensive analysis module is connected to both the hand state judgment module and the fall determination module.

[0025] The center of gravity detection module analyzes whether the center of gravity has shifted downwards and constructs a real-time center of gravity feature parameter set.

[0026] Considering that a downward shift in the center of gravity is a typical physiological characteristic of elderly people before a fall, this study analyzes whether the center of gravity has shifted downward to accurately capture the precursor features of a fall and achieve early identification of abnormal body postures.

[0027] Furthermore, considering the physiological differences among elderly individuals, such as varying heights and body shapes, using a fixed threshold to determine the center of gravity downwards would lead to a lack of specificity in the detection. Therefore, it is necessary to design an allowable height deviation for each detection subject, and to personalize the settings based on the actual body data of the subject when first wearing the smart device, adapting to the physiological characteristics of different elderly individuals and avoiding misjudgments caused by a fixed threshold.

[0028] Based on this, the specific steps of the center of gravity detection module are as follows: S11, obtain the height of the center of gravity from the ground in real time from the smart device worn by the detection object.

[0029] In this embodiment, the smart device includes a smart bracelet worn on the left and right hands of the detection subject, and a smart belt worn around the waist of the detection subject. The left and right smart bracelets and the smart belt are all equipped with multimodal sensors, capable of collecting data such as geographical location, speed, and altitude in real time. Furthermore, the center of gravity mentioned in this invention refers to the geometric center of the smart belt.

[0030] S12. Combining the allowable height deviation of the detected object with analysis of whether the center of gravity has shifted downwards, when a downward shift of the center of gravity is detected, the azimuth angle of the center of gravity offset is calculated based on the displacement vector of the current center of gravity position coordinates in the reference three-dimensional coordinate system relative to the reference time point, and a real-time center of gravity feature parameter set is constructed. The specific implementation steps include: S121. Calculating the absolute difference between the current center of gravity height above the ground and the center of gravity height above the ground at adjacent historical time points. If the current center of gravity height above the ground is less than the center of gravity height above the ground at adjacent historical time points, and the absolute difference is greater than the allowable height deviation of the detected object, it is determined that the center of gravity has shifted downwards.

[0031] In a specific embodiment of the present invention, the specific method for setting the allowable height deviation includes: First, when the subject wears the smart device for the first time, the subject performs standing and walking actions according to the guidance of the caregiver and the smart device, and collects a dataset of the subject's center of gravity height from the ground during the standing and walking process.

[0032] Secondly, the average value of the center of gravity height above the ground during the standing process is calculated and recorded as the standing center of gravity height of the detected object.

[0033] Then, the absolute difference between the height of each center of gravity above the ground and the height of the standing center of gravity of the test subject during the walking process is calculated to form a difference sequence.

[0034] Next, outliers in the difference sequence are removed, and the maximum value of the remaining difference is taken as the allowable height deviation. The steps for removing outliers are as follows: calculate the mean and standard deviation of the difference sequence, and record the differences in the difference sequence that are greater than the sum of the mean and three times the standard deviation as outliers to be removed.

[0035] This invention analyzes whether the center of gravity has shifted downward by extracting the height of the center of gravity in real time and combining it with the allowable height deviation of the detected object. It ensures early identification of fall risk from a time dimension, solves the adaptability defects of traditional fixed thresholds, and makes the judgment results more consistent with the actual physical movement status of the elderly.

[0036] S122. Obtain the coordinates of the current center of gravity and the center of gravity at the reference time point in the established reference three-dimensional coordinate system along the X and Y axes to obtain the horizontal displacement vector.

[0037] In this embodiment, the adjacent historical time points of the current time are used as the reference time points. Based on the geometric center of the smart belt at the reference time point, the origin is defined, the direction pointing to the right of the human body is the positive X-axis direction, the direction pointing directly forward of the human body is the positive Y-axis direction, and the vertical upward direction is the positive Z-axis direction.

[0038] S123. The angle between the horizontal displacement vector and the positive X-axis direction clockwise is recorded as the azimuth angle of the center of gravity offset at the current moment.

[0039] In a specific embodiment of the present invention, the value range of the offset azimuth angle is from 0° to 360°, and the specific calculation formula for the centroid offset azimuth angle is as follows: .

[0040] This represents the azimuth angle of the center of gravity offset. These represent the X-axis and Y-axis coordinates of the horizontal displacement vector, respectively. The ratio of the Y-axis to the X-axis coordinates, representing the horizontal displacement vector, reflects the tangent value of the azimuth angle of the center of gravity offset. This azimuth angle is calculated using the arctangent function (artan). During the calculation, the coordinate quadrant must be corrected to avoid angular errors caused by identical sine values.

[0041] S124. Collect the center of gravity movement speed of the detected object, and combine the current center of gravity height above the ground, center of gravity movement speed, and center of gravity offset azimuth angle to form a real-time center of gravity feature parameter set.

[0042] This invention constructs a real-time set of center of gravity feature parameters by calculating the azimuth angle of the center of gravity offset, thereby realizing the quantification of the direction of the center of gravity movement of the detected object and the comprehensive and real-time perception of the body's movement state. This avoids the one-sidedness of information from monitoring a single part and improves the accuracy of the judgment results.

[0043] The hand state judgment module, in conjunction with the historical normal behavior dataset of the detected object, determines whether the current state is reasonable.

[0044] Considering that it is difficult to distinguish whether a shift in the center of gravity is an abnormal state caused by a fall or a reasonable state of normal daily behavior when monitoring only the center of gravity or the single-dimensional movement data of the hands, this invention sets up a hand state judgment module. After detecting the characteristic of a shift in the center of gravity as a precursor to a fall, it simultaneously collects multi-dimensional movement data of both hands. Combining the set of center of gravity feature parameters with the historical normal behavior dataset of the detected object, it comprehensively judges whether the current state of a shift in the center of gravity is reasonable from the perspective of the correlation between the movement features of the torso and the hands, thus avoiding the one-sidedness of information from a single dimension that leads to incorrect fall judgment.

[0045] Based on this, the specific implementation steps of the hand state judgment module are as follows: S21, obtain the azimuth angle of the two hands and the offset distance of the two hands from the center of gravity within a set time period starting from the reference time point.

[0046] In a specific embodiment of the present invention, the method for obtaining the azimuth angles of the two hands includes: first, designating the current reference three-dimensional coordinate system as the reference coordinate system, and obtaining the position coordinates of the two hands in the reference coordinate system at each time point within a set time period starting from the reference time point. Specifically, this is achieved by obtaining actual geographical location data through the GPS of the smart belt and smart bracelet, and then converting the actual geographical location data of the two smart bracelets relative to the smart belt at the waist to obtain the position coordinates of the two hands.

[0047] Secondly, based on the left-hand and right-hand position coordinates at each time point within a set time period and the corresponding hand position coordinates at the reference time point, the left-hand displacement vector and the right-hand displacement vector are obtained. In this embodiment, the set time period is 3 seconds, but the implementer can also set other specific values.

[0048] Then, the left-hand displacement vector and the right-hand displacement vector are projected onto the XY plane, and the clockwise angle between each projection vector and the positive X-axis is recorded as the left-hand offset azimuth angle and the right-hand offset azimuth angle at the corresponding time point.

[0049] The formula for calculating the hand offset azimuth angle is the same as the formula for calculating the center of gravity offset azimuth angle mentioned above.

[0050] In addition, the offset distance between the hands and the center of gravity is obtained as follows: obtain the X-axis and Y-axis coordinates of the left and right hands, as well as the X-axis and Y-axis coordinates of the center of gravity, and calculate the distance between the left and right hands and the center of gravity using the Euclidean distance calculation formula. This distance is recorded as the offset distance between the hands and the center of gravity.

[0051] S22. Combining the real-time centroid feature parameter set and the historical normal behavior dataset of the detected object, determine whether the current state is reasonable.

[0052] The method for constructing the historical normal behavior dataset of the detection object includes: W1, removing all historical time periods corresponding to fall records from the historical monitoring data of the detection object.

[0053] In this embodiment, the historical monitoring data refers to all monitoring data of the detection object from the first time it wears the smart device to the reference time point. The historical time period corresponding to the fall record refers to the time period consisting of 30 seconds before and after the time of the fall. The implementer can set other specific values, but it cannot be too short so that the fall data is not completely removed and pollutes the historical normal behavior dataset, thereby affecting the reasonable judgment result of the subsequent state.

[0054] W2. Obtain the offset distance sequence of both hands from the center of gravity, the movement speed sequence of both hands, and the offset azimuth sequence of both hands within the remaining historical time period, and form a set of hand feature parameters.

[0055] W3. Obtain the set of centroid feature parameters for each time point within the remaining historical time period, combine it with the set of hand feature parameters to form a historical normal behavior dataset, and store it in the backend database of the elderly fall detection and alarm system.

[0056] like Figure 2 As shown, in a specific embodiment of the present invention, the specific method for determining whether the current state is reasonable includes: S221, comparing the collected real-time center of gravity feature parameter set with the center of gravity feature parameter set in the historical normal behavior dataset to determine whether the real-time center of gravity features are reasonable. The specific implementation steps are as follows: S2211, obtaining the range of center of gravity height above the ground, the range of center of gravity movement speed, and the range of center of gravity offset azimuth angle based on the center of gravity feature parameter sets at various times in the historical normal behavior dataset.

[0057] S2212. If the current center of gravity height above the ground, center of gravity movement speed, and center of gravity offset azimuth angle are all within the range of center of gravity height above the ground, center of gravity movement speed, and center of gravity offset azimuth angle in the historical normal behavior dataset, then the real-time center of gravity characteristics of the detected object are determined to be reasonable.

[0058] S2213. Conversely, if the real-time centroid feature of the detected object is deemed unreasonable, it will be determined that the object is not.

[0059] S222. If the real-time center of gravity characteristics are unreasonable, then the current state is determined to be unreasonable.

[0060] S223. Conversely, the sequence of offset distances from the hands to the center of gravity, the sequence of movement speeds of the hands, and the sequence of offset azimuth angles of the hands within the set time period are matched with the set of hand feature parameters in the historical normal behavior dataset to calculate the similarity between the set time period and each historical time period.

[0061] In this embodiment, the similarity matching is performed using a dynamic time warping algorithm. The specific steps are as follows: taking the offset distance sequence from both hands to the center of gravity as an example, the current offset distance sequence from both hands to the center of gravity collected within a set time period is taken as the first time series, and the historical offset distance sequence from both hands to the center of gravity of the detected object during normal behavior is taken as the second time series; using the length of the first time series as the window length, sliding on the second time series with a step size of 1, all candidate subsequences of equal length are extracted.

[0062] Calculate the Euclidean distance between the i-th point in the first time series and the j-th point in each candidate subsequence to construct the initial distance matrix. Calculate M(i,j) based on the initial distance matrix and the dynamic programming recursive formula. Sequentially fill the cumulative cost matrix to construct the cumulative distance matrix corresponding to each candidate subsequence in the first and second time series. Denote the last element in each cumulative distance matrix as the DTW distance, and normalize it to convert it into a normalized similarity in the interval of 0 to 1.

[0063] The specific formula for similarity is as follows: .

[0064] in, Represents similarity, Represents DTW distance, The mean of the first time series. This is achieved by performing a ratio analysis between the DTW distance and the sequence mean, thus eliminating the influence of sequence dimensions on the DTW distance. (The denominator is used for this purpose.) The constant 1 in the formula ensures that two sequences are completely identical, i.e., when the DTW distance is 0, their similarity is 1. (This is achieved through...) The larger the DTW distance, the smaller the similarity, and the similarity is mapped to the interval between 0 and 1.

[0065] The matching method for the movement speed sequence and the offset azimuth sequence of both hands is the same as the offset distance sequence of both hands from the center of gravity.

[0066] S224. When the similarity of the offset distance of both hands from the center of gravity, the movement speed of both hands, and the offset azimuth angle of both hands in a certain historical time period in the historical normal behavior dataset is all greater than the corresponding similarity threshold, the current state is determined to be reasonable; otherwise, the current state is unreasonable. The specific steps are as follows: If the DTW distance between the first time series and a candidate subsequence in the second time series is less than the set normalized similarity threshold, the current action sequence is determined to match similar daily behaviors; otherwise, it is determined not to match. The closer the normalized similarity is to 1, the more similar the two sequences are; the closer it is to 0, the less similar the two sequences are. In this embodiment, the normalized similarity threshold is set to 0.9, but the implementer can also set other specific values.

[0067] This invention combines a dataset of historical normal behaviors of the detected object to determine whether the downward shift of the center of gravity is reasonable. This allows for the exclusion of downward shifts of the center of gravity caused by normal daily behavior, and the identification of unreasonable downward shifts of the center of gravity for further analysis. This reduces the probability of normal behavior being misjudged as abnormal and improves the recognition accuracy of the detection system.

[0068] The comprehensive analysis module analyzes the overall offset consistency determination results.

[0069] Considering that even if the downward shift of the center of gravity is determined to be unreasonable, there may still be random abnormal limb movements that are not caused by a fall. Not all abnormal downward shifts of the center of gravity have the limb movement characteristics of a fall. If we directly enter the fall determination process, it will increase the amount of invalid analysis and reduce the detection efficiency. Therefore, it is necessary to further screen out the abnormal movement states that truly have the characteristics of a fall by analyzing the consistency between the center of gravity and the direction of the hand shift.

[0070] S31. When the current state is detected to be unreasonable, the overall offset consistency judgment result is analyzed based on the center of gravity offset azimuth angle and the offset azimuth angle of both hands.

[0071] The specific analysis steps for determining the overall offset consistency include: S311, recording the time period from the reference time point to when the center of gravity's height above the ground begins to remain constant as the center of gravity change analysis window. In this embodiment, when the center of gravity's height above the ground is equal for n or more consecutive time points after a certain time point, that time point is recorded as the time point when the center of gravity's height above the ground begins to remain constant. In this embodiment, n is 3, but the implementer can also set other values.

[0072] S312. Obtain the azimuth angle of the center of gravity shift and the azimuth angle of the hands at each time point in the center of gravity change analysis window, and calculate the difference in azimuth angle between the center of gravity and the left hand, and between the center of gravity and the right hand.

[0073] S313. Analyze the overall offset consistency judgment result based on the difference in offset azimuth angle between the center of gravity and the left hand and between the center of gravity and the right hand.

[0074] In a specific embodiment of the present invention, when the difference between the offset azimuth angle of the center of gravity and the left hand is less than the allowable deviation of the offset azimuth angle, it is recorded that the offset azimuth of the center of gravity and the left hand are consistent.

[0075] Similarly, if the difference between the offset azimuth angle of the center of gravity and the right hand is less than the allowable azimuth angle deviation, it is recorded as the offset azimuth of the center of gravity and the right hand being consistent.

[0076] When the difference in azimuth angle between the center of gravity and the left hand, and between the center of gravity and the right hand, is less than the allowable azimuth deviation, it is recorded that the center of gravity is in the same azimuth as the hands.

[0077] Conversely, it is recorded as the orientation of the hands' offset being inconsistent with the orientation of the center of gravity's offset.

[0078] In this embodiment, the percentage of the range of offset azimuth angle values ​​is used as the allowable deviation of the offset azimuth angle. The implementer may also set other specific values.

[0079] This invention analyzes the overall offset consistency judgment results based on the center of gravity offset azimuth angle and the offset azimuth angle of both hands. After initially judging the state to be unreasonable, the offset consistency analysis further filters out abnormal movement states with fall characteristics, excludes random abnormal limb movements that are not fall-related, and reduces invalid analysis in subsequent fall judgments.

[0080] The fall detection module analyzes whether the personnel have fallen and issues a corresponding alarm.

[0081] Considering that determining fall posture solely based on offset consistency cannot accurately reflect whether the detected object has actually fallen, and that the core characteristics of a human fall also include changes in the height of the torso and hands above the ground, a single orientation consistency analysis has limitations in judgment and cannot distinguish between fall posture and the imbalance state of limbs that have not actually fallen, it is necessary to combine the temporal sequence of the center of gravity and the height of the hands above the ground to comprehensively judge from both the posture and the actual fall state, thereby improving the accuracy of fall detection.

[0082] Based on this, the specific implementation steps of the fall determination module are as follows: S41, based on the time sequence of the center of gravity height from the ground and the height of both hands from the ground, combined with the determination result of offset consistency, a comprehensive analysis is conducted to determine whether a fall has occurred.

[0083] like Figure 3As shown, the specific steps of the comprehensive analysis of whether a fall has occurred include: S411, recording the center of gravity offset azimuth angle at the last time point of the center of gravity change analysis window as the full-range center of gravity offset azimuth angle, and determining the offset type by combining the offset azimuth angle range of each offset type.

[0084] In this embodiment, the determined offset type includes forward offset, left offset, right offset, and backward offset. The offset azimuth angle range is divided into four equal parts to obtain the offset azimuth angle ranges corresponding to the four offset types. Specifically, when the center of gravity offset azimuth angle range is... The time offset type is right offset, when When the offset type is forward offset, when When the offset type is left-skewed, At that time, the offset type is back offset.

[0085] S412. When the offset type is forward and the overall offset consistency judgment result is that the offset direction of both hands is consistent with the offset direction of the center of gravity, it is judged as a falling posture.

[0086] S413. When the offset type is leftward or rightward and the overall offset consistency judgment result is that the offset direction of the corresponding hand or both hands is consistent with the offset direction of the center of gravity, it is judged as a falling posture. Among them, when the human body falls to the left or rightward, the corresponding side hand or both hands are usually consistent with the body offset direction.

[0087] S414. When the offset type is backward offset, it is determined to be a falling posture. It should be noted that after excluding historical normal behavior, if the offset type of the detected object is backward offset, it can be determined to be a falling posture.

[0088] The correspondence between the offset type and the overall offset consistency determination results in the above-mentioned fall posture determination process was obtained through the analysis of a large amount of motion capture data of elderly people falling. For example, by analyzing the motion capture data of elderly people falling forward, it was found that when the body loses balance and leans forward, the torso and arms will instinctively extend forward to try to support themselves, resulting in the center of gravity and the offset azimuth of the hands being consistent.

[0089] S415. Conversely, if the posture is not a fall, it is determined to be a non-fall posture. When it is determined to be a fall posture, the detection object is determined to be a fall based on the time sequence of the center of gravity height above the ground and the time sequence of the hand height above the ground in the center of gravity change analysis window. The specific implementation steps are as follows: S4151. When the offset type is forward offset, left offset, or right offset, obtain the center of gravity height above the ground at the final moment of the time sequence of the center of gravity height above the ground, record it as the final center of gravity height above the ground, and analyze the ground contact time point in the time sequence of the hand height above the ground.

[0090] In a specific embodiment of the present invention, the method for analyzing the ground contact time point is as follows: After the subject wears the smart device for the first time, the subject performs a full hand contact action, and the height of the subject's hands from the ground is collected. The maximum value is taken as the base ground contact height. The height of the hands from the ground in the subject's historical fall records is collected, and the maximum value is taken as the actual ground contact height. The average value of the actual ground contact height and the base ground contact height is taken as the ground contact height threshold. The time points in the time sequence of the height of the hands from the ground that are lower than the ground contact height threshold are recorded as ground contact time points.

[0091] S4152. If the final center of gravity is lower than the torso touching the ground height threshold and there is a point of contact with the ground, the subject is judged to have fallen; otherwise, the subject is judged not to have fallen.

[0092] S4153. When the offset type is backward offset and the final center of gravity height from the ground is lower than the torso touching the ground height threshold, the detected object is determined to have fallen.

[0093] The torso-to-ground height threshold is obtained as follows: the height of the subject's center of gravity from the ground when seated is used as the torso-to-ground height threshold to avoid missing falls while seated due to an excessively low threshold. This method adaptively adjusts the threshold based on the height and body shape of different subjects to ensure targeted detection.

[0094] S42. Issue an alarm when a fall is detected.

[0095] This invention comprehensively analyzes whether a fall has occurred by combining the time sequence of the center of gravity height above the ground and the height of both hands above the ground with the determination result of offset consistency. When a fall is determined, a corresponding alarm is issued. This breaks through the limitations of a single height threshold or a single movement feature determination, making the fall determination more in line with the actual limb movement patterns of a fall, effectively reducing the probability of misjudgment and missed detection, and ensuring the personal safety of the elderly.

[0096] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, in the form of a computer program product.

[0097] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0098] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0099] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0100] Finally, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A fall detection and alarm system for the elderly based on multimodal sensors, characterized in that, include: The center of gravity detection module collects the height of the center of gravity above the ground in real time to analyze whether the center of gravity has shifted downward. When the downward shift of the center of gravity is detected, it calculates the azimuth angle of the center of gravity offset based on the displacement vector of the current center of gravity in the reference three-dimensional coordinate system and constructs a real-time center of gravity feature parameter set. The hand state judgment module obtains the azimuth angle of the two hands and the offset distance of the two hands from the center of gravity within a set time period starting from the reference time point. Combined with the real-time center of gravity feature parameter set and the historical normal behavior dataset of the detected object, it determines whether the current state is reasonable. The comprehensive analysis module, when it detects that the current state is unreasonable, analyzes the overall offset consistency judgment result based on the center of gravity offset azimuth angle and the offset azimuth angle of both hands. The fall detection module analyzes whether a fall has occurred based on a time sequence of the center of gravity height above the ground and the height of both hands above the ground, combined with the result of offset consistency. When a fall is detected, a corresponding alarm is issued.

2. The fall detection and alarm system for the elderly based on a multimodal sensor according to claim 1, characterized in that, The specific contents of the center of gravity detection module include: Calculate the absolute difference between the current center of gravity height above the ground and the center of gravity height above the ground at adjacent historical time points. If the current center of gravity height above the ground is less than the center of gravity height above the ground at adjacent historical time points, and the absolute difference is greater than the allowable height deviation of the object being tested, then the center of gravity is determined to have shifted downwards. Obtain the coordinates of the current center of gravity and the center of gravity at the reference time point within the established reference three-dimensional coordinate system along the X and Y axes to obtain the horizontal displacement vector; The angle between the horizontal displacement vector and the positive X-axis direction clockwise is recorded as the azimuth angle of the center of gravity offset at the current moment; The system collects the center of gravity movement speed of the detected object and combines the current center of gravity height above the ground, center of gravity movement speed, and center of gravity offset azimuth angle to form a real-time center of gravity feature parameter set.

3. A fall detection and alarm system for the elderly based on a multimodal sensor according to claim 2, characterized in that, The specific method for setting the allowable height deviation includes: When the subject first wears the smart device, they perform standing and walking actions under the guidance of the caregiver and the smart device, and the data set of the subject's center of gravity height from the ground during the standing and walking process is collected. Calculate the average value of the center of gravity height above the ground during the standing process, and record it as the standing center of gravity height of the detected object; Calculate the absolute difference between the height of each center of gravity above the ground and the height of the standing center of gravity of the object during the walking process, and form a difference sequence; By removing outliers from the difference sequence, the maximum value of the remaining difference is taken as the allowable height deviation.

4. A fall detection and alarm system for the elderly based on a multimodal sensor according to claim 1, characterized in that, The methods for obtaining the azimuth angles of the hand offsets include: The current reference three-dimensional coordinate system is designated as the reference coordinate system. The coordinates of the hand positions in the reference coordinate system are obtained at each time point within a set time period starting from the reference time point. Based on the left-hand position coordinates, right-hand position coordinates at each time point within a set time period and the corresponding hand position coordinates at the reference time point, the left-hand displacement vector and right-hand displacement vector are obtained. Project the left-hand displacement vector and the right-hand displacement vector onto the XY plane, and record the clockwise angle between each projection vector and the positive X-axis as the left-hand offset azimuth angle and the right-hand offset azimuth angle at the corresponding time point.

5. A fall detection and alarm system for the elderly based on a multimodal sensor according to claim 1, characterized in that, The method for constructing the historical normal behavior dataset of the detected object includes: Remove all historical time periods corresponding to fall records from the historical monitoring data of the monitored subjects; The sequence of offset distances from the hands to the center of gravity, the sequence of hand movement speeds, and the sequence of hand offset azimuths are obtained within the remaining historical time period to form a set of hand feature parameters; Obtain the set of centroid feature parameters for each time point within the remaining historical time period, combine them with the set of hand feature parameters to form a historical normal behavior dataset, and store it in the backend database of the elderly fall detection and alarm system.

6. A fall detection and alarm system for the elderly based on a multimodal sensor according to claim 5, characterized in that, The specific method for determining whether the current state is reasonable includes: The real-time center of gravity feature parameter set is compared with the center of gravity feature parameter set in the historical normal behavior dataset to determine whether the real-time center of gravity feature is reasonable. If the real-time center of gravity features are unreasonable, then the current state is determined to be unreasonable. Conversely, the sequence of offset distances from the hands to the center of gravity, the sequence of movement speeds of the hands, and the sequence of offset azimuth angles of the hands within a set time period are matched with the set of hand feature parameters in the historical normal behavior dataset to calculate the similarity between the set time period and each historical time period. If the similarity of the offset distance of both hands from the center of gravity, the movement speed of both hands, and the offset azimuth angle of both hands in a certain historical period in the historical normal behavior dataset are all greater than the corresponding similarity threshold, then the current state is determined to be reasonable; otherwise, the current state is unreasonable.

7. A fall detection and alarm system for the elderly based on a multimodal sensor according to claim 6, characterized in that, The methods for determining whether the real-time center of gravity feature is reasonable include: Based on the set of center of gravity feature parameters at each time point in the historical normal behavior dataset, the range of center of gravity height above the ground, the range of center of gravity movement speed, and the range of center of gravity offset azimuth angle are obtained. If the current center of gravity height above the ground, center of gravity movement speed, and center of gravity offset azimuth angle are all within the range of center of gravity height above the ground, center of gravity movement speed, and center of gravity offset azimuth angle in the historical normal behavior dataset, then the real-time center of gravity characteristics of the detected object are deemed reasonable. Conversely, if the real-time centroid feature of the detected object is deemed unreasonable, it will be determined that the object is not.

8. A fall detection and alarm system for the elderly based on a multimodal sensor according to claim 1, characterized in that, The specific analysis steps for the overall offset consistency determination result include: The time period from the reference time point to when the center of gravity's height above the ground begins to remain constant is recorded as the center of gravity change analysis window; Obtain the azimuth angles of the center of gravity shift and the azimuth angles of both hands at each time point in the center of gravity change analysis window, and calculate the difference in azimuth angles between the center of gravity and the left hand, and between the center of gravity and the right hand, respectively. The overall offset consistency determination result is determined by analyzing the difference in offset azimuth angle between the center of gravity and the left hand, and between the center of gravity and the right hand.

9. A fall detection and alarm system for the elderly based on a multimodal sensor according to claim 8, characterized in that, The specific method for comprehensive analysis to determine whether a fall has occurred is as follows: The azimuth angle of the center of gravity shift at the last time point in the center of gravity change analysis window is recorded as the full-range center of gravity shift azimuth angle. The shift type is determined by combining the azimuth angle range of each shift type. When the offset type is forward and the overall offset consistency judgment result is that the offset position of both hands is consistent with the offset position of the center of gravity, it is judged as a falling posture; When the offset type is left or right and the overall offset consistency judgment result is that the offset position of the corresponding hand or both hands is consistent with the offset position of the center of gravity, it is judged as a falling posture. When the offset type is backward offset, it is determined to be a falling posture; Conversely, if the posture is not a fall, it is determined to be a non-fall posture. If it is determined to be a fall posture, the detection object is determined to be a fall based on the time sequence of the center of gravity height above the ground and the time sequence of the height of both hands above the ground in the center of gravity change analysis window.

10. A fall detection and alarm system for the elderly based on a multimodal sensor according to claim 9, characterized in that, The specific steps for determining whether the object being detected has fallen are as follows: When the offset type is forward offset, left offset, or right offset, obtain the center of gravity height above the ground at the final moment of the time series of the center of gravity height above the ground, and record it as the final center of gravity height above the ground. Analyze the ground contact time points in the time series of the hand height above the ground. If the final height of the center of gravity from the ground is lower than the threshold for the height of the torso touching the ground, and there is a point of contact with the ground, then the subject is judged to have fallen; otherwise, the subject is judged not to have fallen. When the offset type is backward offset and the final center of gravity height from the ground is lower than the torso-to-ground height threshold, the detected object is determined to have fallen.

Images