Fall prediction and positioning method

Abstract

The invention provides a fall prediction and positioning method, and relates to the technical field of old people health, and the method comprises the steps: collecting a map of a room where a user is located, and carrying out the coordinate conversion of the map, and obtaining a map distance; acquiring a step length and an inertial navigation orientation when the user moves in a map range; establishing a behavior detection model, and obtaining an image orientation in the video image when the user moves through the trained behavior detection model; collecting the floor where the user is located, the air pressure of each floor, the floor air pressure of the building and the outdoor temperature when the user moves; establishing a neural network model, wherein the output of the neural network model is an acceleration calibration parameter k, an orientation calibration parameter j and a height calibration parameter h; according to k, j and h, identifying a risk area where the user may fall on a map and predicting whether the user falls, if so, sending an alarm and a falling position to a guardian, and if not, continuing monitoring; the system is suitable for long-term monitoring of indoor activities of old people, is suitable for indoor complex environments, and has an indoor positioning function.

Description

Technical Field

[0001] The present invention relates to the field of health technology for the elderly, and in particular to a fall prediction and positioning method. Background Art

[0002] With the aging population, falls among the elderly have gradually become an important public safety issue that has attracted much attention. As the elderly are not mobile, their main activities are carried out indoors, which directly leads to about 70% of falls among the elderly occurring indoors. In reality, the guardians of the elderly cannot provide 24-hour all-round monitoring, so a series of devices for judging falls have been produced, which are mainly divided into four categories: wearable devices, visual sensors, environmental sensors, and multi-sensor fusion. Solutions based on wearable devices and visual sensors are more popular and common. The former is mainly due to its advantages of flexible wearing, easy carrying and low cost; the latter is worse than wearable devices in terms of accuracy, but not as good as the former in terms of portability, and is affected by the visual range and has a relatively limited scene. The method based on environmental sensors generally uses WIFI or RADAR to establish the relationship between environmental signal changes and human behavior through wireless communication technology to detect falls. However, this method is not suitable for long-term use at home because it requires the installation of multiple devices and daily maintenance, and the cost is relatively high. In addition, the current fall prediction technology solutions used indoors rarely consider the issue of indoor positioning. After predicting a fall, they are unable to report the location of the elderly in a timely manner like other solutions used outdoors. Therefore, when an elderly person falls, if they are in a blind spot, they will miss the time for rescue because they cannot know whether they are indoors.

[0003] In the prior art, it is more popular to use inertial navigation elements or a combination of inertial navigation elements and other sensors to predict falls. A Chinese patent with patent number CN202311745869.4 discloses an intelligent sensing alarm system for preventing patients from falling for nursing care, which integrates bed pressure sensors, depth cameras, thermal imaging sensors and motion sensors to achieve comprehensive collection of patient behavior. However, this invention focuses on bedside activities, and the deployed sensors are limited to single wards, which are not suitable for the more complex environment of the home and cannot achieve seamless switching between indoors and outdoors. A method for warning of falls and similar actions based on inertial sensing is disclosed in a document with publication number CN116189383A. It determines whether there is weightlessness through an acceleration threshold, and performs a fall warning in combination with the instantaneous displacement or tilt angle in the vertical direction. However, this solution does not take into account that the error of the inertial navigation sensor will gradually increase, and lacks corresponding calibration solutions and indoor positioning solutions. A fall detection method and system are disclosed in the document with publication number CN117496664A, which predicts falls by repairing blocked video data. However, the positioning problem is still not solved and the corresponding heart rate monitoring equipment needs to be worn additionally. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a fall prediction and positioning method, which is suitable for long-term monitoring of indoor activities of the elderly, adaptable to complex indoor environments, and has the function of indoor positioning.

[0005] The present invention provides a fall prediction and positioning method, comprising the following steps:

[0006] S1. Collect the map of the indoor environment where the user is located, and perform coordinate conversion on the map to obtain the map distance;

[0007] S2, collecting the step length and inertial navigation direction of the user when he / she is active within the map range;

[0008] S3. Establish a behavior detection model, train the behavior detection model through the falling posture data and non-falling posture data in the open source data set, and obtain the trained behavior detection model; collect video images of the user during activities, obtain the image orientation of the user during activities in the video image through the YOLOv8 model, and then obtain the user's behavior according to the image orientation through the behavior detection model, and predict the user's behavior through the slope classifier;

[0009] S4, collecting the floor where the user is located, the air pressure of each floor, the air pressure of the ground floor where the user is located, and the outdoor temperature;

[0010] S5. Establish a neural network model, the input of the neural network model is the comprehensive acceleration, the inertial navigation orientation, the image orientation, the relative height of the floor, the air pressure of each floor, the ground air pressure of the floor and the outdoor temperature, and the output of the neural network model is the acceleration calibration parameter k, the orientation calibration parameter j and the height calibration parameter h;

[0011] S6, predicting whether the user has fallen and locating the user at the same time according to the acceleration calibration parameter k, the orientation calibration parameter j and the height calibration parameter h, if yes, sending an alarm and the fall location to the guardian, if no, continuing monitoring;

[0012] S7. Establish a fall risk prediction model. The input of the fall risk prediction model is the recent fall location, and the output is a hot spot map of indoor fall risk areas.

[0013] Compared with the prior art, the present application has the following advantages: since the entire algorithm is based on a map, the user's activity trajectory can be effectively known based on the step length, inertial navigation orientation, and image orientation. When an elderly person falls, the elderly person's last stop of activity can be determined as the fall position, that is, the position where the activity trajectory stops, so that the function of indoor positioning can be provided, and the position of the elderly person can be quickly determined. The acceleration calibration parameter k, the orientation calibration parameter j, and the height calibration parameter h are necessary parameters for accurately detecting the fall of the elderly indoors. At the same time, after the video image is first acquired, the orientation calibration parameter j can be fixed, so that when the video image cannot be acquired, the neural network model can still continue to calibrate the acceleration calibration parameter k and the height calibration parameter h when the video image cannot be acquired, and maintain accuracy. When the video image can be acquired, the acceleration calibration parameter k, the orientation calibration parameter j, and the height calibration parameter h can be calibrated at the same time to accurately monitor the elderly; the behavior detection model can determine whether the user falls through the video image, and work simultaneously with the neural network model to accurately monitor the user's trajectory and fall situation, and adapt to complex indoor environments.

[0014] In a possible implementation, step S1 includes the following steps:

[0015] S11, obtaining the CAD format map where the user is active;

[0016] S12. Establish the WGS1984 coordinate system on the map and measure the map distance using the WGS1984 coordinate system.

[0017] Compared with the existing technology, the CAD format map using the above technical solution can facilitate viewing the specific distribution of the entire room, and can establish the WGS1984 coordinate system on the CAD format map. The WGS1984 coordinate system is a standardized coordinate system with good compatibility, thereby effectively measuring the map distance, which can be used to calibrate the user's walking distance and achieve seamless switching between indoor and outdoor positioning.

[0018] In a possible implementation, step S2 includes the following steps:

[0019] S21, the user carries a smart phone, and uses an accelerometer and a gyroscope inside the smart phone to collect acceleration and angular velocity of the user during activities;

[0020] S22, converting the acceleration and angular velocity into step length and inertial navigation heading respectively through the PDR algorithm.

[0021] Compared with the existing technology, the above technical solution is adopted because smartphones are indispensable electronic devices in modern life, and they are equipped with accelerometers and gyroscope modules. Therefore, the acceleration and angular velocity of the user during activities can be directly obtained by using a third-party APP in the smartphone, and converted into step length and inertial navigation direction through the PDR algorithm. This does not require additional wearable devices, which is very convenient.

[0022] In a possible implementation, the behavior detection model in step S3 is a YOLOv8 model.

[0023] Compared with the existing technology, the YOLOv8 model using the above technical solution can effectively detect portraits in video images.

[0024] In a possible implementation, step S3 includes the following steps:

[0025] S31. Install surveillance cameras in the room where the user is active;

[0026] S32, collecting video images of user activities through a surveillance camera;

[0027] S33, in the video image, select the user position through the portrait recognition extraction frame, obtain the coordinates of the upper left corner and the lower right corner of the portrait recognition extraction frame, generate step points according to the midpoint of the bottom edge of the portrait recognition extraction frame, and generate an activity track according to the step points;

[0028] S34. Obtaining the image orientation according to the orientation formed between two adjacent step points in the activity trajectory.

[0029] Compared with the prior art, the surveillance camera using the above technical solution is because surveillance cameras are often installed in indoor areas where most elderly people are active. They are commonly found in hospitals, nursing homes or living rooms of some families. If they are not available, they can be installed. Therefore, the surveillance camera can be used directly to obtain video images of elderly people's activities. No additional equipment is required, which is very convenient. In the portrait recognition of video images, the portrait is generally identified by the portrait recognition extraction frame. When the portrait moves, the portrait recognition extraction frame also moves accordingly. Therefore, the movement of the portrait recognition extraction frame represents the movement of the portrait, and the upper left and lower right horizontal and vertical coordinates of the portrait can be used to extract the moving points of the human body, thereby forming an activity trajectory, and then obtaining a more accurate image orientation. It can also be used to extract the height and width of the portrait recognition extraction frame, which is used to obtain the slope classifier for fall prediction, so as to achieve the purpose of data sharing. At the same time, the self-trained behavior detection model is used to improve the accuracy of behavior detection by using a double verification method.

[0030] In a possible implementation manner, step S3 further includes the following steps:

[0031] S301, obtaining the height and width of the portrait recognition extraction frame according to the coordinates of the upper left corner and the lower right corner of the portrait recognition extraction frame, and constructing a slope classifier by the following formula:

[0032]

[0033] where t i is the predicted value of the height or width of the extraction box for portrait recognition, x i The height or width observation value of the extraction box for portrait recognition, as well as They are the internal variance of the observed value, the variance of the difference between the observed value and the predicted value, and the internal variance of the predicted value, n is the total number of samples, m and p are the predicted pixel mutation points of the portrait recognition extraction box;

[0034] S302, minimize the SSE variance, obtain the best slope mutation point, and predict the user's behavior based on the slope mutation point and the user's behavior in step S3.

[0035] Compared with the existing technology, the above technical solution can effectively perform double verification through the behavior detection model and accurately predict whether the user's behavior is a fall.

[0036] In a possible implementation, step S4 includes the following steps:

[0037] S41. The air pressure of each floor, the air pressure of the floor where the user is located, and the outdoor temperature of the day are collected through a smart phone during the user's activities; the air pressure of each floor and the air pressure of the floor where the user is located are collected through a barometer.

[0038] Compared with the existing technology, the above technical solution can directly obtain the air pressure on each floor, the ground air pressure on the floor where the user is located, and the outdoor temperature of the day through a third-party APP in a smart phone. The outdoor temperature is relatively accurate when obtained through a third-party APP, while the air pressure on each floor and the ground air pressure on the floor where the user is located are not accurate when obtained through a third-party APP. Therefore, a barometer is required to actually measure the air pressure on each floor and the ground air pressure on the floor where the user is located. Since the air pressure generally does not change much at the same geographical location, the measurement frequency of each air pressure can be once every three months, and the outdoor temperature can be measured once a day.

[0039] In a possible implementation, step S5 includes the following steps:

[0040] Step S5 includes the following steps:

[0041] S51. Calculate the acceleration calibration parameter k according to the following formula:

[0042]

[0043] Where n represents the number of steps, a * Denotes the three-dimensional acceleration, D real The user's activity distance;

[0044] S52, obtaining the orientation calibration parameter j according to the following formula:

[0045]

[0046] Among them, t(j) represents a certain moment, t(i) represents the moment closest to t(j), and D′ 1 Indicates the maximum projection distance of the surveillance camera, ω * Indicates the image orientation, D distance Denotes D′ 1 The corresponding actual map distance, X t(j) ,X t(j+1) Respectively represent the horizontal coordinates of the step point at time t(j) and t(j+1), D′ t(j) and D′ t(j+1) Respectively represent the projection distance between the surveillance camera and the corresponding step point;

[0047] S53, obtain the height calibration parameter h according to the following formula:

[0048]

[0049] Among them, P 0 Indicates the ground pressure of the building on that day, P i Indicates the air pressure on each floor of the day, H i Indicates the current floor height, H g Indicates the ground height, and They represent the lower and upper limits of the relative height of each floor, T represents the outdoor temperature of the day, ΔH i Indicates the relative height of each floor, H Real (i) represents the measured relative height of each floor, and n represents the total number of samples;

[0050] S54, calculate the adaptive residual r by the following formula * :

[0051]

[0052] Among them, r i represents the least square residual, δ i represents the diagonal factor of the least square leverage matrix, n represents the total number of samples; S55, r is calculated by the following formula * Perform normalization:

[0053]

[0054] Where K is a constant and s is the residual r i Parameters determined by the median absolute deviation value;

[0055] S56, calculate the neural network weight value w by the following formula i :

[0056]

[0057] S57. Calculate the regression coefficient b by the following formula:

[0058] b=(X T WX) -1 X T Wy;

[0059] Where W is the weight matrix, X is the independent variable matrix, and y is the observation value response matrix;

[0060] S58. Calculate the weight variance J by the following formula:

[0061]

[0062] Compared with the prior art, when the weight variance J converges to the minimum, that is, the weight parameter of the hidden layer of the neural network model is optimal, the neural network model established by the above formula can effectively and continuously calibrate the acceleration calibration parameter k, the orientation calibration parameter j and the height calibration parameter h.

[0063] In a possible implementation, step S6 includes the following steps:

[0064] S61. Calculate the user's location by the following formula:

[0065]

[0066] Among them, P i represents 2D coordinates, Indicates the direction of due east. Indicates the direction toward due north, M, β, ε represent the rotation, stretching and migration of coordinates respectively;

[0067] S62. Predict falls based on k:

[0068]

[0069] Among them, x i It represents the predicted status value, 1 means falling, 0 means not falling; and Respectively represent the upper and lower limits of the value range after fall recognition. If x i =1, then go to step S63, if x i =0, then go to step S61;

[0070] S63, obtain the coordinates of the falling position (x R ,y R ,F) and sent to the guardian.

[0071] Compared with the existing technology, the above technical solution can be used to visualize the map and accurately inform the guardian of the location where the elderly person fell. It is suitable for long-term monitoring of the elderly's indoor activities and adapt to complex indoor environments.

[0072] In a possible implementation, step S7 includes the following steps:

[0073] S71. Predict the probability of a user falling in each area of ​​the map by the following formula:

[0074]

[0075] Among them, P F (i) represents the probability of falling or not falling at present, P fall (i) represents the probability of falling, y i Indicates the actual status value, 1 means falling, 0 means not falling;

[0076] S72. Identify the indoor fall risk area hotspot map according to the fall probability, and correct the result of step S62.

[0077] Compared with the existing technology, the above technical solution can be used to visualize the map, making it easier for guardians to take preventive measures in high-risk areas. It is suitable for long-term monitoring of the indoor activities of the elderly and adaptable to complex indoor environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 It is a schematic diagram of the process of the present invention;

[0079] Figure 2 A schematic diagram of fall prediction according to the present invention;

[0080] Figure 3 It is a schematic diagram of the trajectory and falling position of the present invention;

[0081] Figure 4 A schematic diagram of the trajectory of recovery walking of the present invention;

[0082] Figure 5 This is a schematic diagram of the present invention using the K-means algorithm for identification:

[0083] Figure 6 For the present invention in the attached Figure 5 Schematic diagram of identification based on SVM algorithm;

[0084] Figure 7 It is a schematic diagram of the trajectory of the going downstairs process and the falling process of the present invention;

[0085] Figure 8 This is a schematic diagram of the fall prediction process of going downstairs according to the present invention;

[0086] Fig. 9 This is a schematic diagram of the fall risk area of ​​the present invention;

[0087] Fig.10 This is a schematic diagram of using a slope classifier to identify the height and width of a portrait recognition extraction frame in the present invention. DETAILED DESCRIPTION

[0088] First, those skilled in the art should understand that these implementations are only used to explain the technical principles of the embodiments of the present application, and are not intended to limit the protection scope of the embodiments of the present application. Those skilled in the art can make adjustments to them as needed to adapt to specific application scenarios.

[0089] The present application is further described in detail below in conjunction with the accompanying drawings and specific embodiments. All of the following algorithms can be executed on a local computer, a smart phone or a tablet computer; the guardian described below is the guardian of the elderly.

[0090] The present application embodiment discloses a fall prediction and positioning method, comprising the following steps:

[0091] S1. Collect the map of the indoor environment where the user is located, and perform coordinate conversion on the map to obtain the map distance;

[0092] S2, collecting the step length and inertial navigation direction of the user when he / she is active within the map range;

[0093] S3. Establish a behavior detection model, train the behavior detection model through the falling posture data and non-falling posture data in the open source data set, and obtain the trained behavior detection model; collect video images of the user during activities, obtain the image orientation of the user during activities in the video image through the YOLOv8 model, and then obtain the user's behavior according to the image orientation through the behavior detection model, and predict the user's behavior through the slope classifier;

[0094] S4, collecting the floor where the user is located, the air pressure of each floor, the air pressure of the ground floor where the user is located, and the outdoor temperature;

[0095] S5. Establish a neural network model, the input of the neural network model is the comprehensive acceleration, the inertial navigation orientation, the image orientation, the relative height of the floor, the air pressure of each floor, the ground air pressure of the floor and the outdoor temperature, and the output of the neural network model is the acceleration calibration parameter k, the orientation calibration parameter j and the height calibration parameter h;

[0096] S6, predicting whether the user has fallen and locating the user at the same time according to the acceleration calibration parameter k, the orientation calibration parameter j and the height calibration parameter h, if yes, sending an alarm and the fall location to the guardian, if no, continuing monitoring;

[0097] S7. Establish a fall risk prediction model. The input of the fall risk prediction model is the recent fall location, and the output is a hot spot map of indoor fall risk areas.

[0098] Step S1 includes the following steps:

[0099] S11, obtaining the CAD format map where the user is active;

[0100] S12. Establish the WGS1984 coordinate system on the map and measure the map distance using the WGS1984 coordinate system.

[0101] Specifically, the CAD format map can facilitate viewing the specific distribution of the entire room. The WGS1984 coordinate system is established on the CAD format map. The WGS1984 coordinate system has global uniformity, high-precision positioning, compatibility and interoperability, so it can effectively measure the map distance, which is used to calibrate the user's walking distance. The map distance refers to the actual distance corresponding to the map. Generally, when a room is built, the various dimensions of the indoor room are designed, such as the length, width and height of the kitchen, so this step is mainly to establish a coordinate system so that various parameters such as inertial navigation orientation and image orientation can be unified when the user is indoors; in addition, even if the user leaves the room, this coordinate system can be seamlessly connected to outdoor monitoring; the map can be stored on a computer or smart phone and transmitted over the network, so that the guardian can view the map at any time.

[0102] Step S2 includes the following steps:

[0103] S21, the user carries a smart phone, and uses an accelerometer and a gyroscope inside the smart phone to collect acceleration and angular velocity of the user during activities;

[0104] S22, converting the acceleration and angular velocity into step length and inertial navigation heading respectively through the PDR algorithm.

[0105] Specifically, the smartphone can be installed at the waist of the elderly, which can be the lower pocket of the shirt or the upper pocket of the pants. Those skilled in the art can flexibly adjust it according to the actual situation. The internal core of existing smartphones generally integrates inertial navigation elements, which include accelerometers and gyroscopes, so acceleration and angular velocity can also be directly obtained through third-party APPs. The PDR algorithm is the Pedestrian Dead Reckoning algorithm, which can calculate the position and direction of the walker through acceleration and angular velocity parameters. The PDR algorithm belongs to the existing algorithm and will not be described in detail in this application. The step length and inertial navigation direction can be selected to be saved for three months or longer, and those skilled in the art can flexibly adjust it according to the actual hardware storage situation; the step length and inertial navigation direction need to be updated and calibrated daily, and the update frequency can be flexibly adjusted according to the actual hardware situation.

[0106] The YOLOv8 model in step S3 is trained using an open source dataset on the open source data platform roboflow, so that portraits in video images can be effectively recognized.

[0107] Step S3 includes the following steps:

[0108] S31. Install surveillance cameras in the room where the user is active;

[0109] S32, collecting video images of user activities through a surveillance camera;

[0110] S33, in the video image, select the user position through the portrait recognition extraction frame, obtain the coordinates of the upper left corner and the lower right corner of the portrait recognition extraction frame, generate step points according to the midpoint of the bottom edge of the portrait recognition extraction frame, and generate an activity track according to the step points;

[0111] S34. Obtaining the image orientation according to the orientation formed between two adjacent step points in the activity trajectory.

[0112] Specifically, surveillance cameras are installed in most indoor areas where the elderly are active. They are commonly found in hospitals, nursing homes or some homes. If they are not available, they can be installed directly. Therefore, surveillance cameras can be used directly to obtain video images of the elderly's activities. No additional equipment is required, and it is very convenient. Surveillance cameras can communicate with computers and smart phones. In portrait recognition in video images, portraits are generally identified by portrait recognition extraction frames. When the portrait moves, the portrait recognition extraction frame also moves with it. Therefore, the movement of the portrait recognition extraction frame represents the movement of the portrait, and the step movement of the portrait is at the feet. Therefore, the midpoint of the bottom edge of the portrait recognition extraction frame needs to be taken as the horizontal coordinate, so that the step points can be accurately generated, thereby forming an activity trajectory, and then obtaining a more accurate image orientation.

[0113] For other changes that affect acceleration, when the portrait is doing different actions, the slope classifier can be used to find the distortion points for identification. If there are actions with large interference, the K-Means algorithm can be superimposed on the binary SVM algorithm for identification. Figure 5 As shown in the figure, it is the effect diagram of K-Means algorithm recognition. Figure 6 As shown in the attached Figure 5 Based on the above, the binary SVM algorithm is used to further identify the effect diagram. The red dot is the moment of falling, which can be used to set the acceleration value range of falling, which is more accurate. In addition, the upper left and lower right horizontal and vertical coordinates of the portrait can be used to extract the moving points of the human body, thereby forming an activity trajectory, and then obtaining a more accurate image orientation. It can also be used to extract the height and width of the portrait recognition extraction box, which is used to obtain the slope classifier for fall prediction, so as to achieve the purpose of data sharing. At the same time, the self-trained behavior detection model is used to improve the accuracy of behavior detection by using double verification.

[0114] Step S3 includes the following steps:

[0115] S301, obtaining the height and width of the portrait recognition extraction frame according to the coordinates of the upper left corner and the lower right corner of the portrait recognition extraction frame, and constructing a slope classifier by the following formula:

[0116]

[0117] where t i is the predicted value of the height or width of the extraction box for portrait recognition, x i The height or width observation value of the extraction box for portrait recognition, as well as They are the internal variance of the observed value, the variance of the difference between the observed value and the predicted value, and the internal variance of the predicted value, n is the total number of samples, m and p are the predicted pixel mutation points of the portrait recognition extraction box;

[0118] S302, minimize the SSE variance, obtain the best slope mutation point, and predict the user's behavior based on the slope mutation point and the user's behavior in step S3.

[0119] Specifically, the above technical solution can share the coordinates of the human body extraction frame, and double-verify with the results of the behavior detection model. As shown in Table 2 below, Table 2 is a comparison of the accuracy of behavior detection in actual measurement. Single-class visual fall detection refers to only judging falls, and all other behaviors are regarded as non-falls. It can be concluded that the slope classifier method is more sensitive to the behavior before the fall and can accurately predict the fall trend, but the sensitivity to the behavior of getting up from the ground after the fall is reduced. At this time, it can be corrected according to the trained visual detection model. Fig.10 As shown, the slope classifier can be used to find the mutation area where the height and width of the portrait recognition extraction box have obvious slope changes at the same time, and the first mutation point is given priority to predict falls.

[0120] Step S4 includes the following steps:

[0121] S41. The air pressure of each floor, the air pressure of the floor where the user is located, and the outdoor temperature of the day are collected through a smart phone during the user's activities; the air pressure of each floor and the air pressure of the floor where the user is located are collected through a barometer.

[0122] Specifically, the air pressure of each floor, the ground air pressure of the floor where the person is located, and the outdoor temperature of the day are directly obtained through a third-party APP in the smartphone. The outdoor temperature is more accurate when obtained through a third-party APP, while the air pressure of each floor and the ground air pressure of the floor where the person is located are not accurate when obtained through a third-party APP. Therefore, it is necessary to use a barometer to actually measure the air pressure of each floor and the ground air pressure of the floor where the person is located. Since the air pressure changes little in the same geographical location, the measurement frequency of each air pressure can be once every three months, and the measurement of the outdoor temperature can be once a day, so as to finally obtain a high-precision height calibration parameter h. In addition, if it is a single-story building, or if the person lives on the first floor, the height calibration parameter h can be simplified, such as ignoring the relative height of each floor and the air pressure of each floor.

[0123] Step S5 includes the following steps:

[0124] S51. Calculate the acceleration calibration parameter k according to the following formula:

[0125]

[0126] Where n represents the number of steps, a * Denotes the three-dimensional acceleration, D real The user's activity distance; without increasing the system calculation pressure and ensuring accuracy, the above parameters are calibrated once a day.

[0127] S52, obtaining the orientation calibration parameter j according to the following formula:

[0128]

[0129] Among them, t(j) represents a certain moment, t(i) represents the moment closest to t(j), and D′ 1 Indicates the maximum projection distance of the surveillance camera, ω * Indicates the image orientation, D distance Denotes D′ 1 The corresponding actual map distance, X t(j) ,X t(j+1) Respectively represent the horizontal coordinates of the step point at time t(j) and t(j+1), D′ t(j) and D′ t(j+1) They respectively represent the projection distance between the surveillance camera and the corresponding step point; without increasing the calculation pressure of the system and ensuring accuracy, the above parameters are calibrated once a day.

[0130] S53, obtain the height calibration parameter h according to the following formula:

[0131]

[0132] Among them, P 0 Indicates the ground pressure of the building on that day, P i Indicates the air pressure on each floor of the day, H i Indicates the current floor height, H g Indicates the ground height, and They represent the lower and upper limits of the relative height of each floor, T represents the outdoor temperature of the day, ΔH i Indicates the relative height of each floor, H Real (i) represents the measured relative height of each floor, and n represents the total number of samples; the above parameters are calibrated every three months without increasing the calculation pressure of the system and ensuring accuracy.

[0133] S54, calculate the adaptive residual r by the following formula * :

[0134]

[0135] Among them, r i represents the least square residual, δ i represents the diagonal factor of the least square leverage matrix, n represents the total number of samples; S55, r is calculated by the following formula * Perform normalization:

[0136]

[0137] Where K is a constant and s is the residual ri Parameters determined by the median absolute deviation value;

[0138] S56, calculate the neural network weight value w by the following formula i :

[0139]

[0140] S57. Calculate the regression coefficient b by the following formula:

[0141] b=(X T WX) -1 X T Wy;

[0142] Where W is the weight matrix, X is the independent variable matrix, and y is the observation value response matrix;

[0143] S58. Calculate the weight variance J by the following formula:

[0144]

[0145] Specifically, the above formula is used to adjust the weight of the hidden layer of the neural network. When the weight variance J converges to the minimum, that is, the hidden layer weight of the neural network model is optimal, the neural network model is established, and the input of the neural network model is the three-dimensional acceleration a * , image orientation, inertial navigation orientation, air pressure on each floor, air pressure on the floor where you are located, and outdoor temperature on that day. The output is the acceleration calibration parameter k, orientation calibration parameter j, and height calibration parameter h.

[0146] Step S6 includes the following steps:

[0147] S61. Calculate the user's location by the following formula:

[0148]

[0149] Among them, P i represents 2D coordinates, Indicates the direction of due east. Indicates the direction toward due north, M, β, ε represent the rotation, stretching and migration of coordinates respectively;

[0150] S62. Predict falls based on k:

[0151]

[0152] Among them, x i It represents the predicted status value, 1 means falling, 0 means not falling; and Respectively represent the upper and lower limits of the value range after fall recognition. If x i=1, then go to step S63, if x i =0, then go to step S61; wherein, and These are the upper and lower limits of the value range after fall recognition in the absence of historical data, that is, in the absence of the heat map output in step S72.

[0153] S63, obtain the coordinates of the falling position (x R ,y R ,F) and sent to the guardian.

[0154] Step S7 includes the following steps:

[0155] S71. Predict the probability of a user falling in each area of ​​the map by the following formula:

[0156]

[0157] Among them, P F (i) represents the probability of falling or not falling at present, P fall (i) represents the probability of falling, y i Indicates the actual status value, 1 means falling, 0 means not falling;

[0158] S72. Identify the indoor fall risk area hotspot map according to the fall probability, and correct the result of step S62.

[0159] Specifically, the indoor fall risk area heat map can be used to correct the fall prediction result of S62, thereby forming a double verification, as shown in the attached figure. Figure 2 As shown in the figure, the red dot is the predicted time of falling. Figure 3 The figure shows the path formed by the steps and the position when the fall occurred. If you resume walking after the fall, as shown in the attached figure Figure 4 As shown in the figure, the track will continue to move forward and the alarm will still be sent to remind the guardian. Figure 7 As shown in the figure, it is a schematic diagram of the fall and trajectory during the downstairs process. Figure 8 As shown in the figure, the red dot is the time when the fall occurs predicted by the integrated acceleration. By collecting the trajectory of the past three months, high-risk areas can be identified, allowing guardians to take preventive measures, such as the attached Fig. 9As shown, the risk level marked at the stairs is slightly higher than that in the room, so the probability of falling at the stairs is higher than that in the room. The risk value is used as an additional environmental factor (0-1). When predicting a fall, the corresponding weight will be assigned according to the risk map. For example, when predicting a fall at the stairs and in the room, a preliminary judgment will be made based on the acceleration value first, and then the additional environmental factor will be superimposed. If it is still greater than 0.5 after superposition, an alarm will be issued; otherwise, a selective alarm will be issued. The recent period can be the past three months. The above-mentioned fall risk prediction model for identifying the indoor fall risk area hotspot map can be implemented by a clustering algorithm.

[0160] As shown in Table 1 below, Table 1 shows the behavior detection results on the public dataset.

[0161] in:

[0162] It can be seen that the accuracy and recall rate of single-class fall detection are better than those of multi-class behavior detection, so single-class behavior detection is adopted. As shown in Table 2 below, Table 2 is a comparison of the correct rate of behavior detection in actual measurement. Single-class visual fall detection refers to only judging falls, and other behaviors are all regarded as non-falls. It can be concluded that the slope classifier method is more sensitive to the behavior before the fall, but the sensitivity to the behavior of getting up from the ground after the fall is reduced. At this time, it can be corrected according to the results of single-class visual fall detection, which is more accurate in fall detection. As shown in Table 3 below, Table 3 is a comparison of fall prediction and positioning information under full visual assistance and without visual assistance. It can be seen that monitoring supervision is higher. The following data is based on 50 experimental data. The specific accuracy may fluctuate up and down according to the specific experimental scenario, but it can also be gradually adjusted after field deployment with daily data calibration.

[0163]

[0164] Table 1: Behavior detection on public datasets

[0165]

[0166] Table 2: Comparison of the correct rate of behavior detection in actual measurement

[0167]

[0168] Table 3: Comparison of fall prediction and location information with full visual assistance and without visual assistance

[0169] In the description of the present application, the description with reference to the terms "one embodiment", "some embodiments", "in the present embodiment", "specific example", or "some examples" etc. means that the specific features, mechanisms, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, mechanisms, materials or characteristics described may be combined in a suitable manner in any one or more embodiments or examples. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0170] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A fall prediction and positioning method, characterized in that: The following steps are involved: S1. Collecting a map of the indoor space where the user is located, and performing coordinate conversion on the map to obtain a map distance; S2, collecting the step length and inertial navigation direction of the user when moving within the map range; S3, establish a behavior detection model, train the behavior detection model through the falling posture data and non-falling posture data in the open source data set, and obtain the trained behavior detection model; collect video images of the user during the activity, obtain the image orientation of the user during the activity in the video image through the YOLOv8 model, and then obtain the user's behavior according to the image orientation through the behavior detection model, and predict the user's behavior through the slope classifier; S4, collecting the floor where the user is located, the air pressure of each floor, the air pressure on the ground of the floor where the user is located, and the outdoor temperature; S5, establishing a neural network model, wherein the input of the neural network model is the comprehensive acceleration, the inertial navigation orientation, the image orientation, the relative height of the floor, the air pressure of each floor, the ground air pressure of the floor and the outdoor temperature, and the output of the neural network model is the acceleration calibration parameter k, the orientation calibration parameter j and the height calibration parameter h; S6, predicting whether the user has fallen according to the acceleration calibration parameter k, the orientation calibration parameter j and the height calibration parameter h and locating the user at the same time, if so, sending an alarm and the fall location to the guardian, if not, continuing monitoring; S7. Establish a fall risk prediction model, wherein the input of the fall risk prediction model is the recent fall location, and the output is a hot spot map of indoor fall risk areas.

2. The fall prediction method according to claim 1, characterized in that: Step S1 includes the following steps: S11, obtaining a CAD format map where the user is active; S12. Establish a WGS1984 coordinate system on the map, and measure the map distance using the WGS1984 coordinate system.

3. The fall prediction method according to claim 1 or 2, characterized in that: Step S2 includes the following steps: S21, carrying a smart phone on the user, and using an accelerometer and a gyroscope inside the smart phone to respectively collect acceleration and angular velocity of the user during activities; S22. Convert the acceleration and angular velocity into the step length and inertial navigation direction respectively through a PDR algorithm.

4. The fall prediction method according to claim 3, characterized in that: Step S3 includes the following steps: S31, installing a surveillance camera in the room where the user is active; S32, collecting video images of the user's activities through the surveillance camera; S33, in the video image, select the user position through the portrait recognition extraction frame, obtain the coordinates of the upper left corner and the lower right corner of the portrait recognition extraction frame, generate step points according to the midpoint of the bottom edge of the portrait recognition extraction frame, and generate an activity track according to the step points; S34, obtaining an image orientation according to the orientation formed between two adjacent step points in the activity trajectory.

5. The fall prediction method according to claim 4, characterized in that: Step S3 also includes the following steps: S301, obtaining the height and width of the portrait recognition extraction frame according to the coordinates of the upper left corner and the lower right corner of the portrait recognition extraction frame, and constructing a slope classifier by the following formula: where t i is the predicted value of the height or width of the portrait recognition extraction box, x i is the height or width observation value of the portrait recognition extraction frame, as well as are the internal variance of the observed value, the variance of the difference between the observed value and the predicted value, and the internal variance of the predicted value, respectively; n is the total number of samples; m and p are the predicted pixel mutation points of the portrait recognition extraction box; S302, minimizing the SSE variance, obtaining the best slope mutation point, and predicting the user's behavior based on the comparison between the slope mutation point and the user's behavior in step S3.

6. The fall prediction method according to claim 5, characterized in that: Step S4 includes the following steps: S41, collecting the air pressure of each floor, the air pressure on the floor where the user is located, and the outdoor temperature of the day through the smart phone during the user's activities; collecting the air pressure of each floor and the air pressure on the floor where the user is located through a barometer.

7. The fall prediction method according to claim 6, characterized in that: Step S5 includes the following steps: S51. Calculate the acceleration calibration parameter k according to the following formula: Where n represents the number of steps, a * Denotes the three-dimensional acceleration, D real The user's activity distance; S52, obtaining the orientation calibration parameter j according to the following formula: Among them, t(j) represents a certain moment, t(i) represents the moment closest to t(j), and D ′ 1 represents the maximum projection distance of the surveillance camera, ω * represents the image orientation, Ddistance represents the actual map distance corresponding to D′1, and X t(j) ,X t(j+1) Respectively represent the horizontal coordinates of the step point at time t(j) and t(j+1), D ′ t ( j ) and D ′ t ( j +1) Respectively represent the projection distance between the surveillance camera and the corresponding step point; S53, obtain the height calibration parameter h according to the following formula: Among them, P0 represents the ground pressure of the building on that day, P i Indicates the air pressure on each floor of the day, H i Indicates the current floor height, H g Indicates the ground height, and They represent the lower and upper limits of the relative height of each floor, T represents the outdoor temperature of the day, ΔH i Indicates the relative height of each floor, H Real (i) represents the measured relative height of each floor, and n represents the total number of samples; S54, calculate the adaptive residual r* by the following formula: Among them, r i represents the least square residual, δ i represents the diagonal factor of the least square leverage matrix, and n represents the total number of samples; S55. Normalize r* using the following formula: Where K is a constant and s is the residual r i Parameters determined by the median absolute deviation value; S56, calculate the neural network weight value w by the following formula i : S57. Calculate the regression coefficient b by the following formula: b=(X T WX) -1 X T Wy; Where W is the weight matrix, X is the independent variable matrix, and y is the observation value response matrix; S58. Calculate the weight variance J by the following formula:

8. The fall prediction method according to claim 7, characterized in that: Step S6 includes the following steps: S61. Calculate the user's location by the following formula: Among them, P i represents 2D coordinates, Indicates the direction of due east. Indicates the direction toward due north, M, β, ε represent the rotation, stretching and migration of coordinates respectively; S62. Predicting falls based on k: Among them, x i It represents the predicted status value, 1 means falling, 0 means not falling; and Respectively represent the upper and lower limits of the value range after fall recognition. If x i =1, then go to step S63, if x i =0, then go to step S61; S63, obtain the coordinates of the falling position (x R ,y R ,F) and sent to the guardian.

9. The fall prediction method according to claim 8, characterized in that: Step S7 includes the following steps: S71, predicting the probability of the user falling in each area of ​​the map by the following formula: Among them, P F (i) represents the probability of falling or not falling at present, P fall (i) represents the probability of falling, y i Indicates the actual status value, 1 means falling, 0 means not falling; S72. Identify the indoor fall risk area heat map according to the fall probability, and correct the result of step S62.

Images