Fall detection method and device, equipment, storage medium and product

Abstract

The invention discloses a fall detection method, device and equipment, a storage medium and a product, and relates to the technical field of Internet of Things, and the method comprises the steps: obtaining point cloud data generated by a millimeter wave radar in a target space, and carrying out the clustering of the point cloud data, and obtaining a corresponding data cluster; based on the number of the data clusters, determining the number of to-be-detected targets in the target space; when the number of the to-be-detected targets accords with a preset value, calculating falling possibility scores of the to-be-detected targets; determining the falling probability of the to-be-detected target based on the falling possibility score of the to-be-detected target; and when the tumble probability is greater than a preset tumble threshold, determining that the to-be-detected target has a tumble risk, and sending alarm information to the to-be-detected target. Through the mode, the human body posture information is extracted from the environment, the falling possibility is scored according to the human body posture, falling is accurately detected, privacy is protected, meanwhile, high detection accuracy is guaranteed, and the technical problem that falling detection accuracy is low is solved.

Description

Technical Field

[0001] The present application relates to the field of Internet of Things technology, and in particular to fall detection methods, devices, equipment, storage media and products. Background Art

[0002] Slippery and steamy restrooms and bathrooms are common places where falls occur. In such scenarios, for privacy reasons, cameras and wearable devices are not suitable for fall detection. Millimeter-wave radar is often used for fall detection. Millimeter-wave radar uses electromagnetic wave reflection to obtain information such as a person's distance, speed, and angle, thereby estimating their posture.

[0003] Currently, most methods of using millimeter-wave radar to detect falls rely on point cloud data to make judgments, usually ignoring the possibility of multiple people in the scene, and not utilizing information from point clouds at past moments. The criteria for judging falls are also relatively simple, with low accuracy and robustness.

[0004] The above content is only used to assist in understanding the technical solution of the present invention and does not constitute an admission that the above content is prior art. Summary of the Invention

[0005] The main purpose of this application is to provide a fall detection method, device, equipment, storage medium and product, aiming to solve the technical problem of low fall detection accuracy in the existing technology.

[0006] To achieve the above objectives, the present application provides a fall detection method, which includes:

[0007] Acquire point cloud data generated by the millimeter-wave radar in the target space, and cluster the point cloud data to obtain corresponding data clusters;

[0008] Determining the number of to-be-detected objects in the object space based on the number of the data clusters;

[0009] When the number of the targets to be detected meets a preset value, calculating the fall possibility scores of the targets to be detected;

[0010] Determining a fall probability of the target to be detected based on the fall possibility score of the target to be detected;

[0011] When the fall probability is greater than a preset fall threshold, it is determined that the target to be detected has a fall risk, and an alarm message is generated.

[0012] In one embodiment, the step of clustering the point cloud data to obtain corresponding data clusters includes:

[0013] Projecting the point cloud data onto a two-dimensional plane to obtain two-dimensional point cloud data;

[0014] Determining a core point in the two-dimensional point cloud data, wherein the number of points within a neighborhood radius of the core point is greater than a threshold value of the number of points within the cluster;

[0015] Determine corresponding boundary points within the neighborhood radius of the core point, divide the boundary points into data clusters corresponding to the core point, and merge data clusters with overlapping neighborhood radius of the core point;

[0016] Calculating the number of points within the cluster, the geometric volume, and the energy of the data cluster, and deleting the data cluster whose number of points within the cluster is less than the threshold of the number of points within the cluster, deleting the data cluster whose geometric volume is greater than a preset upper volume limit or less than a preset lower volume limit, and deleting the data cluster whose energy is less than a preset energy threshold;

[0017] Back-projecting is performed on the core points and boundary points in the data cluster to determine the point cloud data to which each data cluster is divided.

[0018] In one embodiment, when the number of the targets to be detected meets a preset value, the step of calculating the fall probability score of the targets to be detected includes:

[0019] When the number of the targets to be detected meets a preset value, determining corresponding scoring indicators based on the point cloud data in the data cluster, the scoring indicators including at least waist height, vertical speed of the center of mass, relative vertical displacement of the center of mass, and ground inclination;

[0020] Based on the score corresponding to the scoring indicator and the weight corresponding to the scoring indicator, a fall possibility score of the target to be detected is obtained.

[0021] In one embodiment, the step of determining the corresponding scoring index based on the point cloud data in the data cluster includes:

[0022] Determining a current centroid based on the point cloud data in the data cluster;

[0023] Taking the height of the current center of mass as the waist height;

[0024] Based on the angle between the antenna plane of the millimeter-wave radar and the wall of the target space, projecting the radial velocity of the current center of mass into the vertical direction to obtain the vertical velocity of the center of mass;

[0025] Determining the relative vertical displacement of the center of mass based on the height of the current center of mass and the average height of historical center of mass;

[0026] Based on the current center of mass, a human body trunk vector is determined, and the angle between the human body trunk vector and the ground is used as the ground inclination angle.

[0027] In one embodiment, the step of determining the human body trunk vector based on the current center of mass includes:

[0028] determining a covariance matrix based on the current centroid;

[0029] Determining a system of linear equations corresponding to eigenvalues ​​and eigenvectors based on the covariance matrix;

[0030] Solving the linear equations to obtain a target eigenvector;

[0031] The target feature vector is used as a human body vector.

[0032] In one embodiment, the step of determining the fall probability of the target to be detected based on the fall possibility score of the target to be detected includes:

[0033] Obtaining a historical fall probability score of the target to be detected within a preset time range;

[0034] Calculating a weighted fall probability score of the target to be detected based on the fall probability score, the weight of the fall probability score, the historical fall probability score, and the weight of the historical fall probability score;

[0035] The fall probability weighted score of the target to be detected is mapped to a preset probability range to obtain the fall probability of the target to be detected.

[0036] In addition, to achieve the above objectives, the present application also proposes a fall detection device, which includes:

[0037] A point cloud aggregation module is used to obtain point cloud data generated by the millimeter-wave radar in the target space, cluster the point cloud data, and obtain corresponding data clusters;

[0038] a score calculation module, configured to determine the number of to-be-detected objects in the object space based on the number of the data clusters;

[0039] The score calculation module is further configured to calculate the fall probability score of the target to be detected when the number of the target to be detected meets a preset value;

[0040] A fall judgment module, configured to determine a fall probability of the target to be detected based on the fall possibility score of the target to be detected;

[0041] The fall judgment module is further configured to determine that the target to be detected has a fall risk and generate an alarm message when the fall probability is greater than a preset fall threshold.

[0042] In addition, to achieve the above-mentioned purpose, the present application also proposes a fall detection device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the fall detection method described above.

[0043] In addition, to achieve the above-mentioned purpose, the present invention also proposes a storage medium, which is a computer-readable storage medium and stores a computer program. When the computer program is executed by a processor, the steps of the fall detection method described above are implemented.

[0044] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of the fall detection method described above are implemented.

[0045] The present application provides a fall detection method, which obtains point cloud data generated by a millimeter-wave radar in a target space, clusters the point cloud data, and obtains corresponding data clusters; based on the number of data clusters, determines the number of targets to be detected in the target space; when the number of targets to be detected meets a preset value, calculates the fall possibility score of the target to be detected; based on the fall possibility score of the target to be detected, determines the fall probability of the target to be detected; when the fall probability is greater than a preset fall threshold, determines that the target to be detected has a fall risk, and sends an alarm message to the target to be detected. The present application uses millimeter-wave radar to extract human posture information of the target to be detected from the environment, determines the number of targets to be detected by clustering, and in the case of a single person, scores the fall possibility, accurately calculates the fall probability, and thus accurately detects falls. It has high accuracy and precision, high robustness, and can effectively protect privacy. It is suitable for fall detection in privacy scenarios, improves scenario applicability, and solves the technical problem of low fall detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0047] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0048] Figure 1 This is a flow chart of the first embodiment of the fall detection method of the present application;

[0049] Figure 2 This is a flow chart of the second embodiment of the fall detection method of the present application;

[0050] Figure 3 A schematic diagram of a brief flow chart of the fall detection method provided in Example 2 of the present application;

[0051] Figure 4 This is a schematic diagram of the module structure of the fall detection device according to an embodiment of the present application;

[0052] Figure 5 Schematic diagram of the device structure of the hardware operating environment involved in the fall detection method in the embodiment of the present application.

[0053] The realization of the objectives, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0054] It should be understood that the specific embodiments described herein are merely used to explain the technical solutions of the present application and are not intended to limit the present application.

[0055] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0056] The main solution of the embodiment of the present application is: obtaining the point cloud data generated by the millimeter-wave radar in the target space, clustering the point cloud data to obtain corresponding data clusters; determining the number of targets to be detected in the target space based on the number of data clusters; calculating the fall possibility score of the target to be detected when the number of targets to be detected meets the preset value; determining the fall probability of the target to be detected based on the fall possibility score of the target to be detected; when the fall probability is greater than the preset fall threshold, determining that the target to be detected is at risk of falling, and sending an alarm message to the target to be detected.

[0057] Currently, most methods of using millimeter-wave radar to detect falls rely on point cloud data to make judgments, usually ignoring the possibility of multiple people in the scene, and not utilizing information from point clouds at past moments. The criteria for judging falls are also relatively simple, with low accuracy and robustness.

[0058] The present application provides a solution, which uses millimeter-wave radar to extract human posture information of the target to be detected from the environment, determines the number of targets to be detected by clustering, scores the possibility of falling in the case of a single person, accurately calculates the probability of falling, and thus accurately detects falls. It has high accuracy and precision, high robustness, and can effectively protect privacy. It is suitable for fall detection in privacy scenarios, improves scenario applicability, and solves the technical problem of low fall detection accuracy.

[0059] It should be noted that the execution subject of this embodiment may be a computing service device with data processing, network communication, and program execution capabilities, such as a tablet computer, personal computer, mobile phone, etc., or an electronic device capable of performing the above functions, such as a fall detection device, etc., and this embodiment does not specifically limit this. The following uses a fall detection device as an example to illustrate this embodiment and the following embodiments.

[0060] The present application embodiment provides a fall detection method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the fall detection method of the present application.

[0061] In this embodiment, the fall detection method includes steps S10 to S50:

[0062] Step S10, acquiring point cloud data generated by the millimeter-wave radar in the target space, and clustering the point cloud data to obtain corresponding data clusters;

[0063] It should be noted that millimeter-wave radar transmits frequency-modulated continuous waves into space. By analyzing the received signals, it can estimate the distance, velocity, and angle of multiple objects in space. The target space is the environment currently being detected for fall, such as a bathroom, and this embodiment does not specifically limit this.

[0064] It can be understood that the spatial rectangular coordinate system (three-dimensional space) is established with the millimeter wave radar as the origin, the vertical upward direction as the positive direction of the Z axis, and the normal perpendicular to the radar antenna as the positive direction of the X axis. The millimeter wave radar outputs a frame of point cloud data P(t) = {{p5} i＝0…N-1}, point cloud data is a set, which contains N elements, each of which is a point p represented by a spatial rectangular coordinate system. i (x,y,z,v,s), where x, y, and z are the points p i The three-dimensional coordinates of point p i The radial velocity of point p is i signal-to-noise ratio.

[0065] It should be understood that point cloud data reflects the spatial characteristics of the human body. The detected points within the point cloud data are partially derived from reflections from the body and partially from noise. Generally speaking, points belonging to the same individual are closer together, have higher density, and tend to cluster together. Therefore, to remove noise points from the point cloud and separate points belonging to different individuals for subsequent analysis, it is necessary to cluster the point cloud data into different data clusters, each representing a different individual.

[0066] In a feasible implementation, step S10 may include steps S101 to S102:

[0067] Step S101, projecting the point cloud data onto a two-dimensional plane to obtain two-dimensional point cloud data;

[0068] It should be noted that in order to enhance the robustness of clustering, all points in the point cloud data can be projected onto a two-dimensional plane (XOY plane), that is, p i (x,y,z,v,s)→p i (x, y, 0, v, s), the data at this point is the two-dimensional point cloud data. Then cluster the projected point cloud and finally restore the projected points to the three-dimensional space (XYZ space), which can keep the point ownership relationship unchanged.

[0069] Step S102, determining a core point in the two-dimensional point cloud data, wherein the number of points within a neighborhood radius of the core point is greater than a threshold value of the number of points within the cluster;

[0070] It should be noted that the number of points within the neighborhood radius is the number of points existing within the neighborhood radius, and the point whose number of points within the neighborhood radius is greater than the threshold value of the number of points within the cluster is the core point. The threshold value of the number of points within the cluster is the pre-set minimum number of points within the cluster. The specific value can be flexibly adjusted according to actual needs, and this embodiment does not make specific restrictions on this. The neighborhood radius is the set neighborhood range, and the specific value can be flexibly adjusted according to actual needs, and this embodiment does not make specific restrictions on this. For example, assuming that the threshold value of the number of points within the cluster is 5, if the number of points within the neighborhood radius of point A is 8, then point A is the core point, and if the number of points within the neighborhood radius of point B is 4, then point B is not a core point. Each core point corresponds to an initial data cluster, and after merging and deletion, the final data cluster is obtained.

[0071] Step S103, determining corresponding boundary points within the neighborhood radius of the core point, dividing the boundary points into data clusters corresponding to the core point, and merging data clusters with overlapping neighborhood radius of the core point;

[0072] It should be noted that for all points within the neighborhood radius of each core point, if it is not another core point, it is considered a boundary point and is added to the data cluster of the core point. If it is another core point, the data clusters corresponding to the two core points are merged, and the neighborhood radius of the two core points overlaps. The core points are traversed until all the data clusters that can be merged are merged.

[0073] It can be understood that a point is either a core point, a boundary point that has been classified into a data cluster, or a noise point that has not been classified into a data cluster. In other words, these noise points can be removed through clustering. Core points and boundary points reflect the posture information of the human body and are useful points, which can be called target points.

[0074] Step S104, calculating the number of points, geometric volume, and energy of the data clusters, and deleting data clusters whose number of points is less than a threshold value, whose geometric volume is greater than a preset upper limit or less than a preset lower limit, and whose energy is less than a preset energy threshold;

[0075] It should be noted that the number of in-cluster points refers to the number of target points in a data cluster, that is, the sum of the number of core points and boundary points in a data cluster. In this embodiment, it is necessary to delete data clusters whose in-cluster point count is less than the in-cluster point count threshold. For example, assuming that the in-cluster point count threshold is 5, the number of in-cluster points in data cluster A is 4, and the number of in-cluster points in data cluster B is 6, then data cluster A needs to be deleted, but data cluster B does not.

[0076] It can be understood that the point cloud data of each data cluster is regarded as a cuboid, and the point with the largest x, y, and z coordinates and the point with the smallest coordinates are found respectively, and the volume V=(x max -x min )×(y max -y min )×(z max -z min ), where x max 、x min 、y max 、y min 、z max 、z min They respectively represent the x-coordinate of the point with the largest x-coordinate, the x-coordinate of the point with the smallest x-coordinate, the y-coordinate of the point with the largest y-coordinate, the y-coordinate of the point with the smallest y-coordinate, the z-coordinate of the point with the largest z-coordinate, and the z-coordinate of the point with the smallest z-coordinate. The preset volume upper limit is the set maximum value of the geometric volume, and the preset volume lower limit is the set minimum value of the geometric volume. The specific values ​​are usually set according to the normal volume of the human body. For example, the normal volume of the human body is about 0.04-0.08 cubic meters. The preset volume lower limit can be set to 0.02 and the preset volume upper limit can be set to 0.1. This embodiment needs to delete data clusters whose geometric volume is greater than the preset volume upper limit or less than the preset volume lower limit. For example, assuming that the preset volume lower limit is 0.02 and the preset volume upper limit is 0.1, then delete data clusters whose geometric volume is greater than 0.1 cubic meter or whose geometric volume is less than 0.02 cubic meter.

[0077] It should be understood that the sum of the signal-to-noise ratios of all points within a data cluster is the energy of that data cluster. Data clusters with very low energy are likely composed of noise points. Therefore, this embodiment sets a minimum energy requirement, i.e., a preset energy threshold, and deletes data clusters with energy below the preset energy threshold. The specific value of the preset energy threshold can be flexibly adjusted according to actual needs and is not specifically limited in this embodiment.

[0078] In the specific implementation, the number of points, geometric volume and energy of each data cluster are calculated. Invalid data clusters are deleted based on the number of points, geometric volume and energy in the cluster to further ensure the accuracy of the data cluster.

[0079] Step S105 , back-projecting the core points and boundary points in the data cluster to determine the point cloud data to which each data cluster is divided.

[0080] It can be understood that all points are projected back into the XYZ plane to restore the original Z coordinate, that is, p i (x,y,0,v,s)→p i (x, y, z, v, s), while keeping the cluster relationship unchanged. The result of clustering is several data clusters, each of which is a set of target points C(t) = {{p i} i＝0…M-1}, M is the number of points in the cluster.

[0081] Step S20, determining the number of to-be-detected targets in the target space based on the number of the data clusters;

[0082] It should be noted that the target to be detected, i.e., the target in the target space that needs to be judged whether it has fallen, is usually all people in the target space.

[0083] It will be understood that in this embodiment, each data cluster corresponds to a target to be detected. In other words, there are as many targets to be detected as there are data clusters. In other words, the number of targets to be detected is equal to the number of data clusters. The number of targets to be detected can be determined based on the number of data clusters. For example, if the number of data clusters is 1, the number of targets to be detected is 1; if the number of data clusters is 2, the number of targets to be detected is 2.

[0084] Step S30, when the number of the objects to be detected meets a preset value, calculating the fall possibility scores of the objects to be detected;

[0085] It should be noted that if there are no people in the target space, there is no need to detect whether a fall has occurred. If there is more than one person in the target space, the fall behavior will be detected by the other person, and there is no need to detect whether a fall has occurred. Therefore, this embodiment sets the preset value to 1. That is, when the number of targets to be detected is 1, it is necessary to detect whether a fall has occurred. At this time, the fall probability score of the target to be detected is calculated. The fall probability score is a comprehensive score of the probability of falling. In this embodiment, the fall probability score is the fall probability score of the current frame.

[0086] It's understandable that point cloud data can reflect a person's posture and, therefore, determine whether they've fallen. For a given frame of point cloud data, several scoring metrics can be constructed, with higher scores indicating a higher likelihood of a fall. Combining all these scoring metrics, a fall probability score is calculated for the target being detected.

[0087] It should be understood that if there are n scoring indicators, the weight of each scoring indicator is α i , i=1…n, the score of each scoring indicator is x5, i=1…n, the time of the current frame is t, then the fall possibility score can be

[0088] Step S40, determining the fall probability of the target to be detected based on the fall possibility score of the target to be detected;

[0089] In a feasible implementation, step S40 may include: obtaining the historical fall possibility score of the target to be detected within a preset time range; calculating the fall possibility weighted score of the target to be detected based on the fall possibility score, the weight of the fall possibility score, the historical fall possibility score, and the weight of the historical fall possibility score; mapping the fall possibility weighted score of the target to be detected to a preset probability range to obtain the fall probability of the target to be detected.

[0090] It should be noted that the preset time range, i.e., the set historical time range, typically selects several frames from the past and can be flexibly adjusted based on the timeframe, with no specific restrictions. The historical fall probability score is the fall probability score calculated in the past. Both the fall probability score and the historical fall probability score have corresponding weights. Generally speaking, the closer the timeframe, the higher the weight; that is, the fall probability score has the highest weight. The weights of the historical fall probability scores decrease in chronological order of the frames. The specific values ​​can be flexibly adjusted based on the timeframe, with no specific restrictions.

[0091] It can be understood that the fall probability at the current moment is calculated by combining the fall probability scores of several frames. According to the weight of the fall probability score, the weighted score corresponding to the fall probability score is calculated. According to the weight of the historical fall probability score, the weighted score corresponding to the historical fall probability is calculated. The weighted score corresponding to the fall probability score is added to the weighted score corresponding to the historical fall probability. The calculated value is the weighted score of the fall probability. For example, assuming that the fall probability score of the current frame is X1, and the historical fall probability scores of two frames within the preset time range are X2 and X3 respectively, if the weights corresponding to X1, X2, and X3 are w1, w2, and w3 respectively, then the weighted score of the fall probability X = X1*w1+X2*w2+X3*w3.

[0092] In addition, it should be noted that the preset probability range is [0, 1]. By mapping the weighted score of the fall possibility to [0, 1] through a function, the fall probability at the current moment can be obtained.

[0093] Step S50: When the fall probability is greater than a preset fall threshold, it is determined that the target to be detected has a fall risk, and an alarm message is generated.

[0094] It should be noted that the greater the probability of falling, the more likely it is to happen. The preset fall threshold is the threshold set for determining whether a fall has occurred. The specific value is set according to actual needs and is not specifically limited. Alarm information, that is, fall reminder information, is usually sent to the guardian / relatives of the target to be detected, or other designated monitoring personnel, so that appropriate measures can be taken to ensure the safety of the target to be detected.

[0095] It is understood that when the probability of falling is greater than the preset fall threshold, it indicates that the target to be detected has a high probability of falling, that is, there is a risk of falling at this time. Therefore, an alarm message is generated to warn. The alarm message can be in text form, such as: SMS, pop-up window, or voice form, such as: voice prompt, ringtone prompt, etc., without specific limitation. For example, assuming the preset fall threshold is 0.8, if the probability of falling of the target to be detected is 0.85, then the target to be detected is judged to be at risk of falling, and an alarm text message is sent to the relatives of the target to be detected.

[0096] This embodiment provides a fall detection method, which obtains point cloud data generated by a millimeter-wave radar in a target space, clusters the point cloud data, and obtains corresponding data clusters; based on the number of data clusters, determines the number of targets to be detected in the target space; when the number of targets to be detected meets a preset value, calculates the fall possibility score of the target to be detected; based on the fall possibility score of the target to be detected, determines the fall probability of the target to be detected; when the fall probability is greater than a preset fall threshold, determines that the target to be detected is at risk of falling, and sends an alarm message to the target to be detected. The millimeter-wave radar is used to extract human posture information of the target to be detected from the environment, and the number of targets to be detected is determined by clustering. In the case of a single person, the fall possibility is scored, and the fall probability is accurately calculated, thereby accurately detecting falls. The method has high accuracy and precision, high robustness, and can effectively protect privacy. It is suitable for fall detection in privacy scenarios and improves scenario applicability.

[0097] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above embodiment 1 can be referred to the above introduction and will not be described in detail later. Figure 2 , the step S30 may include steps S301 to S302:

[0098] Step S301, when the number of the targets to be detected meets a preset value, determining corresponding scoring indicators based on the point cloud data in the data cluster, the scoring indicators including at least waist height, vertical speed of the center of mass, relative vertical displacement of the center of mass, and ground inclination;

[0099] It should be noted that, in this embodiment, the scoring indicators include at least waist height, vertical speed of the center of mass, relative vertical displacement of the center of mass, and ground inclination.

[0100] In a feasible implementation, step S301 may include steps S3011 to S3015:

[0101] Step S3011, determining the current centroid based on the point cloud data in the data cluster;

[0102] It should be noted that the centroid of the point cloud data in the data cluster is the current centroid of the target to be detected, that is, the current centroid.

[0103] For the data cluster C(t)={{p i} i＝0…M-1}, whose center of mass is Among them, p i ∈C(t), x, y, z are point p i The three-dimensional coordinates of point p i The radial velocity of M is the vector sum of M target points.

[0104] Step S3012, using the height of the current center of mass as the waist height;

[0105] It should be noted that the waist height is the height of the waist of the target to be detected, and the height of the current centroid of the data cluster is used as the waist height, that is, the Z-axis coordinate of the current centroid of the data cluster is used as the waist height.

[0106] It is understandable that, considering that the height of a person falling may be lower than normal, this embodiment sets a height threshold. When the waist height is less than or equal to the height threshold, the score is the difference between the height threshold and the waist height. In this case, the lower the waist height, the higher the corresponding score. When the waist height is greater than the height threshold, the score is 0. The calculation relationship is as follows:

[0107]

[0108] In the formula, x1 represents the score corresponding to waist height, z i Indicates waist height, Indicates the height threshold.

[0109] It should be noted that the specific value of the height threshold can be set according to actual needs. For example, the height threshold is set to half of the average height of Chinese people.

[0110] Step S3013: projecting the radial velocity of the current center of mass to a vertical direction based on the angle between the antenna plane of the millimeter-wave radar and the wall of the target space to obtain the vertical velocity of the center of mass;

[0111] It should be noted that according to the angle θ between the antenna plane of the millimeter wave radar and the wall in the target space, the radial velocity of the center of mass is projected onto the Z axis to obtain the vertical velocity of the center of mass v z =v g cosθ.

[0112] It is understandable that, considering that a human body will have a downward speed when falling, the faster the speed, the greater the energy of the human body's fall. Therefore, this embodiment sets a speed threshold. When the vertical speed of the center of mass is less than or equal to the speed threshold, the score is the square of the vertical speed of the center of mass. At this time, the larger the square of the vertical speed of the center of mass, the higher the corresponding score. When the vertical speed of the center of mass is greater than the speed threshold, the score is 0. The calculation relationship is as follows:

[0113]

[0114] Where x2 represents the fraction corresponding to the vertical velocity of the center of mass, v z represents the vertical velocity of the center of mass, represents the speed threshold, The specific value can be set according to actual needs.

[0115] It should be understood that the dimension of the vertical velocity of the center of mass after being squared is [energy / mass]. The mass of the same individual is fixed and can be ignored. Therefore, the square of the vertical velocity of the center of mass reflects the energy of the human body falling. The bigger you are, the more likely you are to fall.

[0116] Step S3014, determining the relative vertical displacement of the center of mass based on the height of the current center of mass and the average height of historical center of mass;

[0117] It should be noted that when a person falls slowly, although the downward speed of the center of mass is not large, the height of the center of mass will become lower and lower compared to the previous frames. At this time, it is necessary to comprehensively consider the center of mass height of several frames.

[0118] It can be understood that the historical centroid is the centroid of the previous frames, and the average height of the historical centroid is the average height of the centroid of the previous frames. Assuming that the current time is t, the corresponding centroid height is the height of the current centroid, recorded as z t =z g, calculate the center of mass height at time t-1~tT respectively, and record them as z t-1 ~z t-T , as the height of the historical centroid, where T is the length of the set sliding window, and the average height of the centroid of all frames in the sliding window is calculated, that is, the average height of the historical centroid is calculated, and the obtained The relative vertical displacement Δz of the center of mass is the height z of the current center of mass t The average height z from the historical centroid avg The difference between them, that is, Δz=z t -z avg .

[0119] It should be understood that this embodiment sets a displacement threshold. When the relative vertical displacement of the center of mass is less than or equal to the displacement threshold, the score is the difference between the displacement threshold and the relative vertical displacement of the center of mass. At this time, the larger the average value of the center of mass height of all frames in the sliding window, the higher the corresponding score. When the relative vertical displacement of the center of mass is greater than the displacement threshold, the score is 0. The calculation relationship is as follows:

[0120]

[0121] In the formula, x3 represents the fraction corresponding to the relative vertical displacement of the center of mass, Δz represents the relative vertical displacement of the center of mass, represents the displacement threshold, , the specific value can be set according to actual needs.

[0122] Step S3015: determining a human body trunk vector based on the current center of mass, and taking the angle between the human body trunk vector and the ground as the ground inclination angle.

[0123] It should be noted that the ground inclination angle is the angle between the body and the ground. When a person falls slowly, they lie flat on the ground, and the inclination angle between the body and the ground is close to 0 degrees. When a person stands, the inclination angle between the body and the ground is close to 90 degrees. Therefore, the closer the ground inclination angle is to 0, the more likely it is to fall, and the higher the corresponding score.

[0124] It can be understood that if the data cluster is regarded as a collection of vectors, the direction of the human body trunk is the principal component vector of the vectors in the data cluster, and the principal component analysis can be used to find the human body trunk vector.

[0125] In a feasible embodiment, the step of determining the human body trunk vector based on the current center of mass includes: determining the covariance matrix based on the current center of mass; determining the linear equation group corresponding to the eigenvalues ​​and eigenvectors based on the covariance matrix; solving the linear equation group to obtain the target eigenvector; and using the target eigenvector as the human body trunk vector.

[0126] First, calculate the covariance matrix Where “-” is vector subtraction. Next, calculate the eigenvalue λ of the covariance matrix C i With the eigenvector x i Then, solve the linear equation system (λE-C)x=0 and obtain the eigenvector with the largest eigenvalue e=(x e ,y e ,z e ), which is the target feature vector. In this case, the target feature vector is the human body trunk vector.

[0127] It can be understood that the ground inclination is obtained by calculating the angle between the human body's torso vector and the ground. The calculation relationship is as follows:

[0128]

[0129] Where φ is the ground inclination angle, x e 、y e 、z e is the coordinate of the human torso vector e. The score corresponding to the ground inclination angle is calculated using the following formula:

[0130]

[0131] Where x4 represents the score corresponding to the ground inclination, φ represents the ground inclination, and the smaller the ground inclination, the higher the corresponding score.

[0132] Step S302: obtaining a fall probability score of the target to be detected based on the score corresponding to the scoring indicator and the weight corresponding to the scoring indicator;

[0133] It should be noted that all scoring indicators are combined to calculate the final fall probability score. Waist height, vertical velocity of the center of mass, relative vertical displacement of the center of mass, and ground inclination have corresponding weights. Using the weights of the scoring indicators, the scores of the scoring indicators are weighted and summed to obtain the fall probability score. The calculation relationship is as follows:

[0134]

[0135] It can be understood that x1, x2, x3, and x4 are the fractions of waist height, vertical velocity of center of mass, relative vertical displacement of center of mass, and ground inclination, respectively, and α1, α2, α3, and α4 are the weights of waist height, vertical velocity of center of mass, relative vertical displacement of center of mass, and ground inclination, respectively.

[0136] This embodiment provides a fall detection method. When the number of targets to be detected meets a preset value, corresponding scoring indicators are determined based on the point cloud data in the data cluster. The scoring indicators include at least waist height, vertical velocity of the center of mass, relative vertical displacement of the center of mass, and ground inclination. Based on the scores corresponding to the scoring indicators and the weights corresponding to the scoring indicators, a fall probability score of the target to be detected is obtained. Millimeter-wave radar is used to extract human posture information of the targets to be detected from the environment. The number of targets to be detected is determined through clustering. In the case of a single person, multiple scoring indicators are set to comprehensively score the fall probability, accurately calculating the fall probability, thereby accurately detecting falls. This method has high accuracy and precision, is highly robust, and can effectively protect privacy. It is suitable for fall detection in privacy scenarios and improves scenario applicability.

[0137] For example, to help understand the implementation process of the fall detection method obtained by combining this embodiment with the above-mentioned embodiment 2, please refer to Figure 3 , Figure 3 A brief flowchart of a fall detection method is provided, specifically:

[0138] The number of clusters in the point cloud data is used to determine the number of people in the environment at the time. If the number of people is not 1, the fall probability score for the current frame is considered 0. If the number of people is 1, the fall probability score for the current frame is calculated. The fall probability scores of several frames are combined to calculate the current fall probability. When the fall probability exceeds the threshold, an alarm is issued and the user is notified.

[0139] It should be noted that the above examples are only used to understand the present application and do not constitute a limitation on the fall detection method of the present application. More simple transformations based on this technical concept are all within the scope of protection of the present application.

[0140] This application also provides a fall detection device, please refer to Figure 4 , the fall detection device comprises:

[0141] The point cloud aggregation module 10 is used to obtain point cloud data generated by the millimeter wave radar in the target space, cluster the point cloud data, and obtain corresponding data clusters;

[0142] a score calculation module 20, configured to determine the number of to-be-detected objects in the object space based on the number of the data clusters;

[0143] The score calculation module 20 is further configured to calculate the fall probability score of the target to be detected when the number of the target to be detected meets a preset value;

[0144] A fall judgment module 30 is configured to determine a fall probability of the target to be detected based on the fall probability score of the target to be detected;

[0145] The fall judgment module 30 is further configured to determine that the target to be detected has a fall risk and generate an alarm message when the fall probability is greater than a preset fall threshold.

[0146] In a feasible implementation manner, the point cloud aggregation module 10 is further configured to project the point cloud data onto a two-dimensional plane to obtain two-dimensional point cloud data;

[0147] Determining a core point in the two-dimensional point cloud data, wherein the number of points within a neighborhood radius of the core point is greater than a threshold value of the number of points within the cluster;

[0148] Determine corresponding boundary points within the neighborhood radius of the core point, divide the boundary points into data clusters corresponding to the core point, and merge data clusters with overlapping neighborhood radius of the core point;

[0149] Calculating the number of points within the cluster, the geometric volume, and the energy of the data cluster, and deleting the data cluster whose number of points within the cluster is less than the threshold of the number of points within the cluster, deleting the data cluster whose geometric volume is greater than a preset upper volume limit or less than a preset lower volume limit, and deleting the data cluster whose energy is less than a preset energy threshold;

[0150] Back-projecting is performed on the core points and boundary points in the data cluster to determine the point cloud data to which each data cluster is divided.

[0151] In a feasible embodiment, the score calculation module 20 is further configured to determine corresponding scoring indicators based on the point cloud data in the data cluster when the number of the targets to be detected meets a preset value, the scoring indicators including at least waist height, vertical velocity of the center of mass, relative vertical displacement of the center of mass, and ground inclination;

[0152] Based on the score corresponding to the scoring indicator and the weight corresponding to the scoring indicator, a fall possibility score of the target to be detected is obtained.

[0153] In a feasible implementation manner, the score calculation module 20 is further configured to determine a current centroid based on the point cloud data in the data cluster;

[0154] Taking the height of the current center of mass as the waist height;

[0155] Based on the angle between the antenna plane of the millimeter-wave radar and the wall of the target space, projecting the radial velocity of the current center of mass into the vertical direction to obtain the vertical velocity of the center of mass;

[0156] Determining the relative vertical displacement of the center of mass based on the height of the current center of mass and the average height of historical center of mass;

[0157] Based on the current center of mass, a human body trunk vector is determined, and the angle between the human body trunk vector and the ground is used as the ground inclination angle.

[0158] In a feasible implementation manner, the score calculation module 20 is further configured to determine a covariance matrix based on the current centroid;

[0159] Determining a system of linear equations corresponding to eigenvalues ​​and eigenvectors based on the covariance matrix;

[0160] Solving the linear equations to obtain a target eigenvector;

[0161] The target feature vector is used as a human body vector.

[0162] In a feasible implementation manner, the fall judgment module 30 is further configured to obtain a historical fall probability score of the target to be detected within a preset time range;

[0163] Calculating a weighted fall probability score of the target to be detected based on the fall probability score, the weight of the fall probability score, the historical fall probability score, and the weight of the historical fall probability score;

[0164] The fall probability weighted score of the target to be detected is mapped to a preset probability range to obtain the fall probability of the target to be detected.

[0165] The fall detection device provided in this application, employing the fall detection method described in the aforementioned embodiments, can address the technical issue of low fall detection accuracy. Compared to the prior art, the beneficial effects of the fall detection device provided in this application are the same as those of the fall detection method described in the aforementioned embodiments. Other technical features of the fall detection device are the same as those disclosed in the aforementioned embodiments and are not further elaborated here.

[0166] The present application provides a fall detection device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the fall detection method of the above-mentioned embodiment 1.

[0167] Reference below Figure 5, which shows a schematic structural diagram of a fall detection device suitable for implementing embodiments of the present application. The fall detection device in the embodiments of the present application may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5 The fall detection device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0168] like Figure 5 As shown, the fall detection device may include a processing device 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes based on programs stored in a read-only memory (ROM) 1002 or programs loaded from a storage device 1003 into a random access memory (RAM) 1004. RAM 1004 also stores various programs and data required for the operation of the fall detection device. The processing device 1001, ROM 1002, and RAM 1004 are connected to each other via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the fall detection device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a fall detection device with various systems, it should be understood that it is not required to implement or have all of the systems shown. More or fewer systems may be implemented or have alternatively.

[0169] In particular, according to the embodiments disclosed in the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, the embodiments disclosed in the present application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program comprising program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0170] The fall detection device provided in this application utilizes the fall detection method described in the aforementioned embodiment to address the technical issue of low fall detection accuracy. Compared to the prior art, the fall detection device provided in this application achieves the same beneficial effects as the fall detection method described in the aforementioned embodiment. Other technical features of the fall detection device are the same as those disclosed in the aforementioned embodiment and are not further elaborated here.

[0171] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0172] The above description is merely a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

[0173] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, a computer program) stored thereon, wherein the computer-readable program instructions are used to execute the fall detection method in the above-mentioned embodiment.

[0174] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0175] The computer-readable storage medium may be included in the fall detection device, or may exist independently without being incorporated into the fall detection device.

[0176] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the fall detection device, the fall detection device is enabled to: obtain the point cloud data generated by the millimeter-wave radar in the target space, cluster the point cloud data, and obtain corresponding data clusters; determine the number of targets to be detected in the target space based on the number of data clusters; when the number of targets to be detected meets the preset value, calculate the fall possibility score of the target to be detected; determine the fall probability of the target to be detected based on the fall possibility score of the target to be detected; when the fall probability is greater than the preset fall threshold, determine that the target to be detected has a fall risk, and send an alarm message to the target to be detected.

[0177] Computer program code for performing the operations of the present application may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0178] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0179] The modules described in the embodiments of the present application may be implemented in software or hardware, wherein the name of a module does not necessarily limit the unit itself.

[0180] The computer-readable storage medium provided in this application stores computer-readable program instructions (i.e., a computer program) for executing the aforementioned fall detection method, thereby resolving the technical issue of low fall detection accuracy. Compared to the prior art, the beneficial effects of the computer-readable storage medium provided in this application are similar to those of the fall detection method provided in the aforementioned embodiments, and are not further elaborated here.

[0181] The present application also provides a computer program product, comprising a computer program, which implements the steps of the above-mentioned fall detection method when executed by a processor.

[0182] The computer program product provided in this application can solve the technical problem of low fall detection accuracy. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the fall detection method provided in the above embodiment, and will not be repeated here.

[0183] The above description is only part of the embodiments of the present application and does not limit the patent scope of the present application. All equivalent structural transformations made by using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect application in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A fall detection method, characterized in that: The method comprises: Acquire point cloud data generated by the millimeter-wave radar in the target space, and cluster the point cloud data to obtain corresponding data clusters; Determining the number of to-be-detected objects in the object space based on the number of the data clusters; When the number of the targets to be detected meets a preset value, calculating the fall possibility scores of the targets to be detected; Determining a fall probability of the target to be detected based on the fall possibility score of the target to be detected; When the fall probability is greater than a preset fall threshold, it is determined that the target to be detected has a fall risk, and an alarm message is generated.

2. The method according to claim 1, wherein The step of clustering the point cloud data to obtain corresponding data clusters includes: Projecting the point cloud data onto a two-dimensional plane to obtain two-dimensional point cloud data; Determining a core point in the two-dimensional point cloud data, wherein the number of points within a neighborhood radius of the core point is greater than a threshold value of the number of points within the cluster; Determine corresponding boundary points within the neighborhood radius of the core point, divide the boundary points into data clusters corresponding to the core point, and merge data clusters with overlapping neighborhood radius of the core point; Calculating the number of points within the cluster, the geometric volume, and the energy of the data cluster, and deleting the data cluster whose number of points within the cluster is less than the threshold of the number of points within the cluster, deleting the data cluster whose geometric volume is greater than a preset upper volume limit or less than a preset lower volume limit, and deleting the data cluster whose energy is less than a preset energy threshold; Back-projecting is performed on the core points and boundary points in the data cluster to determine the point cloud data to which each data cluster is divided.

3. The method according to claim 1, wherein When the number of the targets to be detected meets a preset value, the step of calculating the fall possibility score of the targets to be detected includes: When the number of the targets to be detected meets a preset value, determining corresponding scoring indicators based on the point cloud data in the data cluster, the scoring indicators including at least waist height, vertical speed of the center of mass, relative vertical displacement of the center of mass, and ground inclination; Based on the score corresponding to the scoring indicator and the weight corresponding to the scoring indicator, a fall possibility score of the target to be detected is obtained.

4. The method according to claim 3, wherein The step of determining corresponding scoring indicators based on the point cloud data in the data cluster includes: Determining a current centroid based on the point cloud data in the data cluster; Taking the height of the current center of mass as the waist height; Based on the angle between the antenna plane of the millimeter-wave radar and the wall of the target space, projecting the radial velocity of the current center of mass into the vertical direction to obtain the vertical velocity of the center of mass; Determining the relative vertical displacement of the center of mass based on the height of the current center of mass and the average height of historical center of mass; Based on the current center of mass, a human body trunk vector is determined, and the angle between the human body trunk vector and the ground is used as the ground inclination angle.

5. The method according to claim 4, wherein The step of determining the human body trunk vector based on the current center of mass comprises: determining a covariance matrix based on the current centroid; Determining a system of linear equations corresponding to eigenvalues ​​and eigenvectors based on the covariance matrix; Solving the linear equations to obtain a target eigenvector; The target feature vector is used as a human body vector.

6. The method according to claim 1, wherein The step of determining the fall probability of the target to be detected based on the fall possibility score of the target to be detected includes: Obtaining a historical fall probability score of the target to be detected within a preset time range; Calculating a weighted fall probability score of the target to be detected based on the fall probability score, the weight of the fall probability score, the historical fall probability score, and the weight of the historical fall probability score; The fall probability weighted score of the target to be detected is mapped to a preset probability range to obtain the fall probability of the target to be detected.

7. A fall detection device, characterized in that: The device comprises: A point cloud aggregation module is used to obtain point cloud data generated by the millimeter-wave radar in the target space, cluster the point cloud data, and obtain corresponding data clusters; a score calculation module, configured to determine the number of to-be-detected objects in the object space based on the number of the data clusters; The score calculation module is further configured to calculate the fall probability score of the target to be detected when the number of the target to be detected meets a preset value; A fall judgment module, configured to determine a fall probability of the target to be detected based on the fall possibility score of the target to be detected; The fall judgment module is further configured to determine that the target to be detected has a fall risk and generate an alarm message when the fall probability is greater than a preset fall threshold.

8. A fall detection device, characterized in that The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the fall detection method according to any one of claims 1 to 6.

9. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the fall detection method according to any one of claims 1 to 6 are implemented.

10. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the steps of the fall detection method according to any one of claims 1 to 6 are implemented.

Images