Fall judgment method and wearable device thereof

Abstract

The invention provides a fall judgment method and a wearable device thereof. The method comprises the following steps: acquiring acceleration data based on a three-axis accelerometer, and acquiring inclination angle data based on an inclination angle sensor; determining a speed variation and a displacement variation of the wearable device in unit time based on the acceleration data, and determining final attitude information of the wearable device in space in combination with the inclination angle data; if it is determined that the posture of the wearable device is an abnormal movement posture based on the final posture information, starting a camera device to collect a front image of the wearable device, and determining the current orientation of the camera device; performing image analysis based on the front image to obtain human body features, and determining a relative position relationship between the human body and the ground based on the current orientation and the position of the human body features in the front image; and performing fall judgment based on the relative position relationship. According to the invention, the accuracy and reliability of fall judgment are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of wearable devices, and in particular to a fall judgment method and a wearable device thereof. Background Art

[0002] In the prior art, most fall detection methods based on wearable devices rely solely on acceleration data collected by a three-axis accelerometer. The principle is to set an acceleration threshold. When the acceleration detected by the device exceeds the preset threshold, it is determined that a fall may have occurred. However, this method has significant flaws. Due to some strenuous exercises in daily life, such as landing on the ground while running, jumping, and other actions, the acceleration generated may also exceed the threshold, resulting in a large number of misjudgments; at the same time, when the elderly fall slowly or fall on soft ground, the acceleration generated may not reach the preset threshold, which will result in missed judgments. Therefore, the existing fall detection methods based only on three-axis accelerometers are insufficient in accuracy and reliability. Summary of the Invention

[0003] The present invention provides a fall judgment method and a wearable device thereof, for improving the accuracy and reliability of fall judgment.

[0004] In a first aspect, the present invention provides a fall detection method, comprising:

[0005] Acceleration data is collected based on the three-axis accelerometer, and inclination data is collected based on the inclination sensor; the acceleration data includes acceleration data of the wearable device in the X-axis, Y-axis, and Z-axis directions; the inclination data includes the pitch angle and roll angle of the wearable device relative to the horizontal plane;

[0006] Determine the velocity change and displacement change of the wearable device per unit time based on the acceleration data, and determine the final posture information of the wearable device in space in combination with the inclination data;

[0007] If it is determined based on the final posture information that the posture of the wearable device is in an abnormal activity posture, starting the camera device to capture a front image of the wearable device and determining the current position of the camera device;

[0008] performing image analysis based on the front image to obtain human body features, and determining a relative positional relationship between the human body and the ground based on the current orientation and the positions of the human body features in the front image;

[0009] A fall is determined based on the relative position relationship.

[0010] In a second aspect, the present invention further provides a wearable device, which is applied to the fall detection method as described in the first aspect; the wearable device integrates a three-axis accelerometer, an inclination sensor, and a camera device; the wearable device includes:

[0011] a data collection module configured to collect acceleration data based on the triaxial accelerometer and collect inclination data based on the inclination sensor; the acceleration data includes acceleration data of the wearable device in X-axis, Y-axis and Z-axis directions; and the inclination data includes a pitch angle and a roll angle of the wearable device relative to a horizontal plane;

[0012] a posture determination module configured to determine a speed change amount and a displacement change amount of the wearable device in a unit time based on the acceleration data, and determine final posture information of the wearable device in space in combination with the inclination data;

[0013] an orientation determination module configured to start the camera to collect a front image of the wearable device and determine a current orientation of the camera if it is determined based on the final posture information that the posture of the wearable device is in an abnormal activity posture;

[0014] a position relationship determination module configured to perform image analysis based on the front image to obtain human body features, and determine a relative position relationship between the human body and the ground based on the current orientation and positions of the human body features in the front image;

[0015] a fall determination module configured to determine a fall based on the relative position relationship.

[0016] In a third aspect, the present application further provides an electronic device, comprising: a memory configured to store a computer software program; and a processor configured to read and execute the computer software program, thereby realizing any one of the fall determination methods described above.

[0017] In a fourth aspect, the present application further provides a non-transitory computer readable storage medium, wherein the storage medium stores a computer software program, and the computer software program is executed by a processor to realize any one of the fall determination methods described above.

[0018] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, and the computer program is executed by a processor to realize any one of the fall determination methods described above.

[0019] The fall judgment method provided by the embodiment of the present invention combines a three-axis accelerometer and an inclination sensor to perform preliminary activity status judgment. It not only takes acceleration into consideration, but also judges whether the device is in a normal activity state based on its posture, thereby reducing misjudgments caused by strenuous exercise. For example, when running, although the acceleration may be large, the device posture is in a normal movement posture and will not be misjudged as a fall. Secondly, the front image of the wearable device is obtained by a camera device and the relative position relationship between the human body and the ground is analyzed. For slow falls or falls on soft ground, even if the acceleration is not obvious, the camera device can capture the situation of the human body approaching or contacting the ground, thereby avoiding missed judgments. Therefore, the embodiment of the present invention comprehensively judges the fall situation from multiple dimensions, thereby improving the accuracy and reliability of fall judgment. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 1 is a flow chart of a fall determination method provided by an embodiment of the present invention;

[0021] Figure 2 is a schematic structural diagram of a wearable device provided by an embodiment of the present invention;

[0022] Figure 3 An embodiment diagram of an electronic device provided by an embodiment of the present invention;

[0023] Figure 4 An embodiment diagram of a computer-readable storage medium provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0024] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making any creative efforts shall fall within the scope of protection of the present invention.

[0025] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Therefore, a feature specified as "first" or "second" may explicitly or implicitly include one or more of the specified features. In the description of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0026] In the description of the present invention, the term "for example" is used to mean "used as an example, illustration or illustration". Any embodiment of the present invention described as "for example" is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is given to enable any person skilled in the art to implement and use the present invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other examples, well-known structures and processes are not elaborated in detail to avoid obscuring the description of the present invention with unnecessary details. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed herein.

[0027] Optional, see Figure 1 , Figure 1 FIG. 1 is a flow chart of a fall detection method provided by the present invention. In an embodiment of the present invention, the execution subject of the fall detection method is a wearable device. Therefore, the fall detection method includes:

[0028] Step 10: Collect acceleration data using a three-axis accelerometer and inclination data using an inclination sensor. The acceleration data includes acceleration data of the wearable device in the X, Y, and Z axes. The inclination data includes the pitch and roll angles of the wearable device relative to the horizontal plane.

[0029] Optionally, a three-axis accelerometer uses a built-in microelectromechanical system (MEMS) to sense changes in acceleration of the wearable device in the X, Y, and Z axes, converting the physical acceleration signal into an electrical signal, which is then converted into digital acceleration data through analog-to-digital conversion. Inclination sensors are usually based on the principle of gravitational acceleration. By measuring the components of gravity on different axes, they calculate the pitch and roll angles of the wearable device relative to the horizontal plane. Wearable devices continuously collect acceleration and inclination data at a certain frequency (e.g., 100Hz).

[0030] In one embodiment, a smart wristband is used as an example. Its built-in three-axis accelerometer and inclinometer begin operating. When a user walks normally, the three-axis accelerometer collects acceleration data on the X-axis that fluctuates between -1g and 1g (g is the acceleration due to gravity), with corresponding dynamic changes on the Y and Z axes. The inclinometer measures the wristband's pitch angle relative to the horizontal plane, which varies within a range of ±15 degrees, and its roll angle, which varies within a range of ±10 degrees. This data is timestamped and stored in real time in the wristband's buffer.

[0031] Step 20: Determine the velocity change and displacement change of the wearable device per unit time based on the acceleration data, and determine the final posture information of the wearable device in space in combination with the inclination data.

[0032] Furthermore, according to the kinematic formula, by integrating the acceleration data, the speed change of the wearable device per unit time can be obtained. The speed change per unit time is integrated again to calculate the displacement change of the wearable device per unit time.

[0033] Furthermore, the final posture information of the wearable device in space is determined based on the speed change and the displacement change in unit time in combination with the inclination data, as specifically described in the process from step 201 to step 203.

[0034] Step 30: If it is determined based on the final posture information that the posture of the wearable device is in an abnormal activity posture, the camera device is started to capture the front image of the wearable device and determine the current position of the camera device.

[0035] Furthermore, the final posture information is used to determine whether the posture of the wearable device is in an abnormal activity posture. When the posture of the wearable device is determined to meet the abnormal posture condition based on the final posture information, the camera device is immediately activated, and the current position of the camera device is determined using inertial navigation or other positioning technology.

[0036] Step 40 , performing image analysis based on the front image to obtain human body features, and determining the relative position relationship between the human body and the ground based on the current orientation and the position of the human body features in the front image.

[0037] Furthermore, computer vision techniques, such as object detection algorithms (e.g., YOLO and Faster R-CNN), are used to analyze the front image and identify key features of the human body, such as the head, torso, and limbs. Based on the current orientation of the camera and the pixel positions of the human features in the image, combined with camera calibration parameters, the relative position of each part of the human body and the ground is calculated, as shown in the process from step 401 to step 403.

[0038] Step 50: determine whether a person has fallen down based on the relative position relationship.

[0039] Furthermore, whether a fall occurs is determined based on the relative position relationship, specifically as in the process from step 501 to step 503 .

[0040] The embodiment of the present invention combines a three-axis accelerometer and an inclinometer to perform preliminary activity status judgment. It not only takes acceleration into consideration, but also judges whether the device is in a normal activity state based on its posture, thereby reducing misjudgments caused by strenuous exercise. For example, when running, although the acceleration may be large, the device posture is in a normal movement posture and will not be misjudged as a fall. Secondly, the front image of the wearable device is obtained through a camera device and the relative position relationship between the human body and the ground is analyzed. For slow falls or falls on soft ground, even if the acceleration is not obvious, the camera device can capture the situation of the human body approaching or contacting the ground, thereby avoiding missed judgments. Therefore, the fall situation is comprehensively judged from multiple dimensions, which improves the accuracy and reliability of fall judgment.

[0041] In one embodiment, the process from step 201 to step 203 includes:

[0042] Step 201 determines a velocity vector change and a displacement vector change based on the velocity change and displacement change at adjacent moments in time. A velocity-displacement plane is constructed based on the velocity vector change and the displacement vector change. The plane normal to the velocity-displacement plane is obtained by the cross product of the velocity vector change and the displacement vector change.

[0043] Optionally, the velocity change and displacement change at adjacent moments in the unit time obtained in step 20 are calculated by vector subtraction. Based on the velocity vector change and displacement vector change, they are expressed as three-dimensional vectors and The velocity displacement plane is constructed using the cross multiplication operation, and the plane normal vector of the plane is Depend on We get, × represents the cross product, where the plane normal vector The calculation formula is:

[0044]

[0045] in, are the unit vectors of the x, y, and z axes in the Cartesian coordinate system.

[0046] In one embodiment, the speed vector change of the smart bracelet in a certain unit time is Displacement vector change Calculate the plane normal vector of the velocity displacement plane according to the above cross product formula:

[0047] Step 202: Convert the pitch angle and roll angle into an initial posture vector of the wearable device in space. The initial posture vector represents the initial tilt state of the wearable device.

[0048] Furthermore, the pitch angle θ and roll angle collected in step 10 are Convert to the initial posture vector in space. Using the rotation matrix theory, the angle information is mapped to the three-dimensional vector space. First, define the rotation matrix R corresponding to the pitch angle and roll angle θ and

[0049]

[0050] Initial pose vector The unit vector (such as represents the initial vertical direction) and multiply the above rotation matrix by the left to obtain:

[0051]

[0052] In one embodiment, if the pitch angle θ of the smart bracelet is 30°, the roll angle Convert degrees to radians: θ rad =π / 6, Calculate the rotation matrix:

[0053]

[0054] Then the initial posture vector is:

[0055]

[0056] Step 203: Determine the final posture information of the wearable device in space based on the initial posture vector and the velocity displacement plane.

[0057] Furthermore, the final posture information of the wearable device in space is determined according to the initial posture vector and the velocity displacement plane, as specifically described in the process from step 2031 to step 2033.

[0058] The embodiment of the present invention deeply integrates acceleration and displacement changes with inclination data, realizing the conversion from raw sensor data to precise spatial posture information. It can accurately capture the posture changes of wearable devices during dynamic movement, and improve the accuracy and robustness of posture information.

[0059] In one embodiment, the process from step 2031 to step 2033 includes:

[0060] Step 2031 : Project the initial posture vector onto the velocity displacement plane to obtain the projection vector of the initial posture vector on the velocity displacement plane.

[0061] Optional, plane normal vector of the plane of displacement with known velocity and the initial pose vector According to the vector projection principle, the initial posture vector Projected onto the velocity displacement plane, the projection vector is obtained The calculation of vector projection is based on the vector dot product and the normalization of the normal vector. First, calculate the component of the initial posture vector in the normal vector direction, and then subtract this component from the initial posture vector to obtain the projection vector. The normalized vector of in, n x 、n y 、n z is the normal vector The components on the three axes of the Cartesian coordinate system, the projection vector calculation formula is: in, Represents the dot product of the initial pose vector and the normalized normal vector.

[0062] In one embodiment, the normal vector of the velocity displacement plane Initial pose vector First calculate the modulus of the normal vector: Normalized normal vector So the projection vector

[0063] Step 2032: Calculate a second angle between the projection vector and the velocity vector change. The second angle represents the relationship between the device posture and the motion direction.

[0064] Furthermore, given the projection vector and velocity vector change The second angle α between the two is calculated according to the vector angle formula. The vector angle formula is based on the vector dot product and the vector modulus. The formula is:

[0065]

[0066] in, is the dot product of the projection vector and the change in velocity vector, and are the modulus of the projection vector and velocity vector changes, respectively. The angle α can be obtained through the inverse cosine function, which represents the relationship between the device posture and movement direction.

[0067] Step 2033: Calculate the posture adjustment of the device relative to the initial tilt state based on the angle combined with the displacement vector change, and superimpose the posture adjustment amount with the initial posture vector to obtain the final posture information of the wearable device in space.

[0068] Furthermore, according to the angle α and the displacement vector change Calculate the attitude adjustment of the device relative to the initial tilt state. Using vector rotation and space transformation theory, convert the angle and displacement vector change into a rotation matrix or quaternion for attitude adjustment. Assuming the direction of the displacement vector change is the rotation axis, construct the rotation matrix R based on the angle. α , and then the initial posture vector The adjusted posture vector is obtained by transformation through the rotation matrix, and finally superimposed with the initial posture vector (the superposition here refers to the synthesis based on the rotated vector, which is essentially to update the posture vector through the rotation operation) to obtain the final posture information of the wearable device in space For a unit vector (Here the displacement vector change can be normalized to obtain the unit vector) The rotation matrix R of the rotation angle α α , according to the Rodriguez rotation formula:

[0069]

[0070] Where I is the identity matrix, is a vector The antisymmetric matrix of is the square of the antisymmetric matrix. The final posture vector is:

[0071] In one embodiment, the displacement vector change Normalized to get unit vector Calculate the rotation matrix R according to the Rodriguez rotation formula α :

[0072]

[0073] Given that α≈47.3°, convert it to radians α rad ≈0.826, sinα rad ≈0.737, cosα rad ≈0.676, so

[0074] Initial pose vector

[0075] The final posture vector is This vector represents the final posture information of the smart bracelet in space.

[0076] The embodiment of the present invention starts from the initial posture vector, uses the velocity-displacement plane to construct a reference benchmark, and deeply couples the device posture with the motion state. Therefore, it fully considers the impact of speed and displacement changes on the posture during the device movement process, and can accurately capture the real-time changes of the device posture in complex motion scenes, thereby improving the accuracy and reliability of the wearable device's posture perception.

[0077] In one embodiment, the process from step 301 to step 304 includes:

[0078] Step 301: Construct a coordinate transformation matrix based on the components of the final posture information on the horizontal axis, the vertical axis, and the vertical axis. The diagonal elements of the coordinate transformation matrix are the components on each coordinate axis.

[0079] Optionally, the final posture information of the known wearable device can be expressed as a three-dimensional vector According to its components on the horizontal axis, vertical axis and vertical axis, the coordinate transformation matrix T is constructed. The diagonal elements of the coordinate transformation matrix T correspond to the components of the final posture information on the three coordinate axes, and the non-diagonal elements are 0. Therefore, the coordinate transformation matrix T is expressed as:

[0080]

[0081] This matrix is ​​used to subsequently perform spatial transformation on the difference vector to analyze its characteristics in different dimensions.

[0082] Continuing with the above embodiment, the final posture vector of the smart bracelet is obtained 0.717), then the constructed coordinate transformation matrix T is:

[0083]

[0084] Step 302 : Project the difference vector between the final posture information and the standard active posture vector onto each coordinate axis to obtain the degree of deviation on each coordinate axis. Based on the degree of deviation on each coordinate axis and the coordinate transformation matrix, determine the degree of spirality of the difference vector in space.

[0085] Furthermore, first, define the standard activity posture vector Represents the typical posture of the device in normal activity state (for example, the posture vector of the bracelet when the user is walking normally). Calculate the difference vector between the final posture information and the standard activity posture vector The difference vector Project them onto the three coordinate axes respectively to obtain the degree of deviation Δv on different coordinate axes x , Δv y , Δv z The difference vector is transformed using the coordinate transformation matrix z, and the degree of spirality of the difference vector in space is determined by calculating the rate of change of the vector's modulus and the change of the vector's direction after the transformation.

[0086] In the specific calculation, suppose that after transformation The vector is The degree of helicity S can be calculated by the following formula (where β is and Angle of

[0087] In one embodiment, assuming the standard activity posture vector Then the difference vector

[0088] calculate for:

[0089]

[0090] calculate Calculated by vector dot product and The included angle cosβ≈0.214, therefore, β=arccos(0.214)≈77.6°≈1.35 radians.

[0091] Therefore, the degree of helicity S=0.033 is calculated.

[0092] Step 303 : determining the spatial curvature based on the first-order derivative and the second-order derivative of the final posture information, and determining the posture abnormality feature value based on the spiral degree and the spatial curvature.

[0093] Furthermore, the final posture information vector Find the first-order derivatives respectively and the second-order derivative According to the curve curvature formula in differential geometry, calculate the curvature of the final posture information in space in, is the cross product of the first and second order derivatives, The modulus of the vector is represented by the helicity S and spatial curvature K. The posture anomaly characteristic value E is determined by the formula: E = S + λK, where λ is a weight coefficient that is adjusted according to the actual application scenario to balance the influence of the helicity and spatial curvature on posture anomaly judgment.

[0094] In one embodiment, the final posture vector is calculated by numerical differentiation method. The first derivative of Second-order derivative therefore, Calculate the space curvature K = 0.00024 / 0.05 3 ≈0.196, assuming the weight coefficient λ = 0.5, the calculated posture abnormality feature value E = 0.131.

[0095] In step 304, if the posture abnormality feature value is greater than or equal to the preset abnormal threshold, it is determined that the posture of the wearable device is in an abnormal activity posture. If the posture abnormality feature value is less than the preset abnormal threshold, it is determined that the posture of the wearable device is in a normal activity posture.

[0096] Further, a preset abnormal threshold τ is set in advance, and the calculated posture abnormality feature value E is compared with the preset abnormal threshold τ. If E≥τ, it is determined that the posture of the wearable device is in an abnormal activity posture. If E<τ, it is determined that the posture of the wearable device is in a normal activity posture.

[0097] In the embodiment of the application, the dynamic change of the device posture and the difference between the standard posture are converted into a quantifiable posture abnormality feature value from the perspective of multi-dimensional space transformation and geometric features, so that the abnormal posture in a complex scene can be effectively recognized, and the accuracy and robustness of posture abnormality detection are improved.

[0098] In an embodiment, the process of steps 401 to 403 includes:

[0099] In step 401, a three-dimensional space coordinate system with the imaging device as the origin is established based on the current orientation of the imaging device, and the position of the human feature in the front image is converted into the three-dimensional space coordinate system based on the imaging principle and device parameters of the imaging device, to obtain the space coordinates of the human feature.

[0100] Optionally, a three-dimensional space coordinate system O-XYZ with the imaging device as the origin is established according to the current orientation (information such as the angle between the direction thereof and the north direction) of the imaging device, wherein the X axis, the Y axis and the Z axis comply with the right-hand rule. The imaging principle of the imaging device is based on the pinhole imaging model, and the pixel coordinates (p, q) of the human feature in the front image are converted into the three-dimensional space coordinate system by using the perspective projection transformation formula in combination with device parameters (such as the focal length f, the image sensor size, the principal point coordinates (p0, q0), etc.).

[0101] Suppose that the space point coordinates corresponding to the pixel coordinates (p, v) are (X, Y, Z), and the perspective projection transformation formula is:

[0102] X=[(p-p0)(Z)] / f.

[0103] Y=[(q-q0)(Z)] / f.

[0104] Wherein Z is the distance from the space point to the optical center of the camera (i.e. the origin of the coordinate system), which can be obtained by depth information or estimated based on scene prior knowledge, to obtain the space coordinates of the human feature.

[0105] In one embodiment, the camera on the smart bracelet has a focal length of f = 5 mm, principal point coordinates (p0, q0) = (320, 240) (image resolution is 640×480), the pixel coordinates of the human head detected in the image are (p = 400, q = 300), and the estimated distance Z from the head to the optical center of the camera is 2000 mm. Therefore, according to the perspective projection transformation formula, the coordinates of the human head in the three-dimensional space coordinate system are calculated as: X = 32000 mm, Y = 24000 mm.

[0106] Step 402: Taking the ground as a plane, derive the ground plane equation in the three-dimensional space coordinate system based on the position information and orientation information of the camera device.

[0107] Furthermore, the position information of the camera device (coordinates (x0, y0, z0) in the three-dimensional space coordinate system) and orientation information (such as pitch angle, roll angle, etc., which can be used to determine the camera posture) are known. Taking the ground as a plane, assuming that the ground is a horizontal plane, its normal vector (vertically upwards).

[0108] According to the plane's point normal equation Ax+By+Cz+D=0 (where (A, B, C) is the plane normal vector and (x, y, z) is the coordinate of any point on the plane), substituting the camera's point (x0, y0, z0) into the equation yields: 0*x+0*y+1*z+D=0. Substituting (x0, y0, z0) into the equation to solve for D: D=-z0, so the ground plane equation is z-z0=0. Continuing with the above example, if the camera's coordinates in the three-dimensional coordinate system are (1000, 1000, 1500), then the ground plane equation is z-1500=0.

[0109] Step 403: Determine the relative position relationship between the human body and the ground based on the ground plane equation and the spatial coordinates of the feature points in the human body features.

[0110] Furthermore, the relative position relationship between the human body and the ground is determined based on the ground plane equation and the spatial coordinates of the feature points in the human body features, as specifically described in the process from step 4031 to step 4034 .

[0111] The embodiment of the present invention is based on the camera imaging principle and spatial geometry theory, and associates the position of human body features in the image with the ground plane, which can accurately determine the relative position of various parts of the human body and the ground. Therefore, it fully considers the position and orientation information of the camera equipment, eliminates the errors caused by image distortion and perspective differences, and provides reliable data support for subsequent fall judgment based on positional relationships, thereby improving the accuracy and reliability of human posture and behavior analysis.

[0112] In one embodiment, the process from step 4031 to step 4034 includes:

[0113] Step 4031 , based on the ground plane equation and the spatial coordinates of the human feature points in the human body features, calculate the vertical distance between the vertical projection of the human feature points and the ground plane.

[0114] Optionally, the ground plane equation is known to be Ax+By+Cz+D=0, and the spatial coordinates of the human feature points are (x i ,y i ,z i ). According to the distance formula from point to plane, calculate the vertical distance d from the vertical projection of the human feature point to the ground plane i , calculated using the distance formula from a point to a plane. In the common case where the ground is a horizontal plane, the plane equation is z-z0=0 (in this case A=0, B=0, C=1, D=-z0), and the distance formula can be simplified to d i =|z i -z0|, this distance intuitively reflects the height relationship between the human body feature points and the ground.

[0115] Continuing with the above embodiment, the ground plane equation is z-1500=0, and the spatial coordinates of the human head feature point are (32000, 24000, 2000). According to the simplified distance formula: d head =500mm, that is, the vertical distance from the vertical projection of the human head to the ground plane is 500mm.

[0116] Step 4032: Calculate the projection coordinates of the human body feature points on the ground plane based on the ground plane equation and the spatial coordinates of the human body feature points in the human body features.

[0117] Furthermore, let the ground plane equation be Ax+By+Cz+D=0, and the coordinates of the human body feature points be (x i ,y i ,z i ). Draw a straight line perpendicular to the ground plane through this feature point. The equation of the line can be expressed as:

[0118] (t is a parameter).

[0119] Therefore, solve the parameter t=-(Ax i +By i +Cz i +D) / (A 2 +B 2 +C 2 ), substitute the t value back into the linear equation to obtain the projection coordinates of the human feature points on the ground plane (x proj ,y proj ,z proj), x proj =x i +At,y proj =y i +Bt,z proj =z i +Ct.

[0120] When the ground is a horizontal plane z-z0=0, the z component of the projection coordinate z proj =z0,x proj =x i ,y proj =y i .

[0121] In one embodiment, for the human head coordinates (32000, 24000, 2000) and the ground plane equation z-1500=0 (where A=0, B=0, C=1, D=-1500), the calculation parameter t=-500, the projection coordinate is x proj =32000,y proj =24000, z proj =1500, that is, the projection coordinates of the head on the ground plane are (32000, 24000, 1500).

[0122] Step 4033 constructs a feature reference vector based on the coordinate vectors of any three non-collinear points on the ground plane, and calculates a first angle between the human feature point and the ground plane based on the feature reference vector and the projected coordinates of the human feature point. The first angle represents the inclination of the human posture relative to the ground.

[0123] Furthermore, we select any three non-collinear points P1 = (x1, y1, z1), P2 = (x2, y2, z2), and P3 = (x3, y3, z3) on the ground plane and construct two coordinate vectors and Get the feature reference vector

[0124] Assume that the projection coordinates of the human feature points on the ground plane are (x proj ,y proj ,z proj ), construct a vector from the projection point to a point on the ground (such as P1) According to the vector dot product formula The first angle θ1 between the human body feature point and the ground plane is calculated, and the angle value is obtained by the arc cosine function. The angle represents the inclination of the human body posture relative to the ground.

[0125] In one embodiment, three points P1 (30000, 20000, 1500), P2 (31000, 20000, 1500), and P3 (30000, 21000, 1500) are selected on the ground plane. Construct the vector: Feature reference vector The head projection coordinates are (32000, 24000, 1500), then Calculating the dot product Calculate the module length Therefore, the first angle is 90°.

[0126] Step 4034: Determine the relative position relationship between the human body and the ground based on the vertical distance, the projection coordinates, and the first angle.

[0127] Furthermore, the integrated vertical distance d i 、Projection coordinates (x proj ,y proj ,z proj ) and the first angle θ1 comprehensively describe the relative position of the human body and the ground. The vertical distance reflects the height difference between the human feature point and the ground, the projected coordinates determine the feature point's horizontal position on the ground, and the first angle θ1 reflects the degree of inclination of the human posture relative to the ground. Using this information from these three dimensions, we can determine the specific relationship between the human body and the ground in different postures, such as standing, bending, and falling.

[0128] Continuing with the above embodiment, for the human head, the vertical distance d head =500mm, the projection coordinates are (32000, 24000, 1500), and the first angle θ1 = 90°. Combining this information, we can see that the head is 500mm above the ground, projected onto the ground at position (32000, 24000), and perpendicular to the ground. If combined with the corresponding information of other parts of the body, the overall posture of the human body can be further determined.

[0129] The embodiment of the present invention achieves a refined description of the relative position relationship between the human body and the ground through vertical distance calculation, projection coordinate derivation, angle solution and multi-dimensional information fusion. Starting from the perspective of spatial geometry, this solution associates the human body feature points with the ground plane in multiple dimensions, breaking through the limitations of single distance or angle judgment. Through rigorous mathematical calculations and vector operations, it can accurately capture the relative relationship between the posture changes of the human body in space and the ground, providing rich and accurate data support for applications such as fall detection and motion posture analysis, effectively improving the accuracy and reliability of human behavior judgment, and having stronger adaptability and robustness in complex scenarios than traditional methods.

[0130] In one embodiment, the process from step 501 to step 504 includes:

[0131] Step 501: Calculate the vertical distance change rate per unit time based on the continuous vertical distances. The distance change rate represents the movement trend of the human body in the vertical direction.

[0132] Optionally, in a continuous time series t1,t2,...,t n In the figure, the vertical distances corresponding to the human feature points are d1, d2, ..., d n The vertical distance change rate per unit time r d It is used to reflect the movement trend of the human body in the vertical direction. It can be calculated by the ratio of the difference in vertical distance between adjacent moments to the time interval. If the time interval is Δy, the vertical distance change rate calculation formula is: d =(d i+1 -d i ) / Δt.

[0133] A positive value indicates that the human feature point is vertically away from the ground, while a negative value indicates that it is close to the ground. The larger the absolute value of the rate of change, the faster the movement speed in the vertical direction.

[0134] Continuing with the example of data collected by the camera on a smart bracelet, assume that the collection interval Δt = 0.1s. At time t1, the vertical distance between the head and the ground is d1 = 1500mm; at time t2 = t1 + 0.1s, the vertical distance between the head and the ground is d2 = 1200mm. Calculate the vertical distance change rate r according to the formula d =-3000mm / s.

[0135] The results show that the head approaches the ground at a speed of 3000 mm / s in the vertical direction.

[0136] Step 502 determines a projection coordinate offset vector based on the projection coordinates of the human feature point on the ground plane at adjacent moments, and determines an angle change based on the difference between the first angles between the human feature point and the ground plane at adjacent moments. The projection coordinate offset vector represents the movement of the human body on the ground plane, and the angle change represents the change in the tilt of the human body posture.

[0137] Furthermore, for adjacent time t i and t i+1 , the projection coordinates of the human body feature points on the ground plane are (x i ,y i ,z0) and (x i+1 ,y i+1 ,z0), projection coordinate offset vector It can be obtained by subtracting the coordinates:

[0138] This vector represents the movement of the human body in the ground plane.

[0139] At the same time, the first angles between the human body feature points and the ground plane at adjacent moments are θ i and θ i+1 , the angle change Δθ is: Δθ=θ i+1 -θ i , the angle change reflects the change in the degree of inclination of the human body posture.

[0140] In one embodiment, at time t1, the projection coordinates of the human head on the ground plane are (3000, 4000, 0), with a first angle θ1 of 85°. At time t2, the projection coordinates become (3050, 4020, 0), with a first angle θ2 of 70°. Calculate the projection coordinate offset vector: The calculated angle change is: Δθ = 70° - 85° = -15°, indicating that the head has a certain horizontal movement on the ground plane and the inclination relative to the ground has decreased by 15°.

[0141] Step 503: Using the vertical distance change rate, the projection coordinate offset vector, and the angle change as the fall trend vector, and combining the vertical distance and the first angle, determine a fall possibility index.

[0142] Furthermore, the vertical distance change rate r d , projection coordinate offset vector and the angle change Δθ to form the falling trend vector In order to quantify the possibility of falling, the fall probability index I is introduced f By building a complex mathematical model, the fall tendency vector is combined with the vertical distance d and the first angle θ. Assuming that a calculation method based on vector norm and geometric relationship is adopted, the fall probability index calculation formula is: Among them, max r 、max p 、max θ are the maximum values ​​of vertical distance change rate, projection coordinate offset vector modulus, and angle change within the normal range of activity; d fall ,θ fall is the preset critical vertical distance and critical angle for falling; d max ,θ max is the maximum possible value of the vertical distance and angle; α and β are weight coefficients, which are adjusted according to the actual situation to balance the influence of various factors on fall judgment.

[0143] Step 504: If the fall probability index is greater than or equal to the preset fall threshold, it is determined that a fall has occurred. If the fall probability index is less than the preset fall threshold, it is determined that no fall has occurred.

[0144] Furthermore, a preset fall threshold T is set in advance. f , the calculated fall probability index I f and the preset fall threshold T f Compare. If I f ≥T f , it is determined that a fall has occurred; if I f <T f , it is determined that no fall has occurred. In one embodiment, a preset fall threshold T is set. f =1.5, due to the calculated fall probability index I f ≈1.9465>5, so it is determined that the person has fallen.

[0145] This embodiment of the present invention constructs a fall trend vector from the vertical distance change rate, the projected coordinate offset vector, and the angle change, and then generates a fall probability index by combining the vertical distance and the first angle. This solution overcomes the limitations of single-metric judgment. Through rigorous mathematical calculations and parameter settings, it fully considers the human body's characteristics during a fall, such as vertical fall, horizontal movement, and sudden changes in posture. It can effectively distinguish between normal activities and falls, significantly improving the accuracy and reliability of fall detection compared to traditional methods, and providing powerful technical support for scenarios such as elderly care and sports safety protection.

[0146] Furthermore, the wearable device provided by the present invention is described below. The wearable device described below and the fall judgment method described above can be referred to in correspondence with each other.

[0147] Optional, see Figure 2 , Figure 2 It is a structural diagram of the wearable device provided by the present invention, in which a three-axis accelerometer, an inclination sensor and a camera device are integrated, including.

[0148] The data acquisition module 210 is configured to acquire acceleration data based on a three-axis accelerometer and inclination data based on an inclination sensor; the acceleration data includes acceleration data of the wearable device in the X-axis, Y-axis, and Z-axis directions; and the inclination data includes the pitch angle and roll angle of the wearable device relative to the horizontal plane.

[0149] The posture determination module 220 is used to determine the speed change and displacement change of the wearable device per unit time based on the acceleration data, and to determine the final posture information of the wearable device in space in combination with the inclination data;

[0150] The orientation determination module 230 is configured to activate a camera to capture a front image of the wearable device and determine a current orientation of the camera if it is determined based on the final attitude information that the wearable device is in an abnormal activity attitude.

[0151] a positional relationship determination module 240 for performing image analysis based on the front image to obtain human features, and determining the relative positional relationship between the human body and the ground based on the current orientation and the position of the human features in the front image;

[0152] The fall determination module 250 is used to determine a fall based on the relative position relationship.

[0153] The embodiment of the present invention combines a three-axis accelerometer and an inclinometer to perform preliminary activity status judgment. It not only takes acceleration into consideration, but also judges whether the device is in a normal activity state based on its posture, thereby reducing misjudgments caused by strenuous exercise. For example, when running, although the acceleration may be large, the device posture is in a normal movement posture and will not be misjudged as a fall. Secondly, the front image of the wearable device is obtained through a camera device and the relative position relationship between the human body and the ground is analyzed. For slow falls or falls on soft ground, even if the acceleration is not obvious, the camera device can capture the situation of the human body approaching or contacting the ground, thereby avoiding missed judgments. Therefore, the fall situation is comprehensively judged from multiple dimensions, which improves the accuracy and reliability of fall judgment.

[0154] See also Figure 3 , Figure 3 This is a diagram of an embodiment of an electronic device provided by an embodiment of the present invention. Figure 3 As shown, an embodiment of the present invention provides an electronic device 300, including a memory 310, a processor 320, and a computer program 311 stored in the memory 310 and executable on the processor 320. When the processor 320 executes the computer program 311, the following steps are implemented:

[0155] Acceleration data is collected based on a three-axis accelerometer, and inclination data is collected based on an inclination sensor. The acceleration data includes acceleration data of the wearable device in the X-axis, Y-axis, and Z-axis directions. The inclination data includes the pitch angle and roll angle of the wearable device relative to the horizontal plane.

[0156] Determine the speed change and displacement change of the wearable device per unit time based on the acceleration data, and determine the final posture information of the wearable device in space in combination with the inclination data;

[0157] If it is determined based on the final posture information that the posture of the wearable device is in an abnormal activity posture, starting the camera device to capture a front image of the wearable device and determining the current orientation of the camera device;

[0158] Perform image analysis based on the front image to obtain human body features, and determine the relative position relationship between the human body and the ground based on the current orientation and the position of the human body features in the front image;

[0159] Fall judgment is performed based on relative position relationship.

[0160] See also Figure 4 , Figure 4 Detailed description of an embodiment of a computer-readable storage medium provided by an embodiment of the present invention. Figure 4 As shown, this embodiment provides a computer-readable storage medium 400 on which a computer program 311 is stored. When the computer program 311 is executed by a processor, the following steps are implemented:

[0161] Acceleration data is collected based on a three-axis accelerometer, and inclination data is collected based on an inclination sensor. The acceleration data includes acceleration data of the wearable device in the X-axis, Y-axis, and Z-axis directions. The inclination data includes the pitch angle and roll angle of the wearable device relative to the horizontal plane.

[0162] Determine the speed change and displacement change of the wearable device per unit time based on the acceleration data, and determine the final posture information of the wearable device in space in combination with the inclination data;

[0163] If it is determined based on the final posture information that the posture of the wearable device is in an abnormal activity posture, starting the camera device to capture a front image of the wearable device and determining the current orientation of the camera device;

[0164] Perform image analysis based on the front image to obtain human body features, and determine the relative position relationship between the human body and the ground based on the current orientation and the position of the human body features in the front image;

[0165] Fall judgment is performed based on relative position relationship.

[0166] On the other hand, the present invention further provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can perform the fall detection method provided by the above methods, which includes:

[0167] Acceleration data is collected based on a three-axis accelerometer, and inclination data is collected based on an inclination sensor. The acceleration data includes acceleration data of the wearable device in the X-axis, Y-axis, and Z-axis directions. The inclination data includes the pitch angle and roll angle of the wearable device relative to the horizontal plane.

[0168] Determine the speed change and displacement change of the wearable device per unit time based on the acceleration data, and determine the final posture information of the wearable device in space in combination with the inclination data;

[0169] If it is determined based on the final posture information that the posture of the wearable device is in an abnormal activity posture, starting the camera device to capture a front image of the wearable device and determining the current orientation of the camera device;

[0170] Perform image analysis based on the front image to obtain human body features, and determine the relative position relationship between the human body and the ground based on the current orientation and the position of the human body features in the front image;

[0171] Fall judgment is performed based on relative position relationship.

[0172] The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment. Persons of ordinary skill in the art will be able to understand and implement the present invention without inventive effort.

[0173] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0174] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A fall judgment method, characterized in that: Applied to a wearable device, the wearable device includes a three-axis accelerometer, an inclination sensor, and a camera device; the fall detection method includes: Acceleration data is collected based on the three-axis accelerometer, and inclination data is collected based on the inclination sensor; the acceleration data includes acceleration data of the wearable device in the X-axis, Y-axis, and Z-axis directions; the inclination data includes the pitch angle and roll angle of the wearable device relative to the horizontal plane; Determine the velocity change and displacement change of the wearable device per unit time based on the acceleration data, and determine the final posture information of the wearable device in space in combination with the inclination data; If it is determined based on the final posture information that the posture of the wearable device is in an abnormal activity posture, starting the camera device to capture a front image of the wearable device and determining the current position of the camera device; performing image analysis based on the front image to obtain human body features, and determining a relative positional relationship between the human body and the ground based on the current orientation and the positions of the human body features in the front image; A fall is determined based on the relative position relationship.

2. The fall judgment method according to claim 1, characterized in that: The determining the relative position relationship between the human body and the ground based on the current orientation and the position of the human body feature in the front image includes: Establishing a three-dimensional spatial coordinate system with the camera device as the origin based on the current camera orientation, and converting the position of the human feature in the front image into the three-dimensional spatial coordinate system based on the imaging principle and device parameters of the camera device to obtain the spatial coordinates of the human feature; Taking the ground as a plane, deriving a ground plane equation of the ground in the three-dimensional space coordinate system according to the position information and orientation information of the camera device; Based on the ground plane equation and the spatial coordinates of the feature points in the human body features, the relative position relationship between the human body and the ground is determined.

3. The fall judgment method according to claim 2, characterized in that: The determining of the relative position relationship between the human body and the ground based on the ground plane equation and the spatial coordinates of the feature points in the human body features includes: Calculating a vertical distance between a vertical projection of the human body feature point and the ground plane based on the ground plane equation and the spatial coordinates of the human body feature point in the human body feature; Calculating the projection coordinates of the human body feature points on the ground plane based on the ground plane equation and the spatial coordinates of the human body feature points in the human body features; Constructing a feature reference vector based on the coordinate vectors of any three non-collinear points on the ground plane, and calculating a first angle between the human feature point and the ground plane based on the feature reference vector and the projected coordinates of the human feature point; the first angle represents the inclination of the human posture relative to the ground; The relative position relationship between the human body and the ground is determined based on the vertical distance, the projection coordinates and the first angle.

4. The fall judgment method according to claim 3, characterized in that: The performing fall judgment based on the relative position relationship includes: Calculating a vertical distance change rate per unit time based on continuous vertical distances; the distance change rate represents a movement trend of the human body in the vertical direction; Determine a projection coordinate offset vector based on the projection coordinates of the human body feature points on the ground plane at adjacent moments, and determine an angle change based on the difference between the first angles between the human body feature points and the ground plane at adjacent moments; the projection coordinate offset vector represents the movement of the human body on the ground plane, and the angle change represents the change in the tilt degree of the human body posture; Determine a fall probability index by taking the vertical distance change rate, the projection coordinate offset vector, and the angle change as a fall tendency vector, and combining the vertical distance and the first angle; If the fall possibility index is greater than or equal to the preset fall threshold, it is determined that a fall has occurred; if the fall possibility index is less than the preset fall threshold, it is determined that no fall has occurred.

5. The fall judgment method according to claim 1, characterized in that: The determining of the velocity change and displacement change of the wearable device per unit time based on the acceleration data, and determining the final posture information of the wearable device in space in combination with the inclination data, includes: Determine a velocity vector change and a displacement vector change based on velocity changes and displacement changes at adjacent moments in unit time, and construct a velocity-displacement plane based on the velocity vector change and the displacement vector change; a plane normal vector of the velocity-displacement plane is obtained based on a cross product of the velocity vector change and the displacement vector change; Converting the pitch angle and roll angle into an initial posture vector of the wearable device in space; the initial posture vector represents an initial tilt state of the wearable device; Based on the initial posture vector and the velocity displacement plane, final posture information of the wearable device in space is determined.

6. The fall judgment method according to claim 5, characterized in that: The determining, based on the initial posture vector and the velocity displacement plane, final posture information of the wearable device in space includes: Projecting the initial posture vector onto the velocity displacement plane to obtain a projection vector of the initial posture vector on the velocity displacement plane; Calculating a second angle between the projection vector and the velocity vector change; wherein the second angle represents a relationship between the device posture and the movement direction; Based on the angle combined with the displacement vector change, the posture adjustment amount of the device posture relative to the initial tilt state is calculated, and the posture adjustment amount is superimposed on the initial posture vector to obtain the final posture information of the wearable device in space.

7. The fall judgment method according to any one of claims 1 to 6, characterized in that: The specific steps of determining whether the posture of the wearable device is in an abnormal activity posture based on the final posture information include: Constructing a coordinate transformation matrix based on the components of the final posture information on the horizontal axis, the vertical axis, and the vertical axis; the diagonal elements of the coordinate transformation matrix are the components on each coordinate axis; Projecting the difference vector between the final posture information and the standard active posture vector onto each coordinate axis, obtaining the degree of deviation on each coordinate axis, and determining the degree of spirality of the difference vector in space based on the degree of deviation on each coordinate axis in combination with the coordinate transformation matrix; determining a spatial curvature based on a first-order derivative and a second-order derivative of the final posture information, and determining a posture abnormality feature value based on the spiral degree and the spatial curvature; If the abnormal posture characteristic value is greater than or equal to the preset abnormal threshold, it is determined that the posture of the wearable device is in an abnormal activity posture; if the abnormal posture characteristic value is less than the preset abnormal threshold, it is determined that the posture of the wearable device is in a normal activity posture.

8. A wearable device, characterized in that: Applicable to the fall detection method according to any one of claims 1 to 7; the wearable device integrates a three-axis accelerometer, an inclination sensor, and a camera device; the wearable device includes: A data acquisition module, configured to acquire acceleration data based on the three-axis accelerometer and inclination data based on the inclination sensor; the acceleration data includes acceleration data of the wearable device in the X-axis, Y-axis, and Z-axis directions; the inclination data includes the pitch angle and roll angle of the wearable device relative to the horizontal plane; A posture determination module, configured to determine a velocity change and a displacement change of the wearable device per unit time based on the acceleration data, and determine final posture information of the wearable device in space in combination with the inclination data; an orientation determination module, configured to activate the camera device to capture a front image of the wearable device and determine a current orientation of the camera device if it is determined based on the final attitude information that the wearable device is in an abnormal activity attitude; a positional relationship determination module, configured to perform image analysis based on the front image to obtain human body features, and determine a relative positional relationship between the human body and the ground based on the current orientation and the positions of the human body features in the front image; A fall determination module is used to determine a fall based on the relative position relationship.

9. An electronic device comprising: Memory for storing computer software programs; A processor, configured to read and execute the computer software program, wherein when the processor executes the computer software program, it implements the fall determination method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing a computer software program, wherein: When the computer software program is executed by a processor, the fall determination method according to any one of claims 1 to 7 is implemented.

Images