Human balance assessment system and method

Abstract

This invention discloses a system and method for analyzing human balance based on digital video. The system includes: a human keypoint acquisition module for detecting multiple human keypoints in a single frame of the digital video; a human balance point calculation module for calculating a human balance point based on multiple human keypoints in at least two consecutive frames of the digital video; a human support zone calculation module for calculating a human support zone based on multiple human keypoints in a single frame of the digital video; and a human balance assessment module for assessing whether the human body is in a balanced state based on the human balance point and the human support zone.

Description

Technical Field

[0001] This invention relates to a human balance assessment system, and more particularly to a human balance assessment system and method based on digital images. Background Technology

[0002] Balance is a crucial factor in maintaining daily activities, and accurate balance assessment is essential for fall prevention and rehabilitation planning. Traditional balance assessment methods typically rely on specialized equipment and manual observation, which are costly, subjective, and lack real-time feedback. Some methods utilize pressure sensors and 3D motion sensors to analyze balance; however, these devices are usually bulky or expensive, limiting their use to specific medical institutions or laboratories and making them difficult to apply in daily life. Therefore, a more convenient, objective balance assessment system and method that provides real-time feedback is needed. Summary of the Invention

[0003] This invention provides a human balance assessment system that assesses whether a human body in a digital video is in a balanced state. The system includes: a human key point acquisition module for detecting multiple human key points in a frame of the digital video; a human balance point calculation module for calculating a human balance point based on multiple human key points in at least two consecutive frames of the digital video; a human support area calculation module for calculating a human support area based on multiple human key points in a frame of the digital video; and a human balance assessment module for assessing whether the human body is in a balanced state based on the human balance point and the human support area.

[0004] The present invention provides a method for assessing human balance, which evaluates whether a human body in the digital video is in a balanced state based on a digital video. The method includes the following steps: (a) calculating a human balance point based on multiple human key points in at least two consecutive frames of the digital video; (b) calculating a human support area based on multiple human key points in one frame of the digital video; and (c) evaluating whether the human body is in a balanced state based on the human balance point and the human support area. Attached Figure Description

[0005] Figure 1 A block diagram of a human stability point estimation system according to one embodiment is shown; Figure 2 It shows 23 key points of the human body; Figure 3 An embodiment of the present invention is shown, in which four new endpoints Q19, Q20, Q22, and Q21 are calculated based on the four projection points S19, S20, S22, and S21 to form a human body support area; Figure 4AThis shows an example where the human body's balance point falls within the body's support area; Figure 4B Examples showing that the human body's balance point does not fall within the body's support area; and Figure 5 A flowchart is shown for a method for estimating the human body's stability point according to one embodiment.

[0006] Explanation of reference numerals in the attached figures 100…Human stability point estimation system 101…Human Key Point Acquisition Module 102…Human Balance Point Calculation Module 103…Human Support Area Calculation Module 104…Human Balance Assessment Module …No. Personal body key point coordinates, …for the first The first frame of the image Coordinates of key points on an individual's body …No. The first frame of the image Speed ​​of key points of the individual body …No. Acceleration of key points in the human body …No. The first frame of the image Acceleration of key points in the human body …gravitational acceleration, …number of key points in the human body, …No. The proportion of an individual's body parts relative to their total body weight ...number of body parts, …related to the first The weight ratio of key points of an individual's body P0~P23…Key points of the human body Q19, Q20, Q21, Q22… new endpoints S19, S20, S21, S22… Projection points of key human body points P19, P20, P21, P22, Steps S101~S104… …No. The coordinates of the center of gravity of an individual body part …No. Acceleration of individual body parts ...coordinates of the zero torque point, Z...human balance point …the time difference between each frame of the image. Detailed Implementation

[0007] Exemplary embodiments of the invention will now be described more fully with reference to the accompanying drawings. These exemplary embodiments can be implemented in various forms, and are provided so that the invention will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art.

[0008] Please refer to Figure 1 The diagram illustrates a block diagram of a human stability point estimation system according to an embodiment. The human stability point estimation system 100 includes: a human key point acquisition module 101, a human balance point calculation module 102, a human support zone calculation module 103, and a human balance assessment module 104.

[0009] The human body keypoint acquisition module 101, based on a digital video, uses a human skeleton joint model to calculate and acquire human body keypoint parameters in each frame of the digital video, including the coordinate values ​​of each human body keypoint in the human skeleton joint model and the coordinate positions of each human body keypoint on the digital video. For example, the BlazePose model can be used to estimate the positions of 33 human body keypoints in each frame of the video through a neural network. According to one embodiment of the present invention, only 23 of these human body keypoints need to be acquired, such as... Figure 2 As shown, P0 represents the nose, P1 / P2 represents the left / right eye, P3 / P4 represents the left / right ear, P5 / P6 represents the left / right corner of the mouth, P7 / P8 represents the left / right shoulder, P9 / P10 represents the left / right elbow, P11 / P12 represents the left / right hand, P13 / P14 represents the left / right hip, P15 / P16 represents the left / right knee, P17 / P18 represents the left / right ankle, P19 / P20 represents the left / right heel, and P21 / P22 represents the left / right toe.

[0010] The human balance point calculation module 102 is used to calculate a human balance point based on multiple human key point parameters in each frame of the digital video. During human limb movement, the position of the human balance point changes. In the most stable state of the human body, its center of pressure equals its center of gravity, which is also equal to the point where the center of mass is projected onto the ground. Physically, this is the point where the torque is zero, called the zero-moment point (ZMP). According to one embodiment of the present invention, this zero-moment point is taken as the human balance point Z. Zero-moment point coordinates. It can be calculated using the following known formula:

[0011] in, The number of body parts. For the first The proportion of an individual's body part relative to the total body weight. For the first The coordinates of the center of gravity of a part of an individual's body. For the first Acceleration of individual body parts, acceleration due to gravity .

[0012] In order to apply the human body key point parameters to the zero torque point formula, this invention transforms the above formula as follows to facilitate calculation:

[0013] in, The number of key points in the human body. For the connection to the first The weight proportions of key points on an individual's body For the first Personal body key point coordinates, For the first Acceleration of key human body points. Coordinates of key human body points in each frame of the image. The parameters are calculated and generated by the human body key point acquisition module 101. and The calculation method is illustrated below with an example.

[0014] Table 1 shows the percentage of body weight by different body parts. The statistical sample consisted of 35 males and 100 females. In Table 1, the percentage of body weight by different body parts is as follows: head: 8.21%, chest: 18.56%, abdomen: 12.65%, pelvis: 14.81%, arms: 10.67%, and legs: 17.55%. Table 1 is an example of statistical figures; it is understood that other versions of statistical figures are also applicable to this invention.

[0015] Table 1

[0016] According to one embodiment of the present invention, the weight percentage associated with each key point of the human body can be calculated by using the key points of the human body contained in each part of the human body. A key point of the human body is a single point on the human body. Although a single point itself has no weight, the present invention associates each key point of the human body with a weight percentage, so the calculation formula (2) can be used for calculation. The weight percentage of the head is 8.21%, including key points P0 to P6. For ease of estimation, this weight percentage is evenly distributed to the key points of the human body in this part, so key points P0 to P6 are each assigned a weight percentage of 8.21 / 7 = 1.173. The weight percentage of the chest is 18.56%, which is evenly distributed to the key points P7, P8, P13, and P14 of the chest, each assigned a weight percentage of 18.56 / 4 = 4.64%. The weight percentage of the abdomen is 12.65%, which is evenly distributed to the key points P7, P8, P13, and P14 of the abdomen, each assigned a weight percentage of 12.65 / 4 = 3.163%. The upper arm accounts for 3.075% of the body weight. This weight is evenly distributed among the key points P7 and P9 on the upper arm, resulting in each receiving 3.075 / 2 = 1.5375%. Similarly, by evenly distributing the weight percentage of each body part to its corresponding key points, the weight percentage associated with each key point can be calculated cumulatively. For example, the left shoulder P7 belongs to the chest, abdomen, and upper arm; therefore, the weight percentage associated with the left shoulder P7 is 4.64 + 3.163 + 1.5375 = 9.3405. This process can be repeated to calculate the weight percentages associated with the remaining key points.

[0017] A digital video is composed of consecutive frames, with a generally fixed time difference between each frame. Because of the numerator and denominator of formula (2) They will eventually cancel each other out, so for ease of calculation, we can set... This does not affect the final calculation result. According to one embodiment of the present invention, utilizing this characteristic of digital video, the first... The first frame of the image The acceleration of a key point on an individual's body is denoted as . , can be from the first The first frame of the image Speed ​​of key points in an individual's body , with the previous frame, i.e., the first The first frame of the image Speed ​​of key points in an individual's body The difference can be calculated based on the parameters of these two frames, as shown in the following formula:

[0018] In one of the frames of the image, the first The velocity values ​​of key points on an individual's body can be obtained from the first frame of the image. The coordinates of individual body key points and the first frame of the previous image The difference in coordinates of key points on an individual's body is calculated using the following formula:

[0019] in For the first The first frame of the image Coordinates of key points on an individual's body The previous frame, i.e., the first frame The first frame of the image Coordinates of key points on an individual's body For the first The first frame of the image The coordinates of key points on an individual's body. That is, through the first... Frame, First Frame, First The coordinates of key points on the human body in three consecutive frames of imagery can be used to calculate the acceleration of those key points. Understandably, more frames could also be used for calculation, which would allow for more accurate calculations based on the larger amount of data, but would inevitably require more computation. Using three consecutive frames of imagery minimizes the computational burden.

[0020] Since the human body key point acquisition module 101 estimates the coordinates of human body key points based on each frame of the digital video in pixels, the gravitational acceleration in formula (2) is calculated accordingly. Units need to be converted According to one embodiment of the present invention, taking into account that the distance between the left and right ears of the human body key points P3 / P4 is not significantly different for most people (for example, it can be reasonably assumed to be 10cm), when the human body key point acquisition module 101 calculates the coordinates of P3 and P4, and then calculates their distance as D pixels, the correspondence between 10cm and D pixels can be obtained. Therefore, the gravitational acceleration... The following formula can be used for conversion:

[0021] Understandably, the above-described unit conversion for gravitational acceleration is one of the feasible methods, and other methods are also applicable to this invention.

[0022] The human body support area calculation module 103 is used to calculate a human body support area based on multiple human body key points in a frame of image in the digital video. According to one embodiment of the present invention, four key points, namely the left heel P19, the right heel P20, the right toe P22, and the left toe P21, are projected onto four projection points on the ground where the human body is located, and are respectively denoted as S19, S20, S22, and S21. The quadrilateral enclosed by these four points is the human body support area of ​​the frame of image.

[0023] Figure 3 This illustration shows another embodiment of the invention, in which four new endpoints Q19, Q20, Q22, and Q21 are calculated based on the four projection points S19, S20, S22, and S21, such that the area formed by these four new endpoints is an enlarged version of the area formed by the four projection points, and this area is used as the human body support area. The figure shows an example where the four projection points S19, S20, S22, and S21 are enlarged outwards from the center point to obtain four new endpoints S19, S20, S22, and S21. The enlarged human body support area can reduce the misjudgment rate of whether it is in a balanced state, but it should not be enlarged excessively, which would reduce the accuracy of determining whether it is in a balanced state. Understandably, other human body support areas formed by expanding the four projection points S19, S20, S22, and S21 are still polygons and are therefore applicable to the present invention.

[0024] The human balance assessment module 104 assesses whether the human body is in a balanced state based on the human balance point calculated by the human balance point calculation module 102 and the human support area calculated by the human support area calculation module 103. If the human balance point Z falls within the human support area, the human body is considered to be in a balanced state. Figure 4A Example. Conversely, if the body's balance point Z falls outside the body's support area, it is considered an unbalanced state, such as... Figure 4B Example. Whether the human body's balance point lies within the body's support area is a mathematical problem of whether a point on a two-dimensional plane lies within a certain polygon. Various algorithms are available, such as the existing ray casting method, and understandably, these algorithms are also applicable to this invention.

[0025] Please refer to Figure 5 The diagram illustrates a flowchart of a human stability point estimation method according to an embodiment, which evaluates whether a human body in the digital video is in a balanced state based on a digital video, and the method includes the following steps.

[0026] Step S101: Based on the digital video, detect multiple human key points in each frame of the digital video.

[0027] Step S102: Calculate a human balance point based on multiple human key points from at least two consecutive frames of the digital video. The calculation method for the human balance point is the same as the function of the aforementioned human balance point calculation module 102, and will not be described again here. Step S103: Calculate a human support region based on multiple human body key points in a frame of the digital video. The calculation method for the human support region is the same as the function of the aforementioned human support region calculation module 103, and will not be described again here.

[0028] Step S104: Based on the human body balance point obtained in step S102 and the human body support area obtained in step S103, assess whether the human body is in a balanced state. The assessment method is the same as the function of the aforementioned human body balance assessment module 104, and will not be described again here.

[0029] Compared with existing technologies, the system and method provided by this invention do not require additional physical sensing devices, such as pressure-sensing insoles or force plates. They can be operated using devices with camera functions, such as mobile phones and tablets, which is very convenient and provides real-time identification of stability and immediate feedback on the human body's balance status.

[0030] The above description illustrates various embodiments of the present invention. However, these embodiments represent only a portion of the applicable examples of the present invention, and the technical scope of the present invention is not limited to the specific structures of the above embodiments. Those skilled in the art can make various modifications and alterations without departing from the spirit and scope of the present invention, such as changing the order of method steps to achieve the same or equivalent functions. Therefore, the scope of protection of the present invention should be determined by the appended claims.

Claims

1. A human balance assessment system, which assesses whether a human body in the digital video is in a balanced state based on a digital video, wherein, This human balance assessment system includes: A human body key point acquisition module is used to detect multiple human body key points in a frame of the digital video based on the digital video. A human balance point calculation module is used to calculate a human balance point based on multiple human key points in at least two consecutive frames of the digital video. A human body support area calculation module is used to calculate a human body support area based on multiple human body key points in a frame of the digital video; and A human balance assessment module assesses whether the human body is in a balanced state based on the body's balance point and support area.

2. The human balance assessment system according to claim 1, wherein, This human body balance point calculation module further includes the following functions: Based on the coordinates of multiple human body key points in the current frame and the previous frame of the digital video, the velocity of each human body key point is calculated. Based on the velocities of multiple human body key points in the current frame and the previous frame of the digital video, calculate the acceleration of each human body key point. as well as Calculate the balance point of the human body based on the acceleration and coordinates of multiple key points of the human body in the current frame.

3. The human balance assessment system according to claim 2, wherein, The calculation of the human body's balance point is based on the following formula: Among them, These are the coordinates of the human body's balance point on the ground where the human body is located. The number of key points in the human body. For the connection to the first The proportion of weight at key points of an individual's body For the first Coordinates of key points on an individual's body. For the first Acceleration at key points in an individual's body This is the acceleration due to gravity.

4. The human balance assessment system according to claim 1, wherein, The multiple human body key points include four key points: left toe, left heel, right toe, and right heel. The human body support area module calculates the human body support area based on the projection points of these four key points onto the ground where the human body is located.

5. The human balance assessment system according to claim 4, wherein, The four key points are projected onto the ground where the human body is located to form four projection points. The human body support area module calculates four new endpoints based on the four projection points, so that the area formed by the four new endpoints is the magnified result of the area formed by the four projection points, and the area formed by the four new endpoints is used as the human body support area.

6. A method for assessing human balance, which evaluates whether a human body in the digital video is in a balanced state based on a digital video, wherein, The method includes the following steps: (a) Based on the digital video, detect multiple human key points in a frame of the digital video; (b) Calculate a human balance point based on multiple human key points in at least two consecutive frames of the digital video. (c) Calculate a human support area based on multiple human key points in a frame of the digital video. as well as (d) Based on the human body's balance point and support area, assess whether the human body is in a balanced state.

7. The human balance assessment method according to claim 6, wherein, Step (b) further includes: (b1) Calculate the velocity of each human key point based on the coordinates of multiple human key points in the current frame and the previous frame of the digital video. (b2) Based on the velocities of multiple human body key points in the current frame and the previous frame of the digital video, calculate the acceleration of each human body key point; and (b3) Calculate the balance point of the human body based on the acceleration and coordinates of the multiple key points of the human body in the current frame.

8. The human balance assessment method according to claim 7, wherein, The calculation of the human body's balance point is based on the following formula: Among them, These are the coordinates of the human body's balance point on the ground where the human body is located. The number of key points in the human body. For the connection to the first The proportion of weight at key points of an individual's body For the first Coordinates of key points on an individual's body. For the first Acceleration at key points in an individual's body This is the acceleration due to gravity.

9. The human balance assessment method according to claim 6, wherein, In step (b), the multiple human body key points include four key points: left toe, left heel, right toe, and right heel. Based on the projection points of these four key points onto the ground where the human body is located, the human body support area is calculated.

10. The human balance assessment method according to claim 9, wherein, The four key points are projected onto the ground where the human body is located to form four projection points. Based on the four projection points, four new endpoints are calculated so that the area formed by the four new endpoints is an enlarged result of the area formed by the four projection points. The area formed by the four new endpoints is used as the support area of ​​the human body.

Images