Fall detection method and apparatus based on human pose estimation

Abstract

Embodiments of the present application provide a fall detection method and device based on human posture estimation. The method comprises: using a neural network to identify key points of a human in an image, to obtain key points and a bounding box of a skeleton of the human; estimating and classifying a posture of the human according to the obtained key points and bounding box, to obtain a classified posture; arranging the classified postures in time sequence to form a posture sequence, and judging whether the human is in a lying state according to the posture sequence; wherein when the human is in the lying state, judging a lying direction of the human in the posture sequence according to the image, and judging whether the human falls in the lying direction according to the lying direction of the human and the posture sequence. Thus, not only the complexity of fall detection can be reduced and the performance of fall detection can be improved, but also a better fall detection effect can be ensured.

Description

Technical Field

[0001] This application relates to the field of motion recognition technology. Background Technology

[0002] Fall detection is a technology that identifies and predicts the timing of falls by estimating and analyzing human posture and movement. Currently, mainstream fall detection methods can be categorized into wearable device-based, environmental sensor-based, and vision-based posture estimation methods. Vision-based fall detection methods have proven effective in handling falls, and some advanced vision-based methods utilizing neural networks have shown excellent detection results for falls in video clips.

[0003] It should be noted that the above introduction to the technical background is only for the purpose of providing a clear and complete explanation of the technical solutions of this application and for the convenience of those skilled in the art to understand them. It should not be assumed that the above technical solutions are known to those skilled in the art simply because these solutions have been described in the background section of this application. Summary of the Invention

[0004] However, the inventors discovered that current vision-based human pose estimation fall detection methods are difficult to apply widely due to the high complexity of algorithms and models or limitations of hardware devices.

[0005] To address at least one of the aforementioned technical problems, embodiments of this application provide a fall detection method and apparatus based on human posture estimation, which reduces the complexity of fall detection, improves the performance of fall detection, and ensures better fall detection results.

[0006] According to one aspect of the embodiments of this application, a fall detection device based on human posture estimation is provided, comprising:

[0007] The acquisition unit uses a neural network to identify key points of the human body in the image and acquires the key points and bounding boxes of the human body's skeleton.

[0008] The classification unit estimates and classifies the human body's pose based on the acquired key points and bounding boxes to obtain the classified pose.

[0009] The first judgment unit arranges the classified postures in chronological order to form a posture sequence, and determines whether the human body is in a lying position based on the posture sequence; and

[0010] The second judgment unit, when the human body is in a lying position, determines the direction in which the human body lies down in the posture sequence based on the image, and determines whether the human body has fallen in that direction based on the direction in which the human body lies down and the posture sequence.

[0011] According to another aspect of the embodiments of this application, a fall detection method based on human posture estimation is provided, comprising:

[0012] The neural network is used to identify key points of the human body in the image, and the key points and bounding boxes of the human body's skeleton are obtained.

[0013] The pose of the human body is estimated and classified based on the obtained key points and bounding boxes to obtain the classified pose;

[0014] The classified postures are arranged in chronological order to form a posture sequence, and the posture sequence is used to determine whether the human body is in a lying position; and

[0015] When the human body is in a lying position, the direction in which the human body lies down in the posture sequence is determined based on the image, and the human body is determined to have fallen in that direction based on the direction in which the human body lies down and the posture sequence.

[0016] According to another aspect of the embodiments of this application, an electronic device is provided, including a memory and a processor, the memory storing a computer program, and the processor being configured to execute the computer program to implement the fall detection method as described above.

[0017] One of the beneficial effects of this application's embodiments is that by arranging the classified human postures into a posture sequence according to time sequence, and judging the human fall and the direction of fall based on the posture sequence, not only can the complexity of fall detection be reduced and the performance of fall detection be improved, but also a better fall detection effect can be guaranteed.

[0018] Referring to the following description and accompanying drawings, specific implementation methods of the embodiments of this application are disclosed in detail, indicating how the principles of the embodiments of this application can be adopted. It should be understood that the implementation methods of this application are not limited in scope. Within the spirit and scope of the appended claims, the implementation methods of this application include many changes, modifications, and equivalents. Attached Figure Description

[0019] The accompanying drawings, which form part of the specification, are used to provide a further understanding of the embodiments of this application and illustrate the implementation methods of this application, together with the textual description, to explain the principles of this application. Obviously, the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other implementation methods based on these drawings without creative effort. In the drawings:

[0020] Figure 1 This is a schematic diagram of a fall detection method according to an embodiment of this application;

[0021] Figure 2 This is a schematic diagram of the key points and bounding box of the human skeleton in an embodiment of this application;

[0022] Figure 3 This is a schematic diagram of a posture sequence according to an embodiment of this application;

[0023] Figure 4 This is a schematic diagram of a human body lying down in different directions according to an embodiment of this application;

[0024] Figure 5 This is a schematic diagram illustrating the calculation of the upper and lower body ratio according to an embodiment of this application;

[0025] Figure 6 This is a schematic diagram of a human body falling forward according to an embodiment of this application;

[0026] Figure 7 This is a schematic diagram of a human body falling to the side or backward according to an embodiment of this application;

[0027] Figure 8 This is a schematic diagram of the fall detection process according to an embodiment of this application;

[0028] Figure 9 This is a schematic diagram of a fall detection device according to an embodiment of this application;

[0029] Figure 10 This is a schematic diagram of an electronic device according to an embodiment of this application. Detailed Implementation

[0030] Referring to the accompanying drawings, the foregoing and other features of the embodiments of this application will become apparent from the following description. Specific embodiments of this application are specifically disclosed in the description and drawings, illustrating partial implementations in which the principles of the embodiments of this application can be adopted. It should be understood that this application is not limited to the described embodiments; rather, the embodiments of this application include all modifications, variations, and equivalents falling within the scope of the appended claims.

[0031] In the embodiments of this application, the terms "first," "second," etc., are used to distinguish different elements by name, but do not indicate the spatial arrangement or chronological order of these elements, and these elements should not be limited by these terms. The term "and / or" includes any one or more of the terms listed in association and all combinations thereof. The terms "comprising," "including," "having," etc., refer to the presence of the stated features, elements, components, or assemblies, but do not exclude the presence or addition of one or more other features, elements, components, or assemblies.

[0032] In the embodiments of this application, the singular forms "a," "the," etc., including the plural forms, should be broadly understood as "a kind" or "a class" rather than limited to the meaning of "an." Furthermore, the term "the" should be understood to include both the singular and plural forms, unless the context explicitly indicates otherwise. Additionally, the term "according to" should be understood as "at least partially based on…," and the term "based on" should be understood as "at least partially based on…," unless the context explicitly indicates otherwise.

[0033] Features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, combined with features in other embodiments, or substituted for features in other embodiments. The term "comprising / including" as used herein means the presence of a feature, integral, step, or component, but does not exclude the presence or addition of one or more other features, integrals, steps, or components.

[0034] In the embodiments of this application, the object used for motion recognition or fall detection can be a human body of various ages, such as the elderly, children, or elderly and / or caregivers, children and / or guardians. This application is not limited to these; the object used for detection can be a human body or other animal with vital characteristics, or a machine without vital characteristics, etc.

[0035] First aspect of the embodiments

[0036] This application provides a fall detection method based on human posture estimation. Figure 1 This is a schematic diagram of a fall detection method according to an embodiment of this application, as shown below. Figure 1 As shown, the method includes:

[0037] Step 101: Use a neural network to identify key points of the human body in the image and obtain the key points and bounding boxes of the human body's skeleton.

[0038] Step 102: Estimate and classify the pose of the human body based on the obtained key points and bounding boxes to obtain the classified pose;

[0039] Step 103: Arrange the classified postures in chronological order to form a posture sequence, and determine whether the human body is in a lying position based on the posture sequence;

[0040] Step 104: When the human body is in a lying position, determine the direction in which the human body is lying in the posture sequence based on the image, and determine whether the human body has fallen in that direction based on the direction in which the human body is lying and the posture sequence.

[0041] According to the above embodiments, by arranging the classified human postures into a posture sequence in chronological order, and judging the fall and the direction of fall based on the posture sequence, it is possible not only to reduce the complexity of fall detection and improve the performance of fall detection, but also to ensure a better fall detection effect.

[0042] It is worth noting that the above appendix Figure 1 The embodiments of this application have only been illustrated schematically, and the application is not limited thereto. For example, the execution order between various operations can be appropriately adjusted, and other operations can be added or some operations can be removed. Those skilled in the art can make appropriate modifications based on the above description, and are not limited to the above-described embodiments. Figure 1 The records.

[0043] Figure 2 This is a schematic diagram of the key points and bounding box of the human skeleton in an embodiment of this application.

[0044] In some embodiments, in step 101, the shoulders, hips, knees, feet, hands, etc., of the human body can be identified as key points, and a bounding box that includes all key points can be obtained. Other joints of the human body can also be selected as key points for identification. Figure 2 As shown, for example, the feet, knees, hips, shoulders, etc. of the human body are selected as key points of the human body, and a bounding box 201 that can contain these key points is obtained.

[0045] In some embodiments, in step 102, the human body's pose is estimated and classified based on the acquired keypoints and bounding boxes to obtain a classified pose. For example, the classified pose can be standing, walking, bending, sitting, squatting, lying down, upper body, lower body, etc., where upper body indicates that the lower body of the human body is occluded or only the upper body is captured in the image, and lower body indicates that the upper body of the human body is occluded or only the lower body is captured in the image. For each classified pose, it can be obtained solely based on the current frame of that classified pose, without depending on the frames before or after that current frame. Figure 2 As shown, based on the obtained key points and bounding box 201 of the human body, the human pose in the image can be estimated as crawling, and the image is classified as crawling.

[0046] Figure 3 This is a schematic diagram of a posture sequence according to an embodiment of this application.

[0047] In some embodiments, in step 103, the classified poses are arranged into a pose sequence in chronological order, such as... Figure 3 As shown, the classified poses are placed into the pose sequence according to the time sequence. The classified poses can be various classified poses as described above. For the pose sequence of the same human body, if the human body or the classified pose of the human body is not detected in a certain frame, then the classified pose of this frame is set to "None".

[0048] In some embodiments, categorized poses are placed into corresponding pose sequences in chronological order. If the buffer size exceeds the size of the pose sequence, the pose furthest from the current frame is deleted in a "first-in, first-out" manner. Figure 3 For example, when classification pose 301 is put into the pose sequence, it exceeds the buffer size of the pose sequence. At this time, the classification pose 302 that is furthest from the frame to which the current classification pose 301 belongs is deleted in the "first-in, first-out" manner. That is, the earliest input classification pose 302 in the current pose sequence is deleted.

[0049] In some embodiments, a pose sequence corresponds to storing a person's categorized poses. The cache size of a pose sequence can be set according to actual conditions. The cache size can be a fixed value or a variable value. For example, the cache size can be related to the number of frames per second (FPS) of the video.

[0050] In some embodiments, in step 103, it is determined whether the human body is in a lying position based on the posture sequence.

[0051] When the posture sequence meets the first condition, it is determined that the human body is in a lying position. The first condition means that among the N postures closest to the current time in the posture sequence, the number of ending postures is greater than a first threshold, where N is a positive integer.

[0052] In some embodiments, the first condition may further include:

[0053] If, in the N poses closest to the current time in the pose sequence, the number of ending poses is greater than or equal to M2, and these N poses only contain some specific poses, then the human body is also judged to be in a lying position.

[0054] In either of the two situations mentioned above, the human body is considered to be in a lying position.

[0055] For example, the posture after the fall is defined as the ending posture. For different classification criteria, the ending posture can be lying down, crawling, etc. In this application, the ending posture is described as lying down, and the first condition, namely, the human body is in a lying down state, is used as an example for explanation.

[0056] In some embodiments, only some specific poses are included among the N poses, where the specific poses can be "lie", "None", "up body", "down body", etc., and reasonable specific poses can be set according to actual needs.

[0057] In the above embodiments, N, M1, and M2 are all positive integers, and reasonable values ​​can be set according to the actual use scenario.

[0058] When it is determined that the human body is lying down, the direction in which the human body is lying down is further determined.

[0059] In some embodiments, determining the direction in which the human body lies down in the final posture based on the image may include the following:

[0060] If a person lies down with their head facing forward, then the direction in which the person is lying down is determined to be forward.

[0061] If a person lies down with their head facing to the side, then the direction in which the person is lying down is determined to be lying on their side.

[0062] If a person lies down with their head facing backward, then the direction in which the person is lying down is determined to be backward.

[0063] Figure 4 This is a schematic diagram of a human body lying down in different directions according to an embodiment of this application.

[0064] In some embodiments, according to The angle formed by the vertical and upward directions is used to determine the direction in which a person is lying down. This represents a vector pointing from the center point of the hips to the center point of the neck or shoulder.

[0065] For example, according to The three lying positions described above are defined by the different ranges of angles formed by the vertical and upward directions. For example, the different angle ranges can be set as follows:

[0066] Front lie: 0°~45°;

[0067] Side lie: 45°~135°;

[0068] Lying back: 135°~180°;

[0069] Based on the above range of included angles, such as Figure 4 As shown, with Taking the vector representing the direction from the center of the buttocks to the center of the neck as an example, Figure 4 (a) The angle θ1 between the angle and the vertically upward direction Y is less than 45°, therefore, Figure 4 In (a), the human body is lying in a forward-facing position; Figure 4 (b) The angle θ2 between the angle and the vertically upward direction Y is greater than 45° and less than 135°, therefore, Figure 4 (b) The human body is lying on its side; Figure 4 (c) The angle θ3 between the angle and the vertically upward direction Y is greater than 135° and less than 180°, therefore, Figure 4 (c) shows the human body lying down in the direction of backwards.

[0070] In addition to the angle range mentioned above, other angle ranges can be set according to specific scenario requirements to determine the direction in which a person lies down. Furthermore, besides classifying the direction in which a person lies down into three categories, other types of lying down directions can also be classified according to specific needs, and the angle range can be set according to specific requirements to determine the corresponding lying down direction.

[0071] In some embodiments, when the human body is in a lying position, it is determined whether the human body meets the second condition. The second condition means that the number of intermediate postures n found in the posture sequence from the N+1th posture closest to the current time, or from the last ending posture in the first direction among the N postures, is greater than the second threshold.

[0072] When the human body meets the second condition, the window stops sliding, where n is a positive integer, the length of W1 is greater than the frame length of the pose sequence, and the first direction represents the direction away from the current time in the pose sequence.

[0073] by Figure 3 For example, a window 303 with a length greater than one frame length is used to slide in the first direction X to find intermediate poses. For example, if N is 3, the window 303 starts sliding from the fourth classified pose in the first direction, or from the last ending pose in the first direction. When the number of intermediate poses falling into the window is greater than the second threshold, the window 303 stops sliding. At this time, the human body meets the second condition.

[0074] In the above embodiments, intermediate postures include sitting, squatting, or bending over.

[0075] The following explains how to determine whether a person will fall forward when lying down in a forward-facing direction.

[0076] In some embodiments, when the human body falls forward, it is determined whether the human body meets the third condition, wherein the third condition represents:

[0077] When the human body meets the second condition mentioned above, in the posture sequence, in the classified postures located in the first direction of the window mentioned above, M3 starting postures can be found in a continuous sequence of W2 postures.

[0078] When the human body does not meet the second condition mentioned above, in the pose sequence, among the N+1 poses most recent in the current time, or among the categorized poses in the first direction of the last ending pose in the first direction among the N poses, it is possible to find M3 starting poses among a continuous sequence of W2 poses.

[0079] Among them, the upper and lower body ratios of the above M3 starting postures satisfy the third threshold. The third threshold is related to the average body ratio of the ending posture. M3 and W2 are both positive integers, and W2 is greater than M3.

[0080] by Figure 3 For example, when the human body meets the second condition, that is, there is an intermediate pose in window 303 that meets the second threshold, if M3 starting poses can be found in the first direction X of window 303 among W2 consecutive poses, then the human body meets the third condition; when the human body does not meet the second condition, that is, there is not a sufficient number of intermediate poses in window 303, then the ending pose closest to the first direction is found in the first direction of the fourth pose closest to the current time, or in the three poses closest to the current time, and M3 starting poses can be found in the first direction of the W2 consecutive poses.

[0081] Figure 5 This is a schematic diagram illustrating the calculation of the upper and lower body ratio according to an embodiment of this application.

[0082] The length ratio of the upper and lower body in an image can be calculated by taking the average distance from the center point of the neck or shoulders to both hips, and the average of the sum of the distances from both hips to the knees and from the knees to the ankles. Figure 5 As shown, for example, the length ratio of the upper and lower body of the human body can be calculated using the following formula:

[0083]

[0084] Where d represents the distance between two points, d(17,11) represents the distance from the left side of the hip 11 to the center point of the shoulder 17, and d(17,12) represents the distance from the right side of the hip 12 to the center point of the shoulder 17. The average of these two distances gives the length of the upper body. d(11,13)+d(13,15) represents the sum of the distances from the left side of the hip 11 to the left knee 13 and from the left knee 13 to the left ankle 15, and d(11,13)+d(13,15) represents the sum of the distances from the right side of the hip 12 to the right knee 14 and from the right knee 14 to the right ankle 15. The average of these two distances gives the length of the lower body. Dividing the length of the upper body by the length of the lower body gives the ratio of the upper and lower body lengths in the image.

[0085] In the above embodiments, the relative positions of key points of the human body are calculated by comparing the ratio of the upper and lower body, and some easily confused specific postures are distinguished, such as distinguishing between "lying down" and "standing" / "walking". This allows for the measurement of the reliability of the posture and further improves the accuracy of fall detection.

[0086] In some embodiments, the starting posture includes standing or walking.

[0087] In the above embodiments, the starting posture can be sitting, squatting, or bending over, in addition to standing and walking. However, sitting, squatting, or bending over does not actually indicate that the human body is in that state. It may just be an approximate movement that occurs when the human body loses balance before falling. In this case, the intermediate movement can be ignored, and only the starting posture and the ending posture can be considered. That is, the second condition is not considered.

[0088] In some embodiments, when the human body is lying down in a forward-facing direction, it is determined whether the human body meets the fourth condition, which means:

[0089] When the first condition is met, in the N postures of the posture sequence, the number of postures in which the human body lies forward is greater than the number of postures in which the human body lies in other directions.

[0090] Among these, the human body meets the first condition, that is, among N postures, the number of ending postures is greater than a first threshold. Figure 3 For example, in Figure 3 In the posture sequence shown, for example, if N is 3, then among the three postures most recent to the current time, the number of postures in which the human body lies forward is greater than the number of postures in which the human body lies in other directions. In this case, the human body satisfies the fourth condition.

[0091] In some embodiments, when the human body is lying in a forward-facing position, it is determined whether the human body meets the fifth condition. The fifth condition is:

[0092] When the human body meets the first condition, in the N poses of the pose sequence, the proportion of the bounding boxes corresponding to the N poses that are non-flat shapes exceeds the fourth threshold.

[0093] Among these, the human body meets the first condition, that is, among N postures, the number of ending postures is greater than a first threshold. Figure 3 For example, in Figure 3 In the pose sequence shown, for example, if N is 3, and the proportion of non-flattened bounding boxes in the three poses most recent to the current time exceeds the fourth threshold, then the human body satisfies the fifth condition.

[0094] The term "non-flat bounding box" means that when the height of the bounding box is greater than its width, and the difference between the height and width is greater than a pre-set threshold, the current bounding box is considered non-flat. Figure 5 For example, Figure 5 If the height of the bounding box 501 in the image shown is greater than its width, and the difference between the height and the width is greater than a preset threshold, then the bounding box 501 is a non-flat bounding box.

[0095] In the example above, the bounding box can be a box that includes the entire human body or a box that includes all key points, such as... Figure 2 As shown, bounding box 201 contains all the key points of the human body. In addition, in some special cases, the bounding box may not contain all the key points of the human body. For example, when the human body loses its center of gravity and causes its arms to open, the arms affect the height and width of the bounding box. In this case, the bounding box may not contain the arms or the key points on the arms.

[0096] In the above embodiments, the fourth threshold can be preset with different values ​​according to different scenario requirements.

[0097] In some embodiments, when the human body is lying down in a forward-facing direction and the human body meets the third, fourth, and fifth conditions mentioned above, it is determined that the human body has fallen forward.

[0098] Figure 6 This is a schematic diagram of a human body falling forward according to an embodiment of this application.

[0099] like Figure 6 As shown, when a person meets all five conditions, it is determined that the person has fallen forward. When the initial posture is sitting, squatting, or bending over, the second condition can be omitted. Furthermore, the third to fifth conditions all have the premise that "the person is in a lying position," which implicitly includes the first condition. Therefore, to determine that a person has fallen forward, at least the third, fourth, and fifth conditions must be met.

[0100] In addition, when a person is lying down in a forward direction, but the person does not fully meet the third, fourth, and fifth conditions mentioned above, other conditions can be used to determine whether the person has fallen forward.

[0101] In some embodiments, when the human body is lying forward and the third, fourth, and fifth conditions are fully met, it is determined whether the human body meets the sixth condition. The sixth condition means:

[0102] When the human body meets the second condition, in the posture sequence, in the classification postures located in the first direction of the intermediate posture, we can find M4 starting postures in a continuous W3 postures.

[0103] When the human body does not meet the second condition, in the posture sequence, among the classified postures in the first direction of the ending posture, we can find M4 starting postures out of W3 consecutive postures.

[0104] Where W3 and M4 are both positive integers, and W3 is greater than M4.

[0105] by Figure 3 For example, the sixth condition means: when the human body meets the second condition, that is, there is an intermediate posture in window 303 that meets the second threshold, there can be M4 starting postures in a continuous W3 postures in the classified postures of the first direction X of window 303; when the human body does not meet the second condition, that is, the number of intermediate postures in window 303 does not meet the second threshold, there can be M4 starting postures in a continuous W3 postures in the classified postures of the first direction of the ending posture, that is, in the classified postures of the first direction of the three postures closest to the current time. For example, if W3 equals 6 and M4 equals 3, then three starting postures can be found in the above six continuous postures.

[0106] In the above embodiments, the reliability of the initial posture can be determined by the posture score or the score of the key point and the relative position.

[0107] In some embodiments, when the human body is lying forward and the third, fourth, and fifth conditions are fully met, it is determined whether the human body meets the seventh condition. The seventh condition means:

[0108] The falling angle θ>D1, where D1 is a preset angle threshold based on the actual situation.

[0109] In the above embodiment, the falling angle θ represents the starting posture that satisfies the sixth condition mentioned above. The ending posture that satisfies the first condition mentioned above The angle between them, where, This represents a vector pointing from the center point of the hips to the center point of the neck or shoulder.

[0110] In some embodiments, when the human body is lying in a forward-facing direction and the human body meets the sixth and seventh conditions, it is determined that the human body has fallen forward.

[0111] In the above embodiments, when a human body falls forward, it can be determined whether the human body meets the third, fourth, and fifth conditions, or whether the human body meets the sixth and seventh conditions. Either condition is sufficient.

[0112] The following explains how to determine whether a person has fallen when they fall to their side or backward.

[0113] In some embodiments, when the human body lies down to its side or backward, it is determined whether the human body meets the eighth condition. The eighth condition is:

[0114] When the human body meets the second condition, in the posture sequence, in the first direction of the classification posture located in the middle posture, we can find M5 starting postures in a continuous W4 postures.

[0115] When the human body does not meet the second condition, in the posture sequence, among the classified postures in the first direction of the ending posture, we can find M5 starting postures out of a continuous W4 postures.

[0116] Where W4 and M5 are both positive integers, and W4 is greater than M5.

[0117] In some embodiments, when the direction in which a person falls is either sideways or backward, it is determined whether the person meets the ninth condition. The ninth condition means:

[0118] The falling angle θ > D2, where D2 is a preset angle threshold based on the actual situation.

[0119] Figure 7 This is a schematic diagram of a human body falling to the side according to an embodiment of this application.

[0120] like Figure 7 As shown, taking a human body falling to the side as an example, the initial posture that satisfies the eighth condition corresponds to... for The ending posture that satisfies the first condition for and The angle between them is the falling angle θ4. If the falling angle θ4 is greater than the preset angle threshold D2, then the human body meets the ninth condition.

[0121] In the above embodiments, the eighth and ninth conditions for determining whether a human body has fallen to the side or backward are the same as the sixth and seventh conditions for determining whether a human body has fallen forward, except that the preset values ​​W4, M5 and W3, M4 and the setting of posture reliability can be independently set according to specific parameters such as video clarity, camera shooting angle, and camera shooting range (wide angle) in actual situations. Other operation procedures are the same. For other specific content about the eighth and ninth conditions, please refer to the sixth and seventh conditions.

[0122] In some embodiments, when the direction in which the human body lies down is to the side or backward, and the human body meets the eighth and ninth conditions, it is determined that the human body has fallen to the side or backward.

[0123] like Figure 7 As shown, when a person meets all of the first, second, eighth, and ninth conditions, it is determined that the person has fallen forward. When the starting posture is sitting, squatting, or bending over, the second condition can be omitted. In addition, the eighth and ninth conditions both have the premise that "the person is in a lying position", which implies the first condition. Therefore, to determine that a person has fallen to the side or backward, at least the eighth and ninth conditions must be met.

[0124] In some embodiments, a detection status marker is set on the human body to indicate whether the human body currently needs to undergo fall detection. The detection marker may include: fallen, skipped detection, or needs detection.

[0125] When a person is marked as having fallen, the fall detection is skipped. For skipping, if it is determined that the person is lying down but it cannot be determined that the person has fallen, the fall detection is skipped. Skipping is an optional marker, that is, the detection marker can also include only two types: fallen and checking.

[0126] In some embodiments, when there are at least W6 starting poses among the W5 poses closest to the current time in the pose sequence, the state flag is reset to require detection, where W5 and W6 are both positive integers, and W5 is greater than W6.

[0127] by Figure 3 For example, W5 equals N. When there are at least W6 starting poses among the N poses closest to the current time in the pose sequence, the state flag is reset to require detection. Here, N and W6 are both positive integers and N is greater than W6.

[0128] In the above embodiments, in some examples, the reliability of the pose can be further judged. For example, based on the above conditions, the reliability is further calculated in W5 poses and a certain threshold is set. If the threshold is met, the state flag is reset to the point where detection is required.

[0129] In the embodiments of this application, for all starting, intermediate, and ending postures involved in fall detection, a corresponding reliability can be introduced for further judgment. The reliability of the posture can be calculated by the posture score or the key point score. For example, the relative position of the key points can be used, or the relative position of the key points can be calculated for some easily confused specific postures, such as calculating the upper and lower body ratio. In addition, the reliability of the posture can also be determined by the bounding box detector and different posture classifier models. This application does not impose any restrictions. Figure 8 This is a schematic diagram of the fall detection process according to an embodiment of this application.

[0130] like Figure 8 As shown, after obtaining the posture sequence, the system first determines whether a fall detection is needed. When a person is marked as needing detection, the system determines whether the person is lying down and the direction of the fall based on the first condition, and then determines whether a fall has occurred based on the different lying directions. When the person is lying forward, the system checks whether all three conditions (the third, fourth, and fifth) are met. If all three conditions are met, the person is judged to have fallen forward. If the person does not meet all three conditions, the system further checks whether the person meets the sixth and seventh conditions. If both conditions are met, the person is also judged to have fallen forward. When the person is lying sideways or backward, the system checks whether the eighth and ninth conditions are met. If both conditions are met, the person is judged to have fallen sideways or backward.

[0131] If, during fall detection, the human body does not meet any of the above conditions—that is, if the human body is determined to be in a lying position (i.e., the first condition is met) but it cannot be determined whether a fall has occurred—the human body is marked as skipping the detection.

[0132] In the above embodiments, when it is determined that a human body has fallen or has been marked as skipping the detection, it is further detected whether the human body needs to be marked for reset. If so, the human body is marked for reset to require detection and fall detection is performed again. If not, a new posture sequence is obtained and fall detection is performed on the posture sequence.

[0133] In the above embodiments, one pose sequence can only be used for fall detection on one human body. This application is not limited to this; if fall detection is required on multiple human bodies, multiple pose sequences can be set up to perform fall detection simultaneously. Figure 8 The fall detection process is shown below.

[0134] Table 1 illustrates some of the effects of embodiments of this application. It shows the performance of the methods of embodiments of this application on different detectors and classifiers.

[0135] Table 1

[0136]

[0137] As shown in Table 1, test 1 indicates falling detection of a human body using method 1 (e.g., using a human body detector model (YOLOX) and a human pose estimation model (CPN) and the original classifier trained based on CPN keypoints) and the original fall detection method; test 2 indicates falling detection of a human body using method 2 (e.g., using a human body detector model (YOLOX) and a human pose estimation model (CPN) and the original classifier trained based on CPN keypoints) and the fall detection method described in the embodiments of this application; test 3 indicates falling detection of a human body using method 3 (e.g., using a human pose estimation model (trtpose) and a pose classifier trained based on trtpose keypoints) and the fall detection method described in the embodiments of this application; test 4 indicates falling detection of a human body using method 4 (e.g., using a human body detector model (YOLOX), a human pose estimation model (CPN), and an improved pose classifier) ​​and the fall detection method described in the embodiments of this application). In addition, recall represents the recall rate, that is, the probability of being effectively detected in the data, precision represents the accuracy of fall detection, and NVR1, NVR2, NVR3, and NVR4 represent four long videos of about 20 minutes each, each containing multiple falls. The videos contain a series of continuous actions such as walking, falling, and getting up, which can fully test the fall detection effect of this application.

[0138] It is evident that, for different detectors and classifiers, the accuracy of fall detection can be effectively improved after using the method described in the embodiments of this application.

[0139] Tables 2 and 3 exemplarily illustrate some other effects of the embodiments of this application. They show the impact of the methods of the embodiments of this application on fall detection performance.

[0140] Table 2

[0141]

[0142] Table 3

[0143]

[0144] Taking the YOLOv3 detector model as an example, Table 2 shows the performance of the fall detection system for different video files using the original fall detection method, and Table 3 shows the performance of the fall detection system for different video files using the fall detection method shown in the embodiments of this application. It can be seen that the fall detection method shown in the embodiments of this application improves the performance of fall detection while having almost no impact on the running speed and storage of the test system (Actlyzer).

[0145] The above description only covers the steps or processes related to this application, but this application is not limited thereto. The fall detection method may also include other steps or processes; for details of these steps or processes, please refer to existing technologies. Furthermore, the above description only uses some structural examples of the fall detection model to illustrate the embodiments of this application, but this application is not limited to these structures, and appropriate modifications can be made to these structures. All such modifications should be included within the scope of the embodiments of this application.

[0146] The above embodiments are merely illustrative examples of embodiments of this application, but this application is not limited thereto, and appropriate modifications can be made based on the above embodiments. For example, the above embodiments can be used alone, or one or more of the above embodiments can be combined.

[0147] As can be seen from the above embodiments, by arranging the classified human postures into a posture sequence according to time sequence, and judging the fall and the direction of fall based on the posture sequence, the complexity of fall detection can be reduced and the performance of fall detection can be improved, and a better fall detection effect can be guaranteed.

[0148] Second aspect of the embodiments

[0149] This application provides a fall detection device based on human posture estimation, the content of which is the same as that in the first aspect and will not be repeated.

[0150] Figure 9 This is a schematic diagram of a fall detection device according to an embodiment of this application, as shown below. Figure 9 As shown, the fall detection device 900 includes:

[0151] The acquisition unit 901 uses a neural network to identify key points of the human body in the image and acquires the key points and bounding boxes of the human body's skeleton.

[0152] The classification unit 902 estimates and classifies the human body's pose based on the acquired key points and bounding boxes to obtain a classified pose.

[0153] The first judgment unit 903 arranges the classified postures in chronological order to form a posture sequence, and judges whether the human body is in a lying position based on the posture sequence.

[0154] The second judgment unit 904, when the human body is in a lying position, determines the direction in which the human body is lying in the posture sequence based on the image, and determines whether the human body has fallen in that direction based on the direction in which the human body is lying and the posture sequence.

[0155] In some embodiments, when the posture sequence satisfies a first condition, the first judgment unit 903 determines that the human body is in a lying position, wherein the first condition indicates that among the N postures in the posture sequence that are closest to the current time, the number of ending postures is greater than a first threshold, where N is a positive integer.

[0156] In some embodiments, the ending posture indicates that the classified posture is lying down or prone.

[0157] In some embodiments, the second determining unit 904 determines the direction in which the human body is lying down by:

[0158] Determine the direction in which the body lies down in the final posture based on the image.

[0159] If the human body lies down with its head facing forward, then the direction in which the human body lies down is determined to be forward.

[0160] If the human body lies down with its head facing to the side, then the direction in which the human body lies down is determined to be lying on its side.

[0161] If the human body lies down with its head facing backward, then the direction in which the human body lies down is determined to be backward.

[0162] In some embodiments, when the human body is in a lying position, the second judgment unit 904 determines whether the human body meets a second condition. The second condition means that, starting from the (N+1)th pose closest to the current time in the pose sequence, or from the last ending pose in the first direction among the N poses, a window of length W1 is slid in the first direction, and the number n of intermediate poses found among the poses falling into the window is greater than a second threshold. When the human body meets the second condition, the window stops sliding, where n is a positive integer, the length of W1 is greater than a frame length in the pose sequence, and the first direction represents the direction away from the current time in the pose sequence.

[0163] In some embodiments, the intermediate posture includes sitting, squatting, or bending over.

[0164] In some embodiments, when the human body is lying down in a forward-lying position, the second determining unit 904 determines whether the human body meets a third condition, wherein the third condition represents:

[0165] When the human body meets the second condition, in the posture sequence, in the classified postures located in the first direction of the window, M3 starting postures can be found out from W2 consecutive postures.

[0166] When the human body does not meet the second condition, in the posture sequence, among the N+1th posture closest to the current time, or the classified postures in the first direction of the last ending posture in the first direction among the N postures, it is possible to find M3 starting postures among a continuous sequence of W2 postures.

[0167] Among them, the upper and lower body ratios of the M3 initial postures satisfy a third threshold. This third threshold is related to the average upper and lower body ratio of the final posture.

[0168] Where W2 and M3 are both positive integers, and W2 is greater than M3.

[0169] In some embodiments, the starting posture includes standing or walking.

[0170] In some embodiments, when the human body is lying down in a forward-lying position, the second determination unit 904 determines whether the human body meets a fourth condition, wherein the fourth condition represents:

[0171] When the human body meets the first condition, in the N postures in the posture sequence, the number of postures in which the human body lies forward is greater than the number of postures in which it lies in other directions.

[0172] In some embodiments, when the human body is lying down in a forward-lying position, the second determining unit 904 determines whether the human body meets a fifth condition, wherein the fifth condition represents:

[0173] When the human body meets the first condition, in the N poses in the pose sequence, the proportion of the bounding boxes corresponding to the N poses that are non-flat shapes exceeds the fourth threshold.

[0174] In some embodiments, when the human body is lying down in a forward-facing direction, and the human body meets the third, fourth, and fifth conditions, the second judgment unit 904 determines that the human body has fallen forward.

[0175] In some embodiments, when the human body is lying down in a forward-facing direction and the human body does not fully meet the third, fourth, and fifth conditions, the second judgment unit 904 determines whether the human body meets the sixth condition, wherein the sixth condition represents:

[0176] When the human body meets the second condition, in the posture sequence, among the classified postures in the first direction located in the intermediate posture, M4 starting postures can be found in a continuous sequence of W3 postures.

[0177] When the human body does not meet the second condition, in the posture sequence, among the classified postures located in the first direction of the ending posture, M4 starting postures can be found out from W3 consecutive postures.

[0178] Where W3 and M4 are both positive integers, and W3 is greater than M4.

[0179] In some embodiments, when the human body is lying down in a forward-facing direction and the human body does not fully meet the third, fourth, and fifth conditions, the second judgment unit 904 determines whether the human body meets the seventh condition, wherein the seventh condition represents:

[0180] The falling angle θ>D1, where D1 is a preset angle threshold based on the actual situation.

[0181] In some embodiments, when the human body is lying down in a forward-facing direction and the human body meets the sixth and seventh conditions, the second judgment unit 904 determines that the human body has fallen forward.

[0182] In some embodiments, when the human body is lying down in a side-lying or backward-lying position, the second determination unit determines whether the human body meets an eighth condition, wherein the eighth condition represents:

[0183] When the human body meets the second condition, in the posture sequence, among the classified postures in the first direction located in the intermediate posture, M5 starting postures can be found in a continuous sequence of W4 postures.

[0184] When the human body does not meet the second condition, in the posture sequence, among the classified postures located in the first direction of the ending posture, M5 starting postures can be found out from W4 consecutive postures.

[0185] Where W4 and M5 are both positive integers, and W4 is greater than M5.

[0186] In some embodiments, when the human body falls to its side or backward, the second determining unit 904 determines whether the human body meets the ninth condition, wherein the ninth condition represents:

[0187] The falling angle θ > D2, where D2 is a preset angle threshold based on the actual situation.

[0188] In some embodiments, when the direction in which the human body lies down is either sideways or backwards, and the human body satisfies the sixth and seventh conditions, the second judgment unit 904 determines that the human body has fallen to the side or backwards.

[0189] In some embodiments, the apparatus further includes:

[0190] The marking unit 905 sets a detection status mark on the human body to indicate whether the human body needs to be detected for a fall. The detection status mark includes: fallen, skipped detection, and needs detection.

[0191] In some embodiments, when at least W6 initial poses exist among the W5 poses closest to the current time in the pose sequence, the marking unit 905 resets the state marker to indicate that detection is required.

[0192] Where W5 and W6 are both positive integers, and W5 is greater than W6.

[0193] It is worth noting that the above description only covers the components or modules relevant to this application, but this application is not limited thereto. The fall detection device 900 may also include other components or modules, and for details regarding these components or modules, please refer to relevant technologies.

[0194] For the sake of simplicity, Figure 9 The diagram only exemplifies the connection relationships or signal flow between various components or modules; however, those skilled in the art should understand that various related technologies, such as bus connections, can be employed. The aforementioned components or modules can be implemented using hardware facilities such as processors and memory; this application does not limit the scope of the embodiments.

[0195] The above embodiments are merely illustrative examples of embodiments of this application, but this application is not limited thereto, and appropriate modifications can be made based on the above embodiments. For example, the above embodiments can be used alone, or one or more of the above embodiments can be combined.

[0196] As can be seen from the above embodiments, by arranging the classified human postures into a posture sequence according to time sequence, and judging the fall and the direction of fall based on the posture sequence, the complexity of fall detection can be reduced and the performance of fall detection can be improved, and a better fall detection effect can be guaranteed.

[0197] Third aspect of the embodiments

[0198] This application provides an electronic device including a fall detection device 900 as described in the embodiments of the second aspect, the contents of which are incorporated herein by reference. This electronic device may be, for example, a computer, server, workstation, laptop computer, smartphone, etc.; however, the embodiments of this application are not limited thereto.

[0199] Figure 10 This is a schematic diagram of an electronic device according to an embodiment of this application. For example... Figure 10As shown, the electronic device 1000 may include a processor (e.g., a central processing unit, CPU) 1010 and a memory 1020; the memory 1020 is coupled to the central processing unit 1010. The memory 1020 can store various types of data; it also stores information processing programs and executes these programs under the control of the processor 1010.

[0200] In some embodiments, the functionality of the fall detection device 900 is integrated into the processor 1010. The processor 1010 is configured to implement the fall detection method as described in the first aspect of the embodiment.

[0201] In some embodiments, the fall detection device 900 is configured separately from the processor 1010. For example, the fall detection device 900 can be configured as a chip connected to the processor 1010, and the functions of the fall detection device 900 can be implemented through the control of the processor 1010.

[0202] For example, the processor 1010 is configured to perform the following control: using a neural network to perform keypoint recognition on a human body in an image, obtaining keypoints and bounding boxes of the human body's skeleton; estimating and classifying the human body's pose based on the obtained keypoints and bounding boxes to obtain classified poses; arranging the classified poses in chronological order to form a pose sequence, and determining whether the human body is in a lying position based on the pose sequence; wherein, when the human body is in a lying position, determining the direction in which the human body is lying in the pose sequence based on the image, and determining whether the human body has fallen in that direction based on the direction in which the human body is lying and the pose sequence.

[0203] In addition, such as Figure 10 As shown, the electronic device 1000 may also include: an input / output (I / O) device 1030 and a display 1040, etc.; the functions of the above components are similar to those in the prior art, and will not be described in detail here. It is worth noting that the electronic device 1000 is not necessarily required to include... Figure 10 All components shown; in addition, the electronic device 800 may also include Figure 10 For components not shown, please refer to relevant technologies.

[0204] This application also provides a computer-readable program, wherein when the program is executed in an electronic device, the program causes the computer in the electronic device to perform the fall detection method as described in the first aspect embodiment.

[0205] This application also provides a storage medium storing a computer-readable program, wherein the computer-readable program causes a computer in an electronic device to perform the fall detection method as described in the first aspect embodiment.

[0206] The apparatus and methods described above in this application can be implemented in hardware or in combination with software. This application relates to a computer-readable program that, when executed by a logic component, enables the logic component to implement the apparatus or components described above, or to implement the various methods or steps described above. This application also relates to storage media for storing the above programs, such as hard disks, magnetic disks, optical disks, DVDs, flash memory, etc.

[0207] The methods / apparatus described in conjunction with the embodiments of this application can be directly embodied in hardware, software modules executed by a processor, or a combination of both. For example, one or more and / or combinations of one or more functional block diagrams shown in the figures can correspond to various software modules in a computer program flow, or to various hardware modules. These software modules can correspond to the various steps shown in the figures, respectively. These hardware modules can be implemented, for example, using a field-programmable gate array (FPGA) to embed these software modules.

[0208] The software module can reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art. A storage medium can be coupled to the processor, enabling the processor to read information from and write information to the storage medium; or the storage medium can be an integral part of the processor. The processor and storage medium can reside in an ASIC. The software module can be stored in the memory of a mobile terminal or in a memory card that can be inserted into the mobile terminal. For example, if the device (such as a mobile terminal) uses a high-capacity MEGA-SIM card or a high-capacity flash memory device, the software module can be stored in the MEGA-SIM card or the high-capacity flash memory device.

[0209] One or more and / or one or more combinations of functional blocks described in the accompanying drawings can be implemented as a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, or any suitable combination thereof for performing the functions described herein. One or more and / or one or more combinations of functional blocks described in the accompanying drawings can also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in communication with a DSP, or any other such configuration.

[0210] The present application has been described above with reference to specific embodiments. However, those skilled in the art should understand that these descriptions are exemplary and not intended to limit the scope of protection of the present application. Those skilled in the art can make various modifications and variations to the present application based on the principles thereof, and these modifications and variations are also within the scope of the present application.

[0211] Regarding the implementation methods including the above embodiments, the following notes are also disclosed:

[0212] 1. A fall detection method based on human posture estimation, comprising:

[0213] The neural network is used to identify key points of the human body in the image, and the key points and bounding boxes of the human body's skeleton are obtained.

[0214] The pose of the human body is estimated and classified based on the obtained key points and bounding boxes to obtain the classified pose;

[0215] The classified postures are arranged in chronological order to form a posture sequence, and the posture sequence is used to determine whether the human body is in a lying position.

[0216] When the human body is in a lying position, the direction in which the human body lies down is determined based on the image and the posture sequence. Based on the direction in which the human body lies down and the posture sequence, it is determined whether the human body has fallen in that direction.

[0217] 2. According to the method described in Appendix 1, wherein,

[0218] When the human body is lying face down, and the human body does not fully meet the third, fourth, and fifth conditions, determine whether the human body meets the sixth condition, wherein the sixth condition means:

[0219] When the human body meets the second condition, in the posture sequence, among the classified postures in the first direction located in the intermediate posture, M4 starting postures can be found in a continuous sequence of W3 postures.

[0220] When the human body does not meet the second condition, in the posture sequence, among the classified postures located in the first direction of the ending posture, M4 starting postures can be found out from W3 consecutive postures.

[0221] Where W3 and M4 are both positive integers, and W3 is greater than M4.

[0222] 3. According to the method described in Appendix 1, wherein,

[0223] When the human body is lying face down, and the human body does not fully meet the third, fourth, and fifth conditions, determine whether the human body meets the seventh condition, wherein the seventh condition means:

[0224] The falling angle θ>D1, where D1 is a preset angle threshold based on the actual situation.

[0225] 4. According to the method described in Appendix 1, wherein,

[0226] When the human body is lying down in a forward-facing direction, and the human body meets the sixth and seventh conditions, it is determined that the human body has fallen forward.

[0227] 5. According to the method described in Appendix 1, wherein,

[0228] When the human body is lying down in a sideways or backward position, it is determined whether the human body meets the sixth condition, wherein the sixth condition means:

[0229] When the human body meets the second condition, in the posture sequence, among the classified postures in the first direction located in the intermediate posture, M5 starting postures can be found in a continuous sequence of W4 postures.

[0230] When the human body does not meet the second condition, in the posture sequence, among the classified postures located in the first direction of the ending posture, M5 starting postures can be found out from W4 consecutive postures.

[0231] Where W4 and M5 are both positive integers, and W4 is greater than M5.

[0232] 6. According to the method described in Appendix 1, wherein,

[0233] When the human body falls to its side or backward, it is determined whether the human body meets the seventh condition, wherein the seventh condition means:

[0234] The falling angle θ > D2, where D2 is a preset angle threshold based on the actual situation.

[0235] 7. According to the method described in Appendix 1, wherein,

[0236] When the direction in which the human body lies down is either sideways or backwards, and the human body meets the sixth and seventh conditions, it is determined that the human body has fallen to the side or backwards.

[0237] 8. The method according to Appendix 1, wherein the method further comprises:

[0238] A detection status marker is set on the human body to indicate whether the human body needs to undergo fall detection. The detection status marker includes: fallen, skipped detection, and needs detection.

[0239] 9. According to the method described in Appendix 1, wherein,

[0240] If at least W6 initial poses are among the W5 poses closest to the current time in the pose sequence, the state flag is reset to indicate that detection is required.

[0241] Where W5 and W6 are both positive integers, and W5 is greater than W6.

[0242] 10. An electronic device comprising a processor and a memory, the processor being configured to implement the fall detection method as described in any one of Appendices 1 to 9.

Claims

1. A fall detection device based on human posture estimation, characterized in that, The device includes: The acquisition unit uses a neural network to identify key points of the human body in the image and acquires the key points and bounding boxes of the human body's skeleton. The classification unit estimates and classifies the human body's pose based on the acquired key points and bounding boxes to obtain the classified pose. The first judgment unit arranges the classified postures in chronological order to form a posture sequence, and determines whether the human body is in a lying position based on the posture sequence; and The second judgment unit, when the human body is in a lying position, determines the direction in which the human body lies down in the posture sequence based on the image, and determines whether the human body has fallen in that direction based on the direction in which the human body lies down and the posture sequence.

2. The apparatus according to claim 1, wherein, When the posture sequence meets a first condition, the first judgment unit determines that the human body is in a lying position. The first condition indicates that among the N postures in the posture sequence closest to the current time, the number of ending postures is greater than a first threshold. Wherein, N is a positive integer, and the ending posture indicates that the classified posture is lying down or prone.

3. The apparatus according to claim 1, wherein, The second determination unit determines the direction in which the human body is lying down by: Determine the direction in which the body lies down in the final posture based on the image. If the human body lies down with its head facing forward, then the direction in which the human body lies down is determined to be forward. If the human body lies down with its head facing to the side, then the direction in which the human body lies down is determined to be lying on its side. If the human body lies down with its head facing backward, then the direction in which the human body lies down is determined to be backward.

4. The apparatus according to claim 3, wherein, When the human body is in a lying position, the second judgment unit determines whether the human body meets the second condition. The second condition means that, starting from the (N+1)th posture closest to the current time in the posture sequence, or from the last ending posture in the first direction among the N postures, a window of length W1 is slid in the first direction, and the number n of intermediate postures found among the postures falling into the window is greater than a second threshold. When the human body meets the second condition, the window stops sliding. Wherein, n is a positive integer, the length of W1 is greater than the length of a frame in the pose sequence, the first direction represents the direction away from the current time in the pose sequence, and the intermediate pose includes sitting, squatting, or bending over.

5. The apparatus according to claim 4, wherein, When the human body is lying down in a forward-facing position, the second judgment unit determines whether the human body meets the third, fourth, and fifth conditions. The third condition means: When the human body meets the second condition, in the posture sequence, among the classified postures located in the first direction of the window, it is possible to find M3 starting postures out of a continuous W2 postures. When the human body does not meet the second condition, in the posture sequence, among the N+1th posture closest to the current time, or the classified postures in the first direction of the last ending posture in the first direction among the N postures, it is possible to find M3 starting postures among a continuous sequence of W2 postures. The upper and lower body ratios of the M3 initial postures satisfy a third threshold, which is related to the average upper and lower body ratio of the final posture. Both W2 and M3 are positive integers, and W2 is greater than M3. The starting posture includes standing or walking. The fourth condition indicates that: When the human body meets the first condition, in the N postures in the posture sequence, the number of postures in which the human body lies forward is greater than the number of postures in which it lies in other directions; The fifth condition indicates that: When the human body meets the first condition, in the N poses in the pose sequence, the proportion of the bounding boxes corresponding to the N poses that are non-flat shapes exceeds the fourth threshold.

6. The apparatus according to claim 5, wherein, When the human body is lying down in a forward-facing direction, and the human body meets the third, fourth, and fifth conditions, the second judgment unit determines that the human body has fallen forward.

7. The apparatus according to claim 4, wherein, When the human body is lying down in a sideways or backward position, the second judgment unit determines whether the human body meets the eighth and ninth conditions. The eighth condition represents: When the human body meets the second condition, in the posture sequence, in the classified postures in the first direction located in the intermediate posture, M5 starting postures can be found in a continuous sequence of W4 postures. When the human body does not meet the second condition, in the posture sequence, in the classified postures located in the first direction of the ending posture, there can be M5 starting postures among W4 consecutive postures, where W4 and M5 are both positive integers, and W4 is greater than M5. The ninth condition indicates that: The falling angle θ>D2, where D2 is a preset angle threshold based on the actual situation.

8. The apparatus according to claim 7, wherein, When the direction in which the human body lies down is either sideways or backwards, and the human body meets conditions eight and nine, the second judgment unit determines that the human body has fallen to the side or backwards.

9. The apparatus according to claim 1, wherein, The device further includes: A marking unit sets a detection status mark on the human body to indicate whether the human body currently needs to undergo fall detection. The detection status mark includes: fallen, skipped detection, and needs detection.

10. A fall detection method based on human pose estimation, comprising: The neural network is used to identify key points of the human body in the image, and the key points and bounding boxes of the human body's skeleton are obtained. The pose of the human body is estimated and classified based on the obtained key points and bounding boxes to obtain the classified pose; The classified postures are arranged in chronological order to form a posture sequence, and the posture sequence is used to determine whether the human body is in a lying position. as well as When the human body is in a lying position, the direction in which the human body lies down in the posture sequence is determined based on the image, and the human body is determined to have fallen in that direction based on the direction in which the human body lies down and the posture sequence.

Images