A fall detection method, device, equipment and computer storage medium

Abstract

The embodiment of the present invention provides a fall detection method, device, equipment and computer storage medium, which relates to the field of artificial intelligence and is used to improve the accuracy of fall detection. The method includes: based on a pre-acquired target image, extracting multiple key point information of a target human body in the target image; determining the estimated height parameter, measured height parameter and posture offset parameters of multiple postures of the target human body according to the multiple key point information; determining the threshold interval of the target human body falling according to the measured height parameter; when the estimated height parameter is less than the minimum value of the threshold interval, determining that the target human body is in a fall state, and when the estimated height parameter is greater than the maximum value of the threshold interval, determining that the target human body is not in a fall state; when the estimated height parameter is within the threshold interval, determining whether the target human body falls based on the relationship information between multiple posture offset parameters and the preset posture threshold.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence, and in particular relates to a fall detection method, device, equipment and computer storage medium. Background Art

[0002] Fall detection is to detect whether the target human body is in a falling state. It is widely used in home care, monitoring and sports training. With the rise of artificial intelligence, many fall detection methods based on deep learning have emerged. The use of deep learning methods can continuously improve the accuracy of the model by expanding the data set, so it has attracted the attention of relevant researchers.

[0003] In the existing deep learning-based fall detection methods, most of them use image processing methods or machine learning methods to capture the human target frame, and determine whether the human body has fallen through a fixed height threshold or machine learning model. However, the above methods lack feature analysis of the relative position relationship of key parts of the human body, resulting in misjudgment of the height and posture of the human body. At the same time, due to the use of unified model parameters for detection, they lack the ability to adapt to different scenarios, resulting in reduced accuracy in some scenarios.

[0004] In summary, the fall detection method in the prior art does not fully utilize the key features of human posture and cannot ensure the accuracy of fall detection. Summary of the invention

[0005] Embodiments of the present invention provide a fall detection method, apparatus, device, and computer storage medium, which can improve the accuracy of fall detection.

[0006] In a first aspect, an embodiment of the present invention provides a fall detection method, the method comprising:

[0007] Based on the pre-acquired target image, multiple key point information of the target human body in the target image is extracted;

[0008] Determine an estimated height parameter, a measured height parameter, and posture offset parameters of a plurality of postures of a target human body according to information of a plurality of key points, wherein the posture offset parameter is used to characterize an angle between a target posture and the ground, and the target posture is any one of the plurality of postures;

[0009] According to the measured height parameters, the threshold range of the target human body falling is determined;

[0010] When the estimated height parameter is less than the minimum value of the threshold interval, the target human body is determined to be in a falling state; when the estimated height parameter is greater than the maximum value of the threshold interval, the target human body is determined to be in a non-falling state;

[0011] When the estimated height parameter is within the threshold range, it is determined whether the target human body falls based on the relationship information between the plurality of posture offset parameters and the preset posture threshold.

[0012] In a second aspect, an embodiment of the present invention provides a fall detection device, the device comprising:

[0013] An extraction unit, used for extracting a plurality of key point information of a target human body in a target image based on a pre-acquired target image;

[0014] A first determination unit is used to determine an estimated height parameter, a measured height parameter and a posture offset parameter of a plurality of postures of a target human body according to information of a plurality of key points, wherein the posture offset parameter is used to characterize an angle between a target posture and the ground, and the target posture is any one of the plurality of postures;

[0015] A second determination unit is used to determine a threshold range for a target human body to fall according to the measured height parameter;

[0016] A third determination unit is used to determine that the target human body is in a falling state when the estimated height parameter is less than a minimum value of the threshold interval, and to determine that the target human body is in a non-falling state when the estimated height parameter is greater than a maximum value of the threshold interval;

[0017] The fourth determination unit is used to determine whether the target human body falls based on the relationship information between the multiple posture offset parameters and the preset posture threshold when the estimated height parameter is within the threshold range.

[0018] In a third aspect, an embodiment of the present invention provides a fall detection device, the device comprising:

[0019] A processor, and a memory storing computer program instructions; the processor reads and executes the computer program instructions to implement the fall detection method provided in the first aspect of the embodiment of the present invention.

[0020] In a fourth aspect, an embodiment of the present invention provides a computer storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the fall detection method provided in the first aspect of the embodiment of the present invention is implemented.

[0021] The fall detection method, device, equipment and computer storage medium provided by the embodiment of the present invention determine the estimated height parameter, measured height parameter and multiple posture offset parameters of the target human body based on multiple key point information acquired in advance. When multiple people exist in the target image at the same time, the key point information is extracted for each target human body, thereby achieving the adaptability of parameter acquisition; the target threshold interval of the target human body is determined according to the measured height parameter, and the threshold for judging whether the target human body has fallen is determined by the various parameters of the target human body, which is applicable to a variety of implementation scenarios; when the estimated height parameter is within the target threshold interval, the relationship between multiple posture offset parameters and the preset posture threshold is combined to detect whether the target human body has fallen. Compared with the prior art, the key features of human body posture are fully utilized to improve the accuracy of fall detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solution of the embodiment of the present invention, the following is a brief introduction to the drawings required for use in the embodiment of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0023] Figure 1 is a flow chart of a fall detection method provided by an embodiment of the present invention;

[0024] Figure 2 is a flow chart of another fall detection method provided by an embodiment of the present invention;

[0025] Figure 3 is a flow chart of another fall detection method provided by an embodiment of the present invention;

[0026] Figure 4 is a structural schematic diagram of a fall detection device provided by an embodiment of the present invention;

[0027] Figure 5 It is a structural schematic diagram of a fall detection device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The features and exemplary embodiments of various aspects of the present invention will be described in detail below. In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present invention, rather than to limit the present invention. For those skilled in the art, the present invention can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present invention by illustrating examples of the present invention.

[0029] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0030] Fall detection, as the name suggests, is to detect whether the target human body is in a falling state. It is widely used in home care, security monitoring, sports training and other fields.

[0031] Most existing technologies use image processing or machine learning methods to capture the human target frame, and use a fixed height threshold or machine learning model to determine whether the human body has fallen. However, the above solutions usually have the following problems: lack of analysis of key features of human posture, such as the relative position relationship of key parts such as the torso, legs, head and neck, resulting in misjudgment of human height and posture, affecting the detection accuracy; lack of adaptive capabilities for different scenes. Due to different camera positions, fields of view, brightness, and resolution, the height, body shape, and posture of the target person in the camera are also different. Using unified model parameters for detection will inevitably lead to a decrease in accuracy in some scenes.

[0032] Based on this, Figure 1 As shown, an embodiment of the present invention provides a fall detection method, including:

[0033] S101, based on a pre-acquired target image, extracting a plurality of key point information of a target human body in the target image.

[0034] In some embodiments, 19 key points of the human body are extracted from the target image according to a preset algorithm, as shown in the following table:

[0035] Table 1 Definition of key points of human body

[0036]

[0037]

[0038] For the key point with serial number i, it is represented by K[i] in the embodiment of the present invention. Each key point information consists of a triple (K[i]x, K[i]y, K[i]p) data. K[i]x represents the horizontal coordinate of the key point, that is, the number of pixels from the left edge of the image, K[i]y represents the vertical coordinate of the key point, that is, the number of pixels from the upper edge of the image, and K[i]p represents the confidence of the key point.

[0039] It should be noted that the confidence interval of K[i]p is [0, 1], where 0 means that the key point is not detected and 1 means that the key point position is 100% certain.

[0040] Specifically, the embodiment of the present invention provides 19 human body key point information, and in other embodiments, it may be other numbers and corresponding positions, and the present invention is not limited to this.

[0041] The fall detection method provided by the embodiment of the present invention calculates posture features such as estimated height parameters, actual height parameters, and posture deviation parameters of the degree of deviation from the upright state of the head, neck, trunk, thighs and calves of the human body through 19 key points of the human body. Based on the above features, combined with the threshold adaptive algorithm, the detection threshold is adjusted during the detection process to improve the detection accuracy of complex scenes.

[0042] S102, determining an estimated height parameter, a measured height parameter, and posture offset parameters of a plurality of postures of a target human body according to information of a plurality of key points, wherein the posture offset parameter is used to characterize an angle between a target posture and the ground, and the target posture is any one of the plurality of postures.

[0043] In some embodiments, determining the estimated height parameter of the target person according to the multiple key point information may include:

[0044] Among the multiple key point information, the key point information of the target human body limbs is screened out to determine the first key point information set;

[0045] The height difference between the highest key point and the lowest key point in the first key point information set is determined as the estimated height parameter of the target human body.

[0046] The fall detection method provided by the embodiment of the present invention takes the extraction of human body posture features as a starting point, and improves the accuracy of the estimated height parameters of the target human body based on the analysis of the key features of the human body posture.

[0047] In some embodiments, before determining the measured height parameter of the target human body and the posture offset parameters of the plurality of postures according to the plurality of key point information, the method may further include:

[0048] Select a target key point information set of a target part from among the multiple key point information, where the target part is any one of the head and neck part, trunk part, left thigh part, left calf part, right thigh part and right calf part of the target human body;

[0049] In the target key point information set, determine the coordinates of the highest point and the coordinates of the lowest point.

[0050] The fall detection method provided by an embodiment of the present invention determines the highest point coordinates and the lowest point coordinates of the position in the set for the target key point information corresponding to each target part, so as to obtain the posture characteristics and length information of each target part by analyzing the highest point coordinates and the lowest point coordinates determined for each target part.

[0051] In some embodiments, determining the measured height parameter of the target person according to the multiple key point information may include:

[0052] Calculate the Euclidean distance between the highest coordinate and the lowest coordinate in each key point information set to determine the length of each target part;

[0053] Compare the sum of the length of the left thigh and the length of the left calf with the sum of the length of the right thigh and the length of the right calf, and use the longer one as the lower limb length of the target human body;

[0054] The length of the head and neck, the length of the torso, and the length of the lower limbs are added together according to a preset ratio to determine the actual height parameters of the target human body.

[0055] It should be noted that the effectiveness of the fall detection method provided in the embodiment of the present invention is based on the following assumptions:

[0056] 1) The target human body proportions roughly conform to the standard human body proportions;

[0057] 2) The posture deviation parameters of the target body when it falls will be significantly different from those when it is standing upright or sitting or lying down;

[0058] 3) At a certain moment, the camera can clearly capture all the key points of the target person.

[0059] The measured height parameters of the target human body can be determined by the following steps:

[0060] Step A, by calculating the Euclidean distance between the facial key points and the neck key points, the length of the target human head and neck is determined.

[0061] In some embodiments, facial key points are determined when one of the following conditions is met, and the priority of the conditions is from high to low. When multiple conditions are met at the same time, the one with the highest priority shall prevail.

[0062] Condition a: If key points 18 and 19 are both clear, take the midpoint between them as the facial key point; if one of them is clear, take the clear point as the facial key point.

[0063] Condition b: Key point 1 is clear and is taken as the facial key point.

[0064] Condition c: If both key points 16 and 17 are clear, the midpoint between them is taken as the facial key point; if one of them is clear, the clear point is taken as the facial key point.

[0065] If the above conditions are not met, the facial key points cannot be calculated. In this case, it is considered that assumption 3 is not met and the threshold is not updated.

[0066] In some embodiments, when one of the following conditions is met, the key point of the neck is determined, and the priority of the conditions is from high to low. When multiple conditions are met at the same time, the one with the highest priority shall prevail.

[0067] Condition a: Key point 2 is clear and is taken as the neck key point.

[0068] Condition b: If key points 3 and 6 are both clear, take the midpoint between them as the neck key point; if one of them is clear, take the clear point as the neck key point.

[0069] If the above conditions are not met, the neck key point cannot be calculated. In this case, it is considered that assumption 3 is not met and the threshold is not updated.

[0070] Step B: Determine the length of the target human torso by calculating the Euclidean distance between the neck key point and the hip key point.

[0071] In some embodiments, when one of the following conditions is met, the hip key point is determined, and the priority of the conditions is from high to low. When multiple conditions are met at the same time, the one with the highest priority shall prevail.

[0072] Condition a: Key point 9 is clear and is taken as the hip key point.

[0073] Condition b: If key points 10 and 13 are both clear, take the midpoint between them as the hip key point; if one of them is clear, take the clear point as the hip key point.

[0074] If the above conditions are not met, the hip key points cannot be calculated. In this case, it is considered that assumption 3 is not met and the threshold is not updated.

[0075] Step C: Determine the length of the left thigh of the target body by calculating the Euclidean distance between the left hip key point and the left knee key point, determine the length of the left calf of the target body by calculating the Euclidean distance between the left knee key point and the left ankle key point, and determine the length of the left leg of the target body by adding the above-mentioned length of the left thigh to the length of the left calf.

[0076] In some embodiments, the left thigh length refers to the Euclidean distance between key points 13 and 14 , and the left calf length refers to the Euclidean distance between key points 14 and 15 .

[0077] If the above key points are not clear, it is considered that assumption 3 is not met and the threshold is not updated.

[0078] Step D: Determine the length of the right thigh of the target body by calculating the Euclidean distance between the right hip key point and the right knee key point, determine the length of the right calf of the target body by calculating the Euclidean distance between the right knee key point and the right ankle key point, and determine the length of the right leg of the target body by adding the above-mentioned length of the right thigh to the length of the right calf.

[0079] Step E: Compare the length of the left leg and the length of the right leg of the target human body, and use the longer leg length as the lower limb length of the target human body.

[0080] In some embodiments, the right thigh length refers to the Euclidean distance between key points 10 and 11 , and the left calf length refers to the Euclidean distance between key points 11 and 12 .

[0081] If the above key points are not clear, it is considered that assumption 3 is not met and the threshold is not updated.

[0082] Step F: Add the length of the head and neck, the length of the torso, and the length of the lower limbs according to a preset ratio to determine the actual height parameter of the target human body.

[0083] In some embodiments, determining the posture offset parameters of multiple postures of the target human body according to the multiple key point information may include:

[0084] Obtain multiple target key point information sets;

[0085] For each target key point information set, the absolute value of the ratio of the difference between the horizontal coordinate and the vertical coordinate of the highest position coordinate and the lowest position point coordinate is taken to determine the posture offset parameter of the target part.

[0086] In some real-time examples, the posture offset parameters corresponding to multiple postures can be determined by the following steps:

[0087] ① Head and neck posture offset parameter F[1]

[0088] When one of the following conditions is met, F[1] is calculated. The priority of the conditions is from high to low. When multiple conditions are met at the same time, the one with the highest priority takes precedence.

[0089] Condition a: k[2].p>0.5 and(k

[18] .p>0.5 or k

[19] .p>0.5);

[0090] At this time, if k

[18] .p and k

[19] .p are both greater than 0.5, let k[m] be the midpoint between k

[18] and k

[19] , otherwise let k[m] be the point greater than 0.5;

[0091] Then F[1]=|k[2].xk[m].x| / (k[2].yk[m].y).

[0092] Condition b: k[1].p>0.5 and k[2].p>0.5;

[0093] At this time, when the confidence of k[1].p and k[2].p both exceed 0.5, it means that the front face can be clearly seen;

[0094] Then F[1]=|k[2].xk[1].x| / (k[2].yk[1].y).

[0095] Condition c: k[2].p>0.5 and(k

[16] .p>0.5 or k

[17] .p>0.5);

[0096] At this time, if k

[16] .p and k

[17] .p are both greater than 0.5, let k[m] be the midpoint between k

[16] and k

[17] , otherwise let k[m] be the point between them that is greater than 0.5;

[0097] Then F[1]=|k[2].xk[m].x| / (k[2].yk[m].y).

[0098] Condition c means that the human eye can be clearly seen, but there may be partial occlusion, resulting in condition a not being met.

[0099] When the above conditions are not met, F[1] is not available.

[0100] It should be noted that in the formula for determining F[1] under conditions a, b, and c, the numerator takes an absolute value, but the denominator does not take an absolute value.

[0101] ② The posture offset parameter F[2] of the trunk

[0102] When one of the following conditions is met, F[2] is calculated. The priority of the conditions is from high to low. When multiple conditions are met at the same time, the one with the highest priority takes precedence.

[0103] Condition a: k[2].p>0.5 and k[9].p>0.5;

[0104] F[2]=|k[9].xk[2].x| / (k[9].yk[2].y).

[0105] Condition b: k[2].p>0.5 and(k

[10] .p>0.5 or k

[13] .p>0.5);

[0106] If k

[10] .p>0.5 and k

[13] .p>0.5, let k[m] be the midpoint between the two, otherwise let k[m] be the point where the p is greater than 0.5;

[0107] F[2]=|k[m].xk[2].x| / (k[m].yk[2].y).

[0108] If the above conditions are not met, F[2] is not available.

[0109] ③ Posture offset parameter F[3] of the right thigh

[0110] k

[10] .p>0.5 and k

[11] .p>0.5;

[0111] F[3]=|k

[11] .xk

[10] .x| / (k

[11] .yk

[10] .y).

[0112] If the above conditions are not met, F[3] is not available.

[0113] ④ Posture offset parameter F[4] of the right calf

[0114] k

[11] .p>0.5 and k

[12] .p>0.5;

[0115] F[4]=|k

[12] .xk

[11] .x| / k

[12] .yk

[11] .y.

[0116] If the above conditions are not met, F[4] is not available.

[0117] ⑤ Posture offset parameter F of the left thigh[5]

[0118] k

[13] .p>0.5 and k

[14] .p>0.5;

[0119] F[5]=|k

[14] .xk

[13] .x| / k

[14] .yk

[13] .y.

[0120] If the above conditions are not met, F[5] is not available.

[0121] ⑥ Posture offset parameter F[6] of the left calf

[0122] k

[14] .p>0.5 and k

[15] .p>0.5;

[0123] F[6]=|k

[15] .xk

[14] .x| / (k

[15] .yk

[14] .y).

[0124] If the above conditions are not met, F[6] is not available.

[0125] The fall detection method provided by the embodiment of the present invention, in the process of detecting the estimated height parameters and the measured height parameters of the target human body, provides implementation methods under multiple conditions for the length detection of each target part, thereby improving the adaptability of the method and the accuracy of the measured height parameters.

[0126] S103, determining a threshold range for the target person to fall according to the measured height parameter.

[0127] In some embodiments, the minimum value of the threshold interval may be 0.3*actual height parameter, and the maximum value of the threshold interval may be 0.6*actual height parameter, which is not limited in the present invention.

[0128] It should be noted that, in the fall detection method provided by the embodiment of the present invention, if multiple people appear in the same frame image, each person is taken as a target person, and the threshold range for the target person to fall is determined by measuring the height parameters, and adaptive threshold detection is generated for each person.

[0129] S104, when the estimated height parameter is less than the minimum value of the threshold interval, it is determined that the target human body is in a falling state; when the estimated height parameter is greater than the maximum value of the threshold interval, it is determined that the target human body is not in a falling state.

[0130] S105, when the estimated height parameter is within the threshold range, determining whether the target human body falls based on relationship information between a plurality of posture offset parameters and preset posture thresholds.

[0131] In some embodiments, when the measured height parameter is within the threshold range, determining whether the target human body falls based on the relationship information between the plurality of posture offset parameters and the preset posture threshold value may include:

[0132] When the measured height parameter is within the target threshold range, at least one of the following conditions is met, and the target person is determined to have fallen:

[0133] The posture deviation parameters of the trunk are within the posture threshold range;

[0134] The posture deviation parameters of the left thigh and left calf are both within the posture threshold range;

[0135] The posture deviation parameters of the right thigh and right calf are both within the posture threshold range;

[0136] The posture deviation parameters of the head and neck are within the posture threshold range.

[0137] In some embodiments, the posture determination rule may include the following steps:

[0138] Step A, first calculate conditions a, b and c

[0139] Condition a: F[2] can be used AND 0<=F[2]<=T3;

[0140] Condition b: F[3] is available AND F[4] is available 0<=F[3]<=T3 AND 0<=F[4]<=T3;

[0141] Condition c: F[5] is available AND F[6] is available 0<=F[5]<=T3 AND 0<=F[6]<=T3;

[0142] If one of the three conditions is met, it is determined not to be a fall and the algorithm ends, otherwise it goes to step B.

[0143] Step B: If F[2] satisfies AND(NOT 0<=F[2]<=T3), it is determined to be a fall, otherwise, go to step C.

[0144] Step C: If F[1] satisfies AND(NOT 0<=F[1]<=T3), it is determined to be a fall and the algorithm ends. Otherwise, it goes to step D.

[0145] Step D: When none of the above conditions are met, it is determined to be a non-fall.

[0146] The fall detection method proposed in the embodiment of the present invention mainly consists of three aspects: extraction of human posture features, fall detection method and threshold adaptive algorithm. Through 19 key points of the human body, the posture features such as estimated height parameters, actual height parameters, and posture deviation parameters of the degree of deviation from the upright state of the head, neck, trunk, thighs and calves are calculated. Based on the above features, combined with the threshold adaptive algorithm, the detection threshold is adjusted during the detection process to improve the detection accuracy of complex scenes.

[0147] The fall detection method provided by the embodiment of the present invention is described in detail below with reference to the accompanying drawings and specific embodiments.

[0148] like Figure 2 As shown, the fall detection method provided by the embodiment of the present invention may include the following steps:

[0149] S201, reading a target image in a pre-acquired video, where the target image may be any frame image in the pre-acquired video.

[0150] S202, determining a target human body in a target image, and extracting a plurality of key points of the human body.

[0151] In some embodiments, 19 key points of the human body are extracted from the target image according to a preset algorithm, as shown in the following table:

[0152] Table 1 Definition of key points of human body

[0153]

[0154] For the key point with serial number i, it is represented by K[i] in the embodiment of the present invention. Each key point information consists of a triple (K[i]x, K[i]y, K[i]p) data. K[i]x represents the horizontal coordinate of the key point, that is, the number of pixels from the left edge of the image, K[i]y represents the vertical coordinate of the key point, that is, the number of pixels from the upper edge of the image, and K[i]p represents the confidence of the key point.

[0155] It should be noted that the confidence interval of K[i]p is [0, 1], where 0 means that the key point is not detected and 1 means that the key point position is 100% certain.

[0156] Specifically, the embodiment of the present invention provides 19 human body key point information, and in other embodiments, it may be other numbers and corresponding positions, and the present invention is not limited to this.

[0157] S203, determining an estimated height parameter of a human body posture and posture offset parameters corresponding to a plurality of postures through posture feature extraction.

[0158] 1) The estimated height parameters can be determined by the following steps:

[0159] Step A: obtain the information of 15 other human key points besides K[4], K[5], K[7], and K[8].

[0160] Among them, K[4], K[5], K[7], and K[8] represent the key point information of the right elbow, right wrist, left elbow, and left wrist of the human body respectively.

[0161] Step B: The height difference between the highest point and the lowest point of the above 15 human body key point information is used as the estimated height parameter h of the target human body.

[0162] 2) The posture offset parameters corresponding to multiple postures can be determined by the following steps:

[0163] ① Head and neck posture offset parameter F[1]

[0164] When one of the following conditions is met, F[1] is calculated. The priority of the conditions is from high to low. When multiple conditions are met at the same time, the one with the highest priority takes precedence.

[0165] Condition a: k[2].p>0.5 and(k

[18] .p>0.5 or k

[19] .p>0.5);

[0166] At this time, if k

[18] .p and k

[19] .p are both greater than 0.5, let k[m] be the midpoint between k

[18] and k

[19] , otherwise let k[m] be the point greater than 0.5;

[0167] Then F[1]=|k[2].xk[m].x| / (k[2].yk[m].y).

[0168] Condition b: k[1].p>0.5 and k[2].p>0.5;

[0169] At this time, when the confidence of k[1].p and k[2].p both exceed 0.5, it means that the front face can be clearly seen;

[0170] Then F[1]=|k[2].xk[1].x| / (k[2].yk[1].y).

[0171] Condition c: k[2].p>0.5 and(k

[16] .p>0.5 or k

[17] .p>0.5);

[0172] At this time, if k

[16] .p and k

[17] .p are both greater than 0.5, let k[m] be the midpoint between k

[16] and k

[17] , otherwise let k[m] be the point between them that is greater than 0.5;

[0173] Then F[1]=|k[2].xk[m].x| / (k[2].yk[m].y).

[0174] Condition c means that the human eye can be clearly seen, but there may be partial occlusion, resulting in condition a not being met.

[0175] When the above conditions are not met, F[1] is not available.

[0176] It should be noted that in the formula for determining F[1] under conditions a, b, and c, the numerator takes an absolute value, but the denominator does not take an absolute value.

[0177] ② The posture offset parameter F[2] of the trunk

[0178] When one of the following conditions is met, F[2] is calculated. The priority of the conditions is from high to low. When multiple conditions are met at the same time, the one with the highest priority takes precedence.

[0179] Condition a: k[2].p>0.5 and k[9].p>0.5;

[0180] F[2]=|k[9].xk[2].x| / (k[9].yk[2].y).

[0181] Condition b: k[2].p>0.5 and(k

[10] .p>0.5 or k

[13] .p>0.5);

[0182] If k

[10] .p>0.5 and k

[13] .p>0.5, let k[m] be the midpoint between the two, otherwise let k[m] be the point where the p is greater than 0.5;

[0183] F[2]=|k[m].xk[2].x| / (k[m].yk[2].y).

[0184] If the above conditions are not met, F[2] is not available.

[0185] ③ Posture offset parameter F of the right thigh[3]

[0186] k

[10] .p>0.5 and k

[11] .p>0.5;

[0187] F[3]=|k

[11] .xk

[10] .x| / (k

[11] .yk

[10] .y).

[0188] If the above conditions are not met, F[3] is not available.

[0189] ④ Posture offset parameter F[4] of the right calf

[0190] k

[11] .p>0.5 and k

[12] .p>0.5;

[0191] F[4]=|k

[12] .xk

[11] .x| / k

[12] .yk

[11] .y.

[0192] If the above conditions are not met, F[4] is not available.

[0193] ⑤ Posture offset parameter F of the left thigh[5]

[0194] k

[13] .p>0.5 and k

[14] .p>0.5;

[0195] F[5]=|k

[14] .xk

[13] .x| / k

[14] .yk

[13] .y.

[0196] If the above conditions are not met, F[5] is not available.

[0197] ⑥ Posture offset parameter F[6] of the left calf

[0198] k

[14] .p>0.5 and k

[15] .p>0.5;

[0199] F[6]=|k

[15] .xk

[14] .x| / (k

[15] .yk

[14] .y).

[0200] If the above conditions are not met, F[6] is not available.

[0201] It should be noted that when the human body tends to be upright, the value of F[i] will tend to 0; when the human body tends to lie horizontally, the value of F[i] will tend to infinity; when the up and down positions of some key points of the human body are reversed, the value of F[i] will be negative.

[0202] S204, adaptively adjusting the judgment threshold of fall detection according to the shape of the target human body in the target image, and determining the threshold range of the target human body falling.

[0203] In some embodiments, the adaptive adjustment of the threshold determines the measured height parameters of the target human body by identifying clear key points of the human body, and further determines the threshold interval [TI, T2] for the target human body to fall based on the measured height parameters.

[0204] It should be noted that the effectiveness of threshold adaptive adjustment is based on the following assumptions:

[0205] 1) The target human body proportions roughly conform to the standard human body proportions;

[0206] 2) The posture deviation parameters of the target body when it falls will be significantly different from those when it is standing upright or sitting or lying down;

[0207] 3) At a certain moment, the camera can clearly capture all the key points of the target person.

[0208] The measured height parameter H of the target human body can be determined by the following steps:

[0209] Step A, by calculating the Euclidean distance between the facial key points and the neck key points, the length of the target human head and neck is determined.

[0210] In some embodiments, facial key points are determined when one of the following conditions is met, and the priority of the conditions is from high to low. When multiple conditions are met at the same time, the one with the highest priority shall prevail.

[0211] Condition a: If key points 18 and 19 are both clear, take the midpoint between them as the facial key point; if one of them is clear, take the clear point as the facial key point.

[0212] Condition b: Key point 1 is clear and is taken as the facial key point.

[0213] Condition c: If both key points 16 and 17 are clear, the midpoint between them is taken as the facial key point; if one of them is clear, the clear point is taken as the facial key point.

[0214] If the above conditions are not met, the facial key points cannot be calculated. In this case, it is considered that assumption 3 is not met and the threshold is not updated.

[0215] In some embodiments, when one of the following conditions is met, the key point of the neck is determined, and the priority of the conditions is from high to low. When multiple conditions are met at the same time, the one with the highest priority shall prevail.

[0216] Condition a: Key point 2 is clear and is taken as the neck key point.

[0217] Condition b: If key points 3 and 6 are both clear, take the midpoint between them as the neck key point; if one of them is clear, take the clear point as the neck key point.

[0218] If the above conditions are not met, the neck key point cannot be calculated. In this case, it is considered that assumption 3 is not met and the threshold is not updated.

[0219] Step B: Determine the length of the target human torso by calculating the Euclidean distance between the neck key point and the hip key point.

[0220] In some embodiments, when one of the following conditions is met, the hip key point is determined, and the priority of the conditions is from high to low. When multiple conditions are met at the same time, the one with the highest priority shall prevail.

[0221] Condition a: Key point 9 is clear and is taken as the hip key point.

[0222] Condition b: If key points 10 and 13 are both clear, take the midpoint between them as the hip key point; if one of them is clear, take the clear point as the hip key point.

[0223] If the above conditions are not met, the hip key points cannot be calculated. In this case, it is considered that assumption 3 is not met and the threshold is not updated.

[0224] Step C: Determine the length of the left thigh of the target body by calculating the Euclidean distance between the left hip key point and the left knee key point, determine the length of the left calf of the target body by calculating the Euclidean distance between the left knee key point and the left ankle key point, and determine the length of the left leg of the target body by adding the above-mentioned length of the left thigh to the length of the left calf.

[0225] In some embodiments, the left thigh length refers to the Euclidean distance between key points 13 and 14 , and the left calf length refers to the Euclidean distance between key points 14 and 15 .

[0226] If the above key points are not clear, it is considered that assumption 3 is not met and the threshold is not updated.

[0227] Step D: Determine the length of the right thigh of the target body by calculating the Euclidean distance between the right hip key point and the right knee key point, determine the length of the right calf of the target body by calculating the Euclidean distance between the right knee key point and the right ankle key point, and determine the length of the right leg of the target body by adding the above-mentioned length of the right thigh to the length of the right calf.

[0228] Step E: Compare the length of the left leg and the length of the right leg of the target human body, and use the longer leg length as the lower limb length of the target human body.

[0229] In some embodiments, the right thigh length refers to the Euclidean distance between key points 10 and 11 , and the left calf length refers to the Euclidean distance between key points 11 and 12 .

[0230] If the above key points are not clear, it is considered that assumption 3 is not met and the threshold is not updated.

[0231] Step F: Add the length of the head and neck, the length of the torso, and the length of the lower limbs according to a preset ratio to determine the actual height parameter H of the target human body.

[0232] In some embodiments, the actual height parameter H of the target human body=1.5*length of the head and neck portion+length of the trunk portion+length of the lower limb portion.

[0233] In some embodiments, the threshold interval [T1, T2] for the target person to fall is determined according to a preset ratio and based on the actual height parameter of the target person.

[0234] For example, the minimum value T1 of the threshold interval is 0.3*actual height parameter, and the maximum value T2 of the threshold interval is 0.6*actual height parameter.

[0235] In some embodiments, the preset posture threshold interval is [0, 1], and the value of the posture threshold is independent of the height and body shape of the target person.

[0236] It should be noted that, in the fall detection method provided by the embodiment of the present invention, if multiple people appear in the same frame image, each person is taken as a target human body and an adaptive threshold detection is generated for each person.

[0237] S205, such as Figure 3 As shown, the fall detection of the target human body is performed in combination with the threshold interval of the target human body falling.

[0238] In some embodiments, when the estimated height parameter is less than the minimum value T1 of the threshold interval, it is determined that the target human body is in a falling state;

[0239] When the estimated height parameter is greater than the maximum value T2 of the threshold interval, determining that the target human body is in a non-falling state;

[0240] When the estimated height parameter is within the threshold range, it is determined whether the target human body has fallen based on a preset posture discrimination rule.

[0241] In some embodiments, when the estimated height parameter is within the threshold range, determining whether the target human body has fallen based on a preset posture discrimination rule may include the following steps:

[0242] Step A, first calculate conditions a, b and c

[0243] Condition a: F[2] can be used AND 0<=F[2]<=T3;

[0244] Condition b: F[3] is available AND F[4] is available 0<=F[3]<=T3 AND 0<=F[4]<=T3;

[0245] Condition c: F[5] is available AND F[6] is available 0<=F[5]<=T3 AND 0<=F[6]<=T3;

[0246] If one of the three conditions is met, it is determined not to be a fall and the algorithm ends, otherwise it goes to step B.

[0247] Step B: If F[2] satisfies AND(NOT 0<=F[2]<=T3), it is determined to be a fall, otherwise, go to step C.

[0248] Step C: If F[1] satisfies AND(NOT 0<=F[1]<=T3), it is determined to be a fall and the algorithm ends. Otherwise, it goes to step D.

[0249] Step D: When none of the above conditions are met, it is determined to be a non-fall.

[0250] In different embodiments, the field of view of the camera varies in distance, and the height and posture of the target human body are also different, so using a unified threshold for fall detection will reduce the accuracy. Therefore, the threshold adaptive algorithm provided by the embodiment of the present invention can adapt to different scenarios, enhance the adaptability of the algorithm, and improve the accuracy of the detection threshold in different scenarios.

[0251] like Figure 4 As shown, an embodiment of the present invention provides a fall detection device, the device comprising:

[0252] The extraction unit 401 is used to extract a plurality of key point information of a target human body in the target image based on the pre-acquired target image;

[0253] A first determining unit 402 is used to determine an estimated height parameter, a measured height parameter and a posture offset parameter of a plurality of postures of a target human body according to the plurality of key point information, wherein the posture offset parameter is used to characterize an angle between a target posture and the ground, and the target posture is any one of the plurality of postures;

[0254] The second determining unit 403 is used to determine the threshold interval of the target human body falling according to the measured height parameter;

[0255] The third determining unit 404 is used to determine that the target human body is in a falling state when the estimated height parameter is less than the minimum value of the threshold interval, and to determine that the target human body is in a non-falling state when the estimated height parameter is greater than the maximum value of the threshold interval;

[0256] The fourth determination unit 405 is used to determine whether the target human body falls based on the relationship information between the multiple posture offset parameters and the preset posture threshold when the estimated height parameter is within the threshold range.

[0257] In some embodiments, the first determining unit 402 is specifically configured to:

[0258] Among the multiple key point information, the key point information of the target human body limbs is screened out to determine the first key point information set;

[0259] The height difference between the highest key point and the lowest key point in the first key point information set is determined as the estimated height parameter of the target human body.

[0260] In some embodiments, the first determining unit 402 is specifically configured to:

[0261] Select a target key point information set of a target part from among the multiple key point information, where the target part is any one of the head and neck part, trunk part, left thigh part, left calf part, right thigh part and right calf part of the target human body;

[0262] In the target key point information set, determine the coordinates of the highest point and the coordinates of the lowest point.

[0263] In some embodiments, the first determining unit 402 is specifically configured to:

[0264] Calculate the Euclidean distance between the highest coordinate and the lowest coordinate in each key point information set to determine the length of each target part;

[0265] Compare the sum of the length of the left thigh and the length of the left calf with the sum of the length of the right thigh and the length of the right calf, and use the longer one as the lower limb length of the target human body;

[0266] The length of the head and neck, the length of the torso, and the length of the lower limbs are added together according to a preset ratio to determine the actual height parameters of the target human body.

[0267] In some embodiments, the first determining unit 402 is specifically configured to:

[0268] Obtain multiple target key point information sets;

[0269] For each target key point information set, the absolute value of the ratio of the difference between the horizontal coordinate and the vertical coordinate of the highest position coordinate and the lowest position point coordinate is taken to determine the posture offset parameter of the target part.

[0270] In some embodiments, the fourth determining unit 405 is specifically configured to:

[0271] When the measured height parameter is within the target threshold range, at least one of the following conditions is met, and the target person is determined to have fallen:

[0272] The posture deviation parameters of the trunk are within the posture threshold range;

[0273] The posture deviation parameters of the left thigh and left calf are both within the posture threshold range;

[0274] The posture deviation parameters of the right thigh and right calf are both within the posture threshold range;

[0275] The posture deviation parameters of the head and neck are within the posture threshold range.

[0276] Other details of the fall detection device provided according to an embodiment of the present invention are combined with the above Figure 1-Figure 3 The fall detection method according to the embodiment of the present invention described is similar and will not be described in detail here.

[0277] Figure 5 A schematic diagram of the hardware structure of fall detection provided by an embodiment of the present invention is shown.

[0278] Combination Figure 1-Figure 4 The fall detection method and apparatus described in the embodiments of the present invention may be implemented by a fall detection device. Figure 5 FIG. 5 is a schematic diagram showing a hardware structure 500 of a fall detection device according to an embodiment of the invention.

[0279] The fall detection device may include a processor 501 and a memory 502 storing computer program instructions.

[0280] Specifically, the processor 501 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiment of the present invention.

[0281] The memory 502 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 502 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In one example, the memory 502 may include a removable or non-removable (or fixed) medium, or the memory 502 is a non-volatile solid-state memory. The memory 502 may be inside or outside the integrated gateway disaster recovery device.

[0282] In one example, the memory 502 may be a read-only memory (ROM). In one example, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically rewritable ROM (EAROM), or a flash memory, or a combination of two or more of these.

[0283] The processor 501 reads and executes the computer program instructions stored in the memory 502 to implement Figure 1 The method / steps S101 to S105 in the embodiment shown in the figure are achieved Figure 1 The corresponding technical effects achieved by executing the methods / steps in the examples shown are not repeated here for the sake of brevity.

[0284] In one example, the fall detection device may further include a communication interface 503 and a bus 510. Figure 5 As shown, the processor 501, the memory 502, and the communication interface 503 are connected via a bus 510 and communicate with each other.

[0285] The communication interface 503 is mainly used to implement the communication between the modules, devices, units and / or equipment in the embodiment of the present invention.

[0286] Bus 510 includes hardware, software or both, and couples the components of online data traffic billing equipment to each other. For example, but not limitation, the bus may include Accelerated Graphics Port (AGP) or other graphics bus, Enhanced Industry Standard Architecture (EISA) bus, Front Side Bus (FSB), Hyper Transport (HT) interconnection, Industry Standard Architecture (ISA) bus, InfiniBand interconnection, Low Pin Count (LPC) bus, Memory bus, Micro Channel Architecture (MCA) bus, Peripheral Component Interconnect (PCI) bus, PCI-Express (PCI-X) bus, Serial Advanced Technology Attachment (SATA) bus, Video Electronics Standards Association Local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 510 may include one or more buses. Although the embodiment of the present invention describes and shows a specific bus, the present invention considers any suitable bus or interconnection.

[0287] The fall detection device provided by the embodiment of the present invention can adapt to different scenarios, avoiding the problem of reduced accuracy of fall detection due to the different heights and postures of the target human body and the different fields of view of the camera, thereby enhancing the adaptability of the algorithm and improving the accuracy of the detection threshold in different scenarios.

[0288] In addition, in combination with the fall detection method in the above embodiments, the embodiment of the present invention can provide a computer storage medium for implementation. The computer storage medium stores computer program instructions; when the computer program instructions are executed by a processor, any one of the fall detection methods in the above embodiments is implemented.

[0289] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0290] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), appropriate firmware, plug-in, function card, etc. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (Radio Frequency, RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0291] It should also be noted that the exemplary embodiments mentioned in the present invention describe some methods or systems based on a series of steps or devices. However, the present invention is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiments, or in a different order from the embodiments, or several steps can be performed simultaneously.

[0292] Aspects of the present disclosure are described above with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0293] The above is only a specific implementation of the present invention. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the system, module and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. It should be understood that the protection scope of the present invention is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be covered within the protection scope of the present invention.

Claims

1. A fall detection method, characterized in that: The method comprises: Based on the pre-acquired target image, extracting a plurality of key point information of the target human body in the target image; According to the multiple key point information, the estimated height parameter, the measured height parameter and the posture offset parameters of multiple postures of the target human body are determined, wherein the posture offset parameter is used to characterize the angle between the target posture and the ground, and the target posture is any one of the multiple postures; wherein the measured height parameter is determined, including: in the target key point information set of the target part of the multiple key point information, calculating the Euclidean distance between the coordinates of the highest point and the coordinates of the lowest point, determining the length of each of the target parts, comparing the sum of the length of the left thigh part and the length of the left calf part with the sum of the length of the right thigh part and the length of the right calf part, wherein the longer one is used as the length of the lower limbs of the target human body, adding the length of the head and neck part, the length of the torso part, and the length of the lower limbs according to a preset ratio to determine the measured height parameter of the target human body, the target part is any one of the head and neck part, the torso part, the left thigh part, the left calf part, the right thigh part and the right calf part of the target human body; Determining a threshold range for a target person to fall according to the measured height parameter; When the estimated height parameter is less than the minimum value of the threshold interval, it is determined that the target human body is in a falling state; when the estimated height parameter is greater than the maximum value of the threshold interval, it is determined that the target human body is in a non-falling state; When the estimated height parameter is within the threshold interval, whether the target human body falls is determined based on relationship information between a plurality of the posture offset parameters and a preset posture threshold.

2. The method according to claim 1, characterized in that Determining the estimated height parameter of the target human body according to the plurality of key point information includes: Among the plurality of key point information, filter out the key point information of the target human body limbs to determine a first key point information set; The height difference between the highest key point and the lowest key point in the first key point information set is determined as the estimated height parameter of the target human body.

3. The method according to claim 1, characterized in that Before determining the measured height parameter of the target human body and the posture offset parameters of multiple postures according to the multiple key point information, the method further includes: Selecting a target key point information set of the target part from the plurality of key point information; In the target key point information set, the coordinates of the highest point of the position and the coordinates of the lowest point of the position are determined.

4. The method according to claim 3, characterized in that: The step of determining the posture offset parameters of the plurality of postures of the target human body according to the plurality of key point information comprises: Acquire a plurality of target key point information sets; For each target key point information set, the absolute value of the ratio of the difference between the horizontal coordinate and the vertical coordinate of the highest point coordinate and the lowest point coordinate is taken to determine the posture offset parameter of the target part.

5. The method according to claim 3, characterized in that: The method of determining whether the target human body falls down based on relationship information between a plurality of the posture offset parameters and a preset posture threshold value when the estimated height parameter is within the threshold interval includes: When the estimated height parameter is within the threshold range, at least one of the following conditions is met, and it is determined that the target human body has fallen: The posture deviation parameter of the torso part is within the posture threshold range; The posture deviation parameters of the left thigh part and the left calf part are both within the posture threshold range; The posture deviation parameters of the right thigh part and the right calf part are both within the posture threshold range; The posture deviation parameter of the head and neck part is within the posture threshold range.

6. A fall detection device, characterized in that: The device comprises: An extraction unit, configured to extract a plurality of key point information of a target human body in the target image based on a pre-acquired target image; A first determination unit is used to determine the estimated height parameter, the measured height parameter and the posture offset parameters of multiple postures of the target human body according to the multiple key point information, wherein the posture offset parameter is used to characterize the angle between the target posture and the ground, and the target posture is any one of the multiple postures; wherein determining the measured height parameter includes: in the target key point information set of the target part of the multiple key point information, calculating the Euclidean distance between the coordinates of the highest point and the coordinates of the lowest point, determining the length of each of the target parts, comparing the sum of the length of the left thigh part and the length of the left calf part with the sum of the length of the right thigh part and the length of the right calf part, wherein the longer one is used as the length of the lower limb of the target human body, adding the length of the head and neck part, the length of the torso part, and the length of the lower limb according to a preset ratio to determine the measured height parameter of the target human body, and the target part is any one of the head and neck part, the torso part, the left thigh part, the left calf part, the right thigh part and the right calf part of the target human body; a second determination unit is used to determine the threshold interval of the target human body falling according to the measured height parameter; A third determining unit is used to determine that the target human body is in a falling state when the estimated height parameter is less than the minimum value of the threshold interval, and to determine that the target human body is in a non-falling state when the estimated height parameter is greater than the maximum value of the threshold interval; The fourth determination unit is used to determine whether the target human body falls based on the relationship information between the multiple posture offset parameters and the preset posture threshold when the estimated height parameter is within the threshold range.

7. The device according to claim 6, characterized in that The fourth determining unit is specifically configured to: When the estimated height parameter is within the threshold range, at least one of the following conditions is met, and it is determined that the target human body has fallen: The posture deviation parameter of the torso part is within the posture threshold range; The posture deviation parameters of the left thigh part and the left calf part are both within the posture threshold range; The posture deviation parameters of the right thigh part and the right calf part are both within the posture threshold range; The posture deviation parameter of the head and neck part is within the posture threshold range.

8. A fall detection device, characterized in that: The device comprises: a processor, and a memory storing computer program instructions; the processor reads and executes the computer program instructions to implement the fall detection method according to any one of claims 1 to 5.

9. A computer storage medium, characterized in that: The computer storage medium stores computer program instructions, and when the computer program instructions are executed by the processor, the fall detection method according to any one of claims 1 to 5 is implemented.

Images