A human body posture determination method and system for complex multi-person scenes

Abstract

The present invention discloses a method and system for determining human posture in complex multi-person scenes. The method first divides the obtained human body key points into several sets, such as: a set of key points of the upper body and a set of key points of the lower body, etc.; then extracts the statistical features of the geometric distribution of the human body key points in each set, such as: a minimum circumscribed rectangle, a convex hull, etc.; finally, the human body posture is determined based on the calculated statistical features, such as: the rotation angle of the minimum circumscribed rectangle, the horizontal angle of the convex hull, etc. The advantage of the present invention is that it can quickly and accurately determine the human body posture when the position of the human body key points is inaccurate or some key points are missing, and provide accurate information for the detection of abnormal behaviors such as falls and conflicts.

Description

Technical Field

[0001] The present invention belongs to the field of pattern recognition and artificial intelligence technology, and specifically relates to a robust human posture determination method, and more particularly to a human posture determination method and system for complex multi-person scenes. Background Art

[0002] Video surveillance equipment is widely deployed in various public places, such as kindergartens, nursing homes, train stations, and shopping malls. These devices generate massive amounts of surveillance data during daily operation, making manual monitoring of video scenes impractical. Semantic descriptions of human posture in surveillance scenes—that is, identifying different human postures—can help people quickly understand the status of individuals and events in the scene, significantly enhancing the real-time detection of unexpected and sudden situations.

[0003] Traditional manual monitoring methods are not only inefficient but also difficult to respond to emergencies promptly. With the development of artificial intelligence and pattern recognition technologies, several methods for human posture recognition have emerged. Depending on whether or not the human body is considered a "connecting rod" model, these methods can be roughly divided into two categories: those based on video feature extraction and those based on key point extraction. The following sections discuss these two types of methods.

[0004] Among methods based on video feature extraction, the paper "Action recognition by dense trajectories [C]. IEEE Conference on Computer Vision and Pattern Recognition" proposes using optical flow to track dense sampling points to obtain human motion trajectories and subsequently perform posture recognition. This method requires real-time tracking of sampling points to obtain human motion trajectories, but this tracking is prone to errors in scenes with pedestrians. The paper "Human posture recognition based on projection histogram and Support Vector Machine" uses an ellipse to fit the human contour, then constructs a histogram along the major and minor axes of the ellipse to describe the human shape. Finally, a support vector machine is used to recognize human posture. In situations where accurate bounding box extraction is impossible, such as when multiple people are biting or occluded, the extracted human shape will be severely altered, resulting in a sharp decline in posture recognition performance. The paper "A bio-inspired event-based size and position invariant human posture recognition algorithm" proposes a human posture recognition method based on the simplified segment Hausdorff distance. This method takes two consecutive image frames from a video sequence as input, compares the differences between the two frames to identify active objects, and decomposes the active object's contour into vector segments. This method simplifies calculations and improves efficiency, but it cannot detect body parts that do not move between previous and subsequent frames, resulting in a decrease in recognition accuracy. The paper "Posture recognition invariant to background, cloth textures, body size, and camera distance using morphological geometry" uses the length and width of the extracted human silhouette to identify human posture, avoiding the influence of detailed information such as clothing background. Since the length and width of the human silhouette can only roughly describe the human posture, the accuracy of posture recognition by this method is not high. Methods based on video feature extraction require that the video be tracked and shot before determining the human posture, or that the target's motion process be extracted through a tracking algorithm first. This makes such methods difficult to apply to complex scenes with large numbers of people where target tracking is difficult.

[0005] Among methods based on human keypoint extraction, the paper "Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields" proposes a model that simultaneously predicts the positions and positional relationships of body parts to extract keypoints. The paper "A Coarse-Fine Network for Keypoint Localization" proposes a coarse-to-fine multi-level supervised network (CFN) for extracting keypoints. These methods do not require target tracking and are therefore suitable for human pose recognition in multi-person scenarios. The paper "Neural Network Approach for 2-Dimension Person Pose Estimation With Encoded Mask and Keypoint Detection" uses a convolutional deep neural network to extract keypoints from image segmentation masks and learn the interconnected relationships between keypoints, achieving a combination of image segmentation and a bottom-up approach to extract keypoints. Most methods based on human keypoint extraction simply extract feature points without determining the human pose. However, in practical applications, simply obtaining a person's feature points is often insufficient. Determining the person's posture can help people quickly understand the state of individuals in a scene and the events occurring. It also serves as an important basis for intelligent monitoring systems to automatically analyze and judge the situation. Therefore, several methods have emerged for determining human posture based on key point information. The paper "Human posture classification using skeleton information" uses the distance between key points and the angle between the lines connecting them to determine human posture. However, in practical applications, errors in extracted key points and missing key points due to occlusion often occur, which can significantly reduce the performance of existing posture determination methods, or even lead to their failure. Summary of the Invention

[0006] The purpose of the present invention is to address the problems existing in the above-mentioned prior art and provide a method and system for determining human posture in complex multi-person scenes, so as to quickly understand the status of each individual in the scene and the events that have occurred.

[0007] The technical solution for achieving the purpose of the present invention is: a method for determining human posture in complex multi-person scenes, the method comprising the following steps:

[0008] Step 1: Detect key points of the human body and divide the key point sets;

[0009] Step 2: extract the statistical features of the geometric distribution of human body key points in each set;

[0010] Step 3: Determine the human body posture based on statistical features.

[0011] Furthermore, the step 1 of detecting the key points of the human body and dividing the key point sets includes the following specific processes:

[0012] Step 1-1: Construct a sample training set, which includes several human body images with labeled key points of the human body;

[0013] Step 1-2, training a deep convolutional neural network using the sample training set;

[0014] Steps 1-3: Use the trained deep convolutional neural network to detect key points in the human body image to be detected;

[0015] Steps 1-4, divide the key point set, specifically:

[0016] Eliminate the key points of non-human torso, including hands, to form a set of key points of the whole body;

[0017] The key points of the human torso are divided into an upper body key point set, a lower body key point set, a left arm key point set and a right arm key point set;

[0018] The lower body key points are further divided into a left leg key point set, a right leg key point set, a left thigh key point set, and a right thigh key point set.

[0019] Furthermore, the step 2 extracts the statistical features of the geometric distribution of the human body key points in each set, and the specific process includes:

[0020] Step 2-1-1, calculate the convex hull of each key point set;

[0021] Step 2-1-2, find the two key points with the largest distance from the key points that constitute the convex hull;

[0022] Step 2-1-3, calculate the angle between the line connecting the two key points and the horizontal direction when rotated in the clockwise direction, as the horizontal angle of the convex hull.

[0023] Furthermore, the human body posture is determined based on statistical features in step 3, specifically: the postures of the arms, legs, and the whole body are determined based on the horizontal angles of the convex hulls of the key point sets:

[0024] Assume that the horizontal angle of the convex hull of the whole body key point set is α0; the horizontal angle of the convex hull of the upper body key point set is α1; the horizontal angle of the convex hull of the lower body key point set is α2; the horizontal angle of the convex hull of the left arm key point set is α3; the horizontal angle of the convex hull of the right arm key point set is α4; the horizontal angle of the convex hull of the left leg key point set is α5; the horizontal angle of the convex hull of the right leg key point set is α6; the horizontal angle of the convex hull of the left thigh key point set is α7; the horizontal angle of the convex hull of the right thigh key point set is α8;

[0025] If |tanα3|∈(a3,b3) or |tanα4|∈(a3,b3), it is judged as an arm-raising posture; if |tanα5|∈(a5,b5) or |tanα6|∈(a5,b5), it is judged as a kicking posture; if |tanα0|∈(a0,b0), |tanα1|∈(a1,b1) and |tanα2|∈(a2,b2), it is judged as a standing posture; if |tanα7|∈[a7,b7] or |tanα8|∈[a7,b7], it is judged as a squatting posture; if |tanα0|∈[c0,d0], |tanα1|∈[c1,d1] and |tanα2|∈[c2,d2], it is judged as a lying posture.

[0026] Furthermore, the step 2 extracts the statistical features of the geometric distribution of the human body key points in each set, and the specific process includes:

[0027] Step 2-2-1, calculate the minimum bounding rectangle of each key point set;

[0028] Step 2-2-2, respectively calculate the angles formed by the long side and the short side of the minimum circumscribed rectangle rotated clockwise with the horizontal direction;

[0029] In step 2-2-3, take the smaller of the two included angles as the rotation angle of the minimum circumscribed rectangle.

[0030] Furthermore, the human body posture is determined based on statistical features in step 3, specifically: the posture of the arms, legs, and whole body is determined based on the rotation angle of the minimum circumscribed rectangle of each key point set:

[0031] Assume that the horizontal angle of the minimum circumscribed rectangle of the whole body key point set is β0; the horizontal angle of the minimum circumscribed rectangle of the upper body key point set is β1; the horizontal angle of the minimum circumscribed rectangle of the lower body key point set is β2; the horizontal angle of the minimum circumscribed rectangle of the left arm key point set is β3; the horizontal angle of the minimum circumscribed rectangle of the right arm key point set is β4; the horizontal angle of the minimum circumscribed rectangle of the left leg key point set is β5; the horizontal angle of the minimum circumscribed rectangle of the right leg key point set is β6; the horizontal angle of the minimum circumscribed rectangle of the left thigh key point set is β7; the horizontal angle of the minimum circumscribed rectangle of the right thigh key point set is β8;

[0032] like or It is determined to be an arm-raising posture; if or It is determined to be a kicking posture; if and If β1|∈[λ1,γ1], it is judged as a standing posture; if or It is judged as a squatting posture; if |β0|∈[θ0,Θ0], β1|∈[θ1,Θ1] and β2|∈[θ2,Θ2], it is judged as a lying posture.

[0033] A human body posture determination system for complex multi-person scenes, the system comprising:

[0034] Key point segmentation module, used to detect key points of the human body and divide the key point sets;

[0035] Statistical feature extraction module, used to extract statistical features of geometric distribution of key points of human body in each set;

[0036] The posture determination module is used to determine the human posture based on statistical features.

[0037] Compared with the existing technology, the present invention has the following significant advantages: 1) The present invention adopts the idea of ​​dividing the human body key point set. Compared with the method based on video feature extraction, it avoids the difficulty of target extraction and can be applied to human posture judgment in multi-person scenes; 2) Compared with the existing method based on human body key point extraction, the human body posture judgment problem is converted into the statistical relationship of the key point set, which makes the present invention insensitive to the absence and displacement of some key points. Even when part of the body is blocked and the key point is inaccurate, the correct posture judgment result can still be obtained based on the statistical features of the set, thereby greatly improving the robustness of posture judgment; 3) The judgment speed of the present invention is relatively fast, and real-time judgment of human posture within the monitoring range can be realized, helping people quickly understand the individual status and events in the current scene.

[0038] The present invention is further described in detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 The figure is a flow chart of a method for determining human posture in a complex multi-person scene according to an embodiment.

[0040] Figure 2 Schematic diagram of the division of a set of key points of a human body in one embodiment.

[0041] Figure 3A schematic diagram of the horizontal angle of calculating the convex hull of a key point set in one embodiment.

[0042] Figure 4 A schematic diagram of calculating the rotation angle of the minimum circumscribed rectangle of a key point set in one embodiment.

[0043] Figure 5 1 is a diagram showing the basic posture determination results of a single person in an embodiment; wherein the key points of the human body are marked with dots; the posture determination results are marked on the person's head, and Figures (a) to (c) are standing posture, squatting posture, and bending posture, respectively.

[0044] Figure 6 This is a diagram showing the determination results of a complex posture of a single person in an embodiment, in which the key points of the human body are marked with dots, and the posture determination results are marked on the person's head. Figures (a) to (c) are standing arm-raising posture, standing kicking posture, and standing arm-raising kicking posture, respectively.

[0045] Figure 7 This is a diagram showing the determination results of multiple people's complex postures in one embodiment, where key points of the human body are marked with dots, and the determination results of the posture are marked on the person's head. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0047] It should be noted that if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of such features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in this field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0048] In one embodiment, combined Figure 1 , provides a human body posture determination method for complex multi-person scenes, the method comprising the following steps:

[0049] Step 1: Detect key points of the human body and divide the key point sets;

[0050] Step 2: extract the statistical features of the geometric distribution of human body key points in each set;

[0051] Step 3: Determine the human body posture based on statistical features.

[0052] Furthermore, in one embodiment, the step 1 of detecting key points of the human body and dividing the key point sets includes:

[0053] Step 1-1: Construct a sample training set, which includes several human body images with labeled key points of the human body;

[0054] Step 1-2, training a deep convolutional neural network using the sample training set;

[0055] Steps 1-3: Use the trained deep convolutional neural network to detect key points in the human body image to be detected;

[0056] Steps 1-4, combined Figure 2 , divide the key point set into:

[0057] Eliminate the key points of non-human torso, including hands, to form a set of key points of the whole body;

[0058] The key points of the human torso are divided into an upper body key point set, a lower body key point set, a left arm key point set and a right arm key point set;

[0059] The lower body key points are further divided into a left leg key point set, a right leg key point set, a left thigh key point set, and a right thigh key point set.

[0060] Here, the method in the document "Realtime Multi-person 2D Pose Estimation Using PartAffinity Fields" is used to extract human key points.

[0061] Here, the key point set of the human torso includes the left eye, right eye, mouth, neck, left hip joint, right hip joint, left knee joint, right knee joint, left ankle joint, and right ankle joint, a total of ten key points;

[0062] The upper body key point set includes the left eye, right eye, mouth, neck, left hip joint, and right hip joint, a total of six key points;

[0063] The key point set of the lower body includes the left hip joint, right hip joint, left knee joint, right knee joint, left ankle joint, and right ankle joint, a total of six key points;

[0064] The left arm key point set includes the left wrist joint, left elbow joint, and left shoulder joint, a total of three key points;

[0065] The right arm key point set includes the right wrist joint, right elbow joint, and right shoulder joint, a total of three key points;

[0066] The left leg key point set includes the left ankle joint, left knee joint, and left hip joint, a total of three key points;

[0067] The right leg key point set includes the right ankle joint, right knee joint, and right hip joint, a total of three key points;

[0068] The left thigh key point set includes the left knee joint and the left hip joint, a total of two key points;

[0069] The right thigh key point set includes the right knee joint and the right hip joint, a total of two key points.

[0070] Furthermore, in one embodiment, in combination Figure 3 , step 2 extracts the statistical features of the geometric distribution of the human body key points in each set, and the specific process includes:

[0071] Step 2-1-1, calculate the convex hull of each key point set;

[0072] Step 2-1-2, find the two key points with the largest distance from the key points that constitute the convex hull;

[0073] Step 2-1-3, calculate the angle between the line connecting the two key points and the horizontal direction when rotated in the clockwise direction, as the horizontal angle of the convex hull.

[0074] Furthermore, in one embodiment, the step 3 of determining the posture of the human body based on the statistical features is specifically as follows: determining the posture of the arms, legs, and the whole body based on the horizontal angle of the convex hull of each key point set:

[0075] Assume that the horizontal angle of the convex hull of the whole body key point set is α0; the horizontal angle of the convex hull of the upper body key point set is α1; the horizontal angle of the convex hull of the lower body key point set is α2; the horizontal angle of the convex hull of the left arm key point set is α3; the horizontal angle of the convex hull of the right arm key point set is α4; the horizontal angle of the convex hull of the left leg key point set is α5; the horizontal angle of the convex hull of the right leg key point set is α6; the horizontal angle of the convex hull of the left thigh key point set is α7; the horizontal angle of the convex hull of the right thigh key point set is α8;

[0076] If |tanα3|∈(a3,b3) or |tanα4|∈(a3,b3), it is judged as an arm-raising posture; if |tanα5|∈(a5,b5) or |tanα6|∈(a5,b5), it is judged as a kicking posture; if |tanα0|∈(a0,b0), |tanα1|∈(a1,b1) and |tanα2|∈(a2,b2), it is judged as a standing posture; if |tanα7|∈[a7,b7] or |tanα8|∈[a7,b7], it is judged as a squatting posture; if |tanα0|∈[c0,d0], |tanα1|∈[c1,d1] and |tanα2|∈[c2,d2], it is judged as a lying posture.

[0077] Preferably here, (a3, b3) = (0.25, +∞), (a5, b5) = (0.25, 5), (a0, b0) = (a1, b1) = (a2, b2) = (4, +∞), [a7, b7] = [0, 0.25], [c0, d0] = [c1, d1] = [c2, d2] = [0, 0.25].

[0078] Furthermore, in one embodiment, in combination Figure 4 , step 2 extracts the statistical features of the geometric distribution of the human body key points in each set, and the specific process includes:

[0079] Step 2-2-1, calculate the minimum bounding rectangle of each key point set;

[0080] Step 2-2-2, respectively calculate the angles formed by the long side and the short side of the minimum circumscribed rectangle rotated clockwise with the horizontal direction;

[0081] In step 2-2-3, take the smaller of the two included angles as the rotation angle of the minimum circumscribed rectangle.

[0082] Furthermore, in one embodiment, the step 3 of determining the human body posture based on statistical features is specifically as follows: determining the posture of the arms, legs, and whole body based on the rotation angle of the minimum circumscribed rectangle of each key point set:

[0083] Assume that the horizontal angle of the minimum circumscribed rectangle of the whole body key point set is β0; the horizontal angle of the minimum circumscribed rectangle of the upper body key point set is β1; the horizontal angle of the minimum circumscribed rectangle of the lower body key point set is β2; the horizontal angle of the minimum circumscribed rectangle of the left arm key point set is β3; the horizontal angle of the minimum circumscribed rectangle of the right arm key point set is β4; the horizontal angle of the minimum circumscribed rectangle of the left leg key point set is β5; the horizontal angle of the minimum circumscribed rectangle of the right leg key point set is β6; the horizontal angle of the minimum circumscribed rectangle of the left thigh key point set is β7; the horizontal angle of the minimum circumscribed rectangle of the right thigh key point set is β8;

[0084] like or It is determined to be an arm-raising posture; if or It is determined to be a kicking posture; if and If β1|∈[λ1,γ1], it is judged as a standing posture; if or It is judged as a squatting posture; if |β0|∈[θ0,Θ0], β1|∈[θ1,Θ1] and β2|∈[θ2,Θ2], it is judged as a lying posture.

[0085] Here preferably, [λ1,γ1]=[0°,25°], [θ0, Θ0] = [θ1, Θ1] = [θ2, Θ2] = [0°, 25°].

[0086] In one embodiment, a human body posture determination system for complex multi-person scenes is provided, the system comprising:

[0087] Key point segmentation module, used to detect key points of the human body and divide the key point sets;

[0088] Statistical feature extraction module, used to extract statistical features of geometric distribution of key points of human body in each set;

[0089] The posture determination module is used to determine the human posture based on statistical features.

[0090] Furthermore, in one embodiment, the key point division module includes:

[0091] A training set construction unit is used to construct a sample training set, which includes a number of human body images with key points of the human body marked;

[0092] A training unit, configured to train a deep convolutional neural network using the sample training set;

[0093] The key point detection unit uses the trained deep convolutional neural network to detect the key points in the human body image to be detected;

[0094] The division unit is used to divide the key point set, including:

[0095] The first division sub-unit is used to remove key points other than the human torso, including the hands, to form a set of key points for the whole body;

[0096] The second division subunit is used to divide the key points of the human body trunk into an upper body key point set, a lower body key point set, a left arm key point set and a right arm key point set;

[0097] The third division subunit is used to further divide the lower body key points into a left leg key point set, a right leg key point set, a left thigh key point set and a right thigh key point set.

[0098] Furthermore, in one embodiment, the statistical feature extraction module includes:

[0099] A first calculation unit is used to calculate the convex hull of each key point set;

[0100] A key point screening unit is used to find the two key points with the largest distance from the key points that constitute the convex hull;

[0101] A second calculation unit is used to calculate the angle between the line connecting the two key points and the horizontal direction when rotated in the clockwise direction as the horizontal angle of the convex hull;

[0102] The posture determination module is used to determine the posture of the arms, legs and the whole body based on the horizontal angle of the convex hull of each key point set:

[0103] Assume that the horizontal angle of the convex hull of the whole body key point set is α0; the horizontal angle of the convex hull of the upper body key point set is α1; the horizontal angle of the convex hull of the lower body key point set is α2; the horizontal angle of the convex hull of the left arm key point set is α3; the horizontal angle of the convex hull of the right arm key point set is α4; the horizontal angle of the convex hull of the left leg key point set is α5; the horizontal angle of the convex hull of the right leg key point set is α6; the horizontal angle of the convex hull of the left thigh key point set is α7; the horizontal angle of the convex hull of the right thigh key point set is α8;

[0104] If |tanα3|∈(a3,b3) or |tanα4|∈(a3,b3), it is judged as an arm-raising posture; if |tanα5|∈(a5,b5) or |tanα6|∈(a5,b5), it is judged as a kicking posture; if |tanα0|∈(a0,b0), |tanα1|∈(a1,b1) and |tanα2|∈(a2,b2), it is judged as a standing posture; if |tanα7|∈[a7,b7] or |tanα8|∈[a7,b7], it is judged as a squatting posture; if |tanα0|∈[c0,d0], |tanα1|∈[c1,d1] and |tanα2|∈[c2,d2], it is judged as a lying posture.

[0105] Furthermore, in one embodiment, the statistical feature extraction module includes:

[0106] A third calculation unit is used to calculate the minimum bounding rectangle of each key point set;

[0107] a fourth calculation unit, configured to respectively calculate the angles formed by the long side and the short side of the minimum circumscribed rectangle rotated in the clockwise direction with the horizontal direction;

[0108] a fifth calculating unit, configured to take the smaller of the two included angles as the rotation angle of the minimum circumscribed rectangle;

[0109] The posture determination module is used to determine the posture of the arms, legs and the whole body according to the rotation angle of the minimum circumscribed rectangle of each key point set:

[0110] Assume that the horizontal angle of the minimum circumscribed rectangle of the whole body key point set is β0; the horizontal angle of the minimum circumscribed rectangle of the upper body key point set is β1; the horizontal angle of the minimum circumscribed rectangle of the lower body key point set is β2; the horizontal angle of the minimum circumscribed rectangle of the left arm key point set is β3; the horizontal angle of the minimum circumscribed rectangle of the right arm key point set is β4; the horizontal angle of the minimum circumscribed rectangle of the left leg key point set is β5; the horizontal angle of the minimum circumscribed rectangle of the right leg key point set is β6; the horizontal angle of the minimum circumscribed rectangle of the left thigh key point set is β7; the horizontal angle of the minimum circumscribed rectangle of the right thigh key point set is β8;

[0111] like or It is determined to be an arm-raising posture; if or It is determined to be a kicking posture; if and If β1|∈[λ1,γ1], it is judged as a standing posture; if or It is judged as a squatting posture; if |β0|∈[θ0,Θ0], β1|∈[θ1,Θ1] and β2|∈[θ2,Θ2], it is judged as a lying posture.

[0112] Regarding the specific limitations of the human posture determination system for complex multi-person scenes, please refer to the limitations of the human posture determination method for complex multi-person scenes above, which will not be repeated here. The various modules in the above-mentioned human posture determination system for complex multi-person scenes can be implemented in whole or in part through software, hardware, and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above-mentioned modules.

[0113] As a specific example, in one embodiment, the present invention is further verified and explained, the above minimum bounding rectangle method is used to determine the human posture in the surveillance video, and the test results are as follows: Figure 5 、 Figure 6 、 Figure 7 and as shown in Table 1.

[0114] Table 1 Statistics of human posture determination results

[0115]

[0116] Depend on Figure 5 The judgment results of multiple basic postures of a single person and Figure 6 The results of judging a single person's multiple complex postures show that the present invention can more accurately judge a single person's posture. Figure 7 The results of judging complex postures of multiple people with mutual occlusion show that the present invention can simultaneously judge the postures of multiple people while being insensitive to the effects of occlusion. As shown in Table 1, the average accuracy rate of human posture judgment by the present invention is 92.833%. The accuracy rate is higher for postures with clear distinctions, such as standing, squatting, raising arms, and lying flat, while the accuracy rate is slightly lower for more confusing postures, such as bending over and kicking legs.

[0117] The basic principles, main features, and advantages of the present invention are shown and described above. Those skilled in the art should understand that the present invention is not limited to the foregoing embodiments. The foregoing embodiments and descriptions are merely illustrative of the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims and their equivalents.

Claims

1. A human body posture determination method for complex multi-person scenes, characterized in that: The method comprises the following steps: Step 1, detect the key points of the human body and divide the key point sets; Step 2: extract the statistical characteristics of the geometric distribution of the key points of the human body in each set; the specific process includes: Step 2-1-1, calculate the convex hull of each key point set; Step 2-1-2, find the two key points with the largest distance from the key points that constitute the convex hull; Step 2-1-3, calculating the angle between the line connecting the two key points and the horizontal direction when rotated in the clockwise direction as the horizontal angle of the convex hull; Step 3: Determine the posture of the human body based on statistical features, specifically: determine the posture of the arms, legs, and the whole body based on the horizontal angle of the convex hull of each key point set: Assume that the horizontal angle of the convex hull of the whole body key point set is α0; the horizontal angle of the convex hull of the upper body key point set is α1; the horizontal angle of the convex hull of the lower body key point set is α2; the horizontal angle of the convex hull of the left arm key point set is α3; the horizontal angle of the convex hull of the right arm key point set is α4; the horizontal angle of the convex hull of the left leg key point set is α5; the horizontal angle of the convex hull of the right leg key point set is α6; the horizontal angle of the convex hull of the left thigh key point set is α7; the horizontal angle of the convex hull of the right thigh key point set is α8; If |tanα3|∈(a3,b3) or |tanα4|∈(a3,b3), it is judged as the arm-raising posture; if |tanα5|∈(a5,b5) or |tanα6|∈(a5,b5), it is judged as the kicking posture; if |tanα0|∈(a0,b0), |tanα1|∈(a1,b1) and |tanα2|∈(a2,b2), it is judged as the standing posture; if |tanα7|∈[a7,b7] or |tanα8|∈[a7,b7], it is judged as the squatting posture; if |tanα0|∈[c0,d0], |tanα1|∈[c1,d1] and |tanα2|∈[c2,d2], it is judged as the lying posture.

2. The human body posture determination method for complex multi-person scenes according to claim 1 is characterized in that: Step 1 detects the key points of the human body and divides the key point sets. The specific process includes: Step 1-1, construct a sample training set, which includes several human images with marked key points of the human body; Step 1-2, training a deep convolutional neural network using the sample training set; Step 1-3, using the trained deep convolutional neural network to detect key points in the human body image to be detected; Steps 1-4, divide the key point set, specifically: Eliminate the key points that are not on the human torso, including the hands, to form a set of key points for the whole body; The key points of the human body trunk are divided into an upper body key point set, a lower body key point set, a left arm key point set and a right arm key point set; The key points of the lower body are further divided into a left leg key point set, a right leg key point set, a left thigh key point set and a right thigh key point set.

3. A human body posture determination method for complex multi-person scenes, characterized in that: The method comprises the following steps: Step 1, detect the key points of the human body and divide the key point sets; Step 2: extract the statistical features of the geometric distribution of the key points of the human body in each set. The specific process includes: Step 2-2-1, calculate the minimum bounding rectangle of each key point set; Step 2-2-2, respectively calculate the angles formed by the long side and the short side of the minimum circumscribed rectangle rotating in the clockwise direction with the horizontal direction; Step 2-2-3, take the smaller value of the two angles as the rotation angle of the minimum circumscribed rectangle; Step 3: Determine the human body posture based on statistical features, specifically: According to the rotation angle of the minimum circumscribed rectangle of each key point set, the posture of the arms, legs and whole body is determined: Assume that the horizontal angle of the minimum circumscribed rectangle of the whole body key point set is β0; the horizontal angle of the minimum circumscribed rectangle of the upper body key point set is β1; the horizontal angle of the minimum circumscribed rectangle of the lower body key point set is β2; the horizontal angle of the minimum circumscribed rectangle of the left arm key point set is β3; the horizontal angle of the minimum circumscribed rectangle of the right arm key point set is β4; the horizontal angle of the minimum circumscribed rectangle of the left leg key point set is β5; the horizontal angle of the minimum circumscribed rectangle of the right leg key point set is β6; the horizontal angle of the minimum circumscribed rectangle of the left thigh key point set is β7; the horizontal angle of the minimum circumscribed rectangle of the right thigh key point set is β8; like or It is judged as the arm-raising posture; if or It is judged as a kicking posture; if and If |β1|∈[λ1,γ1], it is judged as a standing posture; if |β1|∈[λ1,γ1], it is judged as a bending posture; or It is judged as a squatting posture; if |β0|∈[θ0,Θ0], |β1|∈[θ1,Θ1] and |β2|∈[θ2,Θ2], it is judged as a lying posture.

4. A human posture determination system for complex multi-person scenes, characterized in that: The system comprises: Key point division module, used to detect key points of the human body and divide the key point sets; A statistical feature extraction module is used to extract the statistical features of the geometric distribution of the key points of the human body in each set; the statistical feature extraction module includes: A first calculation unit, used for calculating the convex hull of each key point set; A key point screening unit is used to find the two key points with the largest distance from the key points constituting the convex hull; A second calculation unit is used to calculate the angle between the line connecting the two key points rotated in the clockwise direction and the horizontal direction as the horizontal angle of the convex hull; The posture determination module is used to determine the posture of the arms, legs and the whole body according to the horizontal angle of the convex hull of each key point set: Assume that the horizontal angle of the convex hull of the whole body key point set is α0; the horizontal angle of the convex hull of the upper body key point set is α1; the horizontal angle of the convex hull of the lower body key point set is α2; the horizontal angle of the convex hull of the left arm key point set is α3; the horizontal angle of the convex hull of the right arm key point set is α4; the horizontal angle of the convex hull of the left leg key point set is α5; the horizontal angle of the convex hull of the right leg key point set is α6; the horizontal angle of the convex hull of the left thigh key point set is α7; the horizontal angle of the convex hull of the right thigh key point set is α8; If |tanα3|∈(a3,b3) or |tanα4|∈(a3,b3), it is determined to be an arm-raising posture; if |tanα5|∈(a5,b5) or |tanα6|∈(a5,b5), it is determined to be a kicking posture; if |tanα0|∈(a0,b0), |tanα1|∈(a1,b1) and |tanα2|∈(a2,b2), it is determined to be a standing posture; if |tanα7|∈[a7,b7] or |tanα8|∈[a7,b7], it is determined to be a squatting posture; if |tanα0|∈[c0,d0], |tanα1|∈[c1,d1] and |tanα2|∈[c2,d2], it is determined to be a lying posture; The posture determination module is used to determine the human posture based on statistical features.

5. The human body posture determination system for complex multi-person scenes according to claim 4, characterized in that: The key point division module comprises: A training set construction unit, used to construct a sample training set, the set including a number of human images with key points of the human body annotated; A training unit, used for training a deep convolutional neural network using the sample training set; A key point detection unit detects key points in the human body image to be detected using a trained deep convolutional neural network; The division unit is used to divide the key point set, including: The first division sub-unit is used to remove key points other than the human torso, including the hands, to form a set of key points for the whole body; The second division subunit is used to divide the key points of the human body trunk into an upper body key point set, a lower body key point set, a left arm key point set and a right arm key point set; The third division subunit is used to further divide the lower body key points into a left leg key point set, a right leg key point set, a left thigh key point set and a right thigh key point set.

6. A human posture determination system for complex multi-person scenes, characterized in that: The system comprises: Key point division module, used to detect key points of the human body and divide the key point sets; A statistical feature extraction module is used to extract the statistical features of the geometric distribution of the key points of the human body in each set; the statistical feature extraction module includes: A third calculation unit is used to calculate the minimum circumscribed rectangle of each key point set; A fourth calculation unit, used to respectively calculate the angles formed by the long side and the short side of the minimum circumscribed rectangle rotating in the clockwise direction and the horizontal direction; A fifth calculation unit, used for taking the smaller value of the two included angles as the rotation angle of the minimum circumscribed rectangle; The posture determination module is used to determine the posture of the arms, legs and the whole body according to the rotation angle of the minimum circumscribed rectangle of each key point set: Assume that the horizontal angle of the minimum circumscribed rectangle of the whole body key point set is β0; the horizontal angle of the minimum circumscribed rectangle of the upper body key point set is β1; the horizontal angle of the minimum circumscribed rectangle of the lower body key point set is β2; the horizontal angle of the minimum circumscribed rectangle of the left arm key point set is β3; the horizontal angle of the minimum circumscribed rectangle of the right arm key point set is β4; the horizontal angle of the minimum circumscribed rectangle of the left leg key point set is β5; the horizontal angle of the minimum circumscribed rectangle of the right leg key point set is β6; the horizontal angle of the minimum circumscribed rectangle of the left thigh key point set is β7; the horizontal angle of the minimum circumscribed rectangle of the right thigh key point set is β8; like or It is judged as the arm-raising posture; if or It is judged as a kicking posture; if and If |β1|∈[λ1,γ1], it is judged as a standing posture; if |β1|∈[λ1,γ1], it is judged as a bending posture; or If |β0|∈[θ0,Θ0], |β1|∈[θ1,Θ1] and |β2|∈[θ2,Θ2], then it is judged as a lying posture; The posture determination module is used to determine the human posture based on statistical features.

Images