A method and device for human posture recognition
By combining a pre-set tracker and a human body detection network for human bounding box detection, and adding an interference part classification network during key point prediction, the problem of the inability to balance recognition accuracy and speed in existing technologies is solved, achieving high-precision and high-speed human pose recognition.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- E SURFING VISION TECHNOLOGY CO LTD
- Filing Date
- 2022-11-30
- Publication Date
- 2026-05-26
AI Technical Summary
Existing technologies cannot simultaneously achieve both high recognition accuracy and fast recognition speed in human posture recognition.
Human bounding box detection is performed by combining a pre-set tracker and a human detection network. A pre-set key point prediction network is used to predict human key points, including a baseline pose estimation network and an interference part classification network. Humans are tracked by IOU and threshold to reduce redundant calculations, and interference analysis is performed to improve prediction accuracy.
It achieves both high recognition accuracy and fast recognition speed in human pose recognition, while reducing the amount of computation and improving the processing speed.
Smart Images
Figure CN115909497B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of machine vision technology, and in particular to a method and apparatus for human posture recognition. Background Technology
[0002] Human pose recognition primarily involves extracting the coordinates of joints of various parts of the human body from video or images captured by cameras. Human pose recognition currently has numerous applications, including virtual reality, human-computer interaction, sports training and analysis, and abnormal behavior detection.
[0003] Current human pose recognition algorithms can be mainly divided into two categories: top-down and bottom-up. Top-down algorithms first detect the bounding box of each human body and then predict the skeletal keypoints of each person using a keypoint regression network. Bottom-up algorithms directly predict the skeletal keypoints of all human bodies and then connect the keypoints to represent different human figures. The former consumes a lot of computational resources in the human target detection stage, resulting in a slow overall algorithm speed; while the latter has poor anti-interference capabilities and lower recognition accuracy. Summary of the Invention
[0004] This application provides a human posture recognition method and apparatus to solve the technical problem that the prior art cannot simultaneously achieve high recognition accuracy and fast recognition speed.
[0005] In view of this, the first aspect of this application provides a human pose recognition method, comprising:
[0006] Human detection is performed on the target person image frame by combining a pre-set tracker and a human detection network to obtain the target human bounding box. The pre-set tracker is a network that tracks the human body based on IOU and threshold.
[0007] A preset key point prediction network is used to predict human key points in the target human body frame. The preset key point prediction network includes a baseline pose estimation network and an interference part classification network.
[0008] The human posture in the target person image frame is identified based on the human body key points to obtain the human posture recognition result.
[0009] Preferably, the step of combining a pre-set tracker and a human detection network to perform human detection on the target person image frame to obtain the target human bounding box, wherein the pre-set tracker is a network that tracks the human body based on IOU and threshold, and further includes:
[0010] Acquire multiple image frames of people from surveillance video;
[0011] The image frame of the person is preprocessed to obtain the target image frame of the person. The preprocessing includes cropping, mean reduction and normalization.
[0012] Preferably, the step of combining a pre-set tracker and a human detection network to perform human detection on the target person image frame to obtain the target human bounding box, wherein the pre-set tracker is a network that tracks the human body based on IOU and a threshold, including:
[0013] The system uses a preset tracker to determine whether the historical IOU of the previous frame is less than a threshold. If so, it uses a human detection network to perform human detection on the target person image frame, obtains the target human bounding box, and updates the current IOU.
[0014] If not, the expanded bounding rectangle of the human body key points in the previous frame is expanded by a preset number of pixels to obtain the target human body bounding box, and the current IOU is updated.
[0015] Preferably, the preset key point prediction network predicts human body key points within the target human body bounding box. The preset key point prediction network includes a baseline pose estimation network and an interference part classification network, comprising:
[0016] Reference key points are obtained within the target human body frame using a reference pose estimation network;
[0017] The interference vector is obtained by performing interference analysis on the human torso in the target human body frame based on the interference part classification network.
[0018] Human body key points are obtained by weighted calculation based on the reference key points and the interference vector.
[0019] Preferably, the step of recognizing the human posture in the target person image frame based on the human body key points to obtain the human posture recognition result includes:
[0020] The key points of the human body are connected to form a human skeleton diagram based on the key point matching algorithm.
[0021] The human posture in the target person image frame is identified based on the human skeleton diagram to obtain the human posture recognition result.
[0022] A second aspect of this application provides a human posture recognition device, comprising:
[0023] The human bounding box detection unit is used to perform human detection on the target person image frame by combining a preset tracker and a human detection network to obtain the target human bounding box. The preset tracker is a network that tracks the human body based on IOU and threshold.
[0024] A key point prediction unit is used to predict human key points in the target human body frame using a preset key point prediction network. The preset key point prediction network includes a baseline pose estimation network and an interference part classification network.
[0025] The human posture recognition unit is used to recognize the human posture in the target person image frame based on the human body key points, and obtain the human posture recognition result.
[0026] Preferred options also include:
[0027] The image frame acquisition unit is used to acquire multiple image frames of people in the surveillance video;
[0028] The image preprocessing unit is used to perform preprocessing operations on the image frame of the person to obtain the target image frame of the person. The preprocessing includes cropping, mean reduction and normalization.
[0029] Preferred options also include:
[0030] The image frame acquisition unit is used to acquire multiple image frames of people in the surveillance video;
[0031] The image preprocessing unit is used to perform preprocessing operations on the image frame of the person to obtain the target image frame of the person. The preprocessing includes cropping, mean reduction and normalization.
[0032] Preferably, the human body frame detection unit includes:
[0033] The first judgment subunit is used to determine whether the historical IOU of the previous frame is less than the threshold by using a preset tracker. If so, the human body detection network is used to perform human body detection on the target person image frame to obtain the target human body bounding box and update the current IOU.
[0034] The second judgment subunit is used to, if not, expand the minimum bounding rectangle of the human body key points in the previous frame by a preset number of pixels to obtain the expanded box as the target human body box, and update the current IOU.
[0035] Preferably, the key point prediction unit includes:
[0036] The baseline prediction subunit is used to obtain baseline key points in the target human body frame through the baseline pose estimation network;
[0037] The interference analysis subunit is used to perform interference analysis on the human torso in the target human body frame based on the interference part classification network to obtain the interference vector;
[0038] The weighted calculation subunit is used to perform weighted calculations based on the reference key points and the interference vector to obtain human body key points.
[0039] Preferably, the human posture recognition unit includes:
[0040] A connection subunit is used to connect the key points of the human body into a human skeleton diagram based on a key point matching algorithm.
[0041] The recognition subunit is used to recognize the human posture in the target person image frame based on the human skeleton diagram, and obtain the human posture recognition result.
[0042] As can be seen from the above technical solutions, the embodiments of this application have the following advantages:
[0043] This application provides a human pose recognition method, comprising: performing human detection on a target person image frame by combining a pre-set tracker and a human detection network to obtain a target human bounding box; the pre-set tracker being a network that tracks the human body based on IOU and a threshold; predicting human key points in the target human bounding box using a pre-set key point prediction network, the pre-set key point prediction network including a baseline pose estimation network and an interference part classification network; and recognizing the human pose in the target person image frame based on the human key points to obtain a human pose recognition result.
[0044] The human pose recognition method provided in this application uses a tracker in conjunction with a human detection network to detect human bounding boxes. Tracking the human body based on Intersection over Union (IOU) and a threshold reduces the number of human detections, lowers redundant computations, and accelerates processing. Furthermore, an interference part classification network is incorporated into the keypoint prediction process for interference analysis. Predicting keypoints based on the interference analysis results improves prediction accuracy. Through targeted improvements to the human pose recognition process, both high recognition accuracy and fast recognition speed can be achieved simultaneously. Therefore, this application solves the technical problem of existing technologies being unable to simultaneously achieve both high recognition accuracy and fast recognition speed. Attached Figure Description
[0045] Figure 1 A flowchart illustrating a human posture recognition method provided in this application embodiment;
[0046] Figure 2 This is another schematic flowchart illustrating a human posture recognition method provided in an embodiment of this application;
[0047] Figure 3 This is a schematic diagram of the structure of a human posture recognition device provided in an embodiment of this application;
[0048] Figure 4 A schematic diagram of the human bounding box detection process of the MMpose network model provided in the embodiments of this application;
[0049] Figure 5A schematic diagram of the human bounding box detection process combining a pre-set tracker and a human detection network, provided for an embodiment of this application;
[0050] Figure 6 This is a schematic diagram illustrating the process of a preset key point prediction network predicting key points, as provided in an embodiment of this application. Detailed Implementation
[0051] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present application.
[0052] For easier understanding, please refer to Figure 1 An embodiment of a human posture recognition method provided in this application includes:
[0053] Step 101: Combine the preset tracker and the human detection network to perform human detection on the target person image frame to obtain the target human bounding box. The preset tracker is a network that tracks the human body based on IOU and threshold.
[0054] The pre-set tracker is a network that tracks human bodies in image frames based on IOU (Intersection over Union) and a corresponding threshold. The threshold can be set based on practical experience or specific circumstances, and is not limited here. IOU is a standard for measuring the accuracy of detecting corresponding objects in a specific dataset. Any task that yields a predicted range (bounding boxes, hereinafter referred to as bboxes) in the output can be measured using IoU. By tracking human bodies, the pre-set tracker avoids the large computational burden caused by repeated detections in human detection networks, thus accelerating the algorithm's processing speed.
[0055] The specific detection principle is that when the preset tracker can track a specific human body, the human body detection network only needs to detect the human body bounding box once, until the tracked human body is lost, that is, it is outside the preset tracking range. Then the human body detection network is restarted to detect the human body bounding box. This mechanism can greatly reduce the amount of computation and improve the running speed.
[0056] The improved algorithm in this application is based on the MMpose network model, a top-down algorithm. Its human bounding box detection mechanism obtains the human bounding boxes in the image through a detection network, and then directly uses the human region map as the input map for the human skeleton keypoint prediction network to perform keypoint prediction calculations. This method involves continuously rereading the human bounding box detection, resulting in a large computational load and slowing down the process.
[0057] Step 102: Use a preset key point prediction network to predict human body key points in the target human body bounding box. The preset key point prediction network includes a baseline pose estimation network and an interference part classification network.
[0058] The preset keypoint prediction network includes two important network layers: a baseline pose estimation network and an interference part classification network. The baseline pose estimation network can directly predict human keypoints within the target human bounding box. However, in multi-person scenes, multiple overlapping human body parts can interfere with the identification of the main person, leading to inaccurate keypoint prediction. Therefore, this embodiment adds an interference part classification network to extract vector features for each part, thereby optimizing keypoint prediction. Specifically, to avoid increasing the overall computational burden of the algorithm, a lightweight network can be selected to construct the interference part classification network, which improves prediction accuracy while reducing computational cost.
[0059] Step 103: Identify the human posture in the target person image frame based on human key points to obtain the human posture recognition result.
[0060] The predicted key points of the human body are marked. Based on the marks, the points can be connected to form the human skeleton lines. Based on the direction of the lines or the geometric shape formed by the lines, the human posture in the target person image frame can be identified, and the human posture recognition result can be obtained.
[0061] The human pose recognition method provided in this application uses a tracker in conjunction with a human detection network to detect human bounding boxes. Tracking the human body based on Intersection over Union (IOU) and thresholds reduces the number of human detections, lowers redundant computations, and accelerates processing. Furthermore, an interference part classification network is incorporated into the keypoint prediction process for interference analysis. Predicting keypoints based on the interference analysis results improves prediction accuracy. Through targeted improvements to the human pose recognition process, both high recognition accuracy and fast recognition speed can be achieved simultaneously. Therefore, this application's embodiments solve the technical problem that existing technologies cannot simultaneously achieve both high recognition accuracy and fast recognition speed.
[0062] For easier understanding, please refer to Figure 2 This application provides a second embodiment of a human posture recognition method, including:
[0063] Step 201: Obtain multiple image frames of people from the surveillance video.
[0064] Step 202: Perform preprocessing operations on the image frames of the person to obtain the target image frames of the person. The preprocessing includes cropping, mean subtraction and normalization.
[0065] The monitoring equipment continuously acquires video streams. The monitored video is selected from these streams and decoded using FFmpeg. Multiple image frames can then be extracted from the monitored video. Since there is a temporal relationship between adjacent frames, continuous frames can be detected and keypoints predicted to determine human posture and even human behavior. This embodiment primarily analyzes human movements, so image frames containing human figures are selected.
[0066] Preprocessing is performed to improve the quality of image frames, facilitating subsequent human bounding box detection and keypoint prediction. In addition to the cropping, mean reduction, and normalization processes proposed in this embodiment, other preprocessing procedures can be added as needed, and are not limited here.
[0067] Step 203: Determine whether the historical IOU of the previous frame is less than the threshold using a preset tracker. If so, perform human detection on the target person image frame using a human detection network to obtain the target human bounding box and update the current IOU.
[0068] Step 204: If not, then use the expanded bounding rectangle of the minimum bounding rectangle of the human body key points in the previous frame by a preset number of pixels as the target human body bounding box, and update the current IOU.
[0069] If a human detection network is used to detect a human body in every image frame, then a single surveillance video would require multiple calls to the human detection network, resulting in a significant computational burden. Therefore, whether or not the human detection network has detected the current target human bounding box needs to be determined based on historical IOUs. This involves activating a pre-set tracker to process the data and ultimately decide whether the human detection network is needed to process that frame.
[0070] If the historical IOU of the previous frame is less than the threshold, it means that the minimum bounding rectangle of the human body key points identified in the previous image frame has a small overlap with the minimum bounding rectangle of the human body key points identified in the frame before that. In this case, the IOU calculated based on the two minimum bounding rectangles will be less than the threshold, and the detected target may have moved out of the previous image frame. The human body has moved out of the movement range allowed by the tracker. Therefore, the current frame will no longer use the minimum bounding rectangle of the previous frame to extract the target human body bounding box. Instead, it is necessary to perform human detection on the target human body image frame through the human body detection network to obtain the target human body bounding box.
[0071] If the historical IOU of the previous frame is not less than the threshold, it means that there is a large overlap between the minimum bounding rectangle of the human key points identified in the previous image frame and the minimum bounding rectangle of the human key points identified in the frame before that. Therefore, the historical IOU calculated based on the two minimum bounding rectangles is greater than or equal to the threshold, and the detected target is still within the image frame. In the current frame, the target human bounding box can be extracted by expanding the minimum bounding rectangle of the human key points identified in the previous moment by a preset number of pixels. This does not require redundant detection through the human detection network.
[0072] It should be noted that the first frame image needs to directly use the human bounding box detected by the human detection network, and then use the preset key point prediction network to identify human key points. At this time, the expanded bounding box obtained by expanding the minimum bounding rectangle of the human key point by a certain number of pixels can be regarded as the initial bounding box. Since the human bounding box at this time does not have the corresponding IOU calculated from the human bounding box at the previous moment, the IOU of the first frame image is directly defined as 1.
[0073] It should be explained that the bounding box (bbox) is the expanded bounding rectangle of the human body keypoints after expanding by a preset number of pixels, not the target human body bounding box extracted from the image frame. Therefore, each bbox is obtained after the human body keypoints are predicted and is used to calculate the current IOU for the current frame. The current IOU is then used to determine whether the human body detection network needs to be used for target detection in the next frame.
[0074] It should be noted that the expansion is in pixels, and the specific number of pixels to expand, i.e., the preset number, can be set according to the actual situation and is not limited here. Moreover, the expansion refers to expanding outward from all four sides of the smallest bounding rectangle at the same time, not just one side.
[0075] Please refer to 4. Its human bounding box detection process is a traditional method, directly detecting human bounding boxes through a human detection network (Persondetector). The detection mechanism of this application is described in [reference needed]. Figure 5 The Tracker is a pre-set tracker that can track the human body, and Pose landmarks is a pre-set key point prediction network for subsequent tasks.
[0076] Step 205: Obtain reference key points in the target human body bounding box using the reference pose estimation network.
[0077] Step 206: Perform interference analysis on the human torso in the target human body frame based on the interference part classification network to obtain the interference vector.
[0078] Step 207: Perform weighted calculations based on the baseline key points and interference vectors to obtain human body key points.
[0079] It should be noted that the preset key point prediction network includes a baseline pose estimation network and an interference part classification network. To improve the prediction accuracy of key points for the main target when multiple people's torsos overlap, this embodiment adds an interference part classification network to the key point prediction process. Please refer to [link to relevant documentation]. Figure 6 Here, Image represents the target human bounding box obtained from the above detection. Besides inputting into the baseline pose estimation network for keypoint prediction, the target human bounding box also needs to be input into the interference classification network for interference analysis. The results of both are then weighted by channels to obtain a more accurate prediction of human keypoints. Furthermore, after predicting human keypoints in each current frame, the minimum bounding rectangle of that keypoint needs to be extracted and expanded by a preset number of pixels to obtain an expanded bounding box. The current Interchange of Unit (IOU) is then calculated between this expanded bounding box and the expanded bounding box corresponding to the previous frame.
[0080] The interference part classification network mainly predicts whether each keypoint in the bounding box belongs to the main target, or in other words, it performs a binary classification process for keypoints that belong to the main target and those that do not. In this embodiment, the lightweight network MobileNetV2 is selected as the backbone network for the interference part classification network to extract features suitable for the interference part classification task and obtain the interference vector for each keypoint.
[0081] V = [v0p0, v1p1, ..., v k p k ],v i ∈{0,1},i∈[0,k]
[0082] Where, p i For each joint point in the human body, v i The variable is a binary variable, representing whether the i-th joint is an interfering joint. A value of 0 indicates a non-interfering joint, and a value of 1 indicates an interfering joint. k is the number of joints.
[0083] This embodiment makes appropriate modifications to MobileNetV2 to adapt to the task of classifying interference parts. The 1000-dimensional fully connected classifier used for image classification in MobileNetV2 is replaced with a 1×1 convolution with k output channels. The loss function of the preset keypoint prediction network based on the above is expressed as:
[0084] L = L lm +λL ic
[0085] Among them, L lm L is the loss function of the baseline pose estimation network. ic Let λ be the loss function of the interference part classification network, and λ be the balance factor. Therefore, this embodiment can effectively improve the problem of poor keypoint prediction performance caused by interference from multiple users while introducing a relatively small computational cost.
[0086] Step 208: Connect the key points of the human body to form a human skeleton map based on the key point matching algorithm.
[0087] Step 209: Identify the human posture in the target person image frame based on the human skeleton diagram to obtain the human posture recognition result.
[0088] The keypoint matching algorithm can be selected based on the matching accuracy, and no specific algorithm is specified here. The human skeleton diagram includes lines connecting keypoints and the original target person image frame, which are overlaid for easy identification of corresponding human body parts.
[0089] The method provided in this embodiment retains the high accuracy of the Top-down type while achieving a faster speed than the Top-down type algorithm, reducing computational load and power consumption, making the algorithm more practically applicable in engineering.
[0090] The human pose recognition method provided in this application uses a tracker in conjunction with a human detection network to detect human bounding boxes. Tracking the human body based on Intersection over Union (IOU) and thresholds reduces the number of human detections, lowers redundant computations, and accelerates processing. Furthermore, an interference part classification network is incorporated into the keypoint prediction process for interference analysis. Predicting keypoints based on the interference analysis results improves prediction accuracy. Through targeted improvements to the human pose recognition process, both high recognition accuracy and fast recognition speed can be achieved simultaneously. Therefore, this application's embodiments solve the technical problem that existing technologies cannot simultaneously achieve both high recognition accuracy and fast recognition speed.
[0091] For easier understanding, please refer to Figure 3 This application also provides an embodiment of a human posture recognition device, comprising:
[0092] The human bounding box detection unit 301 is used to perform human detection on the target person image frame by combining a preset tracker and a human detection network to obtain the target human bounding box. The preset tracker is a network that tracks the human body based on IOU and threshold.
[0093] The key point prediction unit 302 is used to predict human key points in the target human body bounding box using a preset key point prediction network. The preset key point prediction network includes a baseline pose estimation network and an interference part classification network.
[0094] The human posture recognition unit 303 is used to recognize the human posture in the target person image frame based on human key points and obtain the human posture recognition result.
[0095] Furthermore, it also includes:
[0096] Image frame acquisition unit 304 is used to acquire multiple image frames of people in a surveillance video;
[0097] The image preprocessing unit 305 is used to perform preprocessing operations on the image frame of the person to obtain the image frame of the target person. The preprocessing includes cropping, mean subtraction and normalization.
[0098] Furthermore, the human body frame detection unit 301 includes:
[0099] The first judgment subunit 3011 is used to determine whether the historical IOU of the previous frame is less than the threshold by using a preset tracker. If so, the target person image frame is detected by the human body detection network to obtain the target human body bounding box and update the current IOU.
[0100] The second judgment subunit 3012 is used to, if not, expand the minimum bounding rectangle of the human body key points in the previous frame by a preset number of pixels to obtain the expanded bounding box as the target human body bounding box, and update the current IOU.
[0101] Furthermore, the keypoint prediction unit 302 includes:
[0102] The baseline prediction subunit 3021 is used to obtain baseline key points in the target human bounding box through the baseline pose estimation network;
[0103] Interference analysis subunit 3022 is used to perform interference analysis on the human torso in the target human body frame based on the interference part classification network to obtain the interference vector;
[0104] The weighted calculation subunit 3023 is used to perform weighted calculations based on the baseline key points and interference vectors to obtain human body key points.
[0105] Furthermore, the human posture recognition unit 303 includes:
[0106] Connecting subunit 3031 is used to connect human key points into a human skeleton map based on a key point matching algorithm;
[0107] The recognition subunit 3032 is used to recognize the human posture in the target person image frame based on the human skeleton diagram, and obtain the human posture recognition result.
[0108] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.
[0109] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0110] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0111] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for executing all or part of the steps of the methods described in the various embodiments of this application through a computer device (which may be a personal computer, server, or network device, etc.). The aforementioned storage medium includes: USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, optical disks, and other media capable of storing program code.
[0112] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.
Claims
1. A method for human posture recognition, characterized in that, include: Human detection is performed on the target person image frame by combining a pre-set tracker and a human detection network to obtain the target human bounding box. The pre-set tracker is a network that tracks the human body based on IOU and threshold. A preset key point prediction network is used to predict human key points in the target human body frame. The preset key point prediction network includes a baseline pose estimation network and an interference part classification network. The human posture in the target person image frame is identified based on the human body key points to obtain the human posture recognition result; The method involves combining a pre-set tracker and a human detection network to perform human detection on the target person image frame, obtaining the target human bounding box. The pre-set tracker is a network that tracks the human body based on IOU and a threshold, including: The system uses a preset tracker to determine whether the historical IOU of the previous frame is less than a threshold. If so, it uses a human detection network to perform human detection on the target person image frame, obtains the target human bounding box, and updates the current IOU. If not, the expanded bounding rectangle obtained by expanding the minimum bounding rectangle of the human body key points in the previous frame by a preset number of pixels is used as the target human body bounding box, and the current IOU is updated.
2. The human posture recognition method according to claim 1, characterized in that, The method involves combining a pre-set tracker and a human detection network to perform human detection on the target person image frame, obtaining the target human bounding box. The pre-set tracker is a network that tracks the human body based on IOU and a threshold. Prior to this, the method also includes: Acquire multiple image frames of people from surveillance video; The image frame of the person is preprocessed to obtain the target image frame of the person. The preprocessing includes cropping, mean reduction and normalization.
3. The human posture recognition method according to claim 1, characterized in that, The method employs a preset keypoint prediction network to predict human body keypoints within the target human body bounding box. This preset keypoint prediction network includes a baseline pose estimation network and an interference part classification network, comprising: Reference key points are obtained within the target human body frame using a reference pose estimation network; The interference vector is obtained by performing interference analysis on the human torso in the target human body frame based on the interference part classification network. Human body key points are obtained by weighted calculation based on the reference key points and the interference vector.
4. The human posture recognition method according to claim 1, characterized in that, The step of identifying the human posture in the target person image frame based on the human body key points to obtain the human posture recognition result includes: The key points of the human body are connected to form a human skeleton diagram based on the key point matching algorithm. The human posture in the target person image frame is identified based on the human skeleton diagram to obtain the human posture recognition result.
5. A human posture recognition device, characterized in that, include: The human bounding box detection unit is used to perform human detection on the target person image frame by combining a preset tracker and a human detection network to obtain the target human bounding box. The preset tracker is a network that tracks the human body based on IOU and threshold. A key point prediction unit is used to predict human key points in the target human body frame using a preset key point prediction network. The preset key point prediction network includes a baseline pose estimation network and an interference part classification network. The human posture recognition unit is used to recognize the human posture in the target person image frame based on the human key points, and obtain the human posture recognition result. The human body frame detection unit includes: The first judgment subunit is used to determine whether the historical IOU of the previous frame is less than the threshold by using a preset tracker. If so, the human body detection network is used to perform human body detection on the target person image frame to obtain the target human body bounding box and update the current IOU. The second judgment subunit is used to, if not, expand the minimum bounding rectangle of the human body key points in the previous frame by a preset number of pixels to obtain the expanded box as the target human body box, and update the current IOU.
6. The human posture recognition device according to claim 5, characterized in that, Also includes: The image frame acquisition unit is used to acquire multiple image frames of people in the surveillance video; The image preprocessing unit is used to perform preprocessing operations on the image frame of the person to obtain the target image frame of the person. The preprocessing includes cropping, mean reduction and normalization.
7. The human posture recognition device according to claim 5, characterized in that, The key point prediction unit includes: The baseline prediction subunit is used to obtain baseline key points in the target human body frame through the baseline pose estimation network; The interference analysis subunit is used to perform interference analysis on the human torso in the target human body frame based on the interference part classification network to obtain the interference vector; The weighted calculation subunit is used to perform weighted calculations based on the reference key points and the interference vector to obtain human body key points.
8. The human posture recognition device according to claim 5, characterized in that, The human posture recognition unit includes: A connection subunit is used to connect the key points of the human body into a human skeleton diagram based on a key point matching algorithm. The recognition subunit is used to recognize the human posture in the target person image frame based on the human skeleton diagram, and obtain the human posture recognition result.