Human body multi-component joint detection model construction method, device and storage medium

Abstract

The application discloses a human multi-component joint detection model construction method and device and a storage medium. The method comprises the following steps: obtaining a training data set; constructing a feature extraction backbone network for extracting multi-level features from an input image; constructing a feature fusion network for receiving multi-level features extracted by the feature extraction backbone network, generating multi-scale feature maps with rich semantic information and different spatial resolutions; constructing a joint detection head and training by using the training data set; and constructing a post-processing module, which is used for executing an inference process and finally outputs an associated triple of a human body frame, a human head frame and a human face frame for each human. The method sets an offset regression branch, discards the idea of directly positioning components, and instead learns the conditional probability distribution of a human head relative to a human body. The offset is used as a spatial pointer to filter out the most matched candidate in the detected human head set in terms of geometric relationship, which can avoid coordinate competition of overlapping targets in principle.

Description

Technical Field

[0001] This invention relates to the field of video image processing technology, and more specifically, to a method, device, and storage medium for constructing a joint detection model of multiple human body parts. Background Technology

[0002] In real-world classroom scenarios, face and body detection solutions typically employ separate face and body detectors to acquire two types of bounding boxes, which are then matched in post-processing based on geometric relationships such as intersection-over-union ratio (IoU) and center distance. However, in densely populated classroom settings, students in the same row sit close together, resulting in overlapping or adjacent bounding boxes. Furthermore, students' postures are varied, often with their heads down, turned to the side, or obscured, meaning they are not always facing the camera. Consequently, face bounding boxes may shift due to posture, making it highly susceptible to incorrect associations if relying solely on geometric rules. This leads to identity confusion, a sharp increase in matching error rates, and severely compromises the accuracy of attendance determination.

[0003] From the perspective of detection models, commonly used methods such as YOLO series and Faster R-CNN can only output target boxes of a single class independently. Their network structures lack the ability to explicitly associate components, requiring external post-processing for pairing. This results in insufficient stability and reliability under dense occlusion conditions. While Transformer-based methods such as DETR implicitly model relationships between targets using attention mechanisms, they lack explicit regression constraints on the spatial offset between the human body and the face. This prevents the network from ensuring the strong prior that "the face is located within a certain offset range above the associated human body bounding box," making component attribution prediction unreliable in complex scenes and pairing errors difficult to avoid. Therefore, existing classroom attendance detection technologies have significant shortcomings in addressing face loss, dense pairing, and component association modeling. Summary of the Invention

[0004] To overcome the high error rate in matching human body, head, and face as described in the prior art, this invention provides a method, device, and storage medium for constructing a joint detection model of multiple human body parts. The technical solution adopted by this invention is as follows.

[0005] In a first aspect, the present invention provides a method for constructing a joint detection model of multiple human body parts, comprising: Obtain the training dataset; Construct a feature extraction backbone network to extract multi-level features from the input image; A feature fusion network is constructed to receive multi-level features extracted by the feature extraction backbone network and generate multi-scale feature maps with rich semantic information and different spatial resolutions. Construct a joint detection head; The joint detection head is trained using the training dataset; A post-processing module is constructed, which is used to execute the inference process and finally outputs the associated triplet of human body bounding box - human head bounding box - human face bounding box for each character; Each photo in the training dataset is hierarchically labeled: Human instance annotation: The full-body outline of each student is annotated with a rectangle, which is defined as the human detection box; Head instance annotation: Inside each human body detection box, annotate the head area of ​​the human body, which is defined as the head detection box. If the head is not visible, the annotation is empty; Face instance annotation: Inside each head detection box, annotate the face region of the human body, which is defined as the face detection box. If the face is not visible, the annotation is empty; The combined detection head includes: a human body detection head, a head detection head, a face detection head, and a head / face offset regression branch; Each detection head is a lightweight convolutional network used to transform each anchor point on the feature map into a specific anchor box and the detection result of each anchor box. The detection result includes: classification confidence and bounding box regression. Classification confidence score: Outputs the confidence score that the corresponding component exists at each spatial location; Bounding box regression: Outputs the coordinate adjustment of the bounding box at each spatial location relative to each Anchor point; The head / face offset regression branch contains a regression network consisting of at least one convolutional layer, which is used to predict the relative offset of the head / face relative to the human body, taking the feature region corresponding to the human body detection box as input. The reasoning process includes: Obtain the candidate bounding boxes for human body, head, and face output by the three detection heads running independently, as well as the detection results of each candidate bounding box. Each candidate bounding box is the Anchor box. Using the relative offset predicted by the head / face offset regression branch, the predicted head reference box and the predicted face reference box are decoded for each human detection box with a confidence level higher than a preset threshold. Based on the positional relationship between the predicted head reference box and each head detection box in the head candidate box list, and the confidence score of each head detection box in the head candidate box list, a head detection box is selected from the head candidate box list as the head detection box associated with the human body detection box that exceeds the preset threshold. Based on the positional relationship between the predicted face reference box and each face detection box in the face candidate box list, and the confidence score of each face detection box in the face candidate box list, a face detection box is selected from the face candidate box list as the face detection box associated with the human body detection box that exceeds the preset threshold. Output the associated triplet of body bounding box, head bounding box, and face bounding box for each character.

[0006] In one implementation, the post-processing module executes the process of determining the positional relationship between the predicted head reference box and each head detection box in the head candidate box list, and the confidence score of each head detection box in the head candidate box list, and selecting a head detection box from the head candidate box list as the head detection box associated with the human body detection box that exceeds a preset threshold, including: Calculate the distance between each head detection box in the candidate head list and the predicted head reference box; Using a preset score model, the score of each head detection box in the candidate head box list is obtained, and the head detection box with the highest score is selected as the head detection box associated with the human body detection box that is higher than the preset threshold. The preset score model is a model with the distance and the confidence score as parameters. The post-processing module executes the process of selecting a face detection box from the face candidate box list based on the positional relationship between the predicted face reference box and each face detection box in the face candidate box list, and the confidence score of each face detection box in the face candidate box list, as the face detection box associated with the human body detection box that exceeds a preset threshold. This process includes: Calculate the distance between each head detection box in the candidate head list and the predicted head reference box; Using a preset score model, the score of each head detection box in the candidate head box list is obtained, and the head detection box with the highest score is selected as the head detection box associated with the human body detection box that is higher than the preset threshold. The preset score model is a model with the distance and the confidence score as parameters.

[0007] In one implementation, the head / face offset regression branch is configured to perform the process of predicting the relative offset of the head / face relative to the human body, taking the feature region corresponding to the human body detection box as input, including: The relative offset of the human head to the body: Center offset: δ_x = (x_head - x_body) / w_body, δ_y = (y_head - y_body) / h_body; Scale offset: δ_w = log(w_head / w_body), δ_h = log(h_head / h_body); Where x_head is the x-axis coordinate of the center point of the head detection box, y_head is the y-axis coordinate of the center point of the head detection box, w_head is the width of the head detection box, h_head is the height of the head detection box, x_body is the x-axis coordinate of the center point of the human body detection box, y_body is the y-axis coordinate of the center point of the human body detection box, w_body is the width of the human body detection box, and h_body is the height of the human body detection box. The relative offset of the face to the body: Center offset: δ_x = (x_face - x_body) / w_body, δ_y = (y_face - y_body) / h_body; Scale offset: δ_w = log(w_face / w_body), δ_h = log(h_face / h_body); Where x_face is the x-axis coordinate of the center point of the head detection box, y_face is the y-axis coordinate of the center point of the head detection box, w_face is the width of the head detection box, h_face is the height of the head detection box, x_body is the x-axis coordinate of the center point of the human body detection box, y_body is the y-axis coordinate of the center point of the human body detection box, w_body is the width of the human body detection box, and h_body is the height of the human body detection box.

[0008] In one embodiment, the method for constructing the human body multi-part joint detection model further includes: Design the loss function of the network and train the network model parameters; The total loss function for model training includes: L = λ_body * L_body + λ_head * L_head + λ_face * L_face + λ_offset *L_offset; L_body, L_head, and L_face are the loss function parts corresponding to the human body, head, and face, respectively. λ_body, λ_head, and λ_face are the loss weights for the human body, head, and face, respectively. The weight of λ_body is higher than that of λ_head and λ_face. If there is no head / face label in the corresponding human body, the value is 0 and it is not included in the training. L_offset = δ_head * L_Smooth_L1(Δhead) + δ_face * L_Smooth_L1(Δface); δ_head and δ_face are the weights for predicting head and face offsets. If the corresponding human body does not have a head / face label, its value is 0 and it is not included in the training. L_Smooth_L1 is calculated using Smooth L1 Loss. Δhead is the difference between the predicted head offset relative to the body and the actual head offset relative to the body, and Δface is the difference between the predicted face offset relative to the body and the actual face offset relative to the body.

[0009] In one embodiment, the method for constructing the human body multi-part joint detection model further includes, in the case of, the total loss function for model training: L_body, L_head, and L_face are composed of the category classification loss Focal_Loss and the bounding box regression loss SmoothL1Loss, with the function: L_det = β * L_Focal + (1- β) * L_Smooth_L1, where L_det represents L_body, L_head, and L_face, respectively.

[0010] In one implementation, the process of training the network model parameters further includes: During the warm-up phase, λ_offset is set to 0; In subsequent joint training phases, λ_offset is gradually increased.

[0011] In one embodiment, the method for constructing the human body multi-part joint detection model further includes: Cluster analysis is performed on the human bounding box size of the training set to obtain the anchor box size of the human detection head, and the sizes of the anchor boxes of the human detection head, the head detection head, and the face detection head decrease sequentially.

[0012] Secondly, the present invention provides a multi-part joint detection model for the human body, comprising: Feature extraction backbone network is used to extract multi-level features from the input image; A feature fusion network is used to receive multi-level features extracted by the feature extraction backbone network and generate multi-scale feature maps with rich semantic information and different spatial resolutions. The joint detection head includes: a human body detection head, a head detection head, a face detection head, and a head / face offset regression branch; Each detection head is a lightweight convolutional network used to transform each anchor point on the feature map into a specific anchor box and the detection result of each anchor box. The detection result includes: classification confidence and bounding box regression. Classification confidence score: Outputs the confidence score that the corresponding component exists at each spatial location; Bounding box regression: Outputs the coordinate adjustment of the bounding box at each spatial location relative to each Anchor point; The head / face offset regression branch contains a regression network consisting of at least one convolutional layer, which is used to predict the relative offset of the head / face relative to the human body, taking the feature region corresponding to the human body detection box as input. The joint detection head is trained using a pre-prepared training dataset, in which each image is hierarchically labeled: Human instance annotation: The full-body outline of each student is annotated with a rectangle, which is defined as the human detection box; Head instance annotation: Inside each human body detection box, annotate the head area of ​​the human body, which is defined as the head detection box. If the head is not visible, the annotation is empty; Face instance annotation: Inside each head detection box, annotate the face region of the human body, which is defined as the face detection box. If the face is not visible, the annotation is empty; The human body multi-part joint detection model also includes: The post-processing module is used to acquire the candidate bounding boxes lists of human body, head, and face output by the three detection heads running independently, as well as the detection results of each candidate bounding box, which is the anchor box. Using the relative offset predicted by the head / face offset regression branch, for each human body detection box with a confidence score higher than a preset threshold, the module decodes the predicted head reference box and the predicted face reference box. Based on the positional relationship between the predicted head reference box and each head detection box in the head candidate box list, and the confidence score of each head detection box in the head candidate box list, the module selects one head detection box from the head candidate box list as the associated head detection box of the human body detection box with a confidence score higher than the preset threshold. Based on the positional relationship between the predicted face reference box and each face detection box in the face candidate box list, and the confidence score of each face detection box in the face candidate box list, the module selects one face detection box from the face candidate box list as the associated face detection box of the human body detection box with a confidence score higher than the preset threshold. Output the associated triplet of body bounding box, head bounding box, and face bounding box for each character.

[0013] Thirdly, the present invention provides a computer device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method of any of the above embodiments.

[0014] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that the program, when executed by a processor, implements the method of any of the above embodiments.

[0015] This invention incorporates an offset regression branch, abandoning the direct location of components and instead learning the conditional probability distribution (i.e., offset) of a head relative to the human body. During the inference phase, this offset is not directly used as the final location coordinates, but rather as a spatial pointer to filter candidates with the best geometric relationship from the independently detected head set. This design transforms the "localization problem" into an "association verification problem," fundamentally avoiding coordinate competition between overlapping targets. In dense scenes, even if the head detection head independently detects multiple overlapping heads, this offset constraint can uniquely associate them back to the correct human body, thus resolving mismatch issues. Attached Figure Description

[0016] Figure 1 This is a schematic diagram of the overall process of one embodiment of the present invention.

[0017] Figure 2 This is a schematic diagram of image layer labeling for the training dataset of this invention.

[0018] Figure 3 This is a structural diagram of the human body multi-part joint detection model according to Embodiment 1 of the present invention.

[0019] Figure 4 This is a schematic diagram of the matching between the human body detection frame and the human head detection frame in Embodiment 1 of the present invention. Detailed Implementation

[0020] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0021] It should be noted that the terms "first, second, ..." used in the embodiments of the present invention are merely used to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, ..." can be interchanged in a specific order or sequence where permissible. It should be understood that the objects distinguished by "first, second, ..." can be interchanged where appropriate so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. Example 1

[0022] Please see Figure 1 , Figure 1This is a flowchart illustrating a method for constructing a multi-part joint detection model of the human body according to Embodiment 1 of the present invention. The method includes steps S110, S120, S130, S140, S150, and S160. It should be noted that steps S110, S120, S130, S140, S150, and S160 are merely reference numerals used to clearly explain the embodiments and accompanying drawings. Figure 1 The correspondence is not intended to limit the order of steps in this embodiment.

[0023] Step S110: Obtain the training dataset; Each image in the training dataset is hierarchically labeled: Human instance annotation: The full-body outline of each student is annotated with a rectangle, which is defined as the human detection box; Head instance annotation: Inside each human body detection box, annotate the head area of ​​the human body, which is defined as the head detection box. If the head is not visible, the annotation is empty; Face instance annotation: Inside each head detection box, annotate the face region of the human body, which is defined as the face detection box. If the face is not visible, the annotation is empty; Step S120: Construct a feature extraction backbone network to extract multi-level features from the input image; Step S130: Construct a feature fusion network to receive multi-level features extracted by the feature extraction backbone network and generate multi-scale feature maps with rich semantic information and different spatial resolutions. Step S140: Construct the joint detection head; The combined detection head includes: a human body detection head, a head detection head, a face detection head, and a head / face offset regression branch; Each detection head is a lightweight convolutional network used to transform each anchor point on the feature map into a specific anchor box and the detection result of each anchor box. The detection result includes: classification confidence and bounding box regression. Classification confidence score: Outputs the confidence score that the corresponding component exists at each spatial location; Bounding box regression: Outputs the coordinate adjustment of the bounding box at each spatial location relative to each Anchor point; The head / face offset regression branch contains a regression network consisting of at least one convolutional layer, which is used to predict the relative offset of the head / face relative to the human body, taking the feature region corresponding to the human body detection box as input. Step S150: Train the joint detection head using the training dataset; Step S160: Construct a post-processing module, which is used to execute the inference process and finally output the associated triplet of the human body bounding box-head bounding box-face bounding box for each character; The reasoning process includes: step S161 Step S161: Obtain the candidate bounding boxes list of human body, human head and human face output by the three detection heads running independently, as well as the detection results of each candidate bounding box. Each candidate bounding box is the Anchor box. Step S162: Using the relative offset predicted by the head / face offset regression branch, decode the predicted head reference box and the predicted face reference box for each human detection box with a confidence level higher than a preset threshold. Step S163: Based on the positional relationship between the predicted head reference box and each head detection box in the head candidate box list, and the confidence score of each head detection box in the head candidate box list, select a head detection box from the head candidate box list as the head detection box associated with the human body detection box that exceeds the preset threshold. Step S164: Based on the positional relationship between the predicted face reference box and each face detection box in the face candidate box list, and the confidence score of each face detection box in the face candidate box list, select a face detection box from the face candidate box list as the face detection box associated with the human body detection box that exceeds the preset threshold. Step S165: Output the associated triplet of body bounding box - head bounding box - face bounding box for each character.

[0024] like Figure 2 As shown, Figure 2 This is a schematic diagram of image layered annotation in the training dataset of this invention. Figure 2 The example provided is simply labeled, but in reality, everyone should be labeled.

[0025] Step S110: Obtain the training dataset, such as... Figure 2 As shown, each image is labeled in the following layers: Human body instance annotation: The full-body outline of each student is annotated with a rectangle and defined as the human body detection box B_body.

[0026] Head instance annotation: Within each B_body, annotate the student's head region (usually including hair and part of the neck), defined as the head detection box B_head. If the head is not visible, it can be empty.

[0027] Facial instance annotation: Within each B_head, the student's facial region (mainly facial features) is further annotated and defined as the face detection bounding box B_face. If the face is not visible, it can be empty.

[0028] Specifically, a student image dataset for classroom scenarios was created, and a dedicated image dataset was constructed. The construction method of this dataset includes the following key design elements: A. The data collection scenario must be a real classroom environment, including but not limited to classrooms, laboratories, and lecture halls, to ensure coverage of the typical application scenarios with high density, multiple poses, and partial occlusion that this method aims to address.

[0029] B. Data Collection Diversity Control: During the data collection process, the following key variables are systematically controlled to improve the generalization ability of the dataset: Density variation: includes sparse (single), medium density (group), and high density (whole class, group activities) student distribution.

[0030] Perspective changes: Covers multiple angle images such as front view of the podium, top view from the back of the classroom, and side view at eye level, simulating different camera positions for monitoring, teaching analysis, etc.

[0031] Lighting variations: including natural light, indoor lighting, mixed light, and some backlighting conditions.

[0032] Occlusion types: Ensure that the dataset contains rich interpersonal occlusion (students occluding each other), object occlusion (occlusion by desks, computers, books), and self-occlusion (postures such as raising hands, turning heads, etc.).

[0033] C. Data Acquisition Method: Real classroom video streams are acquired using high-definition camera equipment, and image frames are obtained by sampling at fixed intervals; at the same time, some publicly available classroom scene images can be included to supplement data diversity.

[0034] like Figure 3 As shown, Figure 3 This is a structural diagram of the human body multi-part joint detection model of the present invention.

[0035] In step S120, a feature extraction backbone network is constructed to extract multi-level features from the input image. This feature extraction backbone network can employ structures such as ResNet, CSPDarknet, or Swin Transformer to extract multi-level features from the input image, such as... Figure 3 Features of P3, P4, and P5 layers in the model.

[0036] In step S130, a feature fusion network is constructed to receive multi-level features extracted by the feature extraction backbone network and generate multi-scale feature maps with rich semantic information and different spatial resolutions. This feature fusion network can employ either FPN or PAN feature fusion structures, receiving multi-level feature maps from the backbone network and generating multi-scale feature maps {P3, P4, P5,...} with rich semantic information and different spatial resolutions to address student targets at different scales.

[0037] Step S140 is the core step of the present invention. The constructed joint detection head includes: a human body detection head, a human head detection head, a human face detection head, and a human head / face offset regression branch.

[0038] The detection head is typically a lightweight convolutional network whose core task is to transform each point (anchor point) on the feature map into a specific detection result.

[0039] Output two results: Classification confidence score: Outputs the confidence score that the corresponding component exists at each spatial location.

[0040] Bounding box regression: Outputs the coordinate offset (Δx, Δy, Δw, Δh) of the bounding box at each spatial location relative to the preset Anchor.

[0041] Each detection head has a similar function; the face detection head will be used as an example below.

[0042] After the face detection head detects a face bounding box, it obtains the basic values ​​(x, y, w, h) of the face bounding box. However, these basic values ​​are not the final values; the box needs to be adjusted. This adjustment is called bounding box regression, which is used to translate and scale the preset anchor boxes to more accurately fit the true boundaries of the face. Additionally, a confidence score is provided to indicate the confidence level of whether a face exists within a given preset anchor box (the corresponding component for the face detection head is the face, the corresponding component for the head detection head is the head, and the corresponding component for the body detection head is the body).

[0043] The face detection head predicts the confidence score and coordinate offset of the anchor corresponding to each grid point in the input feature map, and then performs coordinate decoding on the original anchor to obtain all the corresponding face candidate boxes in the input feature map.

[0044] It should be noted that multiple anchor boxes of different sizes can be set for each feature map grid point. In this case, multiple anchor boxes can be generated for one feature map grid point.

[0045] During the training phase, IoU calculation is performed using candidate face boxes and ground truth labeled face boxes. Candidate boxes with IoU greater than a threshold are defined as positive samples and participate in subsequent loss function calculations. Network parameters are adjusted through backpropagation of the loss gradient.

[0046] During the inference phase, low-confidence non-face bounding boxes and overlapping redundant boxes are removed through confidence filtering and non-maximum filtering, and the best face detection boxes are retained. Each detection box contains the coordinates and confidence of the detection box.

[0047] All of the above classification confidence scores and bounding box regression values ​​are trained through the process in step S150.

[0048] The various adjustments made by the human body detection head, head detection head, and face detection head are internal adjustments implemented by the human body, head, and face themselves, and the relationship between the human body, head, and face remains "independent." The head / face offset regression branch, however, is used to predict the positions of the head and face bounding boxes based on the position of the human body bounding box, thus ending the "independent" situation between the human body, head, and face. The predicted "relative offset of the head / face relative to the human body" is the empirically derived position of the head / face within a given human body bounding box.

[0049] The head / face offset regression branch is also trained through step S150.

[0050] Step S160 constructs a post-processing module, which is used to execute the inference process. This inference process is the process of using the results of the joint detection head to obtain the associated triplet of human body bounding box - human head bounding box - human face bounding box for each human figure.

[0051] Each detection head transforms each anchor point on the feature map into a specific anchor box. At this point, there are several anchor boxes, and these anchor boxes are called candidate boxes.

[0052] The first step in the inference process, namely step S161, is to obtain these candidate boxes and their detection results.

[0053] These candidate bounding boxes include human body candidate boxes, head candidate boxes, and face candidate boxes. In step S162, the human body candidate boxes are first filtered, that is, human body detection boxes with a confidence level higher than a preset threshold are selected from the human body candidate boxes. These human body detection boxes are all reliable human body detection boxes. Then, for these filtered human body detection boxes, the relative offset predicted by the head / face offset regression branch is used to decode the predicted head reference box and the predicted face reference box. As shown above, the predicted head reference box and face reference box here are the positions of the head / face within the human body box that are derived from experience.

[0054] After selecting a batch of reliable human detection boxes in step S162, the best-matching head and face detection boxes are associated with them. Step S163 is the matching process between human detection boxes and head detection boxes, and step S164 is the matching process between human detection boxes and face detection boxes.

[0055] Specifically, in step S163, the correlation between the predicted head reference box and each head detection box in the candidate head box list is determined by the positional relationship between them. For example... Figure 4 As shown, Figure 4 This is a schematic diagram illustrating the matching of the human body detection frame and the human head detection frame. Figure 4 In the diagram, the large dashed box is the human detection box, the small dashed box is the predicted head reference box, and the other small solid boxes are head candidate boxes. Step 163 uses the predicted head reference box (i.e., the small dashed box) as a reference position to infer which head candidate box (the small solid box) is the most reasonable and best-matching head detection box.

[0056] In one implementation, this positional relationship is the distance between the head detection boxes in the head candidate box list and the predicted head reference boxes, such as improved IoU or normalized center distance.

[0057] In addition to positional relationship, the head detection head generates a confidence score for each head candidate bounding box, which is also used in step S163. Finally, based on the two dimensions of positional relationship and confidence score, a head detection box that matches the human body bounding box selected in step S162 is selected from the many head candidate bounding boxes and associated with it.

[0058] In one implementation, the matching selection process may be as follows: using a preset score model, the score of each head detection box in the candidate head box list is obtained, and the head detection box with the highest score is selected as the head detection box associated with the human body detection box that is higher than the preset threshold. The preset score model is a model with the distance and the confidence score as parameters. For example, the preset score model is: Score = ɑ * dist_norm(H_i,H_ref_i) + (1 - ɑ) * Conf_head_i; Where α is a preset coefficient, the head detection box is H_i, the predicted head reference box is H_ref_i, dist_norm(H_i,H_ref_i) represents the distance between the head detection box and the predicted head reference box, and Conf_head_i is the confidence score.

[0059] Step S164 is similar to step S163. Step S163 selects a head detection box as the head detection box associated with the human body detection box that is above the preset threshold. Step S164 selects a face detection box as the face detection box associated with the human body detection box that is above the preset threshold. It can use the process of step S163 and the same scoring model, which will not be explained in detail here.

[0060] After steps S163 and S164, the human body bounding box is associated with the head bounding box and the face bounding box respectively. At this point, the associated triplet of human body bounding box-head bounding box-face bounding box for each human body can be output.

[0061] In this method, taking head detection as an example, existing technologies directly regress the absolute coordinates of the head in the image, which is prone to coordinate confusion due to occlusion or overlap in dense scenes. This method, however, incorporates an offset regression branch, abandoning the direct location of parts and instead learning the conditional probability distribution (i.e., offset) of the head relative to the human body. During the inference phase, this offset is not directly used as the final location coordinates, but rather as a spatial pointer to filter out the candidate with the best geometric relationship from the independently detected head set. This design transforms the "localization problem" into an "association verification problem," fundamentally avoiding coordinate competition between overlapping targets. In dense scenes, even if the head detection head independently detects multiple overlapping heads, this offset constraint can uniquely associate them back to the correct human body, thus solving the mismatch problem.

[0062] In one implementation, the head / face offset regression branch is configured to perform the process of predicting the relative offset of the head / face relative to the human body, taking the feature region corresponding to the human body detection box as input, including: The relative offset of the human head to the body: Center offset: δ_x = (x_head - x_body) / w_body, δ_y = (y_head - y_body) / h_body; Scale offset: δ_w = log(w_head / w_body), δ_h = log(h_head / h_body); Where x_head is the x-axis coordinate of the center point of the head detection box, y_head is the y-axis coordinate of the center point of the head detection box, w_head is the width of the head detection box, h_head is the height of the head detection box, x_body is the x-axis coordinate of the center point of the human body detection box, y_body is the y-axis coordinate of the center point of the human body detection box, w_body is the width of the human body detection box, and h_body is the height of the human body detection box. The relative offset of the face to the body: Center offset: δ_x = (x_face - x_body) / w_body, δ_y = (y_face - y_body) / h_body; Scale offset: δ_w = log(w_face / w_body), δ_h = log(h_face / h_body); Where x_face is the x-axis coordinate of the center point of the head detection box, y_face is the y-axis coordinate of the center point of the head detection box, w_face is the width of the head detection box, h_face is the height of the head detection box, x_body is the x-axis coordinate of the center point of the human body detection box, y_body is the y-axis coordinate of the center point of the human body detection box, w_body is the width of the human body detection box, and h_body is the height of the human body detection box.

[0063] In one embodiment, the method for constructing the human body multi-part joint detection model further includes step S170.

[0064] Step S170: Design the network's loss function and train the network model parameters; The total loss function for model training includes: L = λ_body * L_body + λ_head * L_head + λ_face * L_face + λ_offset *L_offset; L_body, L_head, and L_face are the loss function parts corresponding to the human body, head, and face, respectively. λ_body, λ_head, and λ_face are the loss weights for the human body, head, and face, respectively. The weight of λ_body is higher than that of λ_head and λ_face. If there is no head / face label in the corresponding human body, the value is 0 and it is not included in the training. L_offset = δ_head * L_Smooth_L1(Δhead) + δ_face * L_Smooth_L1(Δface); δ_head and δ_face are the weights for predicting head and face offsets. If the corresponding human body does not have a head / face label, its value is 0 and it is not included in the training. L_Smooth_L1 is calculated using Smooth L1 Loss. Δhead is the difference between the predicted head offset relative to the body and the actual head offset relative to the body, and Δface is the difference between the predicted face offset relative to the body and the actual face offset relative to the body.

[0065] In this embodiment, the human body bounding box is used as the primary anchor point. Its detection accuracy directly affects the subsequent offset decoding of the human head / face. Therefore, λ_body is usually given a high weight.

[0066] L_offset: Association regression loss, using Smooth L1 Loss. The offset predicted by the network, approximating the ground truth offset between the human body and the head / face in the labels.

[0067] λ_head and λ_face are the weights for the head and face loss, respectively. If the corresponding human body does not have a head / face label, the value is 0, and they are not included in the training to avoid propagating errors. δ_head and δ_face are the weights for the head and face offset prediction loss, respectively. If the corresponding human body does not have a head / face label, the value is 0, and they are not included in the training to avoid propagating errors.

[0068] In one implementation, the total loss function for model training further includes: L_body, L_head, and L_face are composed of the category classification loss Focal_Loss and the bounding box regression loss SmoothL1Loss, with the function: L_det = β * L_Focal + (1- β) * L_Smooth_L1, where L_det represents L_body, L_head, and L_face, respectively.

[0069] In one implementation, the process of training the network model parameters further includes: During the warm-up phase, λ_offset is set to 0; In subsequent joint training phases, λ_offset is gradually increased.

[0070] During the warm-up phase (Epochs 1-10), λ_offset is set to 0 to allow the network to learn to detect each part independently. Then, in subsequent joint training phases, λ_offset is gradually increased to improve the unified detection capability of multiple parts through offset prediction association learning tasks.

[0071] In one embodiment, the method for constructing the human body multi-part joint detection model further includes step S180.

[0072] Step S180: Cluster analysis of the human body bounding box size in the training set to obtain the anchor box size of the human body detection head, and then decrease the size of the anchor box of the human body detection head, the head detection head, and the face detection head in that order.

[0073] In this embodiment, in order to accurately match the huge differences in scale between the human body, head, and face in the classroom scene, this method configures a set of Anchors with decreasing size for the three detection heads and optimized for the corresponding parts.

[0074] The anchors for the human body detection head need to match the student's full-body scale. Based on the human body bounding box size clustering analysis of the training set, larger-sized anchors are set to avoid interference from small scales and focus on locating the complete human body.

[0075] The head detection head anchor needs to match the student's head size. Its size is significantly smaller than the human body anchor but larger than the face anchor, accurately locating the head region within the human body bounding box, unaffected by complex backgrounds.

[0076] The face detection head needs to be matched to the student's facial scale. It is set to a minimal set of anchors to capture fine facial features, effectively detecting small faces even at low resolution or long distances. Example 2

[0077] Corresponding to the method in Example 1, such as Figure 3 As shown, the present invention also provides a joint detection model for multiple human body parts, including: a feature extraction backbone network, a feature fusion network, a joint detection head, and a post-processing module.

[0078] Feature extraction backbone network is used to extract multi-level features from the input image; A feature fusion network is used to receive multi-level features extracted by the feature extraction backbone network and generate multi-scale feature maps with rich semantic information and different spatial resolutions. The joint detection head includes: a human body detection head, a head detection head, a face detection head, and a head / face offset regression branch; Each detection head is a lightweight convolutional network used to transform each anchor point on the feature map into a specific anchor box and the detection result of each anchor box. The detection result includes: classification confidence and bounding box regression. Classification confidence score: Outputs the confidence score that the corresponding component exists at each spatial location; Bounding box regression: Outputs the coordinate adjustment of the bounding box at each spatial location relative to each Anchor point; The head / face offset regression branch contains a regression network consisting of at least one convolutional layer, which is used to predict the relative offset of the head / face relative to the human body, taking the feature region corresponding to the human body detection box as input. The joint detection head is trained using a pre-prepared training dataset, in which each image is hierarchically labeled: Human instance annotation: The full-body outline of each student is annotated with a rectangle, which is defined as the human detection box; Head instance annotation: Inside each human body detection box, annotate the head area of ​​the human body, which is defined as the head detection box. If the head is not visible, the annotation is empty; Face instance annotation: Inside each head detection box, annotate the face region of the human body, which is defined as the face detection box. If the face is not visible, the annotation is empty; The human body multi-part joint detection model also includes: The post-processing module is used to acquire the candidate bounding boxes lists of human body, head, and face output by the three detection heads running independently, as well as the detection results of each candidate bounding box, which is the anchor box. Using the relative offset predicted by the head / face offset regression branch, for each human body detection box with a confidence score higher than a preset threshold, the module decodes the predicted head reference box and the predicted face reference box. Based on the positional relationship between the predicted head reference box and each head detection box in the head candidate box list, and the confidence score of each head detection box in the head candidate box list, the module selects one head detection box from the head candidate box list as the associated head detection box of the human body detection box with a confidence score higher than the preset threshold. Based on the positional relationship between the predicted face reference box and each face detection box in the face candidate box list, and the confidence score of each face detection box in the face candidate box list, the module selects one face detection box from the face candidate box list as the associated face detection box of the human body detection box with a confidence score higher than the preset threshold. Output the associated triplet of body bounding box, head bounding box, and face bounding box for each character.

[0079] In one implementation, the post-processing module executes the process of determining the positional relationship between the predicted head reference box and each head detection box in the head candidate box list, and the confidence score of each head detection box in the head candidate box list, and selecting a head detection box from the head candidate box list as the head detection box associated with the human body detection box that exceeds a preset threshold, including: Calculate the distance between each head detection box in the candidate head list and the predicted head reference box; Using a preset score model, the score of each head detection box in the candidate head box list is obtained, and the head detection box with the highest score is selected as the head detection box associated with the human body detection box that is higher than the preset threshold. The preset score model is a model with the distance and the confidence score as parameters. The post-processing module executes the process of selecting a face detection box from the face candidate box list based on the positional relationship between the predicted face reference box and each face detection box in the face candidate box list, and the confidence score of each face detection box in the face candidate box list, as the face detection box associated with the human body detection box that exceeds a preset threshold. This process includes: Calculate the distance between each head detection box in the candidate head list and the predicted head reference box; Using a preset score model, the score of each head detection box in the candidate head box list is obtained, and the head detection box with the highest score is selected as the head detection box associated with the human body detection box that is higher than the preset threshold. The preset score model is a model with the distance and the confidence score as parameters.

[0080] In one implementation, the head / face offset regression branch is configured to perform the process of predicting the relative offset of the head / face relative to the human body, taking the feature region corresponding to the human body detection box as input, including: The relative offset of the human head to the body: Center offset: δ_x = (x_head - x_body) / w_body, δ_y = (y_head - y_body) / h_body; Scale offset: δ_w = log(w_head / w_body), δ_h = log(h_head / h_body); Where x_head is the x-axis coordinate of the center point of the head detection box, y_head is the y-axis coordinate of the center point of the head detection box, w_head is the width of the head detection box, h_head is the height of the head detection box, x_body is the x-axis coordinate of the center point of the human body detection box, y_body is the y-axis coordinate of the center point of the human body detection box, w_body is the width of the human body detection box, and h_body is the height of the human body detection box. The relative offset of the face to the body: Center offset: δ_x = (x_face - x_body) / w_body, δ_y = (y_face - y_body) / h_body; Scale offset: δ_w = log(w_face / w_body), δ_h = log(h_face / h_body); Where x_face is the x-axis coordinate of the center point of the head detection box, y_face is the y-axis coordinate of the center point of the head detection box, w_face is the width of the head detection box, h_face is the height of the head detection box, x_body is the x-axis coordinate of the center point of the human body detection box, y_body is the y-axis coordinate of the center point of the human body detection box, w_body is the width of the human body detection box, and h_body is the height of the human body detection box.

[0081] In one implementation, the training process of the human body multi-part joint detection model includes: Design the loss function of the network and train the network model parameters; The total loss function for model training includes: L = λ_body * L_body + λ_head * L_head + λ_face * L_face + λ_offset *L_offset; L_body, L_head, and L_face are the loss function parts corresponding to the human body, head, and face, respectively. λ_body, λ_head, and λ_face are the loss weights for the human body, head, and face, respectively. The weight of λ_body is higher than that of λ_head and λ_face. If there is no head / face label in the corresponding human body, the value is 0 and it is not included in the training. L_offset = δ_head * L_Smooth_L1(Δhead) + δ_face * L_Smooth_L1(Δface); δ_head and δ_face are the weights for predicting head and face offsets. If the corresponding human body does not have a head / face label, its value is 0 and it is not included in the training. L_Smooth_L1 is calculated using Smooth L1 Loss. Δhead is the difference between the predicted head offset relative to the body and the actual head offset relative to the body, and Δface is the difference between the predicted face offset relative to the body and the actual face offset relative to the body.

[0082] In one implementation, the total loss function for model training further includes: L_body, L_head, and L_face are composed of the category classification loss Focal_Loss and the bounding box regression loss SmoothL1Loss, with the function: L_det = β * L_Focal + (1- β) * L_Smooth_L1, where L_det represents L_body, L_head, and L_face, respectively.

[0083] In one embodiment, the training process of the human body multi-part joint detection model further includes: During the warm-up phase, λ_offset is set to 0; In subsequent joint training phases, λ_offset is gradually increased.

[0084] In one implementation, the anchor boxes of the human body detection head are obtained based on cluster analysis of the human body box sizes in the training set; the sizes of the anchor boxes of the human body detection head, the head detection head, and the face detection head decrease sequentially.

[0085] This model incorporates an offset regression branch, abandoning the direct approach of locating parts and instead learning the conditional probability distribution (i.e., offset) of a head relative to the human body. During inference, this offset is not directly used as the final location coordinates but rather as a spatial pointer to filter candidates with the best geometric relationship from the independently detected head set. This design transforms the "localization problem" into an "association verification problem," fundamentally avoiding coordinate competition between overlapping targets. In dense scenes, even if the head detector independently detects multiple overlapping heads, this offset constraint can uniquely associate them back to the correct human body, thus resolving mismatch issues. Example

[0086] This invention also provides a storage medium storing computer instructions that, when executed by a processor, implement the method for constructing a human multi-part joint detection model according to any of the above embodiments.

[0087] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, random access memory (RAM), read-only memory (ROM), magnetic disks, or optical disks.

[0088] Alternatively, if the integrated units of the present invention are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present invention, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, terminal, or network device, etc.) to execute all or part of the methods of the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, RAM, ROM, magnetic disks, or optical disks.

[0089] Corresponding to the computer storage medium described above, one embodiment also provides a computer device, which includes a memory, an encoder, and a computer program stored in the memory and executable on the encoder, wherein the encoder executes the program to implement any of the methods for constructing a joint detection model of multiple human body parts as described in the above embodiments.

[0090] The aforementioned computer equipment incorporates an offset regression branch, thus abandoning the direct location of components and instead learning the conditional probability distribution (i.e., offset) of a head relative to the human body. During the inference phase, this offset is not directly used as the final location coordinates, but rather as a spatial pointer to filter candidates with the best geometric relationship from the independently detected head set. This design transforms the "localization problem" into an "association verification problem," fundamentally avoiding coordinate competition between overlapping targets. In dense scenes, even if the head detection head independently detects multiple overlapping heads, this offset constraint can uniquely associate them back to the correct human body, thereby resolving mismatch issues.

[0091] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0092] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A method for constructing a joint detection model for multiple human body parts, characterized in that, include: Obtain the training dataset; Construct a feature extraction backbone network to extract multi-level features from the input image; A feature fusion network is constructed to receive multi-level features extracted by the feature extraction backbone network and generate multi-scale feature maps with rich semantic information and different spatial resolutions. Construct a joint detection head; The joint detection head is trained using the training dataset; A post-processing module is constructed, which is used to execute the inference process and finally outputs the associated triplet of human body bounding box - human head bounding box - human face bounding box for each character; Each photo in the training dataset is hierarchically labeled: Human instance annotation: The full-body outline of each student is annotated with a rectangle, which is defined as the human detection box; Head instance annotation: Inside each human body detection box, annotate the head area of ​​the human body, which is defined as the head detection box. If the head is not visible, the annotation is empty; Face instance annotation: Inside each head detection box, annotate the face region of the human body, which is defined as the face detection box. If the face is not visible, the annotation is empty; The combined detection head includes: a human body detection head, a head detection head, a face detection head, and a head / face offset regression branch; Each detection head is a lightweight convolutional network used to transform each anchor point on the feature map into a specific anchor box and the detection result of each anchor box. The detection result includes: classification confidence and bounding box regression. Classification confidence score: Outputs the confidence score that the corresponding component exists at each spatial location; Bounding box regression: Outputs the coordinate adjustment of the bounding box at each spatial location relative to each Anchor point; The head / face offset regression branch contains a regression network consisting of at least one convolutional layer, which is used to predict the relative offset of the head / face relative to the human body, taking the feature region corresponding to the human body detection box as input. The reasoning process includes: Obtain the candidate bounding boxes for human body, head, and face output by the three detection heads running independently, as well as the detection results of each candidate bounding box. Each candidate bounding box is the Anchor box. Using the relative offset predicted by the head / face offset regression branch, the predicted head reference box and the predicted face reference box are decoded for each human detection box with a confidence level higher than a preset threshold. Based on the positional relationship between the predicted head reference box and each head detection box in the head candidate box list, and the confidence score of each head detection box in the head candidate box list, a head detection box is selected from the head candidate box list as the head detection box associated with the human body detection box that exceeds the preset threshold. Based on the positional relationship between the predicted face reference box and each face detection box in the face candidate box list, and the confidence score of each face detection box in the face candidate box list, a face detection box is selected from the face candidate box list as the face detection box associated with the human body detection box that exceeds the preset threshold. Output the associated triplet of body bounding box, head bounding box, and face bounding box for each character.

2. The method for constructing a joint detection model for multiple human body parts according to claim 1, characterized in that, The post-processing module executes the process of determining the positional relationship between the predicted head reference box and each head detection box in the head candidate box list, and the confidence score of each head detection box in the head candidate box list, and selecting a head detection box from the head candidate box list as the head detection box associated with the human body detection box that exceeds a preset threshold, including: Calculate the distance between each head detection box in the candidate head list and the predicted head reference box; Using a preset score model, the score of each head detection box in the candidate head box list is obtained, and the head detection box with the highest score is selected as the head detection box associated with the human body detection box that is higher than the preset threshold. The preset score model is a model with the distance and the confidence score as parameters. The post-processing module executes the process of selecting a face detection box from the face candidate box list based on the positional relationship between the predicted face reference box and each face detection box in the face candidate box list, and the confidence score of each face detection box in the face candidate box list, as the face detection box associated with the human body detection box that exceeds a preset threshold. This process includes: Calculate the distance between each head detection box in the candidate head list and the predicted head reference box; Using a preset score model, the score of each head detection box in the candidate head box list is obtained, and the head detection box with the highest score is selected as the head detection box associated with the human body detection box that is higher than the preset threshold. The preset score model is a model with the distance and the confidence score as parameters.

3. The method for constructing a joint detection model for multiple human body parts according to claim 1, characterized in that, The head / face offset regression branch is configured to perform the process of predicting the relative offset of the head / face relative to the human body, taking the feature region corresponding to the human body detection box as input, including: The relative offset of the human head to the body: Center offset: δ_x = (x_head - x_body) / w_body, δ_y = (y_head - y_body) / h_body; Scale offset: δ_w = log(w_head / w_body), δ_h = log(h_head / h_body); Where x_head is the x-axis coordinate of the center point of the head detection box, y_head is the y-axis coordinate of the center point of the head detection box, w_head is the width of the head detection box, h_head is the height of the head detection box, x_body is the x-axis coordinate of the center point of the human body detection box, y_body is the y-axis coordinate of the center point of the human body detection box, w_body is the width of the human body detection box, and h_body is the height of the human body detection box. The relative offset of the face to the body: Center offset: δ_x = (x_face - x_body) / w_body, δ_y = (y_face - y_body) / h_body; Scale offset: δ_w = log(w_face / w_body), δ_h = log(h_face / h_body); Where x_face is the x-axis coordinate of the center point of the head detection box, y_face is the y-axis coordinate of the center point of the head detection box, w_face is the width of the head detection box, h_face is the height of the head detection box, x_body is the x-axis coordinate of the center point of the human body detection box, y_body is the y-axis coordinate of the center point of the human body detection box, w_body is the width of the human body detection box, and h_body is the height of the human body detection box.

4. The method for constructing a joint detection model for multiple human body parts according to claim 1, characterized in that, Also includes: Design the loss function of the network and train the network model parameters; The total loss function for model training includes: L = λ_body * L_body + λ_head * L_head + λ_face * L_face + λ_offset * L_offset; L_body, L_head, and L_face are the loss function parts corresponding to the human body, head, and face, respectively. λ_body, λ_head, and λ_face are the loss weights for the human body, head, and face, respectively. The weight of λ_body is higher than that of λ_head and λ_face. If there is no head / face label in the corresponding human body, the value is 0 and it is not included in the training. L_offset = δ_head * L_Smooth_L1(Δhead) + δ_face * L_Smooth_L1(Δface); δ_head and δ_face are the weights for predicting head and face offsets. If the corresponding human body does not have a head / face label, its value is 0 and it is not included in the training. L_Smooth_L1 is calculated using Smooth L1 Loss. Δhead is the difference between the predicted head offset relative to the body and the actual head offset relative to the body, and Δface is the difference between the predicted face offset relative to the body and the actual face offset relative to the body.

5. The method for constructing a joint detection model for multiple human body parts according to claim 4, characterized in that, The total loss function for model training also includes: L_body, L_head, and L_face are composed of the category classification loss Focal_Loss and the bounding box regression loss Smooth L1Loss, with the function: L_det = β * L_Focal + (1- β) * L_Smooth_L1, where L_det represents L_body, L_head, and L_face, respectively.

6. The method for constructing a joint detection model for multiple human body parts according to claim 5, characterized in that, The process of training the network model parameters also includes: During the warm-up phase, λ_offset is set to 0; In subsequent joint training phases, λ_offset is gradually increased.

7. The method for constructing a joint detection model for multiple human body parts according to claim 1, characterized in that, Also includes: Cluster analysis is performed on the human bounding box size of the training set to obtain the anchor box size of the human detection head, and the sizes of the anchor boxes of the human detection head, the head detection head, and the face detection head decrease sequentially.

8. A multi-part joint detection model for the human body, characterized in that, include: Feature extraction backbone network is used to extract multi-level features from the input image; A feature fusion network is used to receive multi-level features extracted by the feature extraction backbone network and generate multi-scale feature maps with rich semantic information and different spatial resolutions. The joint detection head includes: a human body detection head, a head detection head, a face detection head, and a head / face offset regression branch; Each detection head is a lightweight convolutional network used to transform each anchor point on the feature map into a specific anchor box and the detection result of each anchor box. The detection result includes: classification confidence and bounding box regression. Classification confidence score: Outputs the confidence score that the corresponding component exists at each spatial location; Bounding box regression: Outputs the coordinate adjustment of the bounding box at each spatial location relative to each Anchor point; The head / face offset regression branch contains a regression network consisting of at least one convolutional layer, which is used to predict the relative offset of the head / face relative to the human body, taking the feature region corresponding to the human body detection box as input. The joint detection head is trained using a pre-prepared training dataset, in which each image is hierarchically labeled: Human instance annotation: The full-body outline of each student is annotated with a rectangle, which is defined as the human detection box; Head instance annotation: Inside each human body detection box, annotate the head area of ​​the human body, which is defined as the head detection box. If the head is not visible, the annotation is empty; Face instance annotation: Inside each head detection box, annotate the face region of the human body, which is defined as the face detection box. If the face is not visible, the annotation is empty; The human body multi-part joint detection model also includes: The post-processing module is used to acquire the candidate bounding boxes lists of human body, head, and face output by the three detection heads running independently, as well as the detection results of each candidate bounding box, which is the anchor box. Using the relative offset predicted by the head / face offset regression branch, for each human body detection box with a confidence score higher than a preset threshold, the module decodes the predicted head reference box and the predicted face reference box. Based on the positional relationship between the predicted head reference box and each head detection box in the head candidate box list, and the confidence score of each head detection box in the head candidate box list, the module selects one head detection box from the head candidate box list as the associated head detection box of the human body detection box with a confidence score higher than the preset threshold. Based on the positional relationship between the predicted face reference box and each face detection box in the face candidate box list, and the confidence score of each face detection box in the face candidate box list, the module selects one face detection box from the face candidate box list as the associated face detection box of the human body detection box with a confidence score higher than the preset threshold. Output the associated triplet of body bounding box, head bounding box, and face bounding box for each character.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1-7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-7.

Images