A method and system for training an image human body detection model

Abstract

The application provides a method and system for training a human body detection model of an image, the model comprising a transformer encoder, a transformer decoder, a first detection head and a second detection head, the method comprising: performing feature interaction on a processed training sample image in the transformer encoder; then decoding predicted first human body target frame information through the first detection head; combining the predicted first human body target frame information with labeled human body target frame information to obtain a first loss function; screening in the transformer encoder, performing feature interaction in the transformer decoder, and then decoding predicted second human body target frame information through the second detection head; then combining the predicted second human body target frame information with the labeled human body target frame information to obtain a second loss function; and finally mixing the first loss function and the second loss function to perform parameter optimization, so as to obtain an optimized human body detection model. The application can obtain a human body detection model with better technical effects.

Description

Technical Field

[0001] This invention relates to the field of target detection technology in computer vision, and in particular to a method and system for training image human detection models. Background Technology

[0002] In the field of computer vision, human detection has significant application value in scenarios such as public safety, intelligent transportation, and scenic area management. The core task of this technology is to accurately locate and identify all human targets from complex scene images. With breakthroughs in deep learning for visual tasks, deep learning-based end-to-end detection methods have gradually become the mainstream solution. However, existing methods still face challenges in human detection scenarios: traditional loss functions struggle to fully capture the fine-grained features of human targets, leading to high false negative and false positive rates. Current mainstream methods primarily improve performance by increasing model complexity or introducing other modalities of data. While these methods improve detection accuracy on benchmark datasets, they suffer from drawbacks such as high computational resource consumption. Summary of the Invention

[0003] The present invention aims to at least partially solve one of the technical problems in the related art.

[0004] This invention proposes a method for training an image human detection model. By introducing a dual-detection-head structure and a multi-stage feature interaction mechanism, the detection accuracy and robustness of the model are significantly improved, effectively realizing image human detection.

[0005] Another objective of this invention is to provide a system for training image human detection models.

[0006] To achieve the above objectives, the present invention provides a method for training an image human detection model, comprising:

[0007] The image human detection model includes a transformer encoder, a transformer decoder, a first detection head, and a second detection head; the method includes:

[0008] Extract multi-scale depth features from each of the multiple training sample images, including labeled human bounding box information;

[0009] The multi-scale depth features of each extracted training sample image are flattened into a feature sequence and positional encoding is added;

[0010] For each flattened feature sequence, a deformable attention mechanism is used in the transformer encoder to perform feature interaction and obtain multiple query sequences;

[0011] The query sequence after feature interaction is decoded by the first detection head to obtain the predicted first human target box information;

[0012] A first loss function is obtained based on the predicted first human bounding box information and the labeled human bounding box information in the training sample image;

[0013] The multiple query sequences obtained are filtered in the converter encoder to output the filtered query sequence;

[0014] The filtered query sequences are used in the transformer decoder for feature interaction by employing a deformable attention mechanism.

[0015] The query sequence after feature interaction is decoded by the second detection head to obtain the predicted second human target box information;

[0016] The predicted second human bounding box information and the labeled human bounding box information in the training sample images are combined to obtain the second loss function; and

[0017] The first loss function and the second loss function are mixed to optimize the parameters of the converter encoder, converter decoder, first detection head, and second detection head, resulting in an optimized human detection model.

[0018] The method for training an image human detection model according to embodiments of the present invention may also have the following additional technical features:

[0019] In one embodiment of the present invention, the predicted first human bounding box information includes the predicted location of the first human bounding box, and wherein combining the predicted first human bounding box information and the labeled human bounding box information in the training sample image to obtain a first loss function includes:

[0020] Calculate the true intersection-union ratio (KUOR) between each predicted first human bounding box location and the labeled human bounding box location in the training sample image; and

[0021] The first loss function is obtained using the true cross-union ratio.

[0022] In one embodiment of the present invention, the first loss function is the negative of the true cross-union ratio plus one.

[0023] In one embodiment of the present invention, the second human target box information includes at least the position of the second human target box, the classification score of the second human target box, and the positioning accuracy of the second human target box.

[0024] In one embodiment of the present invention, the positioning accuracy of the marked second human body target box is calculated using the following method:

[0025] Calculate the intersection-union ratio (IoU) between the predicted second human bounding box location and all annotated human bounding box locations in the training sample images; and

[0026] The highest cross-union ratio (CUI) among all human bounding box locations is taken as the localization accuracy of the predicted second human bounding box.

[0027] In one embodiment of the present invention, a second loss function is obtained by combining the predicted second human bounding box information and the labeled human bounding box information in the training sample image, including:

[0028] In the second detection head, the predicted second human target box position, the predicted second human target box classification score, and the predicted second human target box positioning accuracy are predicted to correspond to the query sequence in the converter decoder.

[0029] Obtain the labeled human bounding box location and the labeled human bounding box classification score from each training sample image;

[0030] The positioning accuracy of the annotated human bounding box is calculated based on the marked human bounding box position and the predicted second human bounding box position; and

[0031] The second loss function is calculated using the predicted second human bounding box position, the predicted second human bounding box classification score, the predicted second human bounding box positioning accuracy, and the labeled human bounding box position, the labeled human bounding box classification score, and the labeled human bounding box positioning accuracy.

[0032] In one embodiment of the present invention, the second loss function is a joint loss function, and the joint loss function is calculated using the following steps:

[0033] The positioning accuracy loss is calculated using the predicted second human target bounding box positioning accuracy and the labeled human target bounding box positioning accuracy.

[0034] The position loss is calculated using the predicted second human bounding box position and the labeled human bounding box position;

[0035] The classification loss is calculated using the predicted second human bounding box classification score and the labeled human bounding box classification score; and

[0036] The joint loss function is calculated using the positioning accuracy loss, the location loss, and the classification loss.

[0037] In one embodiment of the present invention, the optimized human detection model is obtained by mixing the first loss function and the second loss function to optimize the parameters of the converter encoder, converter decoder, first detection head, and second detection head, thereby including:

[0038] The final loss function is obtained by adding the first loss function and the second loss function; and

[0039] Gradient descent is used with the final loss function to optimize the parameters of the transformer encoder, transformer decoder, first detector head, and second detector head.

[0040] To achieve the above objectives, another aspect of the present invention provides a system for training an image human detection model, comprising:

[0041] The preprocessing module is used to extract multi-scale depth features from each of the multiple training sample images, including labeled human target bounding boxes; the extracted multi-scale depth features of each training sample image are flattened into a feature sequence and positional encoding is added;

[0042] The converter encoder module is used to perform feature interaction on each flattened feature sequence in the converter encoder by employing a deformable attention mechanism to obtain multiple query sequences;

[0043] The first detection head module is used to decode the query sequence after feature interaction to obtain the predicted first human target box information;

[0044] The first loss function calculation module is used to obtain a first loss function based on the predicted first human target box information and the labeled human target box information in the training sample image;

[0045] A converter encoder module is used to filter multiple obtained query sequences in the converter encoder to output the filtered query sequences.

[0046] The transformer decoder module is used to perform feature interaction by employing a deformable attention mechanism on the filtered query sequence in the transformer decoder;

[0047] The second detection head module is used to decode the query sequence after feature interaction to obtain the predicted second human target box information;

[0048] The second loss function calculation module is used to combine the predicted second human bounding box information and the labeled human bounding box information in the training sample image to obtain the second loss function; and

[0049] The loss function mixing module is used to mix the first loss function and the second loss function to optimize the parameters of the converter encoder, converter decoder, first detection head, and second detection head, so as to obtain an optimized human detection model.

[0050] The method and system for training an image-based human detection model according to embodiments of the present invention first perform global feature modeling on the training sample images in the transformer encoder, then decode the preliminary human target bounding box prediction results through a first detection head, and calculate a first loss function by combining it with real annotation information; subsequently, key features are selected in the encoder and sent to the transformer decoder for deeper feature interaction, and then the optimized target bounding box prediction results are output by a second detection head, and a second loss function is calculated; finally, the two loss functions are weighted and fused to guide the joint optimization of model parameters. This not only improves the accuracy and robustness of the human detection model, but also has good scalability and practicality, and is suitable for various application scenarios such as intelligent monitoring, human-computer interaction, and autonomous driving.

[0051] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0052] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0053] Figure 1 This is an example of the result of completing a human target bounding box detection task based on an input image according to an embodiment of the present invention.

[0054] Figure 2 This is a block diagram of the existing system for training human detection models in images.

[0055] Figure 3 Existing use is shown Figure 2 The flowchart shown illustrates the method for training an image-based human detection model.

[0056] Figure 4 The diagram shows a comparison of detection performance caused by insufficient strength of the loss function of existing trained image human detection models and systems.

[0057] Figure 5 This is a structural block diagram of a system for training an image human detection model according to an embodiment of the present invention.

[0058] Figure 6 For one use according to an embodiment of the present invention Figure 5 The flowchart shown illustrates the method for training an image-based human detection model.

[0059] Figure 7 This is a flowchart of a method for obtaining a first loss function based on a predicted first human target bounding box and the human target bounding box information labeled in training sample images, according to an embodiment of the present invention.

[0060] Figure 8 A schematic diagram illustrating the calculation of crossover ratio in the prior art is shown.

[0061] Figure 9 This is a flowchart illustrating a method for calculating a second loss function based on predicted second human bounding box information and labeled human bounding box information in training sample images, according to an embodiment of the present invention.

[0062] Figure 10 To implement the trained image human detection model according to embodiments of the present invention Figure 5 The image shows the effect of the detection results obtained by the application of System 500. Detailed Implementation

[0063] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0064] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0065] The method and system for training an image human detection model according to embodiments of the present invention are described below with reference to the accompanying drawings.

[0066] Figure 1 An example of the results of a human bounding box detection task based on an input image is shown. Figure 1 The input image 110 contains 15 human targets. Inputting image 110 into an image human detection system (not shown in the figure) that includes an image human detection model can output detection results 120 for human target bounding boxes. These detection results include at least the human target bounding box location 130 and the human target bounding box classification 140 (only one human target bounding box result is shown in the figure; the values ​​are for illustrative purposes only).

[0067] Image human detection models require training on multiple training sample images, including labeled human bounding boxes, to obtain a well-trained model. Only then can the image human detection system use this trained model to detect the aforementioned human bounding boxes in applications. Training the image human detection model can employ a dedicated system; however, as described in the background, existing systems for training image human detection models face challenges in improving detection performance. Figure 2 A block diagram of a system 200 for training image human detection models is shown. According to... Figure 2 System 200 includes a preprocessing module 210, a transformer encoder module 220, a transformer decoder module 230, a detection head module 240, and a loss function calculation module 250. The transformer encoder module 220, the transformer decoder module 230, and the detection head module 240 together form an image human detection model containing training parameters. By training to obtain optimized parameters for the transformer decoder, the transformer decoder, and the detection head, a trained image human detection model is obtained.

[0068] Figure 3 Existing use is shown Figure 2 The flowchart shown is for a method 300 for training an image human detection model using system 200. According to... Figure 2 and Figure 3 In step 310, the preprocessing module 210 receives multiple training sample images 202 including labeled human target bounding box information 204, and extracts multi-scale depth features for each training sample image.

[0069] In step 320, the preprocessing module 210 flattens the multi-scale depth features of the extracted training sample images into a feature sequence and adds positional encoding to input the sequence into the transformer encoder module 220.

[0070] In step 330, in the converter encoder module 220, each flattened feature sequence undergoes feature interaction using a deformable attention mechanism to obtain multiple query sequences. As those skilled in the art will know, the converter encoder in the converter encoder module 220 can be represented by the following mathematical model:

[0071]

[0072] X out =FFN E (Attn E (X))+X

[0073] Where X is the input flattened feature sequence, X out The output query sequence is represented by softmax(·), where softmax(·) denotes the softmax function, ReLU(·) denotes the linear activation function, and W... * ,b * It is the weight matrix. The parameters to be optimized in the mathematical model include...

[0074] In step 340, the converter encoder module 220 filters the obtained multiple query sequences and outputs the filtered query sequences, which can be input to the converter decoder module 230.

[0075] In step 350, the transformer decoder module 230 applies a deformable attention mechanism to the filtered query sequence for feature interaction, which is then input into the detection head module 240. As those skilled in the art will know, the transformer decoder in the transformer decoder module 230 can be represented by the following mathematical model:

[0076]

[0077]

[0078] Y out =FFN(CrossAttn(Y,X) out ))+Y

[0079] Where Y is the filtered query sequence input, X out Y is the query sequence output by the converter encoder. out For the query sequence after feature interaction, softmax(·) represents the softmax function, ReLU(·) represents the linear activation function, and W * ,b * It is the weight matrix. The parameters to be optimized in the mathematical model include the weight matrix.

[0080] In step 360, the detection head module 240 decodes the predicted human target box information using the query sequence after feature interaction, and inputs it into the loss function calculation module 250. As those skilled in the art will know, the detection head in the detection head module 240 can be represented by the following mathematical model:

[0081]

[0082] in It is the predicted location of the human body bounding box. It is the predicted human bounding box classification, Sigmoid(·) is the Sigmoid function, W * ,b * It is the weight matrix. The parameters to be optimized in the mathematical model include the weight matrix W. P W C ,b P ,b C .

[0083] In step 370, the loss function calculation module 250 combines the predicted human bounding box information and the labeled human bounding box information in the training sample images to calculate the loss function, optimize the parameters of the transformer decoder, the transformer decoder, and the detection head, thereby obtaining the trained image human detection model. As those skilled in the art know, the loss function is represented by the following mathematical model:

[0084]

[0085] Where P is the labeled human bounding box location, C is the labeled human bounding box classification, Loss1 represents the loss function for human bounding box location, and there are various types of loss functions in existing technologies, including L1 regression loss, L2 regression loss, etc.; Loss2 represents the loss function for human bounding box classification, and there are various types of loss functions in existing technologies, including cross-entropy loss function, Focal loss function, etc.

[0086] Existing technologies include optimization methods at least using gradient descent, employing the following mathematical formula for optimization:

[0087]

[0088] Where L represents the value of the loss function, α represents the descent step size, and W... * ,b * It refers to all the weight matrices in the above mathematical model, including W P W C ,b P ,b C .

[0089] like Figure 1-3 As shown, the loss function of the existing training image human detection model method is calculated only in step 370 (corresponding to loss function calculation module 250). This loss function originates from the detection head module 240 after the transformer decoder module 230 and the labeled human bounding boxes in the training sample images. On one hand, there is a lack of a directly acting loss function calculation module after the transformer encoder module 220. The optimization effect generated by the distant loss function calculation module 250 has already attenuated by the time it reaches the transformer encoder module 220, resulting in insignificant parameter optimization for the transformer encoder. On the other hand, the loss function calculated in step 370 is relatively simple, only considering the human bounding box position and the human bounding box classification score, thus having insignificant parameter optimization effects on the transformer decoder and detection head. Therefore, the strength of the loss function in the existing training image human detection model method and system is insufficient, resulting in unsatisfactory detection results from the trained image human detection model. Figure 4The diagram shows a comparison of detection performance caused by insufficient loss function strength in existing methods and systems for training human detection models. Figure 4 In the image, 'a' represents the correct human detection result, which includes 15 human bounding boxes. Figure 4 In the diagram, 'b' represents the image human detection result predicted by an existing image human detection model with insufficient loss function strength, which only includes 6 human bounding boxes. Therefore, there is a need to improve existing methods and systems for training image human detection models.

[0090] This invention proposes a system and method for training an image-based human detection model by enhancing the loss function. The method introduces a new first loss function into the output of the transformer encoder via a first detection head, and a second loss function, more complex than existing loss functions, into the output of the transformer decoder via a second detection head. Each of these loss functions includes a combination of the target bounding box and the ground truth bounding box, thereby strengthening the model's perception of localization accuracy and compensating for the insufficient strength of loss functions in existing technologies. This allows for better optimization of the human detection model's parameters using the enhanced loss function, resulting in a more technically effective human detection model.

[0091] In one implementation, Figure 5 A structural block diagram of a system 500 for training an image human detection model according to an embodiment of the present invention is shown. Figure 5 System 500 includes a preprocessing module 510, a converter encoder module 520, a first detection head module 530, a first loss function calculation module 540, a converter decoder module 550, a second detection head module 560, a second loss function calculation module 570, and a loss function mixing module 580. Compared with the prior art, system 500 adds the first detection head module 530, the first loss function calculation module 540, and the loss function mixing module 580, and improves the prior art detection head module 240 into a second detection head module, and improves the prior art loss function calculation module 250 into a second loss function calculation module 570.

[0092] Specifically, the preprocessing module 510 is used to extract multi-scale depth features from each of the multiple training sample images, including labeled human target bounding box information; flatten the extracted multi-scale depth features of each training sample image into a feature sequence and add position encoding;

[0093] The converter encoder module 520 is used to perform feature interaction on each flattened feature sequence in the converter encoder by employing a deformable attention mechanism to obtain multiple query sequences;

[0094] The first detection head module 530 is used to decode the query sequence after feature interaction into the predicted first human target box information through the first detection head.

[0095] The first loss function calculation module 540 is used to obtain a first loss function based on the predicted first human target box information and the labeled human target box information in the training sample image;

[0096] The converter encoder module 520 is used to filter multiple obtained query sequences in the converter encoder to output the filtered query sequences.

[0097] Transformer decoder module 550 is used to perform feature interaction on the filtered query sequence in the transformer decoder by employing a deformable attention mechanism.

[0098] The second detection head module 560 is used to decode the query sequence after feature interaction into the predicted second human target box information through the second detection head.

[0099] The second loss function calculation module 570 is used to combine the predicted second human target bounding box information and the labeled human target bounding box information in the training sample image to obtain a second loss function; and

[0100] The loss function mixing module 580 is used to mix the first loss function and the second loss function to optimize the parameters of the converter encoder, converter decoder, first detection head, and second detection head, so as to obtain an optimized human detection model.

[0101] In one implementation, Figure 6 An example of use according to an embodiment of the present invention is shown. Figure 5 The flowchart shown is for the method 600 of training an image human detection model in system 500. According to... Figure 5 and Figure 6 In step 605, the preprocessing module 510 receives multiple training sample images 502, each including labeled human target bounding box information 504, and extracts multi-scale depth features from each training sample image. This step can be performed using methods such as... Figure 3 The method used in step 310 is not described in detail here.

[0102] In step 610, the preprocessing module 510 flattens the multi-scale depth features of each extracted training sample image into a feature sequence and adds positional encoding before inputting it into the transformer encoder module 520. This step can be performed as follows: Figure 3 The method used in step 320 is not described in detail here.

[0103] In step 615, the transformer encoder module 520 performs feature interaction on each flattened feature sequence using a deformable attention mechanism, and obtains multiple query sequences. This step can be performed as follows: Figure 3 The method used in step 330 is not described in detail here.

[0104] In step 620, the first detection head module 530 decodes the flattened feature sequence after feature interaction to obtain the predicted first human target bounding box information. Here, the first detection head module 530 and... Figure 2 The detection head module 240 is similar to that in the previous one. In one embodiment, the first human target bounding box information includes the position of the first human target bounding box. This step is similar to... Figure 3 Compared to step 360 in the previous step, the predicted objects did not include human bounding boxes. Only the human target bounding box The specific methods used are similar.

[0105] In step 625, the first loss function calculation module 540 combines the predicted first human bounding box information with the labeled human bounding box information in the training sample image to obtain the first loss function. How this step is implemented will be described in detail later.

[0106] In step 630, the converter encoder module 520 filters the multiple output query sequences to output a filtered query sequence, which can be input into the converter decoder module 550. This step can be performed as follows: Figure 3 The method used in step 340 is not described in detail here.

[0107] In step 635, the transformer decoder module 550 employs a deformable attention mechanism to perform feature interaction on the filtered query sequence. This step can be achieved using, for example... Figure 3 The method used in step 350 is not described in detail here.

[0108] In step 640, the second detection head module 560 decodes the query sequence after feature interaction to obtain the predicted second human target bounding box information. How this step is implemented will be described in detail later.

[0109] In step 645, the second loss function calculation module 570 combines the predicted second human bounding box information with the labeled human bounding box information in the training sample image to obtain the second loss function. How this step is implemented will be described in detail later.

[0110] In step 650, the loss function mixing module 580 mixes the first loss function and the second loss function to optimize the parameters of the transformer encoder, transformer decoder, first detection head, and second detection head, thereby obtaining an optimized human detection model. How this step is implemented will be described in detail later.

[0111] In the aforementioned method 600, a new first loss function is introduced into the output of the transformer encoder via a first detection head in step 625, and a more complex second loss function than the existing loss function is introduced into the output of the transformer decoder via a second detection head in step 645. Each of these two loss functions includes a combination of the target bounding box and the ground truth bounding box. By mixing the first and second loss functions, the problem of insufficient strength of loss functions in existing technologies is overcome. Optimizing the human detection model using the hybrid loss function not only considers the optimization objective of introducing feature sequences into the transformer encoder, but also compensates for the semantic bias in the transformer decoder between feature vector localization and classification tasks. This strengthens the model's perception of localization accuracy, thereby better utilizing the enhanced loss function to optimize the parameters of the human detection model and obtain a human detection model with better technical performance. Furthermore, although method 600 increases the difficulty and training time, the resource consumption in the application of the trained model does not increase. Compared with mainstream methods that improve performance by increasing model complexity or introducing other modal data, it has the advantage of low resource consumption in the system application where the model is located.

[0112] In one implementation, regarding step 625 above, Figure 7 A flowchart of a method 700 for obtaining a first loss function based on a predicted first human bounding box and human bounding box information annotated in training sample images, according to an embodiment of the present invention, is shown. Figure 7 In step 710, the first detection head module 530 calculates the true intersection-union ratio (CIU) between each predicted first human bounding box position (obtained in step 620) and the labeled human bounding box position in the training sample image; in step 720, the first loss function calculation module 540 uses the true CIU to obtain the first loss function.

[0113] Intersection over Union (IoU) is an existing technique in computer vision and object detection, defined as the ratio of the area of ​​intersection to the area of ​​union of two regions. In one implementation, Figure 8A schematic diagram illustrating the calculation of the intersection-union ratio (IU / U) in the prior art is shown. The diagram includes a complete target bounding box 800 (including overlapping portions), another complete target bounding box 810 (including overlapping portions), and their overlapping portion 830. The union area of ​​the target bounding boxes is defined as the area of ​​target bounding box 800 plus the area of ​​target bounding box 810 minus the area of ​​the overlapping portion 830. The intersection area of ​​the target bounding boxes is defined as the area of ​​the overlapping portion 830. The ratio of the intersection area to the union area is the IU / U. The IU / U in step 720, and the IU / U mentioned below, all apply the aforementioned calculation method, that is, by using the predicted human target bounding box position and the actual human target bounding box position, the intersection area and union area of ​​the corresponding regions can be obtained, thus yielding the IU / U.

[0114] In one implementation, the first loss function can be expressed as the negative of the obtained true cross-union ratio plus one. This can be represented by the following mathematical model:

[0115] Loss enc =1-IoU1

[0116] Where IoU1 is the calculated true crossover-union ratio.

[0117] Those skilled in the art will know that, for step 625, the first loss function calculation module 540 can combine the predicted first human target box information and the labeled human target box information in the training sample image to obtain the first loss function, and other combination methods can also be used, such as using the norm of the difference between the predicted human target box position and the actual human target box position as the first loss function, etc.

[0118] In one implementation, regarding steps 640-645 above, the second human target bounding box information includes at least the location of the second human target bounding box, the classification score of the second human target bounding box, and the localization accuracy of the second human target bounding box. In other words, the predicted second human target bounding box information includes at least the predicted location of the second human target bounding box, the predicted classification score of the second human target bounding box, and the predicted localization accuracy of the second human target bounding box. The information of the labeled second human body target bounding box includes at least the location of the labeled second human body target bounding box, the classification score of the labeled second human body target bounding box, and the IoU accuracy of the labeled second human body target bounding box. Figure 9 A flowchart of a method 900 for calculating a second loss function based on predicted second human bounding box information and labeled human bounding box information in training sample images, according to an embodiment of the present invention, is shown. Figure 9In step 910, the second detection head module 560 predicts the predicted second human target box information corresponding to the query sequence in the converter encoder, including the predicted second human target box position, the predicted second human target box classification score, and the predicted second human target box positioning accuracy. The second detection head module 560 here, as described in step 620, is... Figure 2 The detection head module 240 is similar to that in the previous step. This step is similar to... Figure 3 Compared to step 360 in the previous step, the predicted object includes an additional prediction of positioning accuracy. The weight parameter can also be denoted as W. * ,b * .

[0119] In step 920, the labeled human bounding box location and the labeled human bounding box classification score are obtained from each training sample image.

[0120] In step 930, the second loss function calculation module 570 calculates the IoU (Intersection over Union) of the labeled human bounding box based on the labeled human bounding box position and the predicted second human bounding box position. In one embodiment, the IoU of the labeled second human bounding box is calculated as follows: the intersection-union ratio (IoU) of the predicted second human bounding box position and all labeled human bounding box positions in the training sample image is calculated; the largest IoU among all the corresponding IoUs of the human bounding box positions is taken as the IoU of the labeled second human bounding box.

[0121] In step 940, the second loss function calculation module 570 uses the predicted second human target box position, the predicted second human target box classification score, the predicted second human target box positioning accuracy, and the labeled human target box position, the labeled human target box classification score, and the labeled human target box positioning accuracy to calculate the second loss function.

[0122] In one implementation, the second loss function is a joint loss function, which can be calculated in the following way:

[0123] Loss dec =β1Loss IoU +β2Loss P +β3Loss C

[0124] Among them, Loss dec It is the joint loss function, Loss IoU It is a loss of positioning accuracy, Loss P It is the position loss in existing technology, Loss C This is the classification loss in existing technology. β1, β2, and β3 are the parameters of the function, which can be defined by the user.

[0125] Positioning accuracy loss IoU The following mathematical model can be used for calculation:

[0126]

[0127] Where IoU2 is the localization accuracy of the labeled human target bounding box. This represents the localization accuracy of the predicted second human bounding box. α and β are function parameters that can be defined by the user.

[0128] Positioning accuracy loss IoU Another mathematical model can be used for calculation:

[0129]

[0130] The meanings of the symbols are related to the aforementioned positioning accuracy loss (Loss). IoU The symbols used have the same meaning.

[0131] Position loss P The following mathematical model can be used for calculation:

[0132]

[0133] Position loss P Another mathematical model can be used for calculation:

[0134]

[0135] The meanings of the symbols are the same as those of the aforementioned positional loss. P The symbols used have the same meaning.

[0136] Classification Loss C The following mathematical model can be used for calculation:

[0137]

[0138] Where FocalLoss(·) is a commonly used loss function in existing technologies, and C is the classification score of the predicted second human object bounding box. It is the classification score of the labeled human body target bounding box.

[0139] Classification Loss C Another mathematical model can be used for calculation:

[0140]

[0141] CrossEntropyLoss(·) is a commonly used loss function in existing technologies, and C is the classification score of the predicted second human object bounding box. It is the classification score of the labeled human body target bounding box.

[0142] The second loss function can be any of the positioning accuracy loss, position loss, or classification loss included in the joint loss function, and can be combined to create a variety of implementation methods.

[0143] In one implementation, step 650, combining the first loss function and the second loss function to optimize the parameters of the converter encoder, converter decoder, first detection head, and second detection head to obtain an optimized human detection model, further includes: adding the first loss function and the second loss function to obtain a final loss function; and using the final loss function to perform gradient descent to optimize the parameters of the converter encoder, converter decoder, first detection head, and second detection head. This can be represented by the following mathematical model:

[0144] Loss all =Loss enc +Loss dec

[0145]

[0146] Loss all This represents the final value of the loss function, α represents the descent step size, and W... * ,b * These are all the weight matrices in the mathematical model described above. Other symbols have the same meanings as those in the mathematical model. The specific optimization method is similar to that in step 370.

[0147] Figure 10 This demonstrates a method that includes a pre-trained image human detection model. Figure 5 The image shows the effect of the detection results obtained by the application of System 500. (See image for example.) Figure 10 As shown, the System 500, which employs a supervised enhancement human detection model, significantly improves the model's detection performance and its generalization performance in dealing with unknown data.

[0148] Table 1 shows the comparative results of applying System 500 to the training image human detection model and then applying the model to the CrowdHuman validation set for performance metrics testing. Those skilled in the art generally use mean accuracy (AP, where a higher value indicates better performance) and log-mean loss rate (MR). -2The technical performance of the human detection model is measured by the Jaccard coefficient (JI, where a smaller value indicates better performance) and the Jaccard coefficient (JI, where a larger value indicates better performance). As shown in Table 1, compared to three other detection methods based on existing training image human detection models, AP and MR... -2 The performance indicators were significantly improved, and the JI indicator reached a competitive level, clearly superior to the other three methods. Generally speaking, human detection models excel in AP and MR... -2 If the two metrics in JI perform better, the model can be considered more effective.

[0149] Table 1

[0150]

[0151] The method and system for training an image human detection model according to embodiments of the present invention achieve phased target prediction through a dual-detection-head structure, enabling the model to progressively refine detection results at different levels, effectively improving the recognition ability of human bodies in small targets, occlusions, or complex backgrounds. Utilizing the self-attention mechanism of the transformer, multi-level feature interaction is achieved between the encoder and decoder, enhancing the model's understanding and utilization efficiency of global image information. The joint optimization of the two loss functions allows the model to obtain feedback signals from different angles during training, helping to accelerate convergence and improve model stability. Through multi-stage learning strategies and feature selection mechanisms, the model can better adapt to diverse scene changes, such as pose changes and illumination differences, thereby improving generalization performance in practical applications.

[0152] In summary, the training method and system provided by this invention not only improve the accuracy and robustness of human detection models, but also have good scalability and practicality, making them suitable for various application scenarios such as intelligent monitoring, human-computer interaction, and autonomous driving.

[0153] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0154] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

Claims

1. A method for training an image human detection model, characterized in that, The image human detection model includes a transformer encoder, a transformer decoder, a first detection head, and a second detection head; the method includes: Extract multi-scale depth features from each of the multiple training sample images, including labeled human bounding box information; The multi-scale depth features of each extracted training sample image are flattened into a feature sequence and positional encoding is added; For each flattened feature sequence, a deformable attention mechanism is used in the transformer encoder to perform feature interaction and obtain multiple query sequences; The query sequence after feature interaction is decoded by the first detection head to obtain the predicted first human target box information; A first loss function is obtained based on the predicted first human bounding box information and the labeled human bounding box information in the training sample image; wherein, the true intersection-union ratio (CIU) of each predicted first human bounding box position and the labeled human bounding box position in the training sample image is calculated, and the first loss function is obtained using the true CIU. The multiple query sequences obtained are filtered in the converter encoder to output the filtered query sequence; The filtered query sequences are used in the transformer decoder for feature interaction by employing a deformable attention mechanism. The query sequence after feature interaction is passed through the second detection head, and the second detection head predicts the predicted second human target box position, the predicted second human target box classification score and the predicted second human target box positioning accuracy corresponding to the query sequence in the converter decoder. The predicted second human bounding box information and the labeled human bounding box information in the training sample images are combined to obtain a second loss function; wherein, the labeled human bounding box position and the labeled human bounding box classification score are obtained from each training sample image, the labeled human bounding box localization accuracy is calculated based on the labeled human bounding box position and the predicted second human bounding box position, and the second loss function is calculated using the predicted second human bounding box position, the predicted second human bounding box classification score and the predicted second human bounding box localization accuracy, and the labeled human bounding box position, the labeled human bounding box classification score and the labeled human bounding box localization accuracy; and The first loss function and the second loss function are mixed to optimize the parameters of the converter encoder, converter decoder, first detection head, and second detection head, resulting in an optimized human detection model.

2. The method according to claim 1, characterized in that, The first loss function is the negative of the true cross-union ratio plus one.

3. The method according to claim 1, characterized in that, The positioning accuracy of the second human target bounding box is calculated using the following method: Calculate the intersection-union ratio (IoU) between the predicted second human bounding box location and all human bounding box locations labeled in the training sample images; as well as The highest cross-union ratio (CUI) among all human bounding box locations is taken as the localization accuracy of the predicted second human bounding box.

4. The method according to claim 1, characterized in that, The second loss function is the joint loss function, which is calculated using the following steps: The positioning accuracy loss is calculated using the predicted second human target bounding box positioning accuracy and the labeled human target bounding box positioning accuracy. The position loss is calculated using the predicted second human bounding box position and the labeled human bounding box position; The classification loss is calculated using the predicted second human bounding box classification score and the labeled human bounding box classification score. as well as The joint loss function is calculated using the positioning accuracy loss, the location loss, and the classification loss.

5. The method according to any one of claims 1-4, characterized in that, The first loss function and the second loss function are combined to optimize the parameters of the transformer encoder, transformer decoder, first detection head, and second detection head, thereby obtaining the optimized human detection model, which further includes: The final loss function is obtained by adding the first loss function and the second loss function; and Gradient descent is used with the final loss function to optimize the parameters of the transformer encoder, transformer decoder, first detector head, and second detector head.

6. A system for training an image human detection model, characterized in that, include: The preprocessing module is used to extract multi-scale depth features from each of the multiple training sample images, including labeled human target bounding box information. The multi-scale depth features of each extracted training sample image are flattened into a feature sequence and positional encoding is added; The converter encoder module is used to perform feature interaction on each flattened feature sequence in the converter encoder by employing a deformable attention mechanism to obtain multiple query sequences; The first detection head module is used to decode the query sequence after feature interaction to obtain the predicted first human target box information; The first loss function calculation module is used to obtain a first loss function based on the predicted first human bounding box information and the labeled human bounding box information in the training sample image; wherein, the true intersection-union ratio (CIU) of each predicted first human bounding box position and the labeled human bounding box position in the training sample image is calculated, and the first loss function is obtained using the true CIU. A converter encoder module is used to filter multiple obtained query sequences in the converter encoder to output the filtered query sequences. The transformer decoder module is used to perform feature interaction by employing a deformable attention mechanism on the filtered query sequence in the transformer decoder; The second detection head module is used to pass the query sequence after feature interaction through the second detection head, and predict the predicted second human target box position, the predicted second human target box classification score and the predicted second human target box positioning accuracy corresponding to the query sequence in the converter decoder in the second detection head. The second loss function calculation module is used to combine the predicted second human bounding box information and the labeled human bounding box information in the training sample images to obtain a second loss function; wherein, the labeled human bounding box position and the labeled human bounding box classification score are obtained from each training sample image, the labeled human bounding box localization accuracy is calculated based on the labeled human bounding box position and the predicted second human bounding box position, and the second loss function is calculated using the predicted second human bounding box position, the predicted second human bounding box classification score and the predicted second human bounding box localization accuracy, and the labeled human bounding box position, the labeled human bounding box classification score and the labeled human bounding box localization accuracy; and The loss function mixing module is used to mix the first loss function and the second loss function to optimize the parameters of the converter encoder, converter decoder, first detection head, and second detection head, so as to obtain an optimized human detection model.

Images