Target detection method and device based on dynamic threshold, equipment and storage medium

Abstract

The application provides a target detection method and device based on a dynamic threshold, equipment and a storage medium. The method comprises: inputting a to-be-detected image into a target detection network, processing the to-be-detected image by using a backbone network of the target detection network to obtain a first feature map; inputting the first feature map into a predetermined dynamic threshold prediction branch, predicting a dynamic threshold corresponding to a prediction box of the to-be-detected image by using the dynamic threshold prediction branch to fuse information of a classification branch and a regression branch; sorting the prediction box based on the dynamic threshold, and traversing according to a sorting result, and when an intersection over union of a subsequent prediction box and a current prediction box is higher than the dynamic threshold of the subsequent prediction box, removing the subsequent prediction box, and taking the prediction box remaining after the traversal as a target detection result. The application can dynamically adjust the threshold of non-maximum suppression, avoid missing detection of a target, and improve the accuracy of target detection.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a target detection method, apparatus, device and storage medium based on dynamic threshold. Background Technology

[0002] Non-maximum suppression (NMS) is a common post-processing technique in mainstream multi-object detection algorithms. Generally, deep learning-based object detection networks densely output many bounding boxes and their scores on an image. Non-maximum suppression sorts the bounding boxes based on their scores and, according to the intersection-union ratio (IUU) between them, removes overlapping bounding boxes on the same object, ensuring that only the highest-scoring bounding box is retained for each object, thereby reducing prediction overlap.

[0003] In existing object detection algorithms, whether using NMS or the improved soft-NMS, the threshold of NMS is fixed. This will lead to the problem of missed detection of actually overlapping objects in the image. That is, the NMS technology in existing object detection algorithms is prone to missing targets and has the problem of inaccurate detection in dense scenes and occluded scenes. Summary of the Invention

[0004] In view of this, embodiments of this application provide a target detection method, apparatus, device, and storage medium based on dynamic thresholds to solve the problems of missed target detection and inaccurate detection in dense and occluded scenes in the prior art.

[0005] A first aspect of this application provides a target detection method based on dynamic thresholding, comprising: inputting an image to be detected into a target detection network, processing the image to be detected using the backbone network of the target detection network to obtain a first feature map; inputting the first feature map into a predetermined dynamic threshold prediction branch, fusing information from a classification branch and a regression branch using the dynamic threshold prediction branch, and predicting a dynamic threshold corresponding to a prediction box of the image to be detected; sorting the prediction boxes based on the dynamic threshold, and traversing according to the sorting result; when the intersection-union ratio (IU) of a subsequent prediction box with the current prediction box is higher than the dynamic threshold of the subsequent prediction box, removing the subsequent prediction box, and using the remaining prediction boxes after traversal as the target detection result.

[0006] A second aspect of this application provides a target detection device based on dynamic thresholding, comprising: an input module configured to input an image to be detected into a target detection network, and to process the image to be detected using the backbone network of the target detection network to obtain a first feature map; a prediction module configured to input the first feature map into a predetermined dynamic threshold prediction branch, and to fuse information from a classification branch and a regression branch using the dynamic threshold prediction branch to predict a dynamic threshold corresponding to a prediction box of the image to be detected; and a traversal module configured to sort the prediction boxes based on the dynamic thresholding, and to traverse the prediction boxes according to the sorting results, wherein when the intersection-union ratio (IU) of a subsequent prediction box with the current prediction box is higher than the dynamic threshold of the subsequent prediction box, the subsequent prediction box is removed, and the remaining prediction boxes after traversal are taken as the target detection result.

[0007] A third aspect of this application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the above-described method.

[0008] A fourth aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the above-described method.

[0009] The above-described technical solutions adopted in the embodiments of this application can achieve the following beneficial effects:

[0010] This method involves inputting the image to be detected into an object detection network. The backbone of the network processes the image to obtain a first feature map. This first feature map is then input into a predetermined dynamic threshold prediction branch. This branch fuses information from the classification and regression branches to predict the dynamic threshold corresponding to the bounding boxes in the image. The predicted bounding boxes are then sorted based on the dynamic threshold, and the process iterates through the boxes. If the intersection-union ratio (IU) of a subsequent predicted bounding box with the current predicted bounding box is higher than the dynamic threshold of that box, the subsequent box is removed. The remaining predicted bounding boxes are then used as the object detection result. This method can dynamically adjust the non-maximum suppression threshold, fundamentally solving the problems of missed object detection and inaccurate detection in dense and occluded scenes. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1This is a flowchart illustrating the target detection method based on dynamic threshold provided in the embodiments of this application;

[0013] Figure 2 This is a schematic diagram of the structure of the dynamic threshold prediction branch provided in the embodiments of this application;

[0014] Figure 3 This is a schematic diagram of the target detection device based on dynamic threshold provided in the embodiments of this application;

[0015] Figure 4 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0016] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0017] As described in the background section, Non-maximum Suppression (NMS) is a common post-processing technique in mainstream multi-object detection algorithms. Generally, deep learning-based object detection networks densely output many bounding boxes and their scores on an image. Non-maximum suppression sorts the bounding boxes based on their scores and, according to the intersection-union ratio (IUU) between them, removes overlapping bounding boxes on the same object, ensuring that only the highest-scoring bounding box is retained for each object, thereby reducing prediction overlap.

[0018] However, the problem with existing non-maximum suppression algorithms lies in their use of a one-size-fits-all threshold (fixed threshold). This can lead to missed detections of actually overlapping targets in the image. For example, using a fixed threshold of 0.5 removes all detection boxes with an intersection-union ratio (IU) greater than 0.5. Therefore, if an IU actually exists in the scene, an error will occur. Consequently, the non-maximum suppression technique in existing object detection algorithms is prone to missed detections and suffers from inaccurate detection in dense and occluded scenes.

[0019] In view of this, this application provides a target detection method based on dynamic thresholding to solve the above problems. This application uses a dynamic threshold prediction branch to fuse information from the classification and regression branches during the target detection network detection process, predicting the dynamic threshold of the bounding box in the image to be detected, and then performing non-maximum suppression processing based on the dynamic threshold of the bounding box. Furthermore, the network architecture and pseudo-label calculation technique proposed in this application ensure the trainability of the dynamic threshold, enabling dynamic threshold adjustment. The threshold can be dynamically adjusted based on local density information in the image, solving the problem of missed detections in dense scenes and occluded scenes.

[0020] The technical solution of this application will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0021] Figure 1 This is a flowchart illustrating the target detection method based on dynamic threshold provided in the embodiments of this application. Figure 1 The target detection method based on dynamic thresholds can be executed by the server. For example... Figure 1 As shown, the target detection method based on dynamic thresholds may specifically include:

[0022] S101, The image to be detected is input into the target detection network, and the backbone network of the target detection network is used to process the image to be detected to obtain the first feature map;

[0023] S102, the first feature map is input into the predetermined dynamic threshold prediction branch, and the information of the classification branch and the regression branch are fused by the dynamic threshold prediction branch to predict the dynamic threshold corresponding to the prediction box of the image to be detected.

[0024] S103: Sort the predicted boxes based on the dynamic threshold and traverse them according to the sorting results. When the intersection-union ratio of the subsequent predicted box with the current predicted box is higher than the dynamic threshold of the subsequent predicted box, the subsequent predicted box is removed and the remaining predicted boxes after traversal are taken as the target detection results.

[0025] Specifically, the object detection network of this application can be a neural network improved based on mainstream object detection networks. Currently, mainstream object detection networks generally include a backbone network and a detection head. The detection head includes a classification branch and a regression branch. This application adds a dynamic threshold prediction branch to fuse the information of the classification branch and the regression branch, thereby realizing the prediction of the dynamic threshold of the prediction box.

[0026] In some embodiments, the dynamic threshold prediction branch is used to fuse information from the classification branch and the regression branch to predict the dynamic threshold corresponding to the prediction box in the image to be detected. This includes: fusing a first feature map with the feature map output by the classification branch, and obtaining a second feature map after convolution; and directly convolving the first feature map to obtain a third feature map; stacking the second and third feature maps and convolving them to obtain a fourth feature map; fusing the fourth feature map with the feature map output by the regression branch and convolving it to obtain a fifth feature map; and directly convolving the fourth feature map to obtain a sixth feature map; stacking the fifth and sixth feature maps and convolving them to obtain a seventh feature map; and convolving the seventh feature map to obtain the dynamic threshold corresponding to each prediction box in the image to be detected.

[0027] Specifically, in addition to the backbone network and the detection head (with classification and regression branches), the target detection network of this application also includes a dynamic threshold prediction branch. The structure of the dynamic threshold prediction branch in the target detection network of this application embodiment will be described in detail below with reference to the accompanying drawings. Figure 2 This is a schematic diagram of the structure of the dynamic threshold prediction branch provided in an embodiment of this application. For example... Figure 2 As shown, predicting a dynamic threshold using the dynamic threshold prediction branch in an object detection network can include:

[0028] First, the backbone network of object detection outputs the first feature map r1. After a 3x3 convolution, it is split into two paths. One path is added to the feature map output by the classification branch and then convolved with a 3x3 convolution to obtain the second feature map r2. The other path is directly processed by a 1x1 convolution to obtain the third feature map r3.

[0029] The second feature map r2 and the third feature map r3 are stacked (concat) and then convolved with 3x3 to obtain the fourth feature map r4. The fourth feature map r4 is then divided into two paths for processing. One path is added to the feature map output by the regression branch and then convolved with 3x3 to obtain the fifth feature map r5. The other path is convolved with 1x1 to obtain the sixth feature map r6.

[0030] Finally, the fifth feature map r5 and the sixth feature map r6 are stacked (concat) and then convolved with 3x3 to obtain the seventh feature map r7; the seventh feature map r7 is then convolved with 1x1 with 1 channel to obtain the final predicted dynamic threshold.

[0031] It should be noted that in the structure of the dynamic threshold prediction branch, since the classification branch of the object detection network contains discriminative feature information and the regression branch contains edge and overall range information, and both of these types of information are helpful for the prediction of dynamic thresholds, they need to be integrated into the threshold prediction process of the dynamic threshold prediction branch.

[0032] In the process of fusing information from the classification and regression branches, it is essential to ensure a certain degree of independence for the dynamic threshold prediction branch. Therefore, a split-path convolution and fusion architecture is designed to guarantee the independence of the dynamic threshold prediction branch. During the threshold prediction process of the aforementioned dynamic threshold prediction branch, the information from the classification and regression branches can be treated as auxiliary information (residual information) and added in, making the threshold prediction more accurate.

[0033] The above embodiments illustrate the threshold prediction process of the dynamic threshold prediction branch in the object detection network. This application also proposes a pseudo-label calculation technique to ensure the trainability of the dynamic threshold prediction branch. The training process of the dynamic threshold prediction branch will be described in detail below with reference to specific embodiments.

[0034] In some embodiments, after predicting the dynamic threshold corresponding to the predicted bounding box of the image to be detected, the method further includes: determining a first dynamic threshold pseudo-label corresponding to each anchor box according to a pre-set anchor box, and using the maximum overlap value of the target box corresponding to the anchor box as the second dynamic threshold pseudo-label of the anchor box; predicting the dynamic threshold of the anchor box using a dynamic threshold prediction branch; establishing a loss function based on the first dynamic threshold pseudo-label, the second dynamic threshold pseudo-label, and the dynamic threshold of the anchor box; and training the target detection network using the loss function.

[0035] Specifically, in an object detection network, N anchor boxes are set in advance. For an image, there will be K ground boxes (GT). To train an object detection network, a mapping scheme between anchor boxes and ground boxes needs to be determined. That is, each anchor box is responsible for predicting which ground box, or the anchor box belongs to the background and does not need to predict any ground box.

[0036] Furthermore, training an object detection network essentially involves training the classification and regression capabilities of each anchor box, while this application requires training the dynamic thresholding capability of each anchor box. During training, each sample image is input into the object detection network. For a given anchor box a1, its corresponding target box is already known, thus allowing the determination of the maximum overlapping object of its target box. This determines the generation pool e1 for the maximum overlapping object, referred to as the maximum overlapping generation pool. In this application, during the training of the object detection network, the first dynamic threshold pseudo-label is determined by calculating the intersection-union ratio (IUU) between the predicted box and other predicted boxes in the "maximum overlapping generation pool".

[0037] In some embodiments, determining a first dynamic threshold pseudo-label corresponding to each anchor box based on pre-set anchor boxes includes: determining a generation pool corresponding to each target box based on pre-set anchor boxes and the mapping relationship between target boxes and anchor boxes; inputting a training image into an object detection network, outputting a predicted box obtained by regressing the anchor boxes, and other predicted boxes corresponding to the anchor boxes in the generation pool; calculating the intersection-union ratio (IUR) between the predicted boxes and other predicted boxes, and using the IUR with the largest value as the first dynamic threshold pseudo-label corresponding to the anchor box.

[0038] Specifically, this application also proposes the concept of a generation pool. Since N anchor boxes are mapped to K ground truth (GT) boxes, each GT box has several corresponding anchor boxes. These anchor boxes corresponding to the GT boxes are used as the "generation pool" for that GT box. For example, if an image has 3 GT boxes, and 5, 9, and 3 anchor boxes corresponding to these 3 GT boxes respectively, then the sizes of the "generation pools" corresponding to the first, second, and third GT boxes in the image are 5, 9, and 3 respectively.

[0039] Furthermore, when determining the first dynamic threshold pseudo-label corresponding to each anchor box, after processing an image through the object detection network, anchor box a1 will regress into the corresponding predicted box b, and the m anchor boxes in the generation pool e1 will regress into m predicted boxes F ~ (f1, f2... f... m By calculating b and m prediction boxes (f1, f2... f... m The cross-union ratio (CUNR) of the anchor frame is used, and the maximum value of the CUNR is taken as the dynamic threshold pseudo-label 1 (i.e. the first dynamic threshold pseudo-label).

[0040] Furthermore, when determining the second dynamic threshold pseudo-label corresponding to each anchor frame, the maximum overlap value of the target frame corresponding to anchor frame a1 is used as the dynamic threshold pseudo-label 2 (i.e., the second dynamic threshold pseudo-label) of that anchor frame.

[0041] In some embodiments, the method further includes: determining all target boxes corresponding to the training image, calculating the intersection-union ratio (IUR) of each target box with other target boxes, taking the other target boxes corresponding to the maximum IUR as the maximum overlapping objects of the target boxes, and taking the value of the maximum IUR as the maximum overlap value.

[0042] Specifically, this application also proposes the concepts of calculating the maximum overlapping object and the maximum overlap value, which are explained in detail below.

[0043] For example, in a specific example, for K bounding boxes in an image, the "maximum intersection-union ratio" (UCR) of each bounding box with other bounding boxes is calculated. For instance, if an image has three bounding boxes, and the UCRs of the first bounding box with the second and third bounding boxes are 0.3 and 0.67 respectively, then the maximum UCR for the first bounding box is 0.67 (i.e., the maximum overlap value). Since the bounding box with the maximum UCR is the third bounding box, it is considered the "maximum overlapping object." Through the above calculations, the "maximum overlapping object" and "maximum overlap value" for each bounding box can be calculated. If a bounding box has no overlapping objects, then the maximum overlapping object for that bounding box is empty, and the maximum overlap value is 0.

[0044] In some embodiments, determining the generation pool corresponding to each target box includes: using the anchor boxes corresponding to the target boxes as the generation pool of the target boxes according to the mapping relationship between the target boxes and anchor boxes, and using the number of anchor boxes as the size of the generation pool.

[0045] Specifically, for an explanation of the concept of the generation pool and how to determine the generation pool of the target box, please refer to the content of the foregoing embodiments, which will not be repeated here.

[0046] Furthermore, this application proposes a dual pseudo-label supervision technique for dynamic thresholding to supervise the dynamic threshold prediction branch of the trained object detection network. As can be seen from the foregoing embodiments, after the first feature map is processed by the dynamic threshold prediction branch, each anchor box will predict a dynamic threshold score, and all anchor boxes will obtain dynamic threshold pseudo-label 1 and dynamic threshold pseudo-label 2.

[0047] It's important to note that the first dynamic threshold pseudo-label is dynamic both for all anchor boxes (i.e., the threshold for each anchor box is different) and during the training process (i.e., the value is different in each training round). This dynamic training ensures that the threshold pseudo-label and model capability evolve synchronously; therefore, the pseudo-label reflects the current model's capability to some extent. The second dynamic threshold pseudo-label is only dynamic for all anchor boxes, but it doesn't change during training. It reflects an ideal state, the state at which the model finally fits the model.

[0048] Furthermore, after obtaining the first dynamic threshold pseudo-label, the second dynamic threshold pseudo-label, and the dynamic threshold of the anchor box, this application establishes the following loss function, which is used to train the dynamic threshold prediction branch in the object detection network. In some embodiments, the loss function is expressed as:

[0049] loss=∑α*(p-y1) 2 +(1-α)*(p-2) 2

[0050] Where α represents a constant, p represents the dynamic threshold of the anchor box, y1 represents the pseudo-label of the first dynamic threshold, and y2 represents the pseudo-label of the second dynamic threshold.

[0051] Specifically, α is set to 0.8 in the early stages of model training and to 0.5 in the later stages of model training.

[0052] Furthermore, after the model training is completed, the original nonmaximum suppression technique is changed to dynamic nonmaximum suppression technique, that is, the predicted boxes are sorted based on the scores of the dynamic threshold, and traversed from the high score to the low score. If the intersection-union ratio (IoU) between the subsequent predicted box and the current predicted box is higher than the dynamic threshold of the subsequent predicted box, then the subsequent predicted box is removed.

[0053] The following are embodiments of the apparatus described in this application, which can be used to execute the embodiments of the method described in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the method described in this application.

[0054] Figure 3 This is a schematic diagram of the target detection device based on dynamic threshold provided in the embodiments of this application.

[0055] like Figure 3 As shown, the target detection device based on dynamic thresholds includes:

[0056] The input module 301 is configured to input the image to be detected into the target detection network, and use the backbone network of the target detection network to process the image to be detected to obtain a first feature map;

[0057] The prediction module 302 is configured to input the first feature map into a predetermined dynamic threshold prediction branch, and use the dynamic threshold prediction branch to fuse the information of the classification branch and the regression branch to predict the dynamic threshold corresponding to the prediction box of the image to be detected.

[0058] The traversal module 303 is configured to sort the predicted boxes based on a dynamic threshold and traverse them according to the sorting results. When the intersection-union ratio of a subsequent predicted box with the current predicted box is higher than the dynamic threshold of the subsequent predicted box, the subsequent predicted box is removed, and the remaining predicted boxes after traversal are taken as the target detection results.

[0059] In some embodiments, Figure 3The prediction module 302 fuses the first feature map with the feature map output by the classification branch, and obtains the second feature map after convolution processing. It also directly convolves the first feature map to obtain the third feature map. The second and third feature maps are stacked and convolved to obtain the fourth feature map. The fourth feature map is fused with the feature map output by the regression branch and obtained the fifth feature map after convolution processing. It also directly convolves the fourth feature map to obtain the sixth feature map. The fifth and sixth feature maps are stacked and convolved to obtain the seventh feature map. The seventh feature map is convolved to obtain the dynamic threshold corresponding to each prediction box in the image to be detected.

[0060] In some embodiments, Figure 3 After the training module 304 predicts the dynamic threshold corresponding to the prediction box of the image to be detected, it determines the first dynamic threshold pseudo-label corresponding to each anchor box according to the pre-set anchor box, and uses the maximum overlap value of the target box corresponding to the anchor box as the second dynamic threshold pseudo-label of the anchor box; it uses the dynamic threshold prediction branch to predict the dynamic threshold of the anchor box, establishes a loss function based on the first dynamic threshold pseudo-label, the second dynamic threshold pseudo-label and the dynamic threshold of the anchor box, and uses the loss function to train the target detection network.

[0061] In some embodiments, Figure 3 The training module 304 determines the generation pool corresponding to each target box based on the pre-set anchor boxes and the mapping relationship between the target boxes and the anchor boxes; it inputs the training image into the target detection network and outputs the predicted box obtained by regressing the anchor boxes, as well as other predicted boxes corresponding to the anchor boxes in the generation pool; it calculates the cross-union ratio between the predicted boxes and other predicted boxes, and uses the cross-union ratio with the largest value as the first dynamic threshold pseudo-label corresponding to the anchor box.

[0062] In some embodiments, Figure 3 The training module 304 determines all target boxes corresponding to the training image, calculates the intersection-union ratio (IUR) of each target box with other target boxes, takes the other target boxes with the maximum IUR as the maximum overlapping objects of the target boxes, and takes the value of the maximum IUR as the maximum overlap value.

[0063] In some embodiments, Figure 3 The training module 304 uses the anchor boxes corresponding to the target boxes as the target box generation pool based on the mapping relationship between the target boxes and the anchor boxes, and uses the number of anchor boxes as the size of the generation pool.

[0064] In some embodiments, the loss function is expressed as:

[0065] loss=∑α*(p-y1) 2 +(1-α)*(p-2) 2

[0066] Where α represents a constant, p represents the dynamic threshold of the anchor box, y1 represents the pseudo-label of the first dynamic threshold, and y2 represents the pseudo-label of the second dynamic threshold.

[0067] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0068] Figure 4 This is a schematic diagram of the structure of the electronic device 4 provided in an embodiment of this application. Figure 4 As shown, the electronic device 4 of this embodiment includes a processor 401, a memory 402, and a computer program 403 stored in the memory 402 and executable on the processor 401. When the processor 401 executes the computer program 403, it implements the steps in the various method embodiments described above. Alternatively, when the processor 401 executes the computer program 403, it implements the functions of each module / unit in the various device embodiments described above.

[0069] For example, computer program 403 may be divided into one or more modules / units, which are stored in memory 402 and executed by processor 401 to complete this application. The one or more modules / units may be a series of computer program instruction segments capable of performing a specific function, which describe the execution process of computer program 403 in electronic device 4.

[0070] Electronic device 4 can be a desktop computer, laptop, handheld computer, cloud server, or other electronic device. Electronic device 4 may include, but is not limited to, processor 401 and memory 402. Those skilled in the art will understand that... Figure 4 This is merely an example of electronic device 4 and does not constitute a limitation on electronic device 4. It may include more or fewer components than shown, or combine certain components, or different components. For example, electronic device may also include input / output devices, network access devices, buses, etc.

[0071] Processor 401 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0072] The memory 402 can be an internal storage unit of the electronic device 4, such as a hard disk or RAM. The memory 402 can also be an external storage device of the electronic device 4, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory 402 can include both internal and external storage units of the electronic device 4. The memory 402 is used to store computer programs and other programs and data required by the electronic device. The memory 402 can also be used to temporarily store data that has been output or will be output.

[0073] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0074] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0075] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0076] In the embodiments provided in this application, it should be understood that the disclosed apparatus / computer devices and methods can be implemented in other ways. For example, the apparatus / computer device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. Multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0077] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0078] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0079] If an integrated module / unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program may include computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0080] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A target detection method based on dynamic threshold, characterized in that, include: The image to be detected is input into the target detection network, and the image to be detected is processed by the backbone network of the target detection network to obtain the first feature map; The first feature map is input into a predetermined dynamic threshold prediction branch, and the information of the classification branch and the regression branch are fused using the dynamic threshold prediction branch to predict the dynamic threshold corresponding to the prediction box of the image to be detected. The predicted boxes are sorted based on their classification probability scores, and then traversed according to the sorting results. When the intersection-union ratio of a subsequent predicted box with the current predicted box is higher than the dynamic threshold of the subsequent predicted box, the subsequent predicted box is removed, and the remaining predicted boxes after traversal are taken as the target detection results. After predicting the dynamic threshold corresponding to the prediction box of the image to be detected, the method further includes: Based on the pre-set anchor frames, determine the first dynamic threshold pseudo-label corresponding to each anchor frame, and use the maximum overlap value of the target box corresponding to the anchor frame as the second dynamic threshold pseudo-label of the anchor frame; The dynamic threshold prediction branch is used to predict the dynamic threshold of the anchor box. A loss function is established based on the first dynamic threshold pseudo-label, the second dynamic threshold pseudo-label, and the dynamic threshold of the anchor box. The target detection network is trained using the loss function.

2. The method according to claim 1, characterized in that, The step of using the dynamic threshold prediction branch to fuse information from the classification and regression branches to predict the dynamic threshold corresponding to the prediction box of the image to be detected includes: The first feature map is fused with the feature map output by the classification branch, and the second feature map is obtained after convolution processing. The third feature map is obtained by directly convolving the first feature map. The second feature map and the third feature map are stacked and then convolved to obtain the fourth feature map; The fourth feature map is fused with the feature map output by the regression branch, and after convolution processing, the fifth feature map is obtained; and the fourth feature map is directly convolved to obtain the sixth feature map. The fifth feature map and the sixth feature map are stacked and then convolved to obtain the seventh feature map; The seventh feature map is convolved to obtain the dynamic threshold corresponding to each prediction box in the image to be detected.

3. The method according to claim 1, characterized in that, The step of determining the first dynamic threshold pseudo-label corresponding to each anchor frame based on the pre-set anchor frames includes: Based on the pre-set anchor frames and the mapping relationship between the target frames and the anchor frames, determine the generation pool corresponding to each target frame; The training image is input into the object detection network, which outputs the predicted bounding box obtained by regressing the anchor box, as well as other predicted bounding boxes corresponding to the anchor boxes in the generation pool. Calculate the intersection-union ratio (IUR) between the predicted bounding box and the other predicted bounding boxes, and use the IUR with the largest value as the first dynamic threshold pseudo-label corresponding to the anchor box.

4. The method according to claim 1, characterized in that, The method further includes: All target boxes corresponding to the training image are determined. The intersection-union ratio (IUR) of each target box with other target boxes is calculated. The other target boxes with the maximum IUR are taken as the maximum overlapping objects of the target boxes, and the value of the maximum IUR is taken as the maximum overlap value.

5. The method according to claim 3, characterized in that, Determining the generation pool corresponding to each of the target boxes includes: Based on the mapping relationship between the target box and the anchor box, the anchor boxes corresponding to the target box are used as the generation pool of the target box, and the number of anchor boxes is used as the size of the generation pool.

6. The method according to claim 3, characterized in that, The loss function is expressed as: in, Represents a constant. This represents the dynamic threshold of the anchor frame. This represents the pseudo-label of the first dynamic threshold. This represents the pseudo-label of the second dynamic threshold.

7. A target detection device based on dynamic threshold, characterized in that, include: The input module is configured to input the image to be detected into the target detection network, and process the image to be detected using the backbone network of the target detection network to obtain a first feature map; The prediction module is configured to input the first feature map into a predetermined dynamic threshold prediction branch, and use the dynamic threshold prediction branch to fuse information from the classification branch and the regression branch to predict the dynamic threshold corresponding to the prediction box of the image to be detected. The traversal module is configured to sort the prediction boxes based on the classification probability scores corresponding to the prediction boxes, and traverse according to the sorting results. When the intersection-union ratio of the subsequent prediction box and the current prediction box is higher than the dynamic threshold of the subsequent prediction box, the subsequent prediction box is removed, and the remaining prediction boxes after traversal are taken as the target detection results. It also includes a training module, which, after predicting the dynamic threshold corresponding to the prediction box of the image to be detected, determines the first dynamic threshold pseudo-label corresponding to each anchor box according to the pre-set anchor box, and uses the maximum overlap value of the target box corresponding to the anchor box as the second dynamic threshold pseudo-label of the anchor box. The dynamic threshold prediction branch is used to predict the dynamic threshold of the anchor box. A loss function is established based on the first dynamic threshold pseudo-label, the second dynamic threshold pseudo-label, and the dynamic threshold of the anchor box. The target detection network is trained using the loss function.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 6.

Images