Image target detection method and device based on dynamic label allocation

Abstract

The present invention discloses an image target detection method and device based on dynamic label allocation, the device includes an image processing module, a model construction module, a model training module, and a target detection module, wherein the model construction module is improved by the normalization method and label allocation method of the end-to-end target detection model, and an improved end-to-end target detection model is constructed, and the target detection of the image is performed by the optimal target detection model obtained by training. The present invention is applied to the field of computer vision technology, so that the end-to-end target detection model can improve its current performance in the label allocation method and target detection of different scales while avoiding the problems caused by manual post-processing, thereby achieving an improvement in detection accuracy.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to an image target detection method and device based on dynamic label allocation. Background Art

[0002] Object detection is an important task in computer vision and the basis of many other computer vision tasks, such as instance segmentation, object tracking, and pose estimation. The rapid development of deep learning has made significant breakthroughs in object detection and has become a hot topic of research.

[0003] Target detection of airport aircraft in satellite images has great military significance. It can obtain military activity intelligence, assess the importance of enemy airports, and provide a basis for precision strikes. Recently, computer vision technology, especially deep learning related methods, has been gradually applied to detect aircraft in satellite images. Deep learning-based methods can be generally classified into one-to-many label matching and one-to-one label matching target detection methods according to label matching. Classic target detection methods such as Faster R-CNN, YOLO series and RetinaNet use one-to-many label matching strategies, but this strategy requires NMS post-processing, resulting in a problem of decreased detection performance in some specific scenarios, such as dense scenes such as airports, ports and parking lots. The one-to-one label matching target detection method is called an end-to-end target detection method because it does not require NMS post-processing. It mainly includes DETR, DeformableDETR and SparseRCNN. The use of end-to-end target detection methods can avoid the disadvantages brought by NMS. However, the label matching of end-to-end object detection methods still uses a fixed label allocation strategy, which makes it impossible to dynamically adjust the label allocation strategy according to different training states. In addition, in the current end-to-end object detection methods, the ground truth is normalized using the image size when calculating the cost and loss, which cannot reflect the true regression state of objects of different scales. Therefore, there is still room for improvement in the label allocation method and the detection performance of objects of different scales in the existing end-to-end object detection methods. Summary of the invention

[0004] In view of the deficiencies in the above-mentioned prior art, the present invention provides an image target detection method and device based on dynamic label allocation, so that the end-to-end target detection model can improve its current performance in label allocation and target detection of different scales while avoiding the problems caused by manual post-processing.

[0005] To achieve the above object, the present invention provides an image target detection method based on dynamic label allocation, comprising the following steps:

[0006] Step 1, preprocessing the image marked with the target to be detected to obtain a training image and a verification image, and collecting the training image into a training image set and collecting the verification image into a verification image set;

[0007] Step 2: Build an end-to-end object detection model based on the SparseRCNN network and improve the end-to-end object detection model, including normalization improvement and improvement of the label assignment method based on regression state;

[0008] Step 3, iteratively train the improved end-to-end object detection model through the training image set, and use the verification image set to verify the accuracy of the end-to-end object detection model in each iteration, and select the model with the highest verification accuracy from all the trained end-to-end object detection models as the optimal object detection model;

[0009] Step 4: Use the required detection image as input of the optimal target detection model, and obtain the target detection result of the detection image.

[0010] As a further improvement of the above technical solution, in step 1, the preprocessing is: after cropping the image to a specified size, multi-scale random scaling, image flipping and image pixel normalization are performed, wherein the probability of flipping the training image is 0.5, and the verification image is not subjected to image flipping processing.

[0011] As a further improvement of the above technical solution, step 3 is specifically as follows:

[0012] Load the pre-trained weights into the backbone of the end-to-end object detection model to obtain a pre-trained object detection model.

[0013] The pre-trained end-to-end object detection model is iteratively trained using the AdamW optimization method and a multi-stage learning rate adjustment strategy to obtain several trained end-to-end object detection models.

[0014] In each round of iteration, the verification image set is used to verify the accuracy of the end-to-end object detection model, and the model with the highest verification accuracy is selected from all trained end-to-end object detection models as the optimal object detection model.

[0015] As a further improvement of the above technical solution, in step 3, after obtaining the optimal target detection model, the optimal target detection model is initialized using a deep learning related framework.

[0016] As a further improvement of the above technical solution, in step 2, the normalization improvement is specifically as follows:

[0017]

[0018] In the formula, Δx is the normalized horizontal coordinate error, x g is the horizontal coordinate of the upper left corner or lower right corner of the ground-truth, x b is the horizontal coordinate of the bounding box (specifically, the horizontal coordinate corresponding to the ground-truth), w g is the width of the ground-truth, Δy is the normalized ordinate error, y g is the ordinate of the upper left corner or lower right corner of the ground-truth, y b is the ordinate of the bounding box (specifically, the ordinate corresponding to the ground-truth), h g is the height of ground-truth.

[0019] As a further improvement of the above technical solution, in step 2, the label allocation method based on regression state is improved as follows:

[0020] In the cost matrix cost calculation process for each bounding box, the corresponding weight is dynamically adjusted according to the regression state of each bounding box, specifically:

[0021]

[0022]

[0023] In the formula, Cost giou Cost is the cost item. l1 is the L1 cost term, w giou is the initial setting weight of the giou cost term, w l1 is the initial setting weight of the L1 cost term, error l1 is the sum of the absolute values ​​of Δx and Δy, and α and β are hyper parameters.

[0024] To achieve the above object, the present invention also provides an image target detection device based on dynamic label allocation, wherein the image target detection device detects image targets using the above method.

[0025] As a further improvement of the above technical solution, the image object detection device includes:

[0026] An image processing module is used to preprocess the image with the target to be detected, obtain a training image and a verification image, and collect the training image into a training image set and collect the verification image into a verification image set;

[0027] A model building module is used to improve the end-to-end object detection model based on the SparseRCNN network and build an improved end-to-end object detection model. The improvements to the end-to-end object detection model include normalization improvements and improvements to the label assignment method based on regression states.

[0028] A model training module is used to iteratively train the improved end-to-end object detection model using the training image set, and verify the accuracy of the end-to-end object detection model using the verification image set in each iteration, and select the model with the highest verification accuracy from all the trained end-to-end object detection models as the optimal object detection model;

[0029] The target detection module is used to take the required detection image as the input of the optimal target detection model and obtain the target detection result of the detection image.

[0030] The present invention provides an image target detection method and device based on dynamic label allocation, which improves the label allocation part of an end-to-end target detection model, constructs an improved end-to-end target detection model, and performs target detection on the image through the optimal target detection model obtained through training, thereby achieving an AP (IoU=0.5:0.95) improvement of 0.6 percentage points, a high-precision detection (IoU=0.7) improvement of 0.1 percentage points, a large target detection improvement of 0.3 percentage points, a medium target detection improvement of 0.2 percentage points, and a small target detection improvement of 0.5 percentage points, so that the end-to-end target detection model can improve its current performance in label allocation methods and target detection of different scales while avoiding problems caused by manual post-processing, thereby achieving an improvement in detection accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0032] Figure 1 A schematic diagram of a flow chart of an image target detection method based on dynamic label allocation in an embodiment of the present invention;

[0033] Figure 2 This is a network structure diagram of SparseRCNN in an embodiment of the present invention;

[0034] Figure 3 It is a module schematic diagram of an image object detection device based on dynamic label allocation in an embodiment of the present invention;

[0035] Figure 4 1 is a schematic diagram comparing the first group of detection results of the end-to-end object detection model before and after the improvement in an embodiment of the present invention, wherein (a) is the detection result of the end-to-end object detection model before the improvement, and (b) is the detection result of the end-to-end object detection model after the improvement;

[0036] Figure 5 2 is a schematic diagram comparing the second set of detection results of the end-to-end target detection model before and after improvement in an embodiment of the present invention, (a) is the detection result of the end-to-end target detection model before improvement, and (b) is the detection result of the end-to-end target detection model after improvement.

[0037] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0038] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0039] It should be noted that all directional indications in the embodiments of the present invention (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0040] In addition, in the present invention, descriptions such as "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0041] In the present invention, unless otherwise clearly specified and limited, the terms "connection", "fixation", etc. should be understood in a broad sense. For example, "fixation" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, an electrical connection, a physical connection, or a wireless communication connection; it can be a direct connection, or an indirect connection through an intermediate medium, or it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0042] In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the fact that ordinary technicians in the field can implement it. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0043] Example 1

[0044] like Figure 1 The present invention discloses an image target detection method based on dynamic label allocation, which is applicable to airport aircraft detection or other image target detection in satellite images. The method specifically includes the following steps:

[0045] Step 1, preprocessing the image marked with the target to be detected to obtain a training image and a verification image, and collecting the training image into a training image set and collecting the verification image into a verification image set;

[0046] Step 2: Build an end-to-end object detection model and improve the end-to-end object detection model, including normalization improvement and improvement of label assignment method based on regression state;

[0047] Step 3, iteratively train the improved end-to-end object detection model through the training image set, and use the verification image set to verify the accuracy of the end-to-end object detection model in each iteration, and select the model with the highest verification accuracy from all the trained end-to-end object detection models as the optimal object detection model;

[0048] Step 4: Use the required detection image as input of the optimal target detection model, and obtain the target detection result of the detection image.

[0049] In step 1, the image with the target to be detected is preprocessed as follows: after the image is cropped to the specified size, multi-scale random scaling, image flipping and image pixel normalization are performed, wherein the probability of flipping the training image is 0.5, and the verification image is not subjected to image flipping. Taking the detection of airport aircraft in satellite images as an example, in the specific implementation process, the satellite airport image is directly obtained, or the airport aircraft image is extracted from the remote sensing image taken by the satellite, and the labelImg annotation tool is used to annotate the target to be detected in the aircraft image, that is, the aircraft target. The annotation is an xml format file in the PASCAL VOC format, which is converted into a json format file in the MS COCO format through a script, and the annotated aircraft image is cropped to the specified size, and then multi-scale random scaling, image flipping and image pixel normalization are performed. The preprocessing is collected into the training image set and the verification image set in a ratio of 7 to 3, wherein the probability of flipping the training image is 0.5, and the verification image set is not subjected to image flipping. This preprocessing process can not only ensure that the parameters such as the size of the training image meet the requirements, but also perform data enhancement, which is conducive to enhancing the robustness of the target detection model.

[0050] In step 2, the end-to-end target detection model uses the SparseRCNN network in the prior art, and its network structure is as follows: Figure 2 As shown, it can be divided into three parts, namely the backbone feature extraction network backbone (corresponding to Figure 2 ResNet50 on top), strengthen feature extraction network (corresponding to Figure 2 FPN on) and prediction network (corresponding to Figure 2SparseRCNNHeads on the network). The function of the first part of the backbone extraction network is to perform preliminary feature extraction. The backbone feature extraction network can obtain four preliminary effective features, namely C2 to C5. The function of the second part of the enhanced feature extraction network is to perform enhanced feature extraction and feature fusion of the four features of the backbone part, so as to extract four more effective features, namely P2 to P5. The prediction network in the third part is six serial SparseRCNNHeads. The first SparseRCNNHead uses the initialized bounding boxes to extract features from P2 to P5 and then predicts the category and position information. Then the next SparseRCNNHead uses the position information predicted by the previous SparseRCNNHead to calculate the bounding boxes to extract features from P2 to P5, and then provide the prediction results to the next SparseRCNNHead until the last SparseRCNNHead predicts the target detection result. The whole process uses deep supervision, that is, the prediction results of each SparseRCNNHead need to be supervised. Before calculating the loss function, it is necessary to calculate the category and regression cost of each position, and finally form a cost matrix as the Hungarian binary matching. The L1 cost in the regression cost of the original cost matrix is ​​normalized using the image size. Since it cannot adapt to targets of various scales and cannot reflect the true regression state of the bounding boxes, the normalization method of the end-to-end target detection model is improved in this embodiment, specifically:

[0051]

[0052] In the formula, Δx is the normalized horizontal coordinate error, x g is the horizontal coordinate of the upper left corner or lower right corner of the ground-truth, x b is the horizontal coordinate of the bounding box (specifically, the horizontal coordinate corresponding to the ground-truth), w g is the width of the ground-truth, Δy is the normalized ordinate error, y g is the ordinate of the upper left corner or lower right corner of the ground-truth, y b is the ordinate of the bounding box (specifically the ordinate corresponding to the ground-truth), h gis the height of ground-truth. The subscripts g and b represent the data related to ground-truth and bounding boxes respectively. The above normalization method is more consistent with the calculation of IoU. In addition, the original cost matrix uses a fixed weight calculation method to calculate the cost of each bounding box. Based on this, the label allocation method is also improved in this embodiment, that is, in the cost matrix cost calculation process for each bounding box, the corresponding weight is dynamically adjusted according to the regression state of each bounding box, specifically:

[0053]

[0054]

[0055] In the formula, Cost giou Cost is the cost item. l1 is the L1 cost term, w giou is the initial setting weight of the giou cost term, w l1 is the initial setting weight of the L1 cost term, error l1 is the sum of the absolute value of Δx and the absolute value of Δy, α and β are hyperparameters, and in this embodiment, α and β are both set to 0.7, wherein giou is a well-known technology in the field of target detection, and therefore will not be described in detail in this embodiment. By setting dynamic weights to achieve the purpose of dynamic label allocation, the target detection model can obtain suitable samples for training at different training stages, and the detection accuracy of the model for aircraft is improved without increasing the number of parameters and substantially without increasing the amount of calculation, and further realizing high-precision target detection.

[0056] In this embodiment, the training process of the end-to-end object detection model is specifically as follows:

[0057] Set the training parameters of the end-to-end object detection model, load the pre-trained weights into the backbone of the end-to-end object detection model, and obtain the pre-trained object detection model.

[0058] The pre-trained end-to-end object detection model is iteratively trained using the AdamW optimization method and a multi-stage learning rate adjustment strategy until the number of training rounds reaches a preset number of iterations, and several trained end-to-end object detection models are obtained;

[0059] After each round of iterative training, the verification image set is used to verify the accuracy of the end-to-end target detection model, and the model with the highest verification accuracy is selected from all trained end-to-end target detection models as the optimal target detection model. After obtaining the optimal target detection model, the deep learning related framework is used to initialize the optimal target detection model.

[0060] In the specific implementation process, the training process of the end-to-end object detection model is as follows: load the ResNet50 weights pre-trained on ImageNet, and use Kaiming initialization to initialize the weights of all other layers. Use the training image set to train the object detection model loaded with pre-trained weights using the AdamW optimization method and multi-stage learning rate adjustment strategy. The training parameters are set as follows:

[0061] The backbone parameter is used to specify the backbone feature extraction network used. It is mainly set to return features as [0,1,2,3], i.e. res2 to res5, and to set the BN layer to be learnable.

[0062] Other parameter settings are as follows:

[0063] The total number of training rounds is 100, the batch size is 8, and the initial learning rate is 0.0001. Since the AdamW optimization method is used, a smaller learning rate can be selected to start; the learning rate will decay at 75 and 90 rounds, respectively, with a decay coefficient of 0.1. The weight decay is set to 0.0001;

[0064] Through the above training, the model with the highest detection accuracy is selected from all trained target detection models as the optimal target detection model.

[0065] In step 4, the specific process of detecting the required detection image is: first perform preprocessing such as image cropping and image scaling on the required detection image to make the required detection image reach the size required by the target detection model, and finally input the optimal target detection model to obtain the target detection result of the detection image.

[0066] Example 2

[0067] like Figure 3The present embodiment discloses an image target detection device based on dynamic label allocation, which is suitable for airport aircraft detection or other image target detection in satellite images. The image target detection device includes an image processing module, a model construction module, a model training module and a target detection module. The image processing module is used to preprocess the image with the target to be detected, obtain the training image and the verification image, and collect the training image into the training image set and the verification image into the verification image set; the model construction module is used to improve the end-to-end target detection model based on the SparseRCNN network, and construct the improved end-to-end target detection model, wherein the improvement of the end-to-end target detection model includes normalization improvement and improvement of the dynamic label allocation method based on the regression state; the model training module uses the training image set to iteratively train the improved end-to-end target detection model, and uses the verification image set in each round of iteration to verify the accuracy of the end-to-end target detection model, and selects the model with the highest verification accuracy from all the trained end-to-end target detection models as the optimal target detection model; the target detection module uses the required detection image as the input of the optimal target detection model, and obtains the target detection result of the detection image.

[0068] As an example, the image processing module preprocesses the image with the target to be detected as follows: after the image is cropped to a specified size, multi-scale random scaling, image flipping and normalization of image pixels are performed, wherein the probability of flipping the training image is 0.5, and the verification image is not subjected to image flipping. Taking the detection of airport aircraft in satellite images as an example, in the specific implementation process, the satellite airport image is directly obtained through the image processing module, or the airport aircraft image is extracted from the remote sensing image taken by the satellite, and the labelImg annotation tool is used to annotate the target to be detected in the aircraft image, that is, the aircraft target. The annotation is an xml format file in the PASCAL VOC format, which is converted into a json format file in the MS COCO format through a script, and the annotated aircraft image is cropped to a specified size, and then preprocessed by multi-scale random scaling, image flipping and normalization of image pixels are performed, and collected into the training image set and the verification image set in a ratio of 7 to 3, wherein the probability of flipping the training image is 0.5, and the verification image set is not subjected to image flipping. This preprocessing process can not only ensure that parameters such as the size of the training image meet the requirements, but also perform data enhancement, which is conducive to enhancing the robustness of the target detection model.

[0069] As an example, the end-to-end target detection model adopts the SparseRCNN network in the prior art, and its network structure is as follows: Figure 2 As shown, it can be divided into three parts, namely the backbone feature extraction network backbone (corresponding to Figure 2ResNet50 on top), strengthen feature extraction network (corresponding to Figure 2 FPN on) and prediction network (corresponding to Figure 2 SparseRCNNHeads on the network). The function of the first part of the backbone extraction network is to perform preliminary feature extraction. The backbone feature extraction network can obtain four preliminary effective features, namely C2 to C5. The function of the second part of the enhanced feature extraction network is to perform enhanced feature extraction and feature fusion of the four features of the backbone part, so as to extract four more effective features, namely P2 to P5. The prediction network in the third part is six serial SparseRCNNHeads. The first SparseRCNNHead uses the initialized bounding boxes to extract features from P2 to P5 and then predicts the category and position information. Then the next SparseRCNNHead uses the position information predicted by the previous SparseRCNNHead to calculate the bounding boxes to extract features from P2 to P5, and then provide the prediction results to the next SparseRCNNHead until the last SparseRCNNHead predicts the target detection result. The whole process uses deep supervision, that is, the prediction results of each SparseRCNNHead need to be supervised. Before calculating the loss function, it is necessary to calculate the category and regression cost of each position, and finally form a cost matrix as the Hungarian binary matching. The L1 cost in the regression cost of the original cost matrix is ​​normalized using the image size. Since it cannot adapt to targets of various scales and cannot reflect the true regression state of the bounding boxes, the normalization method of the end-to-end target detection model is improved in this embodiment, specifically:

[0070]

[0071] In the formula, Δx is the normalized horizontal coordinate error, x g is the horizontal coordinate of the upper left corner or lower right corner of the ground-truth, x b is the horizontal coordinate of the bounding box (specifically, the horizontal coordinate corresponding to the ground-truth), w g is the width of the ground-truth, Δy is the normalized ordinate error, y g is the ordinate of the upper left corner or lower right corner of the ground-truth, y b is the ordinate of the bounding box (specifically the ordinate corresponding to the ground-truth), h gis the height of ground-truth. The subscripts g and b represent the data related to ground-truth and bounding boxes respectively. The above normalization method is more consistent with the calculation of IoU. In addition, the original cost matrix uses a fixed weight calculation method to calculate the cost of each bounding box. Based on this, the label allocation method is also improved in this embodiment, that is, in the cost matrix cost calculation process for each bounding box, the corresponding weight is dynamically adjusted according to the regression state of each bounding box, specifically:

[0072]

[0073]

[0074] In the formula, Cost giou Cost is the cost item. l1 is the L1 cost term, w giou is the initial setting weight of the giou cost term, w l1 is the initial setting weight of the L1 cost term, error l1 is the sum of the absolute value of Δx and the absolute value of Δy, α and β are hyperparameters, and in this embodiment, α and β are both set to 0.7. By setting dynamic weights to achieve the purpose of dynamic label allocation, the target detection model can obtain suitable samples for training at different training stages, and the detection accuracy of the model for aircraft is improved without increasing the number of parameters and the amount of calculation, and high-precision target detection is further achieved.

[0075] In this embodiment, the training process of the model training module for the end-to-end object detection model is specifically as follows:

[0076] Set the training parameters of the end-to-end object detection model, load the pre-trained weights into the backbone of the end-to-end object detection model, and obtain the pre-trained object detection model.

[0077] The pre-trained end-to-end object detection model is iteratively trained using the AdamW optimization method and a multi-stage learning rate adjustment strategy until the number of training rounds reaches a preset number of iterations, and several trained end-to-end object detection models are obtained;

[0078] After each round of iterative training, the verification image set is used to verify the accuracy of the end-to-end target detection model, and the model with the highest verification accuracy is selected from all trained end-to-end target detection models as the optimal target detection model. After obtaining the optimal target detection model, the deep learning related framework is used to initialize the optimal target detection model.

[0079] As an example, the training process of the model training module for the end-to-end object detection model is as follows: load the ResNet50 weights pre-trained on ImageNet, and use Kaiming initialization to initialize the weights of all other layers. Use the training image set to train the object detection model loaded with pre-trained weights using the AdamW optimization method and multi-stage learning rate adjustment strategy. The training parameters are set as follows:

[0080] The backbone parameter is used to specify the backbone feature extraction network used. It is mainly set to return features as [0,1,2,3], i.e. res2 to res5, and to set the BN layer to be learnable.

[0081] Other parameter settings are as follows:

[0082] The total number of training rounds is 100, the batch size is 8, and the initial learning rate is 0.0001. Since the AdamW optimization method is used, a smaller learning rate can be selected to start; the learning rate will decay at 75 and 90 rounds, respectively, with a decay coefficient of 0.1. The weight decay is set to 0.0001;

[0083] Through the above training, the model with the highest detection accuracy is selected from all trained target detection models as the optimal target detection model.

[0084] As an example, the detection process of the target detection module for the required detection image is specifically as follows: first, the required detection image is preprocessed by image cropping, image scaling, etc., so that the required detection image reaches the size required by the target detection model, and finally the optimal target detection model is input to obtain the target detection result of the detection image.

[0085] In the above-mentioned embodiments 1 and 2, the normalization part and the label assignment part of the end-to-end target detection model are improved to construct an improved end-to-end target detection model. The optimal target detection model obtained through training is used to detect the airport aircraft image, and the AP (IoU = 0.5:0.95) is improved by 0.6 percentage points, the high-precision detection (IoU = 0.7) is improved by 0.1 percentage points, the large target detection is improved by 0.3 percentage points, the medium target detection is improved by 0.2 percentage points, and the small target detection is improved by 0.5 percentage points, thereby improving the training accuracy. The final test comparison results are as follows: Figure 4-5 As shown, Figure 4 (a) Figure 5 (a) is the detection result of the end-to-end object detection model before improvement. Figure 4 (b) Figure 5 (b) is the detection result of the end-to-end target detection model before improvement. Figure 4-5 It can be seen that the detection accuracy after improvement is significantly improved.

[0086] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. All equivalent structural changes made by using the contents of the present invention specification and drawings under the inventive concept of the present invention, or directly / indirectly applied in other related technical fields are included in the patent protection scope of the present invention.

Claims

1. An image target detection method based on dynamic label allocation, characterized in that: The steps include: Step 1, preprocessing the image marked with the target to be detected to obtain a training image and a verification image, and collecting the training image into a training image set and collecting the verification image into a verification image set; Step 2: Build an end-to-end object detection model based on the SparseRCNN network and improve the end-to-end object detection model, including normalization improvement and improvement of the dynamic label allocation method based on regression state; The normalization improvement is specifically as follows: In the formula, Δx is the normalized horizontal coordinate error, x g is the horizontal coordinate of the ground-truth, x b is the horizontal coordinate of the bounding box, w g is the width of the ground-truth, Δy is the normalized ordinate error, y g is the ordinate of the ground-truth, y b is the ordinate of the boundingbox, h g is the height of ground-truth; The dynamic label allocation method based on regression state is improved as follows: In the cost matrix cost calculation process for each bounding box, the corresponding weight is dynamically adjusted according to the regression state of each bounding box, specifically: In the formula, Cost giou Cost is the cost item. l1 is the L1 cost term, w giou is the initial setting weight of the giou cost term, w l1 is the initial setting weight of the L1 cost term, error l1 is the sum of the absolute value of Δx and the absolute value of Δy, α and β are hyperparameters; Step 3, iteratively train the improved end-to-end object detection model through the training image set, and use the verification image set to verify the accuracy of the end-to-end object detection model in each iteration, and select the model with the highest verification accuracy from all the trained end-to-end object detection models as the optimal object detection model; Step 4: Use the required detection image as input of the optimal target detection model, and obtain the target detection result of the detection image.

2. According to claim 1, the image target detection method based on dynamic label allocation is characterized in that: In step 1, the preprocessing is: after cropping the image to a specified size, multi-scale random scaling, image flipping and image pixel normalization are performed, wherein the probability of flipping the training image is 0.5, and the verification image is not subjected to image flipping processing.

3. The image target detection method based on dynamic label allocation according to claim 1 is characterized in that: Step 3 is as follows: Load the pre-trained weights into the backbone of the end-to-end object detection model to obtain a pre-trained object detection model. The pre-trained end-to-end object detection model is iteratively trained using the AdamW optimization method and a multi-stage learning rate adjustment strategy to obtain several trained end-to-end object detection models. In each round of iteration, the verification image set is used to verify the accuracy of the end-to-end object detection model, and the model with the highest verification accuracy is selected from all trained end-to-end object detection models as the optimal object detection model.

4. The image target detection method based on dynamic label allocation according to claim 3 is characterized in that: In step 3, after obtaining the optimal target detection model, the optimal target detection model is initialized using a deep learning related framework.

5. An image object detection device based on dynamic label allocation, characterized in that: The image target detection device detects image targets using the method described in any one of claims 1 to 4.

6. The image object detection device based on dynamic label allocation according to claim 5, characterized in that: The image object detection device comprises: An image processing module is used to preprocess the image with the target to be detected, obtain a training image and a verification image, and collect the training image into a training image set and collect the verification image into a verification image set; A model building module is used to improve the end-to-end object detection model based on the SparseRCNN network and build an improved end-to-end object detection model. The improvements to the end-to-end object detection model include normalization improvements and improvements to the label assignment method based on regression states. A model training module is used to iteratively train the improved end-to-end object detection model using the training image set, and verify the accuracy of the end-to-end object detection model using the verification image set in each iteration, and select the model with the highest verification accuracy from all the trained end-to-end object detection models as the optimal object detection model; The target detection module is used to take the required detection image as the input of the optimal target detection model and obtain the target detection result of the detection image.

Images