A directed target detection method based on an efficient label assignment strategy

Abstract

The present application belongs to the technical field of directed target detection, and particularly relates to a directed target detection method based on an efficient label assignment strategy. The present application is based on a two-stage directed target detection model. In the first stage of the model, an efficient label assignment strategy is used to construct a region proposal network to generate a directional region proposal frame. In the second stage of the model, a target detection network is used to perform a detection task. The network uses a full connection layer to combine the features obtained by the region proposal frame in the first stage and performs a detection frame regression. Since the efficient label assignment strategy can assign the most suitable real label to the training frame of different sizes and shapes, the label assignment of the training frame is more reasonable, the problems of the number and quality of the training frame corresponding to the label are solved, and thus the target detection rate is improved. The experimental results verify the effectiveness of the method.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of directed target detection, and particularly relates to a directed target detection method based on an efficient label allocation strategy. BACKGROUND

[0002] With the continuous development of remote sensing technology, it is no longer difficult to obtain high-resolution remote sensing images, and more and more new high-resolution remote sensing images have shown more complex diversity in target scale, image background, fine-grained information and spatial coverage range. The detection of remote sensing images is not only applied to military fields such as target reconnaissance and sea area control and defense, but also widely applied to various civilian fields such as geographic information systems, surveying and mapping systems, agriculture and traffic planning and navigation. Therefore, how to accurately detect targets from more complex and diverse remote sensing images is particularly important.

[0003] At present, the traditional directed target detection algorithm uses a static label allocation strategy or a complex adaptive label allocation strategy. The static label allocation strategy refers to selecting only a fixed number of region proposal boxes or only selecting anchor boxes higher than an IOU (Intersection over Union) threshold as training boxes for training for each target real box. Therefore, the matching degree of the anchor box and the real box is low at the initial stage of training, thereby causing too many low-quality anchor boxes to be selected for training, which is an inefficient allocation strategy. The adaptive label allocation strategy usually uses a complex convolutional neural network to learn the label allocation method of the anchor box autonomously, thereby selecting a certain number of anchor boxes as training boxes for the subsequent network. The quality of the training boxes selected by this method is good, but the required calculation time is significantly increased. SUMMARY

[0004] The purpose of the application is to provide a directed target detection method based on an efficient label allocation strategy, which improves the detection accuracy while keeping the calculation complexity basically unchanged.

[0005] The technical scheme of the application is as follows:

[0006] A directed target detection method based on an efficient label allocation strategy, comprising:

[0007] A target detection model is constructed, including a feature extraction backbone network, a region proposal network and a target detection network, wherein the input of the feature extraction backbone network is a remote sensing image, the feature extraction backbone network includes four parallel feature extraction sub-networks, the structure of each feature extraction sub-network includes: one convolutional layer followed by one activation layer, then one convolutional layer, one activation layer and one pooling layer, the adjacent layers are sequentially connected, the outputs of the four feature extraction sub-networks are merged and then input into a four-layer convolutional sub-network to extract the feature map of the target, the convolutional sub-network includes: one convolutional layer followed by one activation layer, then one convolutional layer and one activation layer, wherein all the convolutional kernel sizes are 3*3 with a sliding step of 1, all the pooling layer sizes are 2*2 with a sliding step of 2, and the activation function is ReLu function;

[0008] The feature map output by the feature extraction backbone network is input into the region proposal network and the target detection network. The region proposal network includes: one convolutional layer with a kernel size of 3*3 and a sliding step of 1, followed by one ReLu function activation layer, the output of the activation layer is input into two convolutional layers with a kernel size of 1*1 and a sliding step of 1, the output of one of the convolutional layers is output after passing through a softmax layer to output the probability that the current proposal frame contains a target, the probability is combined with the output of the other convolutional layer as the input of the high-efficiency label assignment layer, the label assignment of the current proposal frame is completed based on the high-efficiency label assignment strategy in the label assignment layer, and finally the optimal proposal frame is output to the target detection network. The steps of the high-efficiency label assignment strategy include: first, all anchor frames with confidence scores ranked in the top 50% are selected, then the IOU value of each real frame GT and the anchor frames is calculated in turn, the anchor frames corresponding to the IOU value of each real frame GT ranked in the top 20% are taken out, and the sum of the IOU values of these anchor frames is taken as the number k of selected training frames, that is, the labels of the anchor frames ranked in the top k in terms of IOU value are selected as the GT, and the k anchor frames constitute the proposal frames for this round of training;

[0009] The output of the feature extraction backbone network and the output of the region proposal network are taken as the input of the target detection network, the structure of the target detection network includes one ROI pooling layer, one fully connected layer, one activation layer, then one fully connected layer, one activation layer and one fully connected layer, the adjacent layers are sequentially connected, and finally the target detection network outputs the detection frame of the target image;

[0010] Before target detection, a training image set is selected to train the network model. First, the training image data is input into the feature extraction backbone network, and the feature map is extracted as the input of the subsequent region proposal network and target detection network. Then, the region proposal network uses the feature map data to output the optimal proposal frame based on the efficient label assignment strategy, and the optimal proposal frame is used as the input of the target detection network. The target detection network quickly determines the detection frame containing the target according to the input feature map data and the optimal proposal frame. Finally, the error between the detection result and the true value is used to complete the iterative update of a plurality of network model parameters based on the BP algorithm.

[0011] After the network model training is completed, a new image is input into the network, and the feature map is extracted after the feature extraction backbone network. The feature map is directly input into the target detection network for target detection. Finally, the target detection network outputs the detection frame containing the target, and the target detection is completed.

[0012] The known data set is used to train the directional target detection model constructed.

[0013] The beneficial effects of the present application are that the present application solves the problem of the optimal number and quality of the training frame corresponding to the label, thereby improving the target detection rate. BRIEF DESCRIPTION OF DRAWINGS

[0014] Figure 1 is a directional target detection model structure diagram of the present application;

[0015] Figure 2 is a high-efficiency label assignment method schematic diagram. DETAILED DESCRIPTION

[0016] The effectiveness and progress of the present application are proved by the following simulation:

[0017] As shown in Figure 1 , the present application proposes a directional target detection model with higher efficiency and better accuracy. The first stage of the model uses a full convolutional network to constitute a region proposal network (RPN), which can generate directional region proposal frames (also known as anchor frames, AF). The second stage of the model uses a target detection network (TDN) to perform the detection task, merges the features obtained by the region proposal frame in the first stage using the full connection layer, and performs detection frame regression. The specific structure is as follows: Figure 1The feature extraction backbone network is composed of 10 convolutional layers, 10 activation layers and 4 pooling layers, wherein all the convolutional layers have a size of 3*3, a sliding step of 1 and a padding operation number (zero padding operation number) of 1, all the pooling layers have a size of 2*2, a sliding step of 2 and no padding operation, and all the activation layers are ReLu function activation layers. The network is used to extract the features of the input image, and the generated feature maps are input into the region proposal network and the target detection network for the generation of proposal boxes and the regression of detection boxes. The region proposal network is composed of 3 convolutional layers, 1 activation layer and 1 softmax classification layer, wherein the first convolutional layer has a size of 3*3, a sliding step of 1 and a padding operation number of 1, and the other two convolutional layers have a size of 1*1, a sliding step of 1 and no padding operation. The activation layer of the network is a ReLu function activation layer. The target detection network is composed of 3 fully connected layers, 2 activation layers and 1 ROI pooling layer, wherein the activation layer of the network is a ReLu function activation layer, and the ROI pooling layer (Region of Interest) is used to pool the feature maps under the proposal boxes of different dimensions into a uniform dimension, so as to perform subsequent full connection operations.

[0018] The efficient label assignment solves the problem of label assignment in target detection from the perspective of the number and quality of training frames corresponding to each label (i.e. Ground Truth, referred to as GT), so that the label assignment of the training frame is more reasonable, and the most suitable real label can be assigned to the training frame of different sizes and shapes. The specific principle is as follows: Figure 2 As shown in the figure, GT i represents the i-th real frame in the image, and AF i represents the i-th anchor frame (Anchor frame, referred to as AF) in the image.

[0019] Suppose that there are N real frames GT in an original image, and each GT contains a target object. The region proposal network in the first stage generates anchor frames for the original image and gives the confidence score (the probability of whether there is an object in the current anchor frame) of each anchor frame. The higher the confidence score of an anchor frame, the greater the probability of the existence of a target object in the frame. Select the top 50% of all anchor frames Figure 2 (hypothetical M) according to the confidence score, and calculate the IOU value of each GT and the anchor frame in turn. In order to select a suitable number and quality of anchor frames for each GT as the subsequent training frame, the method takes out the top 20% of the anchor frames corresponding to the IOU value of each GT, and takes the sum of the IOU values of these anchor frames as the number k of selected training frames, that is, the label of the anchor frame with the top k IOU value is selected as the GT, and the k anchor frames constitute the training frame of this round. If the above percentage calculation results in a decimal, round up.

[0020] The value of k in this method can reflect the matching degree (IOU value size) and the number between the selected anchor frame and the ground truth GT. In the early stage of training, the network parameters are poor, and the matching degree (anchor frame quality) between the anchor frame and the ground truth GT is low, so the value of k is small, that is, the network will not select too many anchor frames with low matching degree to participate in training; as the training proceeds, the network parameters are constantly optimized, and the quality of the anchor frame will also be improved, and the value of k will increase. Therefore, the better the quality of the anchor frame corresponding to the ground truth GT, the more anchor frames will be selected to participate in training, thereby improving the network effect under the condition of ensuring efficiency. Compared with the traditional method, this strategy can select more appropriate and higher quality anchor frames to participate in subsequent boundary regression.

[0021] Experiments are performed using the airplane category in the FAIR1M dataset. The FAIR1M dataset is currently the largest fine-grained target recognition dataset in the field of remote sensing, with more than 1 million instances, all of which are annotated using target bounding boxes, providing more accurate spatial information for directional targets. Compared with other datasets in the field of remote sensing, the FAIR1M dataset has more comprehensive fine-grained types, larger image sizes and direction ranges, more target-distributed complex scenes, and richer geographical information, so the FAIR1M dataset is more diverse and challenging.

[0022] For airplane models, there are 10 fine-grained categories, covering 34 airports around the world, including Boeing 737, Boeing 777, Boeing 747, Boeing 787, Airbus A320, Airbus A220, Airbus A330, Airbus A350, China Commercial Aircraft C919, China Commercial Aircraft ARJ21, etc. It is the most common several categories in civil aviation. In addition, an Other-airplane category is set, and the specific airplane instance allocation is shown in Table 1 as follows:

[0023] Table 1: Airplane categories and instance numbers

[0024]

[0025]

[0026] The computer system environment used in this experiment is windows 10 system, and the hardware environment is CPU AMD Ryzen 55600H with Radeon Graphics, GPU NVIDIA GeForce RTX 3050, 16GB memory. The two-stage model and the two-stage model using efficient label assignment strategy are used to conduct target detection experiment on 3.1 dataset, and the results are shown in table 2. Among them, the learning rate is 0.001, the iteration number is 10 times, and the loss function adopts Smooth L1 loss function.

[0027] Table 2 detection results of two methods

[0028]

[0029] Note: the training time of 1 epoch refers to the time required to train all pictures in the training set for one time

[0030] From table 2, compared with only using the ordinary two-stage detection model, the two-stage model using efficient label assignment method in this paper has improved the average accuracy rate of aircraft detection, and the training speed of the model is obviously improved after using the efficient label assignment method, therefore, the above results show that the method proposed in this paper is effective.

Claims

1. A directed target detection method based on an efficient label assignment strategy, characterized in that, The application relates to a directional target detection model, which comprises a feature extraction backbone network, a region proposal network and a target detection network. The feature extraction backbone network is inputted with a remote sensing image, and the feature extraction backbone network comprises four feature extraction sub-networks which are sequentially connected, the structure of each feature extraction sub-network is a convolution layer-activation layer-convolution layer-activation layer-pooling layer, the output of the pooling layer of the last feature extraction sub-network is sequentially inputted into a convolution layer, an activation layer, a convolution layer and an activation layer, and the extracted feature map is outputted, wherein the size of all the convolution layers is 3*3, the sliding step is 1, the size of all the pooling layers is 2*2, the sliding step is 2, and all the activation layers are ReLu function activation layers; The feature map outputted by the feature extraction backbone network is inputted into the region proposal network and the target detection network, wherein the feature map inputted into the region proposal network is inputted into two convolution layers with the size of 1*1 and the sliding step of 1 after the feature map is inputted into a convolution layer with the size of 3*3 and the sliding step of 1 and a ReLu function activation layer, the output of one of the two convolution layers is inputted into a softmax layer to output the probability that the current proposal frame contains a target, the probability is combined with the output of the other convolution layer to be inputted into an efficient label assignment layer, the label assignment of the current proposal frame is completed in the efficient label assignment layer based on an efficient label assignment strategy, and the optimal proposal frame is finally outputted into the target detection network; The structure of the target detection network is an ROI pooling layer-full connection layer-activation layer-full connection layer-activation layer-full connection layer, and the target detection network outputs the detection frame of the target; The efficient label assignment strategy is as follows: firstly, all the anchor frames with the confidence score ranking in the first 50% are selected, then the IOU value of each real frame GT and the anchor frames is calculated, the anchor frames corresponding to the IOU value of each real frame GT in the first 20% are taken out, and the sum of the IOU values of the anchor frames is taken as the number k of selected training frames, that is, the label of the anchor frame with the IOU value ranking in the first k is selected as the GT, and the k anchor frames constitute the proposal frame for this round of training; Before target detection, a training image set is selected to train the directional target detection model: firstly, training image data is inputted into the feature extraction backbone network to extract feature maps as the input of the subsequent region proposal network and target detection network, then the region proposal network outputs the optimal proposal frame based on the efficient label assignment strategy by using the feature map data, and the optimal proposal frame is inputted into the target detection network, the target detection network quickly determines the detection frame containing the target according to the input feature map data and the optimal proposal frame, finally, the error between the detection result and the real value is used to complete the iterative update of the network model parameters based on the BP algorithm; After the directional target detection model is trained, a new image is inputted into the network, the feature map is extracted after the feature extraction backbone network, the feature map is directly inputted into the target detection network for target detection, and finally the detection frame containing the target is outputted by the target detection network to complete the detection of the target.

Images