Target detection method based on positive and negative sample learning

Abstract

The invention belongs to the technical field of image target detection, and particularly relates to a target detection method based on positive and negative sample learning. Extracting multi-scale features; jointly inputting the multi-scale features into a classification network and a positioning network, and extracting a classification prediction result and a positioning prediction result; inputting a classification prediction result and a positioning prediction result into a confidence coefficient calculation module, and respectively extracting positive sample confidence coefficient and negative sample confidence coefficient; inputting the multi-scale features into a weight prediction network, and respectively extracting a positive sample weight and a negative sample weight; inputting the positive sample confidence coefficient, the negative sample confidence coefficient, the positive sample weight and the negative sample weight into a positive and negative sample score calculation module, and extracting a positive sample score and a negative sample score; and inputting the positive sample score and the negative sample score to a loss calculation module, and outputting a target detection result.

Description

Technical Field

[0001] The present invention relates to the technical field of image target detection, and in particular to a target detection method based on positive and negative sample learning. Background Art

[0002] Object detection is a hot topic in computer vision and has garnered widespread attention in recent years. Current mainstream object detection methods employ a dense prediction process, classifying extracted candidate regions as positive or negative samples based on their relationship to ground-truth annotations. This process is known as label assignment. Most existing methods employ static label assignment strategies. For example, methods based on anchor boxes (such as S. Ren, K. He, R. Girshick, and J. Sun, “Faster R-CNN: Towards realtime object detection with region proposal networks,” in NIPS, 2015 and T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Doll′ar, “Focal loss for dense object detection,” in ICCV, 2017.) divide positive and negative samples in feature pyramid networks at different levels by presetting the intersection-over-union threshold; while methods based on anchor-free boxes (such as Z. Tian, C. Shen, H. Chen, and T. He, “FCOS: Fully convolutional one-stage object detection,” in ICCV, 2019. and T. Kong, F. Sun, H. Liu, Y. Jiang, L. Li, and J. Shi, “Foveabox: Beyond anchor-based object detection,” IEEE T-IP, 2020.) defines a fixed area near the center of the object as a positive sample, and the rest of the area as a negative sample. In recent years, dynamic label assignment methods have been proposed to alleviate the limitations of static strategies.For example: S. Zhang, C. Chi, Y. Yao, Z. Lei, and S. Z. Li, “Bridging the gap between anchor-based and anchor-free detection via adaptive training sample selection,” in CVPR, 2020 dynamically adjusts the threshold according to the IoU statistics between the candidate box and the true annotation; K. Kim and HSLee, “Probabilistic anchor assignment with iou prediction for object detection,” in ECCV, 2020. Based on the Gaussian mixture model, positive and negative samples are adaptively divided according to the probability distribution of the samples; Z. Ge, S. Liu, Z. Li, O. Yoshie, and J. Sun, “OTA: Optimal transport assignment for object detection,” in CVPR, 2021 models label assignment as an optimal transport problem, and achieves the optimal matching between the candidate box and the true annotation by minimizing the classification and positioning losses.

[0003] Although dynamic methods eliminate the reliance on artificial prior sampling knowledge in static methods, existing dynamic strategies usually have two flaws: first, most methods roughly estimate the spatial distribution of positive samples of the target through a mixture model or implicit branches in the prediction head, but rarely pay attention to the influence of object appearance information; second, these methods still cannot perceive the quality distribution of positive samples, and low-quality samples will have a negative impact on detection performance. Summary of the Invention

[0004] This paper proposes a coarse-to-fine optimization framework that enables the model to perceive the appearance characteristics of objects, and introduces a positive sample selection strategy to explore the quality statistical distribution of positive samples through training with label targets of different designs.

[0005] The technical solution of the present invention is as follows: A target detection method based on positive and negative sample learning includes the following steps:

[0006] Step (1) crop the input image to a uniform size and horizontally flip it to perform data enhancement on the input image; input the data-enhanced input image into the feature extraction network to extract the multi-scale feature F P1 and F P2 ;

[0007] Step (2), multi-scale feature F P1 and F P2Input the classification network and positioning network together to extract the classification prediction results P cls And the positioning prediction result P loc ;

[0008] Step (3), the classification prediction result P cls And the positioning prediction result P loc Input to the confidence calculation module and extract the positive sample confidence F + And the negative sample confidence P - ;

[0009] Step (4), multi-scale feature F P1 and F P2 Input weight prediction network and extract positive sample weight ω + and negative sample weight ω - ;

[0010] Step (5), the confidence of the positive sample P + , negative sample confidence P - , positive sample weight ω + and negative sample weight ω - Input to the positive and negative sample score calculation module to extract the positive sample score s + and negative sample score s - ; Score the positive sample s + and negative sample score s - Input to the loss calculation module, and reversely update each network according to the positive sample loss and negative sample loss; obtain the positive sample score s according to the final updated network + and negative sample score s - , and get the target detection result.

[0011] The feature extraction network includes a feature extraction backbone network and a pyramid network; the data-enhanced input image is passed through the feature extraction backbone network to obtain a single-scale feature F; the single-scale feature is passed through the pyramid network to obtain a multi-scale feature.

[0012] The feature extraction backbone network includes 16 convolution modules, each of which contains 3 convolution layers, including 2 1*1 convolutions and 1 3*3 convolution, and the dimension of each convolution layer is 256.

[0013] The pyramid network includes two 1*1 convolutional layers and two upsampling operations;

[0014] The single-scale feature F is upsampled and the upsampling result is passed through a 1*1 convolution layer to obtain the feature F P1 , the dimension of each convolutional layer is 256;

[0015] The feature F P1Upsampling operation, the upsampling result is passed through a 1*1 convolution layer to obtain the feature F P2 , the dimension of each convolutional layer is 256.

[0016] The classification network includes 4 1*1 convolutions, the first 3 convolution dimensions are 256, and the last convolution dimension is 80, and the classification prediction result P is obtained. cls .

[0017] The positioning network includes 4 1*1 convolutions, the first 3 convolution dimensions are 256, and the last convolution dimension is 4, and the positioning prediction result P is obtained. loc .

[0018] In the confidence calculation module, the positive sample confidence P + And the negative sample confidence P - It is obtained by the following formula:

[0019]

[0020] Among them, β is a parameter; is the classification prediction result P cls The power of β, is the positioning prediction result P loc to the power of 1-β.

[0021] The weight prediction network includes a positive and negative sample feature extraction network, a center weight prediction network and a weighted calculation module;

[0022] The positive and negative sample feature extraction network is composed of feature cascade, convolution and upsampling operations to obtain the output feature

[0023]

[0024] in, Represents the output feature, concat represents the feature cascade operation, W C The convolution size is 1*1, and the output dimension is 2; W S It is a convolution with a size of 1*1 and an output dimension of 256;

[0025] The output features After the sigmoid function, the positive sample feature F is obtained + and negative sample features F - ;

[0026] The center weight prediction network, whose input is the single-scale feature F, passes through the global average pooling layer and two fully connected layers to obtain the Gaussian function mean and Gaussian function variance According to the Gaussian function mean and Gaussian function variance Get the object center weight C;

[0027]

[0028] in, Represents the positive sample feature F + and negative sample features F - The distance between the i-th point in and the center point of the Gaussian function;

[0029] The weighted calculation module obtains the positive sample weight ω + and negative sample weight ω - ;

[0030]

[0031] Among them, G(γ,σ 2 ) represents the Gaussian blur function.

[0032] The positive and negative sample score calculation module, the positive sample score s + and negative sample score s - as follows;

[0033]

[0034] The loss calculation module, the positive sample loss is as follows;

[0035]

[0036] is a sample in the i-th positive sample score, N is the batch size, is the number of positive samples;

[0037] The negative sample loss is as follows;

[0038]

[0039] is a sample in the i-th negative sample score, is the number of negative samples, and FL is the focal loss function.

[0040] The beneficial effects of the present invention are as follows: by calculating the weights and confidence of positive and negative samples, the accuracy of target detection is effectively improved. To improve the expressiveness of features, a positive and negative sample feature extraction network is used to fuse multi-scale features. To effectively distinguish the corresponding weights of positive and negative samples, a weight prediction network is used to calculate the center weights of positive and negative samples using a Gaussian function. To improve the confidence prediction results of positive and negative samples, the classification and positioning results are simultaneously considered in the calculation process, and the classification and positioning results are fused using a power exponential form to obtain the confidence of positive and negative samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 This is the overall structure diagram of the target detection method based on positive and negative sample learning;

[0042] Figure 2 Extract network structure diagram for positive and negative sample features;

[0043] Figure 3 This is the network structure diagram for center weight prediction. DETAILED DESCRIPTION

[0044] Figure 1 This is the main flow chart of the technical solution of the present invention. Figure 1 As shown, the target detection method based on positive and negative sample learning proposed in the present invention includes the following steps:

[0045] (1) The input image size is cropped to 1333*800, and the input image is horizontally flipped for data enhancement. The features of the input image are extracted using a feature extraction network consisting of convolution operation, batch normalization, and ReLU corrected linear unit group.

[0046] (2) Repeat step (1) multiple times to obtain a single-scale feature with a size of 40*25.

[0047] (3-1) Use convolution and upsampling operations to perform size operations on the single-scale features obtained in step (2) to obtain multi-scale features F with dimensions of 80*50 and 40*25 respectively. P1 and F P2 .

[0048] (3-2) According to Figure 2 As shown, cascade and upsampling operations are used to obtain the final output features.

[0049]

[0050] (3-3) Input the feature obtained from (3-2) into the convolution layer with a dimension of 2, and pass it through the sigmoid function to obtain the positive sample feature F + and negative sample features F - .

[0051] (4-1) Input the output features obtained from (3-2) into Figure 3 In the center weight prediction network shown in the figure, its input is a single-scale feature F, which is passed through a global average pooling layer and two fully connected layers to obtain the Gaussian function mean and Gaussian function variance According to the Gaussian function mean and Gaussian function variance Get the object center weight C;

[0052]

[0053] in, Represents the positive sample feature F + and negative sample features F - The distance between the i-th point in and the center point of the Gaussian function;

[0054] The weighted calculation module obtains the positive sample weight ω + and negative sample weight ω - ;

[0055]

[0056] Among them, G(γ,σ 2 ) represents the Gaussian blur function.

[0057] The positive and negative sample score calculation module, the positive sample score s + and negative sample score s - as follows;

[0058]

[0059] The loss calculation module, the positive sample loss is as follows;

[0060]

[0061] is the score of the i-th positive sample, N is the batch size, is the number of positive samples;

[0062] The negative sample loss is as follows;

[0063]

[0064] is the score of the i-th negative sample, is the number of negative samples, and FL is the focal loss function. This paper proposes a positive and negative sample feature extraction network and a center weight prediction network that explicitly model object appearance, achieving coarse-to-fine label assignment optimization. It also designs a positive sample selection strategy based on mass distribution, improving detection performance through differentiated training.

[0065] The performance of our proposed method is evaluated on the MS COCO and PASCAL VOC datasets and compared with existing methods. The experiment uses the FCOS framework as the base detector and uses a variety of feature extraction backbone networks (ResNet, ResNeXt, DCN, etc.) to verify the effectiveness of our method.

[0066]

Claims

1. A target detection method based on positive and negative sample learning, characterized in that: The steps are as follows: Step (1) crop the input image to a uniform size and horizontally flip it to perform data enhancement on the input image; input the data-enhanced input image into the feature extraction network to extract the multi-scale feature F P1 and F P2 ; Step (2), multi-scale feature F P1 and F P2 Input the classification network and positioning network together to extract the classification prediction results P cls And the positioning prediction result P loc ; Step (3), the classification prediction result P cls And the positioning prediction result P loc Input to the confidence calculation module and extract the positive sample confidence P + And the negative sample confidence P - ; Step (4), multi-scale feature F P1 and F P2 Input weight prediction network and extract positive sample weight ω + and negative sample weight ω - ; Step (5), the confidence of the positive sample P + , negative sample confidence P - , positive sample weight ω + and negative sample weight ω - Input to the positive and negative sample score calculation module to extract the positive sample score s + and negative sample score s - ; Score the positive sample s + and negative sample score s - Input to the loss calculation module, and reversely update each network according to the positive sample loss and negative sample loss; obtain the positive sample score s according to the final updated network + and negative sample score s - , and get the target detection result.

2. The target detection method based on positive and negative sample learning according to claim 1, characterized in that The feature extraction network includes a feature extraction backbone network and a pyramid network; the data-enhanced input image is passed through the feature extraction backbone network to obtain a single-scale feature F; the single-scale feature is passed through the pyramid network to obtain a multi-scale feature F P1 and F P2 .

3. The target detection method based on positive and negative sample learning according to claim 2, characterized in that: The pyramid network includes two 1*1 convolutional layers and two upsampling operations; The single-scale feature F is upsampled and the upsampling result is passed through a 1*1 convolution layer to obtain the feature F P1 , the dimension of each convolutional layer is 256; The feature F P1 Upsampling operation, the upsampling result is passed through a 1*1 convolution layer to obtain the feature F P2 , the dimension of each convolutional layer is 256.

4. The target detection method based on positive and negative sample learning according to claim 1, characterized in that In the confidence calculation module, the positive sample confidence P + And the negative sample confidence P - It is obtained by the following formula: Among them, β is a parameter; is the classification prediction result P cls The power of β, is the positioning prediction result P loc to the power of 1-β.

5. The target detection method based on positive and negative sample learning according to claim 1, characterized in that The weight prediction network includes a positive and negative sample feature extraction network, a center weight prediction network and a weighted calculation module; the positive and negative sample feature extraction network is composed of feature cascade, convolution and upsampling operations to obtain output features. in, Represents the output feature, concat represents the feature cascade operation, W C The convolution size is 1*1, and the output dimension is 2; W S It is a convolution with a size of 1*1 and an output dimension of 256; The output features After the sigmoid function, the positive sample feature F is obtained + and negative sample features F - ; The center weight prediction network, whose input is the single-scale feature F, passes through the global average pooling layer and two fully connected layers to obtain the Gaussian function mean and Gaussian function variance According to the Gaussian function mean and Gaussian function variance Get the object center weight C; in, Represents the positive sample feature F + and negative sample features F - The distance between the i-th point in and the center point of the Gaussian function; The weighted calculation module obtains the positive sample weight ω + and negative sample weight ω - ; Among them, G(γ,σ 2 ) represents the Gaussian blur function.

6. The target detection method based on positive and negative sample learning according to claim 1, characterized in that The positive and negative sample score calculation module, the positive sample score s + and negative sample score s - as follows; 7. The target detection method based on positive and negative sample learning according to claim 1, characterized in that The loss calculation module, the positive sample loss is as follows; is the score of the i-th positive sample, N is the batch size, is the number of positive samples; The negative sample loss is as follows; is the score of the i-th negative sample, is the number of negative samples, and FL is the focal loss function.

Images