Remote sensing weak and dense target detection method based on feature imitation learning

Abstract

The invention discloses a remote sensing weak and dense target detection method based on feature imitation learning, and relates to the technical field of optical remote sensing target detection, and the method comprises the steps: selecting a single-stage target detection model, and adding a feature imitation head at the head of the model; according to a classification head, a regression head and a feature imitation head output result, calculating a classification score and an intersection-union ratio between a target prediction result and a real label, and calculating a sample quality measure of each sample feature by using a task alignment learning technology; screening the sample features based on a set threshold value to obtain corresponding high-quality and low-quality sample features; updating a class center feature memory bank online by using high-quality sample features; utilizing the low-quality sample features and the updated class center feature memory bank to construct simulation comparison loss; and training the remote sensing weak and dense target detection model. The method solves the problems that an existing method hinders optimization of a model in an end-to-end mode, and real-time detection cannot be ensured in practical application.

Description

Technical Field

[0001] The present invention relates to the technical field of optical remote sensing target detection, and in particular to a remote sensing weak and dense target detection method based on feature imitation learning. Background Art

[0002] As a core technology in computer vision, remote sensing weak and dense target detection aims to effectively find targets of interest from images and accurately determine their categories and locations. Thanks to the effective balance between detection performance and inference latency achieved by the YOLO series of target detectors, it has received widespread attention in recent years, developed rapidly, and achieved remarkable results. However, YOLOs research mainly focuses on general target detection, and there are still huge challenges in detecting weak, dense remote sensing and aerial photography targets with unclear features, adjacent targets, or overlapping targets. Unlike general targets, targets in remote sensing and aerial photography images are small, blurred, and dense, which will result in unclear target features and mutual interference.

[0003] Researchers have sought solutions from multiple angles. Wu et al. proposed SML, mainly based on Faster R-CNN, designed an imitation loss to force the feature representation of small-scale pedestrian targets to be close to that of large-scale pedestrian targets, and enhanced the representation of small-scale pedestrian targets by imitating the rich representation of large-scale pedestrian targets. Kim et al. proposed a pedestrian detection framework based on Faster R-CNN that imitates cue recall to detect small-scale pedestrian targets. By constructing an offline large-scale pedestrian target feature repository, the features of small-scale pedestrian targets are brought closer to the features of large-scale pedestrian targets obtained in the repository. Yuan et al. proposed CRPN based on a two-stage detector to ensure higher quality small target proposal regions, and added a feature imitation branch to the detector to promote the representation of regions of instances with limited size and that confuse the model in an imitative manner.

[0004] The above methods have tried different improvements for faint targets, but they are all based on two-stage detectors and building offline memory libraries, which hinders the optimization of the model in an end-to-end manner and cannot ensure real-time detection in practical applications. Summary of the invention

[0005] In view of the above-mentioned deficiencies in the prior art, the present invention provides a remote sensing weak and dense target detection method based on feature imitation learning, which solves the problem that the existing methods hinder the model from being optimized in an end-to-end manner and cannot ensure real-time detection in practical applications.

[0006] In order to achieve the above-mentioned invention object, the technical solution adopted by the present invention is: a remote sensing weak and dense target detection method based on feature imitation learning, comprising the following steps: S1: Select the single-stage target detection model YOLO as the basic detection model, and add a feature imitation head to the head of the basic detection model; S2: Based on the output results of the classification head, regression head and feature imitation head of the basic detection model, the classification score and intersection-over-union ratio between the target prediction result and the true label are calculated, and the sample quality metric of each sample feature is calculated using task alignment learning technology; S3: setting a sample high-quality metric threshold and a sample low-quality metric threshold, and filtering the sample quality metrics of each sample feature based on the set threshold to obtain corresponding high-quality sample features and low-quality sample features; S4: Use high-quality sample features to update the class-centric feature memory library online; S5: Use low-quality sample features and the updated class center feature memory library to construct imitation contrast loss; S6: Train the remote sensing weak and dense target detection model based on the imitation contrast loss to obtain the trained remote sensing weak and dense target detection model; S7: Apply the trained remote sensing weak and dense target detection model to actual scenarios to complete remote sensing weak and dense target detection based on feature imitation learning.

[0007] Furthermore, the feature imitation head in S1 includes two convolution modules and a nonlinear mapping head connected in series; The convolution module includes a convolution layer Conv2d, a normalization layer BatchNorm2d and an activation layer SiLU; The nonlinear mapping head includes a convolution layer Conv2d, a normalization layer BatchNorm2d, an activation layer SiLU and a convolution layer Conv2d.

[0008] Furthermore, S2 includes the following sub-steps: S21: Obtain the bounding box information matrix based on the output results of the classification head, regression head, and feature imitation head , category score matrix and the target feature embedding matrix ,in, is the field of real numbers, is the batch size, is the number of anchors in each image, is the maximum range of the anchor parameter, is the number of categories in the data set, is the feature embedding dimension; S22: Calculate the classification score between the target prediction result and the true label based on the bounding box information matrix, the category score matrix and the target feature embedding matrix Compared with intersection IoU , the formula is:

[0009]

[0010] in, For the intersection, For the union, Annotate the ground-truth bounding box; S23: Based on the classification score and intersection-over-union ratio between the target prediction result and the true label, the sample quality metric of each sample feature is calculated using task alignment learning technology. The formula is:

[0011] in, is the sample quality indicator, is the scale normalization function used for training stability, is the classification confidence of the sample, is the intersection-over-union ratio of the samples, and is a hyperparameter.

[0012] Furthermore, in S4, the class center feature memory library is updated online using high-quality sample features, and the formula is:

[0013]

[0014] in, After the update The feature embedding of the class in the class-centric feature memory, Before update The feature embedding of the class in the class-centric feature memory, is the momentum factor, is the first Class center features, is the number of all high-quality examples, For the The target feature embedding of high-quality examples, is the indicator function, For the Is the class label of the high-quality examples , if yes, then is 1, otherwise is 0.

[0015] Furthermore, S5 includes the following sub-steps: S51: Calculate the similarity between the low-quality sample features and the class center features of the updated class center feature memory library based on cosine similarity; S52: Constructing imitation contrastive loss based on similarity.

[0016] Furthermore, the simulated contrast loss constructed in S52 is:

[0017] in, To mimic the contrast loss, is the classification loss in the base detection model, is the regression loss in the basic detection model, is the weight parameter, is the instance-level loss, is the class-level loss;

[0018] in, is the number of low-quality samples, is the cosine similarity function, For the Low-quality sample feature embedding, and the category is , is the temperature parameter;

[0019] in, is the number of categories, Before update The feature embedding of the class in the class-centric feature memory, After the update Feature embedding of classes in class-centric feature memory.

[0020] The beneficial effects of the present invention are as follows: the present invention integrates a plug-and-play feature imitation learning module based on the single-stage target detector YOLO to improve feature learning, especially for faint and dense targets. The FIL paradigm proposed in this paper is based on contrastive learning, and enhances the representation of faint and dense target features without adding additional parameters or computational overhead during the inference process. Compared with the original benchmark detector, the improved detector of the present invention achieves better detection accuracy on faint and dense remote sensing targets while maintaining the same inference delay as the original benchmark detector, and can be extended to any single-stage detector. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 Schematic diagram of the overall framework of the remote sensing weak and dense target detection method based on feature imitation learning of the present invention.

[0022] Figure 2 This is a flow chart of the remote sensing weak and dense target detection method based on feature imitation learning of the present invention.

[0023] Figure 3 It is a schematic diagram of the detection results of the remote sensing weak and dense target detection method based on feature imitation learning of the present invention. DETAILED DESCRIPTION

[0024] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.

[0025] The present invention provides a remote sensing weak and dense target detection method based on feature imitation learning, and the corresponding overall network structure is as follows: Figure 1 As shown in the figure, it mainly includes the construction and use of feature imitation head, high / low quality positive sample selection, class center memory library update, imitation comparison loss construction, detection performance comparison and visualization. Figure 1 The overall network structure shown in the figure, the method mainly includes the following steps: like Figure 2 As shown, a remote sensing weak and dense target detection method based on feature imitation learning includes the following steps: S1: Select the single-stage target detection model YOLO as the basic detection model, and add a feature imitation head to the head of the basic detection model; S2: Based on the output results of the classification head, regression head and feature imitation head of the basic detection model, the classification score and intersection-over-union ratio between the target prediction result and the true label are calculated, and the sample quality metric of each sample feature is calculated using task alignment learning technology; S3: setting a sample high-quality metric threshold and a sample low-quality metric threshold, and filtering the sample quality metrics of each sample feature based on the set threshold to obtain corresponding high-quality sample features and low-quality sample features; S4: Use high-quality sample features to update the class-centric feature memory library online; S5: Use low-quality sample features and the updated class center feature memory library to construct imitation contrast loss; S6: Train the remote sensing weak and dense target detection model based on the imitation contrast loss to obtain the trained remote sensing weak and dense target detection model; S7: Apply the trained remote sensing weak and dense target detection model to actual scenarios to complete remote sensing weak and dense target detection based on feature imitation learning.

[0026] In this embodiment, two currently optimal single-stage detectors, YOLOv10-N and YOLOv11-N, are selected as basic detection models to improve the detection of weak and dense remote sensing targets.

[0027] In order to enhance the detector’s ability to identify faint and densely distributed targets without adding additional computational or parameter overhead during the inference process, a feature imitation head (FIH) is introduced in the head stage of the basic detection model. Figure 1 As shown in (a), FIH works in parallel with the regression head (RH) responsible for object localization and the classification head (CH) responsible for classification. FIH focuses on optimizing target feature embedding to improve detection performance. Its detailed architecture is shown in Figure 1 As shown in (b), it is mainly composed of two convolution modules and a nonlinear mapping head in series. During the training phase, FIH is jointly optimized with RH, CH, and the rest of the network, so that the backbone network and the neck can take advantage of the comprehensive supervision provided by feature imitation learning (FIL), and FIH shares weights with the two convolution modules ConvModule in CH, thereby minimizing parameter usage and computational complexity. It is worth noting that in the inference phase, FIH is discarded and only RH and CH are retained. This ensures that the proposed framework does not incur additional computational cost or parameter burden compared to the baseline detector during inference.

[0028] The feature imitation head in S1 includes two convolution modules and a nonlinear mapping head connected in series; The convolution module includes a convolution layer Conv2d, a normalization layer BatchNorm2d and an activation layer SiLU; The nonlinear mapping head includes a convolution layer Conv2d, a normalization layer BatchNorm2d, an activation layer SiLU and a convolution layer Conv2d.

[0029] In order to further optimize the embedding of high-quality and low-quality object features, three key matrices are obtained by processing the outputs of RH, CH and FIH: , category score matrix and the target feature embedding matrix Based on the three output matrices and task alignment learning, each sample is classified as a positive or negative sample according to the example quality metric (IQM), which comprehensively evaluates the classification confidence and regression quality of the sample and is used to evaluate the instance quality.

[0030] The S2 includes the following sub-steps: S21: Obtain the bounding box information matrix based on the output results of the classification head, regression head, and feature imitation head , category score matrix and the target feature embedding matrix ,in, is the field of real numbers, is the batch size, is the number of anchors in each image, is the maximum range of the anchor parameter, is the number of categories in the data set, is the feature embedding dimension; S22: Calculate the classification score between the target prediction result and the true label based on the bounding box information matrix, the category score matrix and the target feature embedding matrix Compared with intersection IoU , the formula is:

[0031]

[0032] in, For the intersection, For the union, Annotate the ground-truth bounding box; S23: Based on the classification score and intersection-over-union ratio between the target prediction result and the true label, the sample quality metric of each sample feature is calculated using task alignment learning technology. The formula is:

[0033] in, is the sample quality indicator, is the scale normalization function used for training stability, is the classification confidence of the sample, is the intersection-over-union ratio of the samples, and is a hyperparameter. The value indicates that the anchor sample has a higher classification score and more accurate location prediction. For each instance, the m anchors with the highest alignment values ​​are selected as positive samples. By setting an appropriate threshold and , which can effectively distinguish high-quality and low-quality positive samples. This supports the dynamic update of class samples and feature imitation learning, thereby improving the overall quality of feature representation.

[0034] In S4, high-quality sample features are used to update the class center feature memory library online, and the formula is:

[0035]

[0036] in, After the update The feature embedding of the class in the class-centric feature memory, Before update The feature embedding of the class in the class-centric feature memory, is the momentum factor, is the first Class center features, is the number of all high-quality examples, For the The target feature embedding of high-quality examples, is the indicator function, For the Is the class label of the high-quality examples , if yes, then is 1, otherwise is 0.

[0037] Figure 1 (a) shows the structure of the class-centric feature memory (MBCE). The class examples of the current dataset are defined as ,in Corresponding to The class center feature of the class, and the feature dimension is Initially, all class examples are initialized to zero. MBCE operates in the form of an online updatable memory bank, which aims to preserve the learned knowledge and guide the model's feature learning. Specifically, the alignment metric Exceeding the threshold The samples are selected as high-quality positive samples (HQP). The target feature embedding and category label corresponding to each HQP sample are and In the memory, category examples are updated via a momentum-based mechanism. The current category center is first calculated using the high-quality positive samples in the current batch. The category center is then used to update the corresponding category examples in the memory bank, thus ensuring adaptive and incremental improvement of category-level representation.

[0038] The S5 comprises the following sub-steps: S51: Calculate the similarity between the low-quality sample features and the class center features of the updated class center feature memory library based on cosine similarity; S52: Constructing imitation contrastive loss based on similarity.

[0039] The simulated contrast loss constructed in S52 is:

[0040] in, To mimic the contrast loss, is the classification loss in the base detection model, is the regression loss in the basic detection model, is the weight parameter, is the instance-level loss, is the class-level loss;

[0041] in, is the number of low-quality samples, is the cosine similarity function, For the Low-quality sample feature embedding, and the category is , is the temperature parameter;

[0042] in, is the number of categories, Before update The feature embedding of the class in the class-centric feature memory, After the update Feature embedding of classes in class-centric feature memory.

[0043] Based on low quality threshold Screening low quality positive (LQP) sample embedding features Then the low-quality sample features are calculated based on cosine similarity And the updated class center feature The similarity between them is calculated and based on the obtained similarity, a supervised contrastive loss is constructed. The purpose of this process is to effectively pull low-quality feature embeddings closer to their respective corresponding class center features while pushing them away from other class center features. To achieve this, an instance-level loss is introduced.

[0044] To address mode collapse and improve the efficiency of feature imitation learning, a class-level loss is proposed, which mainly exploits the features of class center samples before and after the update. This loss ensures the consistency and diversity of class sample representation. This loss encourages consistency between class examples before and after the update while maintaining separability between classes. The bidirectional formulation ensures that the update does not overly bias the representation, effectively alleviating mode collapse and maintaining the diversity and robustness of class examples.

[0045] By carefully designing these losses, our method not only achieves feature imitation learning but also effectively alleviates mode collapse, thereby enhancing the feature representation of faint and dense objects and improving the detection performance of the detection algorithm.

[0046] In one embodiment of the present invention, in order to demonstrate the effectiveness of the method of the present invention for detecting weak and dense remote sensing targets, DIOR, AI-TOD-v2 and VisDrone2019 are used to train and verify the model. The specific description information of the data set is shown in Table 1, where starting from the third column in the table, the numbers on the left side of each column represent the number of images in different data sets, and the numbers on the right side of each column represent the number of instances in different data sets. The performance comparison of the detection algorithm on different data sets is given, as shown in Table 2, where the darker background color is the detection index corresponding to the present invention on each data. In order to understand the effectiveness of the feature-based imitation learning detection algorithm from a more intuitive perspective, some representative image data are selected to visualize the detection results, as shown in Table 2. Figure 3 The specific performance indicators and visualization results once again verify the effectiveness of the proposed method for weak and dense remote sensing target detection.

[0047] Table 1 Detailed description of the dataset

[0048] Table 2 Performance comparison of detection algorithms on different datasets

[0049] This paper attempts for the first time to use feature imitation learning in a single-stage target detection framework to solve the problem of dense target detection. The proposed feature imitation learning (FIL) is a plug-and-play module that can be seamlessly integrated into any single-stage detector. At the same time, it improves the accuracy of remote sensing weak and dense target detection without introducing additional parameters and computational complexity in the reasoning process.

[0050] Those skilled in the art will appreciate that the embodiments described herein are intended to help readers understand the principles of the present invention, and should be understood that the protection scope of the present invention is not limited to such specific statements and embodiments. Those skilled in the art can make various other specific variations and combinations that do not deviate from the essence of the present invention based on the technical revelations disclosed by the present invention, and these variations and combinations are still within the protection scope of the invention.

Claims

1. A remote sensing weak and dense target detection method based on feature imitation learning, characterized in that: The following steps are involved: S1: Select the single-stage target detection model YOLO as the basic detection model, and add a feature imitation head to the head of the basic detection model; S2: Based on the output results of the classification head, regression head and feature imitation head of the basic detection model, the classification score and intersection-over-union ratio between the target prediction result and the true label are calculated, and the sample quality metric of each sample feature is calculated using task alignment learning technology; S3: setting a sample high-quality metric threshold and a sample low-quality metric threshold, and filtering the sample quality metrics of each sample feature based on the set threshold to obtain corresponding high-quality sample features and low-quality sample features; S4: Use high-quality sample features to update the class-centric feature memory library online; S5: Use low-quality sample features and the updated class center feature memory library to construct imitation contrast loss; S6: Train the remote sensing weak and dense target detection model based on the imitation contrast loss to obtain the trained remote sensing weak and dense target detection model; S7: Apply the trained remote sensing weak and dense target detection model to actual scenarios to complete remote sensing weak and dense target detection based on feature imitation learning.

2. According to claim 1, a remote sensing weak and dense target detection method based on feature imitation learning is characterized in that: The feature imitation head in S1 includes two convolution modules and a nonlinear mapping head connected in series; The convolution module includes a convolution layer Conv2d, a normalization layer BatchNorm2d and an activation layer SiLU; The nonlinear mapping head includes a convolution layer Conv2d, a normalization layer BatchNorm2d, an activation layer SiLU and a convolution layer Conv2d.

3. According to claim 1, a remote sensing weak and dense target detection method based on feature imitation learning is characterized in that: The S2 includes the following sub-steps: S21: Obtain the bounding box information matrix based on the output results of the classification head, regression head, and feature imitation head , category score matrix and the target feature embedding matrix ,in, is the field of real numbers, is the batch size, is the number of anchors in each image, is the maximum range of the anchor parameter, is the number of categories in the data set, is the feature embedding dimension; S22: Calculate the classification score between the target prediction result and the true label based on the bounding box information matrix, the category score matrix and the target feature embedding matrix Compared with intersection IoU , the formula is: in, For the intersection, For the union, Annotate the ground-truth bounding box; S23: Based on the classification score and intersection-over-union ratio between the target prediction result and the true label, the sample quality metric of each sample feature is calculated using task alignment learning technology. The formula is: in, is the sample quality indicator, is the scale normalization function used for training stability, is the classification confidence of the sample, is the intersection-over-union ratio of the samples, and is a hyperparameter.

4. According to claim 1, a remote sensing weak and dense target detection method based on feature imitation learning is characterized in that: In S4, high-quality sample features are used to update the class center feature memory library online, and the formula is: in, After the update The feature embedding of the class in the class-centric feature memory, Before update The feature embedding of the class in the class-centric feature memory, is the momentum factor, is the first Class center features, is the number of all high-quality examples, For the The target feature embedding of high-quality examples, is the indicator function, For the Is the class label of the high-quality examples , if yes, then is 1, otherwise is 0.

5. According to claim 4, a remote sensing weak and dense target detection method based on feature imitation learning is characterized in that: The S5 comprises the following sub-steps: S51: Calculate the similarity between the low-quality sample features and the class center features of the updated class center feature memory library based on cosine similarity; S52: Constructing imitation contrastive loss based on similarity.

6. The remote sensing weak and dense target detection method based on feature imitation learning according to claim 5 is characterized in that: The simulated contrast loss constructed in S52 is: in, To mimic the contrast loss, is the classification loss in the basic detection model, is the regression loss in the basic detection model, is the weight parameter, is the instance-level loss, is the class-level loss; in, is the number of low-quality samples, is the cosine similarity function, For the Low-quality sample feature embedding, and the category is , is the temperature parameter; in, is the number of categories, Before update The feature embedding of the class in the class-centric feature memory, After the update Feature embedding of classes in class-centric feature memory.

Images