Small target detection method based on dynamic sample aggregation and prompt word feature enhancement

Abstract

The application discloses a small target detection method based on dynamic sample aggregation and prompt word feature enhancement, and the method comprises the following steps: acquiring a remote sensing small target image to be detected; acquiring a target detection model; and detecting the remote sensing small target image to be detected by using the target detection model, wherein the target detection model takes YOLOv8 as a basic network, a cascaded multi-scale attention module is added in a backbone network of the YOLOv8, and the backbone network is optimized; the target detection model further comprises a dynamic sample aggregation module and a prompt word-based feature enhancement module, features extracted by the optimized backbone network are input into the dynamic sample aggregation module and the prompt word-based feature enhancement module, the dynamic sample aggregation module is used to evaluate a sample feature distribution aggregation degree, and the prompt word-based feature enhancement module is used to increase a feature quantity of a rare class. The method disclosed by the application effectively improves the accuracy of small target detection.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, including image processing, deep learning, and object detection and recognition technologies, and particularly to a small object detection method based on dynamic sample feature aggregation and cue word feature enhancement. Background Technology

[0002] In the field of remote sensing image processing, sample imbalance is a significant and pervasive challenge. With the continuous advancement of remote sensing technology, the ability to acquire high-resolution images has significantly improved. However, in many practical applications, such as environmental monitoring, disaster relief, and agricultural monitoring, the number of samples for certain target categories is often far lower than that for other categories. This sample imbalance makes it difficult for target detection models to learn effective features of these rare categories during training, thus affecting the accuracy and reliability of detection. Existing deep learning-based target detection methods, such as YOLO (You Only LookOnce), CNN (Convolutional Neural Networks), SSD (Single Shot MultiBox Detector), and VIT (Vision Transformer), typically rely on a large number of labeled samples to achieve high-performance detection results. However, in remote sensing images, specific targets (such as small objects or anomalies) may only account for a very small portion of the total samples, causing the models to perform poorly when faced with sample imbalance. While the aforementioned algorithms offer advantages in detection speed and efficiency, they still face challenges in the identification and classification of rare targets.

[0003] To address this, researchers have proposed various strategies, including oversampling, undersampling, data augmentation, and contrastive learning. While oversampling and undersampling can reduce the negative impact of class imbalance in the training dataset and enrich feature information, they can also lead to a large number of duplicate samples in the training set. This redundant data not only increases the computational burden on the model but may also cause overfitting and reduce the model's generalization ability.

[0004] Research on contrastive learning and other methods primarily focuses on large-size target images, using simple backbone networks for feature extraction, and studying the network's feature perception capabilities only from a single scale. Furthermore, when evaluating feature representations, they do not learn nonlinear representations, making it impossible to dynamically assess the feature clustering of samples. Summary of the Invention

[0005] In view of this, the purpose of this invention is to provide a small target detection method based on dynamic sample feature aggregation and cue word feature enhancement, the method comprising:

[0006] Acquire images of the small remote sensing targets to be detected;

[0007] Obtain the object detection model;

[0008] The target detection model is used to detect small remote sensing targets in the image.

[0009] in,

[0010] The object detection model uses YOLOv8 as the base network and adds a cascaded multi-scale attention module to the YOLOv8 backbone network to optimize the backbone network.

[0011] Cascaded multi-scale attention modules include sequentially connected spatial attention units and channel attention units;

[0012] The object detection model also includes a dynamic sample aggregation module and a cue-based feature enhancement module. The features extracted by the optimized backbone network are input into the dynamic sample aggregation module and the cue-based feature enhancement module. The dynamic sample aggregation module is used to evaluate the clustering degree of sample feature distribution and output a loss function to optimize the backbone network through backpropagation. The cue-based feature enhancement module is used to increase the feature amount of rare classes and pass the processed features to the neck network of YOLOv8. The neck network performs multi-scale fusion of the enhanced features and finally generates object detection results through the detection head.

[0013] Furthermore, the YOLOv8 backbone network sequentially includes a first convolutional module, a second convolutional module, a first C2f module, a third convolutional module, a second C2f module, a fourth convolutional module, a third C2f module, a fifth convolutional module, a fourth C2f module, and an SPPF module. The optimized backbone network has two cascaded multi-scale attention modules: the first cascaded multi-scale attention module is located between the second convolutional module and the first C2f module; the second cascaded multi-scale attention module is located between the third C2f module and the fifth convolutional module.

[0014] Furthermore, the spatial attention unit sequentially comprises two 2D convolutional layers, two 1D convolutional layers, a pooling layer, another convolutional layer, and an activation function.

[0015] The spatial attention unit performs the following operations:

[0016] The large kernel convolution algorithm is used to perform depthwise convolution on the input feature map to obtain spatial feature maps with different kernel sizes;

[0017] The spatial feature maps are connected by channels to obtain the first connection result;

[0018] The first connection result is input into the pooling layer for average pooling and max pooling respectively, to obtain the average pooling result and the max pooling result;

[0019] The average pooling result and the max pooling result are concatenated by channel to obtain the second concatenation result;

[0020] The second connection result is input into the convolutional layer for convolution to obtain a spatial attention map containing N channels;

[0021] Apply the Sigmoid activation function to the spatial attention map of each channel to obtain the spatial selection mask of each decomposed large kernel, and obtain the feature representation of the large kernel sequence.

[0022] The feature representation of each large kernel sequence is weighted using the corresponding spatial selection mask;

[0023] The weighted summation results corresponding to all large kernel sequences are fused through convolution to obtain the spatial attention feature, i.e., the output A of the spatial attention unit.

[0024] Furthermore, the channel attention unit sequentially includes a two-dimensional convolutional layer, a global average pooling layer, a one-dimensional convolutional layer, a sigmoid activation function, and an SR layer;

[0025] The channel attention unit performs the following operations:

[0026] Obtain the output A of the spatial attention unit;

[0027] The output A of the spatial attention unit is passed through a two-dimensional convolutional layer and then through a global average pooling layer to obtain the channel aggregation feature.

[0028] Channel aggregation features are processed by a one-dimensional convolutional layer to generate channel weights.

[0029] The channel weights are applied to the Sigmoid activation function to generate feature weights;

[0030] The feature weights and channel aggregated features are weighted and summed along the channel dimension.

[0031] The weighted summation result is input into the SR layer to obtain the channel attention features, which are the outputs of the channel attention units.

[0032] Furthermore, the dynamic sample aggregation module sequentially includes a linear layer, a batch normalization layer, a first convolutional layer, a normalization layer, and a second convolutional layer.

[0033] Furthermore, the feature enhancement module based on cue words includes a text library, in which each rare category corresponds to a spacing cue word vector; the feature enhancement module based on cue words also includes linear layers and Transformer layers.

[0034] The spacing cue word vectors are transformed into cue word feature vectors through a linear layer;

[0035] The optimized backbone network output features are embedded and input into the prompt word feature vector into the Transformer layer to generate output features of the prompt word-based feature enhancement module. The output features are then input into the classifier head and object detector of the object detection model for detection and classification.

[0036] Furthermore, the method also includes optimizing the target detection model using the CIOU loss function, DFL loss function, and CLS loss function.

[0037] This invention also provides a small target detection system based on dynamic sample feature aggregation and cue word feature enhancement, comprising:

[0038] Memory, configured to store computer programs;

[0039] The processor is configured to execute the computer program to implement the small target detection method based on dynamic sample feature aggregation and cue word feature enhancement as described above.

[0040] The beneficial effects of this invention are:

[0041] This invention proposes a cascaded multi-scale attention module, which optimizes the feature extraction process by combining attention strategies of different dimensions, thereby significantly enhancing the model's ability to recognize small target features. Next, a dynamic sample aggregation module is proposed. By calculating the normalized temperature cross-entropy contrastive loss function, the backbone network can effectively learn feature representations that distinguish between rare and majority class samples, improving the aggregation degree of features of samples of the same class and reducing the aggregation of features of samples of different classes, thus improving the discriminative power of inter-class boundaries. Finally, a cue word-based feature enhancement module is proposed. This module adds learnable cue word vectors to the input space, enabling the model to pay more balanced attention to samples of different classes without adding an additional text feature extractor, thereby improving the sample imbalance problem. This method is suitable for small target detection tasks, and is particularly effective in alleviating the problem of the model ignoring rare class sample features under sample imbalance conditions.

[0042] Other advantages, objectives, and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination, or may be learned from practice of the invention. The objectives and other advantages of the invention can be realized and obtained through the following description. Attached Figure Description

[0043] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will now be described in further detail with reference to the accompanying drawings, wherein:

[0044] Figure 1This is a schematic flowchart of a small target detection method based on dynamic sample feature aggregation and cue word feature enhancement;

[0045] Figure 2 This is a network structure diagram of the object detection model;

[0046] Figure 3 This is a structural diagram of a cascaded multi-scale attention mechanism;

[0047] Figure 4 This is a graph of the model training loss function;

[0048] Figure 5 This is a display image showing the test results. Detailed Implementation

[0049] The preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be understood that the preferred embodiments are for illustrative purposes only and are not intended to limit the scope of protection of the present invention.

[0050] This invention addresses the problem of small target detection and recognition in remote sensing imagery through a small target detection method. The model mainly consists of three parts: a small target feature extraction backbone network, a dynamic sample feature aggregation module, and a sample imbalance feature enhancement module (i.e., a feature enhancement module based on cue words). The small target feature extraction backbone network adopts a deep learning network structure, combining feature aggregation and sample imbalance feature enhancement techniques to effectively extract and fuse multi-scale information. This allows the model to pay more balanced attention to various sample features during training, mitigating misclassification of rare classes in imbalanced samples.

[0051] Small target detection methods based on dynamic sample feature aggregation and cue word feature enhancement may include the following steps:

[0052] Acquire images of the small remote sensing targets to be detected;

[0053] Obtain the object detection model;

[0054] The target detection model is used to detect small remote sensing targets in the image.

[0055] in,

[0056] The object detection model uses YOLOv8 as the base network and adds a cascaded multi-scale attention module to the YOLOv8 backbone network to optimize the backbone network.

[0057] Cascaded multi-scale attention modules include sequentially connected spatial attention units and channel attention units;

[0058] The object detection model also includes a dynamic sample aggregation module and a cue-based feature enhancement module. The features extracted by the optimized backbone network are input into the dynamic sample aggregation module and the cue-based feature enhancement module. The dynamic sample aggregation module is used to evaluate the clustering degree of sample feature distribution and output a loss function to optimize the backbone network through backpropagation. The cue-based feature enhancement module is used to increase the feature amount of rare categories and pass the processed features to the neck network of YOLOv8. The neck network performs multi-scale fusion of the enhanced features and finally generates object detection results through the detection head.

[0059] Figure 1 This is a schematic flowchart of a small target detection method based on dynamic sample feature aggregation and cue word feature enhancement. Figure 2 This is a network architecture diagram of an object detection model. (Example:) Figure 1 and Figure 2 As shown, after acquiring remote sensing images, they can be preprocessed. For example, operations such as cropping, rotation, and brightness / contrast adjustment can be applied to preprocess the remote sensing image data.

[0060] By designing cascaded multi-scale attention modules, the feature extraction process can be optimized, thereby extracting refined small target features;

[0061] Next, the similarity of the extracted features is calculated and the clustering of sample feature distribution is dynamically evaluated;

[0062] Then, a feature enhancement module based on cue words adds learnable cue word vectors to the features extracted from the backbone network, and fuses rare class semantic features with image features to increase the amount of rare class features.

[0063] This method also includes training the model, calculating image category scores, and optimizing object detection accuracy using CIOU, DFL, CLS, and XT loss functions. After model training is complete, the trained model can be used to locate and classify small targets in the remote sensing images to be detected.

[0064] The YOLOv8 backbone network consists of, in sequence, a first convolutional module, a second convolutional module, a first C2f module (i.e., a cross-level local network), a third convolutional module, a second C2f module, a fourth convolutional module, a third C2f module, a fifth convolutional module, a fourth C2f module, and an SPPF module (i.e., Spatial Pyramid Pooling-Fast, corresponding to...). Figure 2In the optimized backbone network (feature pyramid), there are two cascaded multi-scale attention modules. The first cascaded multi-scale attention module is located between the second convolutional module and the first C2f module; the second cascaded multi-scale attention module is located between the third C2f module and the fifth convolutional module.

[0065] A cascaded multi-scale attention mechanism is introduced to optimize the backbone network, with the network structure as follows: Figure 2 As shown, this module integrates attention strategies from different dimensions to optimize the feature extraction process, such as... Figure 3 As shown, this enhances the model's ability to recognize small features of the target; the original image or feature map is input into the CNN for feature extraction as shown in formula (1):

[0066] X i+1 =CNN(X) i ), (1)

[0067] Wherein, CNN(·) is the feature extraction module, i.e., a convolutional neural network, corresponding to the convolutional layer in the YOLOv8 backbone network, X i It is the input of the i-th layer of the CNN neural network, where i ranges from 1 to k, and k represents the total number of layers in the CNN neural network.

[0068] The feature map X obtained after passing the original image through a CNN neural network. k+1 The input consists of a cascaded multi-scale attention module composed of spatial and channel attention mechanisms.

[0069] Figure 3 This is a diagram of the cascaded multi-scale attention module structure. The cascaded multi-scale attention module includes sequentially connected spatial attention units and channel attention units. The spatial attention unit sequentially includes two two-dimensional convolutional layers (i.e., ...). Figure 3 Conv2d in the middle) and two one-dimensional convolutional layers (i.e. Figure 3 Conv1d in the pooling layer (i.e.) Figure 3 Avg-max-pool and convolutional layers (i.e., ...) Figure 3 Conv2d after Avg-max-pool and activation function layer (i.e. Figure 3 For details on the connection methods of the sigmoid function (two 2D convolutional layers and one 1D convolutional layer), please refer to [reference needed]. Figure 3 The spatial attention unit performs the following operations:

[0070] The large kernel convolution algorithm is used to perform depth convolution on the input feature map to obtain spatial feature maps with different kernel sizes (i.e., the result generated by formula (5) below). This step can adaptively mine the spatial information of remote sensing image feature maps in the spatial dimension.

[0071] The spatial feature maps are connected by channels to obtain the first connection result;

[0072] The first connection result is input into the pooling layer for average pooling and max pooling respectively, to obtain the average pooling result and the max pooling result;

[0073] The average pooling result and the max pooling result are concatenated by channel to obtain the second concatenation result;

[0074] The second connection result is input into the convolutional layer for convolution to obtain a spatial attention map containing N channels;

[0075] Apply the Sigmoid activation function to the spatial attention map of each channel to obtain the spatial selection mask of each decomposed large kernel, and obtain the feature representation of the large kernel sequence.

[0076] The feature representation of each large kernel sequence is weighted using the corresponding spatial selection mask;

[0077] The weighted summation results corresponding to all large kernel sequences are fused through convolution to obtain the spatial attention feature, i.e., the output A of the spatial attention unit.

[0078] In this invention, the kernel size r of the i-th convolution is defined. i void ratio e i and feel the wild T i As shown in formulas (2) and (3):

[0079] r i-1 ≤r i ,e=1,e i-1 <e i ≤T i-1 (2)

[0080] T1=r1,T i =e i (r i -1)+T i-1 (3)

[0081] In order to extract from feature map X n (Right now Figure 5 The invention obtains features with rich contextual spatial information by taking the first two-dimensional convolution Conv2d from left to right (the result obtained from the first two-dimensional convolution Conv2d from left to right) within different ranges. The invention uses deep convolutions with different receptive fields as shown in formulas (4) and (5):

[0082] X n+1 =U i (X n (4)

[0083]

[0084] Among them, U i (·) is a depthwise convolution with kernel size k and dilation rate d. It is a 1×1 convolutional layer. That is, spatial feature maps with different kernel sizes.

[0085] Next, the feature maps obtained from different kernels are concatenated along the channel dimension, and then channel-based average pooling S is used. avg and max pooling S max The algorithm then concatenates the feature maps processed by the average pooling layer and the max pooling layer by channel, and passes them through a convolutional layer U. N N spatial attention maps are generated (i.e., spatial attention maps with N channels, where N represents the number of channels in the spatial attention map). Each kernel corresponds to one spatial attention map, and its calculation formula is shown in formula (6):

[0086]

[0087] Spatial attention map for each channel The Sigmoid activation function is applied to obtain a separate spatial selection mask for each decomposed large kernel, thus obtaining the feature representation of the large kernel sequence. Here, 'i' corresponds to different kernels. Decomposing the large kernel breaks down a large convolutional kernel into multiple smaller kernels. Each spatial attention map corresponds to a large kernel and also a spatial selection mask. The features in the decomposed large kernel sequence are weighted through their corresponding spatial selection masks (i.e., the feature representation of the large kernel sequence is weighted). The feature map obtained by formula (5) is multiplied by the feature map and then fused through a convolutional layer U to obtain the spatial attention feature A. This process is shown in formula (7):

[0088]

[0089] Next, the final output A is used as the input to the channel attention module. The channel attention unit consists of a two-dimensional convolutional layer, a global average pooling layer, a one-dimensional convolutional layer, a sigmoid activation function layer, and an SR layer.

[0090] The channel attention unit performs the following operations:

[0091] Obtain the output A of the spatial attention unit;

[0092] The output A of the spatial attention unit is convolved through a two-dimensional convolutional layer, and then global average pooling is performed through a global average pooling layer to obtain the channel aggregation feature.

[0093] Channel aggregation features are processed by a one-dimensional convolutional layer to generate channel weights.

[0094] The channel weights are applied to the Sigmoid activation function to generate feature weights;

[0095] The feature weights and channel aggregated features are weighted and summed along the channel dimension.

[0096] The weighted summation result is input into the SR layer to obtain the channel attention features, which are the outputs of the channel attention units.

[0097] Specifically, the final output A serves as the input to the channel attention module, enabling the network to learn the channel information of the drone image feature map. This is achieved first through a two-dimensional convolutional layer, followed by a global average pooling layer (GAP) to obtain the channel aggregated features. Channel weights are then generated by performing a one-dimensional convolution algorithm of size k1; in this invention, k1 is set to 3. The intermediate inputs are then fed into a sigmoid activation function to extract feature weights. Finally, the channel aggregation features are weighted along the channel dimension to obtain a feature map containing rich spatial and channel information. This map is then input into the SR (SILU-RELU) layer to enhance the feature representation capability of small targets. The SR layer includes two activation functions, SILU and RELU, which are cascaded. The process is shown in formula (8):

[0098]

[0099] Where Y′ is the channel attention output feature and c is the number of channels.

[0100] The dynamic sample aggregation module includes a linear layer and a batch normalization layer (i.e., ... Figure 2 The two-dimensional batch normalization layer, the first convolutional layer, the normalized layer, and the second convolutional layer in Figure (a) are shown.

[0101] The output of the dynamic feature aggregation module (or simply "feature aggregation module") is a loss function. This loss function is used for backpropagation to optimize and update the weight parameters of the backbone network. Specifically, the dynamic sample aggregation module transforms the feature map vectors output by the feature extraction network into a similarity matrix, calculates the loss function to optimize the feature extraction network, and enables the backbone network to learn the feature representations that distinguish between positive and negative samples. This allows the model to learn and differentiate the similarity of sample point codes to learn the general features of samples, thereby mitigating the misclassification problem caused by sample imbalance.

[0102] This module first inputs the feature map containing small target features output from the backbone network into a linear layer to convert it into a feature vector. During the training phase, the same batch of images contains multiple images. Let Y be the feature vector output by the feature extractor of the i-th original image. i (i.e., Y′ generated in formula (8)) and the feature vector emb obtained from the linear layer i As shown in the following formula:

[0103] emb i =Linear(Y i (9)

[0104] Here, Linear represents the processing of a linear layer.

[0105] Then, cosine similarity is calculated for each pair of feature vectors obtained from a batch of images (total number N0).

[0106] The contrastive learning loss function is obtained using cosine similarity.

[0107] By comparing the learning loss functions, the XT loss function is obtained.

[0108] Specifically, this can be expressed using formulas (10) and (11):

[0109]

[0110] Wherein, formula (10) represents the normalized temperature cross-entropy contrast loss function, loss XT Let XT represent the loss function, cosSim ij This represents the cosine similarity between the feature vectors output by the linear layer of the tibia in the i-th and j-th remote sensing images (small target images).

[0111] The feature enhancement module based on prompt words employs a method of adding learnable prompt word vectors to the input space to enhance attention to scarce features. Through textual word vector prompts, it leverages the correlation of textual features to enhance sample features.

[0112] The feature enhancement module based on cue words includes a text library, in which each rare category corresponds to a spacing cue word vector, which is initialized empirically; the feature enhancement module based on cue words also includes a linear layer and a Transformer layer.

[0113] The spacing cue word vectors are transformed into cue word feature vector embeddings through a linear layer, which is expressed by formula (12):

[0114]

[0115] embedding i =Linear(vec i (13)

[0116] D→vec i This represents the process of retrieving and initializing word vector suggestions from the text library, where i ranges from 1 to m, and m represents the number of linear layers.

[0117] After that, as Figure 2 As shown in Figure (b), the optimized backbone network output features are embedded and input into the prompt word feature vector into the Transformer layer to generate the output features of the prompt word-based feature enhancement module. These output features are then input into the classifier and object detector of the object detection model for detection and classification. This process is expressed by formulas (14)-(16):

[0118]

[0119] PE (pos,2i) =cos(pos / 1000) 2id (15)

[0120]

[0121] Among them, E i This represents the vector embedding after feature map decomposition; PE is the position vector embedding, where pos indicates the position of the vector embedding in the feature map, d represents the dimension of PE (same as adding word embedding and feature embedding), 2i indicates an even dimension, Attention is the self-attention mechanism module, and Q, K, and V represent the query, key, and value vectors, respectively. To prevent QK... T Excessive inner product caused misclassification in softmax, so we added... This hyperparameter.

[0122] Figure 2 In Figure (b), P0 represents the learnable cue word vector embedding added to the transformer input space after the word vectors are retrieved and initialized from the text library.

[0123] The small object detection method based on dynamic sample feature aggregation and cue word feature enhancement also includes optimizing the object detection model using CIOU loss function, DFL loss function, and CLS loss function. Specifically, the CIOU loss function is used for bounding box regression to optimize the localization accuracy of object detection; the DFL loss function is a loss function that optimizes bounding box prediction through discrete distribution; and the CLS loss function is used to optimize the accuracy of classification tasks.

[0124] First, the pre-divided dataset can be divided into training and test sets and input into the network;

[0125] The batch size is set to 4, the input size to 800×800, the learning rate to 0.01, the momentum to 0.985, and the weight decay to 0.0001. During training, the model loads pre-trained weights and uses the CIOU loss function. CIOUDFL (Distribution Focal Loss) loss function dfl and CLS loss (i.e., classification loss) function loss cls To optimize target detection accuracy, the loss function results are as follows: Figure 4 As shown, where, Figure 4 The lines connecting the blue dots represent the actual results, and the dashed lines formed by the red dots represent the smoothed results, expressed by formulas (17)-(19):

[0126]

[0127] loss dfl (S i ,S i+1 )=-((y i+1 -y)log(S i )+(yy i )log(S i+1 )), (18)

[0128]

[0129] Where y is the predicted location classification point, y i Here are the other predicted location classification points, IOU is the intersection-union ratio of the predicted bounding box and the ground truth bounding box, and b is the center coordinate of the predicted bounding box. gt The coordinates are the center coordinates of the ground truth bounding box. In formula (17), c represents the Euclidean distance between the two opposite vertices of the smallest rectangular box that can accommodate both the predicted and ground truth bounding boxes. a represents a weight parameter used to balance the effects of shape and position differences. M represents the number of categories. S i This represents the probability value of two adjacent discrete locations predicted by the model, used to model the continuous value distribution of the target bounding box.

[0130] w gt The width of the ground truth bounding box, h gt w represents the height of the ground truth bounding box, w represents the width of the predicted bounding box, and h represents the height of the predicted bounding box.

[0131] Precision, recall, and mAP are selected as metrics to evaluate the model, and are expressed by formulas (20)-(22) respectively:

[0132]

[0133] Where TP is the number of positive samples correctly identified, FP is the number of negative samples correctly identified as positive samples, FN is the number of negative samples correctly identified, and mAP is the average precision of the dataset (AP).j Let represent the average precision of the j-th class of samples, and C represent the total number of classes.

[0134] Figure 4 In this diagram, `train` represents the loss function used during model training; `val` represents the loss function used during model validation; `metric` represents the evaluation metric; the horizontal axis represents the number of training and validation rounds; and the vertical axes of the first three columns represent the corresponding loss functions, where `box_loss` corresponds to the CIOU loss function. CIOU The value of cls_loss corresponds to the CLS loss function loss. cls The value of dfl_loss corresponds to the DFL loss function loss. dfl The values ​​are shown, and the vertical axes of the two columns on the right refer to precision, recall, and mean precision, respectively.

[0135] The trained network model was applied to a remote sensing small target detection task to verify its detection performance in real remote sensing images. By comparing the model's predictions with the ground truth labeled targets, its performance in small target detection, localization accuracy, and recognition ability was evaluated.

[0136] This invention also provides a small target detection system based on dynamic sample feature aggregation and cue word feature enhancement, characterized in that it includes:

[0137] Memory, configured to store computer programs;

[0138] The processor is configured to execute the computer program to implement the small target detection method based on dynamic sample feature aggregation and cue word feature enhancement as described above.

[0139] The advantages of the present invention are illustrated below through specific embodiments. The dataset used in this embodiment is the DOTA-v2.0 dataset released by Wuhan University, consisting of 11,268 remote sensing images. The pixel size of each image ranges from 800×800 to 20,000×20,000. It contains 1,793,658 instances across 18 categories and abbreviations for these instances: plane (PL), ship (SH), storage tank (ST), baseball diamond (BD), tennis court (TC), basketball court (BC), and ground athletics field (T). The categories are: rack-field (GTF), harbor (Harbor), bridge (Bridge), large-vehicle (LV), small-vehicle (SV), helicopter (HC), roundabout (RA), soccer-ball-field (SBF), swimming-pool (SP), container-crane (CC), airport (AP), and helipad (HL). Table 1 shows the tags and the number of instances for each category in DOTA-v2.0.

[0140] Table 1 shows the 18 categories and corresponding instances (per instance) of the DOTAv2.0 dataset.

[0141]

[0142] The training environment was configured as follows: CUDA 11.7, PyTorch 2.0.0 deep learning framework, Intel Core i9-13900k CPU, 64GB of RAM, and an NVIDIA GeForce RTX 4090 GPU (batch size of 4) for model training and testing. The Stochastic Gradient Descent (SGD) optimizer was used for training with a learning rate of 0.01, momentum of 0.985, and weight decay of 0.0001.

[0143] To verify the effectiveness of the technical solution of this invention, firstly... Figure 5 The detection performance of this invention is demonstrated. Subsequently, on the DOTA-2.0 dataset, comparative experiments with other methods were conducted to evaluate the performance of different methods in small object detection tasks. The experimental results are shown in Table 2.

[0144] Table 2 shows the experimental results comparing the unbalanced single-class AP and all-class mAP50 scores in DOTA 2.0.

[0145] Experimental methods BD TC RA mAP50 SASM 40.62 78.58 46.57 44.53 <![CDATA[R 3 It]]> 50.95 78.52 53.2 47.26 S2A-Net 51.31 80.73 52.6 49.86 RRF 49.96 77.88 51.22 53.28 FCOS-O 47.53 78.61 52.38 48.51 This invention 71.63 94.56 59.36 60.85

[0146] In Table 2, the bolded results indicate the best performance for each column.

[0147] The experiment analyzed the mAP50 scores of all categories and the AP scores of imbalanced classes in the dataset, finding that the proposed model showed significant improvements in both metrics compared to other comparative models. In imbalanced classes, such as baseball-diamond (BD), tennis-court (TC), and roundabout (RA), the proposed model achieved the best accuracy in the comparative experiment, enabling more accurate classification and localization. The experimental results demonstrate that the proposed method can alleviate the problem of neglecting scarce sample features under imbalanced conditions in small object and imbalanced class detection tasks, allowing the model to better capture image features of scarce categories and pay more balanced attention to samples from each category during training.

[0148] In summary, the small target detection method based on dynamic sample feature aggregation and prompt word feature enhancement proposed in this invention aims to enhance the model's ability to locate and recognize small target features. In order to enhance the model's ability to locate and recognize small target features, this invention proposes a cascaded multi-scale attention mechanism. By integrating attention strategies of different dimensions, the feature extraction process is optimized to extract more refined small target features and achieve accurate location of small targets. In order to alleviate the class misclassification problem caused by sample imbalance, this invention proposes a dynamic sample feature aggregation module. The cosine similarity of the feature vectors of the original images in a batch after being extracted by the backbone network is calculated, and the normalized temperature cross-entropy contrast loss function is calculated to allow the backbone network to learn to distinguish the feature representations of sparse and majority samples, thereby improving the distribution aggregation degree of similar and dissimilar features. (3) In order to enable the model to pay more balanced attention to samples of each category, this invention proposes a prompt word feature enhancement module. By adding learnable text prompt word vectors to the input space, the text feature correlation is used to enhance sample features, thereby alleviating the class misclassification problem caused by sample imbalance.

[0149] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A small target detection method based on dynamic sample feature aggregation and cue word feature enhancement, characterized in that, include: Acquire images of the small remote sensing targets to be detected; Obtain the object detection model; The target detection model is used to detect small remote sensing targets in the image. in, The object detection model uses YOLOv8 as the base network and adds a cascaded multi-scale attention module to the YOLOv8 backbone network to optimize the backbone network. Cascaded multi-scale attention modules include sequentially connected spatial attention units and channel attention units; The object detection model also includes a dynamic sample aggregation module and a cue-based feature enhancement module. The features extracted by the optimized backbone network are input into the dynamic sample aggregation module and the cue-based feature enhancement module. The dynamic sample aggregation module is used to evaluate the clustering degree of sample feature distribution and output a loss function to optimize the backbone network through backpropagation. The cue-based feature enhancement module is used to increase the feature amount of rare categories and pass the processed features to the neck network of YOLOv8. The neck network performs multi-scale fusion of the enhanced features and finally generates object detection results through the detection head. The dynamic sample aggregation module consists of a linear layer, a batch normalization layer, a first convolutional layer, a normalization layer, and a second convolutional layer. The cue-based feature enhancement module includes a text library, in which each rare category corresponds to a spacing cue vector; the cue-based feature enhancement module also includes linear layers and Transformer layers; The spacing cue word vectors are transformed into cue word feature vectors through a linear layer; The optimized backbone network output features are embedded and input into the prompt word feature vector into the Transformer layer to generate output features of the prompt word-based feature enhancement module. The output features are then input into the classifier head and object detector of the object detection model for detection and classification.

2. The small target detection method based on dynamic sample feature aggregation and cue word feature enhancement according to claim 1, characterized in that, The YOLOv8 backbone network consists of, in sequence, a first convolutional module, a second convolutional module, a first C2f module, a third convolutional module, a second C2f module, a fourth convolutional module, a third C2f module, a fifth convolutional module, a fourth C2f module, and an SPPF module. The optimized backbone network has two cascaded multi-scale attention modules: the first cascaded multi-scale attention module is located between the second convolutional module and the first C2f module; the second cascaded multi-scale attention module is located between the third C2f module and the fifth convolutional module.

3. The small target detection method based on dynamic sample feature aggregation and cue word feature enhancement according to claim 1, characterized in that, The spatial attention unit consists of two 2D convolutional layers, two 1D convolutional layers, a pooling layer, another convolutional layer, and an activation function. The spatial attention unit performs the following operations: The large kernel convolution algorithm is used to perform depthwise convolution on the input feature map to obtain spatial feature maps with different kernel sizes; The spatial feature maps are connected by channels to obtain the first connection result; The first connection result is input into the pooling layer for average pooling and max pooling respectively, to obtain the average pooling result and the max pooling result; The average pooling result and the max pooling result are concatenated by channel to obtain the second concatenation result; The second connection result is input into the convolutional layer for convolution to obtain a spatial attention map containing N channels; Apply the Sigmoid activation function to the spatial attention map of each channel to obtain the spatial selection mask of each decomposed large kernel, and obtain the feature representation of the large kernel sequence. The feature representation of each large kernel sequence is weighted using the corresponding spatial selection mask; The weighted summation results corresponding to all large kernel sequences are fused through convolution to obtain the spatial attention feature, i.e., the output A of the spatial attention unit.

4. The small target detection method based on dynamic sample feature aggregation and cue word feature enhancement according to claim 3, characterized in that, The channel attention unit consists of a two-dimensional convolutional layer, a global average pooling layer, a one-dimensional convolutional layer, a sigmoid activation function, and an SR layer, in sequence. The channel attention unit performs the following operations: Obtain the output A of the spatial attention unit; The output A of the spatial attention unit is passed through a two-dimensional convolutional layer and then through a global average pooling layer to obtain the channel aggregation feature. Channel aggregation features are processed by a one-dimensional convolutional layer to generate channel weights. The channel weights are applied to the Sigmoid activation function to generate feature weights; The feature weights and channel aggregated features are weighted and summed along the channel dimension. The weighted summation result is input into the SR layer to obtain the channel attention features, which are the outputs of the channel attention units.

5. The small target detection method based on dynamic sample feature aggregation and cue word feature enhancement according to claim 1, characterized in that, The method also includes optimizing the target detection model using the CIOU loss function, DFL loss function, and CLS loss function.

6. A small target detection system based on dynamic sample feature aggregation and cue word feature enhancement, characterized in that, include: Memory, configured to store computer programs; The processor is configured to execute the computer program to implement the small target detection method based on dynamic sample feature aggregation and cue word feature enhancement as described in any one of claims 1 to 5.

Images