A method for detecting rotating small targets in remote sensing images by incorporating cognitive features

Abstract

This invention belongs to the field of image processing technology, and specifically relates to a method for detecting rotating small targets in remote sensing images by incorporating cognitive features. It includes: inputting a remote sensing image into a pre-established and trained target detection model, detecting the positional information of all small targets in the image, and predicting the category of the small targets, thereby achieving small target detection; the target detection model uses a single-stage rotating target detection network S. 2 A-Net, as a basic target detection framework, enhances the representation ability of target topological characteristics by introducing a physical cognitive feature extraction module and a cognitive feature and deep feature fusion module. This improves the detection accuracy of weak targets in complex scenes with few samples under low-resolution remote sensing observation conditions, while reducing the false alarm rate. The method of this invention improves the representation ability of target topological characteristics, increases the detection accuracy of weak targets in complex scenes with few samples under low-resolution remote sensing observation conditions, and reduces the false alarm rate.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, and specifically relates to a method for detecting rotating small targets in remote sensing images by fusing cognitive features. Background Technology

[0002] Object detection and classification aims to accurately locate target objects in a given scene image or video, label the target's category, and accurately describe the target's location and category information. Currently, scholars both domestically and internationally have conducted extensive research on object detection and classification methods for remote sensing images. Based on different feature extraction methods, these methods can be mainly divided into two categories: traditional methods based on manually designed features and methods based on deep learning.

[0003] (1) Traditional methods based on the characteristics of manual design

[0004] Traditional object detection and classification methods primarily rely on manually designed feature extractors and classifiers to achieve object detection tasks. Commonly used manually designed features include gray-level statistical features (such as histograms, HOG, etc.), geometric structural features (such as SIFT, SURF, etc.), transform features (FFT, Hough transform, wavelet transform, etc.), and algebraic features (such as PCA, ICA, etc.). For example, Sun Qiao et al. proposed a ship detection and classification method based on histogram of oriented gradients (HOG) and support vector machine (SVM); Ji Xiaofei et al. proposed a multi-object detection and classification algorithm based on scale-invariant features (SIFT) and support vector machine (SVM). However, object detection and classification methods based on manually designed features are greatly affected by feature selection, and their feature representation ability, generalization ability, and robustness are relatively weak, indicating that the performance of object detection and classification needs to be improved.

[0005] (2) Deep learning-based methods

[0006] The application of deep learning technology in object detection mainly benefits from its powerful feature extraction capabilities and end-to-end training method. Compared with traditional object detection methods, deep learning-based object detection methods have certain advantages in terms of accuracy and robustness. In recent years, deep learning-based object detection technology has continued to develop, from early object detection methods based on convolutional neural networks (CNNs), including the RCNN series of two-stage methods and single-stage methods such as YOLO and SSD, to the recently emerging object detection methods based on Transformers.

[0007] Transformer-based object detection methods primarily rely on global attention mechanisms to model image features, offering advantages in global feature modeling. However, they face significant challenges in local feature modeling, computational resource requirements, and training data demands, leading to poor performance on targets with relatively small scales. Therefore, current low-resolution remote sensing image small target detection tasks mainly employ convolutional neural network (CNN)-based methods, with improvements and innovations tailored to specific scenes, targets, and image quality. While CNN-based object detection methods excel in handling local features and hardware requirements, they are relatively weak in global feature modeling. CNN feature extraction, through operations such as convolution and pooling, can only focus on limited information within adjacent pixel regions, failing to capture global topological structure information of the target and lacking the "structural understanding" ability of the human visual system, thus affecting the accuracy of the detection model. Furthermore, current deep learning object detection methods based on CNNs also suffer from a lack of interpretability and a high dependence on labeled samples, highlighting the limitations of deep learning methods in terms of general applicability.

[0008] In summary, current remote sensing image target detection methods based on convolutional neural networks suffer from problems such as weak ability to extract global topological features of targets, lack of interpretability, and high dependence on labeled samples. These issues make it difficult to meet the high-precision detection and classification requirements for small targets in complex scenes with few samples under low-resolution remote sensing observation conditions. Summary of the Invention

[0009] The purpose of this invention is to overcome the shortcomings of the existing technology and provide a remote sensing image target detection method that integrates cognitive features, so as to improve the value of existing low-resolution remote sensing systems for the reconnaissance and monitoring of small, densely arranged, rotating targets in complex scenes with few samples.

[0010] To achieve the above objectives, this invention proposes a method for detecting rotating small targets in remote sensing images by fusing cognitive features, comprising:

[0011] The remote sensing image is input into a pre-established and trained target detection model to detect the location information of all small targets in the image and predict the category of the small targets, thus achieving small target detection;

[0012] The target detection model uses a single-stage rotating target detection network S. 2 A-Net, as a basic target detection framework, enhances the ability to represent the topological characteristics of targets by introducing a physical cognitive feature extraction module and a cognitive feature and deep feature fusion module. This improves the detection accuracy of weak targets in complex scenes with few samples under low-resolution remote sensing observation conditions and reduces the false alarm rate.

[0013] Preferably, the S 2A-Net comprises a backbone network, a feature pyramid network (FPN), a feature alignment module (FAM), and a direction detection module (ODM), all connected in sequence.

[0014] The backbone network is used to extract basic convolutional features of the image;

[0015] The Feature Pyramid Network (FPN) is used to extract multi-scale features;

[0016] The Feature Alignment Module (FAM) is used to extract alignment features that are aligned with the high-quality rotating anchor frame.

[0017] The directional detection module ODM is used to generate the final target detection result.

[0018] Preferably, the backbone network adopts a ResNet network to extract basic features from the input remote sensing image I and generate basic image feature maps {C3, C4, C5}.

[0019] The Feature Pyramid Network (FPN) performs multi-scale feature extraction on the basic feature maps {C3, C4, C5} of the image to generate multi-scale feature maps {P3, P4, P5, P6, P7}.

[0020] Preferably, the Feature Alignment (FAM) module includes an Anchor Box Optimization Network (ARN) and an Alignment Convolutional Layer (ACL). The ARN generates high-quality rotated anchor boxes, and the ACL adaptively aligns the convolutional features with the high-quality rotated anchor boxes.

[0021] The anchor box optimization network performs convolution operations on the multi-scale feature maps {P3, P4, P5, P6, P7} of the image respectively, and outputs the classification and regression results for each feature map:

[0022] cls_pre i =Conv(Conv(Conv(Pi)))i=3,4,...,7

[0023] reg_pre i =Conv(Conv(Conv(Pi)))i=3,4,...,7

[0024] In the formula, cls_pre i and reg_pre i These are the anchor box classification information and anchor box position information output by the classification branch of the ARN network for the i-th scale feature map, respectively;

[0025] The aligned convolutional layer is implemented using aligned convolutional kernels, with multi-scale feature maps as input.

[0026] The rotational anchor frame position information reg_pre for {P3, P4, P5, P6, P7} and feature maps at each scale. i For i = 3, 4, ..., 7, the output is the rotated convolutional feature, i.e., the aligned feature Fd. i :

[0027] Fd i =AlignConv(Pi,reg_pre) i ), i = 3, 4, ..., 7

[0028] Here, AlignConv(·) represents the alignment convolution kernel.

[0029] Preferably, the input to the physical cognition feature extraction module is the anchor box position information reg_pre output by the classification branch of the ARN network. i Using remote sensing image I, anchor frame cognitive features are extracted to obtain high-quality physical cognitive features Fr of the rotated anchor frame. i , as input to the cognitive feature and deep feature fusion module;

[0030] The input to the cognitive feature and deep feature fusion module is the physical cognitive feature Fr. i Alignment feature Fd output by the feature alignment module FAM i The fusion process is performed to obtain the fusion feature F. i Input the orientation detection module ODM.

[0031] Preferably, the processing procedure of the physical cognitive feature extraction module includes:

[0032] Step S1) Use the FAST algorithm to perform point feature detection and obtain a set of feature points;

[0033] Step S2) For the feature point set, perform point topological structure feature description to obtain the target global description vector; including:

[0034] Step S2-1) Construct a complex network and use an autonomous dynamic evolution mechanism that combines edges and points for dynamic evolution;

[0035] Step S2-2) Perform local MS-PIIFD description on the evolution network graphs obtained at each time step during the dynamic evolution process of the complex network;

[0036] Steps S2-3) employ the Bag-of-Five (BOF) method to obtain a domain visual dictionary through feature clustering;

[0037] Step S2-4) Utilize the dictionary and network locality centrality to perform weighted projection processing on the local MS-PIIFD descriptor vectors to generate the global MS-PIIFD descriptor.

[0038] Preferably, the processing procedure of the cognitive feature and deep feature fusion module includes:

[0039] The target's physical cognitive features are mapped to a deep feature space through an embedding layer;

[0040] Physical cognitive characteristics Fr i After embedding layer mapping, it is aligned with the feature Fd i By fusing the data using a multilayer perceptron, a fusion feature F based on cognitive perception is obtained. i :

[0041] F i =MLP(f θ (Fr i ),Fd i )

[0042] Among them, f θ (·) represents the embedding layer mapping, and MLP(·) represents the multilayer perceptron.

[0043] Preferably, the processing procedure of the orientation detection module ODM includes:

[0044] The input fused feature F is obtained by using an active rotating filter (ARF). i Encode directional information to generate directional-sensitive feature Z i ;

[0045] The orientation-invariant feature Z is obtained by merging orientation-sensitive features. i ′;

[0046] The direction-sensitive feature Z i and direction-invariant feature Z i '', respectively input into the bounding box regression branch and the bounding box classification branch, perform three convolutions to obtain the bounding box regression result Reg i and classification results Cls i .

[0047] Preferably, the target detection model further includes a post-processing module, the processing of which includes:

[0048] Regression results of bounding boxes at 5 scale feature layers Reg i and classification results Cls i Decoding is performed, and bounding boxes with category confidence scores higher than the set threshold are selected. Non-maximum suppression is then applied to the bounding boxes, and the detection boxes with the highest confidence scores are retained to obtain category information.

[0049] Preferably, the method further includes a training step for the object detection model, comprising:

[0050] A remote sensing image land vehicle target detection dataset can be constructed by selecting data corresponding to small-sized vehicle targets from the existing publicly available DOTA remote sensing image target detection dataset; or by using satellite and airborne data obtained from Google Earth as the basic data source, image data preparation, target annotation and data augmentation can be carried out to construct a remote sensing image land vehicle target detection dataset.

[0051] A portion of the remote sensing image land vehicle target detection dataset was selected as the training set, and the other portion as the validation set.

[0052] Set the initialization and training parameters for the object detection model;

[0053] The training set data is sequentially input into the object detection model, the training loss function is calculated using the dynamic soft label assignment method, and the model is trained using the stochastic gradient descent (SGD) optimizer until the training requirements are met, resulting in a well-trained object detection model. The model is then validated using the mAP evaluation metric on the validation set.

[0054] Compared with the prior art, the advantages of the present invention are:

[0055] 1. This invention proposes a method for characterizing the topological structure features of target points in complex scenes of remote sensing images by combining dynamically evolving complex networks and MS-PIIFD. This method enhances the ability to capture the global topological structure characteristics of targets, thereby solving the problems of deep convolution features, such as difficulty in obtaining global topological structure information, lack of interpretability, and high dependence on labeled samples. This method supplements target detection with efficient physical cognition features.

[0056] 2. This invention is in S 2 The A-Net rotating target detection and classification framework improves the ability to represent the topological characteristics of targets by introducing target physical cognitive features and fusing them with deep convolutional learning features. This enhances the detection accuracy of weak targets in complex scenes with few samples under low-resolution remote sensing observation conditions, while reducing the false alarm rate. Attached Figure Description

[0057] Figure 1 This is a flowchart illustrating the remote sensing image rotation small target detection method that integrates cognitive features according to the present invention.

[0058] Figure 2 This is a structural diagram of the remote sensing image rotating small target detection network model that integrates cognitive features according to the present invention;

[0059] Figure 3 This is a structural diagram of the S2A-Net rotating target detection network model;

[0060] Figure 4 This is a structural diagram of the ResNet basic feature extraction network model;

[0061] Figure 5 This is a structural diagram of the FPN feature pyramid network model;

[0062] Figure 6 This is a structural diagram of the FAM feature alignment network model;

[0063] Figure 7 This is a structural diagram of the ODM-oriented detection network model;

[0064] Figure 8 It is a path diagram for a method to characterize the topological features of target points in complex scenes of remote sensing images by combining dynamically evolving complex networks and MS-PIIFD.

[0065] Figure 9 This is a structural diagram of a model that integrates deep learning features and physical cognitive features;

[0066] Figure 10 shows the results of the comparative experiment 1: reducing the false alarm rate; where Figure 10(a) is the baseline algorithm and Figure 10(b) is the improved algorithm of this invention;

[0067] Figure 11 shows the results of comparative experiment 2: improved accuracy, where Figure 11(a) is the baseline algorithm and Figure 11(b) is the improved algorithm of this invention. Detailed Implementation

[0068] This invention discloses a method for detecting small rotating targets in remote sensing images by fusing cognitive features. Based on a self-built target topology-cognitive feature, it achieves this by... 2 The A-Net rotating target detection framework improves the detection and classification performance of small targets in complex scenes with few samples under low-resolution remote sensing observation conditions by fusing target cognitive features and deep convolutional learning features.

[0069] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0070] like Figure 1 As shown, Embodiment 1 of the present invention proposes a method for detecting rotating small targets in remote sensing images by fusing cognitive features. Specifically, it includes:

[0071] The system receives the remote sensing image to be detected, inputs a pre-established and trained remote sensing image rotation small target detection network model that integrates cognitive features, and obtains the target detection result, namely the target bounding box and category information.

[0072] A remote sensing image rotation small target detection network model that integrates cognitive features is used to predict the location and category information of the target by performing basic convolutional feature extraction, FPN multi-scale feature extraction, deep feature alignment, physical cognitive feature extraction, fusion of cognitive and deep features, and directional detection processing on the remote sensing image to be detected.

[0073] This embodiment includes four steps: establishing a remote sensing image rotation small target detection model that integrates cognitive features, constructing a sample dataset, model training, and model testing and verification.

[0074] Step 1: Model building;

[0075] Model structure as follows Figure 2 As shown. In this invention, the aim is to detect the location information (x) of all targets in the input remote sensing image I. i ,y i ,w i ,h i ,θ i (i.e., rotate the bounding box) and predict the target category information (cls) i ).

[0076] The target detection model of this invention uses a single-stage rotating target detection network S 2 A-Net as a basic object detection framework (such as...) Figure 2 (As shown in the light green box), it introduces two modules: physical cognitive feature extraction and cognitive feature and deep feature fusion (as shown in the light green box). Figure 2 (As shown in the red box in the middle). S 2 A-Net consists of a backbone network, a feature pyramid network (FPN), a feature alignment module (FAM), and an oriented detection module (ODM). The target detection model of this invention first uses the backbone network to extract basic convolutional features of the image; second, it uses the feature pyramid network to extract features at multiple scales; then, at each scale feature layer of the pyramid, it performs depth feature alignment, physical cognitive feature extraction, fusion of cognitive and depth features, and oriented detection processing to obtain the detection results at each scale feature layer; finally, it performs post-processing such as non-maximum suppression on the detection results at each scale feature layer to obtain the final target detection result.

[0077] The method specifically includes:

[0078] S1, S 2 A-Net serves as the foundational object detection framework. It employs a high-performance single-stage rotating object detection network—S... 2 A-Net is used as the baseline network framework for object detection and classification. 2A-Net, building upon the RetinaNet object detection network framework, proposes a FAM (Feature Alignment) module and an ODM (Orientational Detection) module to alleviate the inconsistency between classification scores and localization accuracy in previous single-stage detectors. The FAM module generates high-quality rotated anchor boxes using an Anchor Refinement Network (ARN) and adaptively aligns convolutional features with these high-quality rotated anchor boxes using an Alignment Convolution Layer (ACL). The ODM module first encodes orientation information using Active Rotating Filters (ARF) to generate orientation-sensitive features, then merges these features to extract orientation-invariant features. Finally, these features are fed back into the classification and regression sub-networks to improve the consistency between classification scores and localization accuracy. Specifically, it includes:

[0079] The structure of the S2A-Net rotating target detection network model is as follows: Figure 3 As shown, the network model consists of four parts: a backbone network, an FPN feature pyramid network, a FAM feature alignment module, and an ODM orientation detection module. FAM and ODM form the detection head, used to process each scale layer of the feature pyramid. First, the backbone network extracts basic convolutional features of the image; then, the FPN feature pyramid network extracts multi-scale features; next, the FAM feature alignment module extracts alignment features aligned with high-quality rotated anchor boxes; finally, the ODM orientation detection module generates the final target detection result.

[0080] 1.1 Basic Convolution Feature Extraction.

[0081] like Figure 4 As shown, the input remote sensing image I undergoes basic convolutional feature extraction to generate basic image feature maps {C3, C4, C5}; specifically including:

[0082] A ResNet network is used as the basic backbone network model for basic convolutional feature extraction. ResNet is a residual network that constructs deep networks by designing and stacking residual modules (units). The residual modules in ResNet, through identity mapping, help the network propagate gradients, avoiding the gradient vanishing problem that occurs when the network is too deep. This allows for training deeper network models, resulting in stronger expression of deep features, smaller model size, and higher training accuracy and efficiency. Therefore, this invention uses a ResNet network to extract basic features from the input remote sensing image I, generating basic image feature maps {C3, C4, C5}.

[0083] C1=MaxPool(ReLU(BN(Conv(I))))

[0084] C2 = Conv2_x(C1)

[0085] C3 = Conv3_x(C2)

[0086] C4 = Conv4_x(C3)

[0087] C5 = Conv5_x(C4)

[0088] Where Conv(·) represents the convolution operation, BN(·) represents the Batch Normalization operation, ReLU(·) represents the ReLU activation function, and MaxPoo(l·) represents the max pooling operation; Conv2_x(·), Conv3_x(·), Conv4_x(·), and Conv5_x(·) represent the residual structures of layers 1, 2, 3, and 4 of ResNet50, respectively.

[0089] 1.2 Multi-scale feature extraction.

[0090] like Figure 5 As shown, multi-scale feature extraction is performed on the basic convolutional features of the image to generate multi-scale feature maps {P3, P4, P5, P6, P7}; specifically including:

[0091] A Feature Pyramid Network (FPN) is employed for multi-scale feature extraction. The feature construction process of the FPN consists of three main stages: bottom-up, top-down, and intra-layer connections. In this invention, the bottom-up stage of the FPN is implemented using a ResNet basic convolutional feature extraction network, specifically as described in Section 1.1, resulting in three scales of basic feature maps {C3, C4, C5}. The top-down process constructs new feature maps by scaling the upper-layer feature maps to maintain a consistent scale with the lower-layer feature maps, thus achieving feature map fusion. Finally, the new feature map is added element-wise to each corresponding element of the original lower-layer feature map, thereby fusing the upper and lower-layer features. Next, each fused feature map is output as a new feature map with a depth of 256. A 3×3 convolution is applied to the fused feature map for smoothing to eliminate the possibility of insufficient fusion caused by direct addition. This invention employs an FPN (Feature Pyramid Network) to extract multi-scale features from the basic convolutional features of an image, generating multi-scale feature maps {P3, P4, P5, P6, P7}.

[0092] P7 = Conv(P6)

[0093] P6 = Conv(C5)

[0094] P5 = Conv(Conv(C5))

[0095]

[0096]

[0097] Where Conv3(·) represents a 3×3 convolution operation, Conv1(·) represents a 1×1 convolution operation, and UpSample(·) represents an upsampling operation. This indicates an element-wise addition operation.

[0098] 1.3 Feature Alignment.

[0099] like Figure 6 As shown, for the image features of each scale layer of the pyramid, feature alignment is performed based on the generated rotated anchor boxes to generate image alignment features; specifically including:

[0100] The FAM feature alignment module consists of an anchor box optimization network and an alignment convolutional layer. The anchor box optimization network (ARN) generates high-quality rotated anchor boxes, and the alignment convolutional layer (ACL) adaptively aligns the convolutional features with the high-quality rotated anchor boxes.

[0101] 1.3.1 Anchor Box Optimization Network (ARN): This network consists of two parallel branches: an anchor box classification branch and an anchor box regression branch. The anchor box classification branch predicts the category of the anchor box, while the anchor box regression branch optimizes the horizontal anchor boxes to generate high-quality rotated anchor boxes. Both branches are implemented using three 3×3 convolutions. The input is the multi-scale feature maps {P3, P4, P5, P6, P7} generated by the FPN. Convolution operations are performed on each feature map, and the output is the classification and regression results for each feature map, as shown in the following equation:

[0102] cls_pre i =Conv(Conv(Conv(Pi)))i=3,4,…,7

[0103] reg_pre i =Conv(Conv(Conv(Pi)))i=3,4,...,7

[0104] Among them, cls_pre i and reg_pre i These are the anchor box classification information and anchor box position information output by the classification branch of the ARN network for the i-th scale feature map, respectively.

[0105] (2) Aligned Convolutional Layer (ACL): This layer is implemented using AlignConv. The input is multi-scale feature maps {P3, P4, P5, P6, P7} and the rotation anchor box position information reg_pre corresponding to each scale feature map. iFor i = 3, 4, ..., 7, the output is the rotated convolutional feature Fd. i This refers to alignment features.

[0106] Fd i =AlignConv(Pi,reg_pre) i ), i = 3, 4, ..., 7

[0107] Here, AlignConv(·) represents the alignment convolution kernel.

[0108] 1.4 Targeted detection.

[0109] like Figure 7 As shown, the alignment features Fd for each scale layer of the input are... i Targeted detection is performed separately, and the final detection and classification results for all anchor frames are output. i ,Cls i Specifically, it includes:

[0110] The ODM orientation detection module first encodes orientation information to generate orientation-sensitive features; then these features are merged to extract orientation-invariant features; finally, these features are fed back into the classification and regression sub-networks to improve the consistency between classification scores and localization accuracy.

[0111] 1.4.1 An active rotation filter (ARF) is used to encode the orientation information of the input alignment feature Fd to generate orientation-sensitive features.

[0112] An ARF (Adaptive Field Redirection Filter) is a filter of size k×k×N that actively rotates N-1 times during feature convolution with the input to generate orientation-sensitive features with N directional channels. For a feature map Fd and an actively rotated filter F, the i-th directional channel of the output orientation-sensitive feature Z can be represented as:

[0113]

[0114] in, F rotates clockwise by θ i version and Fd (n) They are And the nth directional channel of Fd.

[0115] 1.4.2 The orientation-invariant feature Z′ is obtained by merging orientation-sensitive features. This can be achieved simply by selecting the orientation channel feature with the strongest response as the output feature Z′, as shown in the following equation:

[0116] Z′=max Z (n) ,0<n<N-1

[0117] 1.4.3 The orientation-sensitive feature Z and the orientation-invariant feature Z′ are fed into the bounding box regression branch and the bounding box classification branch, respectively, and three convolutions are performed to obtain the bounding box regression result Reg and the classification result Cls.

[0118] Reg = Conv(Conv(Conv(Z)))

[0119] Cls = Conv(Conv(Conv(Z′)))

[0120] 1.5 Post-processing for target detection. For the above S... 2 The output results of the detection head corresponding to the 5 scale feature layers of A-Net are shown in Reg. i and Cls i Decoding is performed to obtain the location and category information of all bounding boxes. Bounding boxes with category confidence scores higher than a set threshold are selected. Then, non-maximum suppression is applied to the bounding boxes to remove redundant bounding boxes, i.e., retaining the detection boxes with the highest confidence scores while suppressing those that overlap with them and have lower confidence scores.

[0121] S2, physical cognitive feature extraction. For example... Figure 2 The remote sensing image rotation small target detection model shown integrates cognitive features and uses the high-quality rotated anchor box position information reg_pre output by the anchor box optimization network (ARN). i Anchor frame cognitive features are extracted from the input image to obtain high-quality rotated anchor frame cognitive features Fr. i It serves as the input for the cognitive feature and deep feature fusion module.

[0122] Convolutional neural network feature extraction, through operations such as convolution and pooling, can only focus on information in a limited region between adjacent pixels, making it difficult to obtain global topological structure representation information of the target. It lacks the "structural understanding" ability of the human visual system, affecting the accuracy of the detection model. Especially in low-resolution remote sensing images, there are problems such as complex background environments, small target scales and weak features, and target scale diversity due to image resolution variations. Deep convolutional learning methods alone are insufficient to obtain enough target representation information. Therefore, this invention constructs a target physical cognition feature extraction branch. Building upon the aforementioned basic deep convolutional feature extraction, i.e., multi-scale feature extraction, this branch enhances the ability to represent the topological structure characteristics of the target, thereby improving the detection performance of small targets.

[0123] This invention proposes a method for characterizing the topological structure of target points in complex scenes of remote sensing images by combining dynamically evolving complex networks and MS-PIIFD (Multi-Scale PartialIntensity Invariant Feature Descriptor), for extracting physical cognitive features of targets. First, the FAST method is used for point feature detection, balancing computational complexity, timeliness, and positioning accuracy. Second, the proposed point topological structure feature description method combining dynamically evolving complex networks and MS-PIIFD is used to describe the detected feature point set, generating a global MS-PIIFD descriptor, ultimately achieving high-precision capture of the global topological structure characteristics of weak targets. Specifically, it includes:

[0124] 2.1 Point feature detection.

[0125] The FAST algorithm is used for point feature detection to obtain a feature point set. Compared with algorithms such as Harris, SIFT, and SURF, the FAST point feature detection algorithm has the advantages of low computational cost and high speed, and also shows good performance in terms of background interference resistance, corner point richness, and localization accuracy. SIFT and SURF algorithms, because they process in multi-scale space, result in significant errors in corner point localization in small-scale images; the Harris algorithm detects relatively sparse corner points.

[0126] 2.2 Description of point topology features.

[0127] This invention proposes a point topology feature description method combining dynamically evolving complex networks and MS-PIIFD. The specific method path is as follows: Figure 8 For the aforementioned set of detected feature points, a point topology feature description is performed to obtain the global target description vector. First, a complex network is constructed and dynamically evolved using an autonomous dynamic evolution mechanism combining edges and points. Second, local MS-PIIFD descriptions are performed on the evolved network graphs obtained at each time step during the dynamic evolution process of the complex network. MS-PIIFD exhibits optimal robustness to differences in nonlinear intensity, rotational distortion, and scale changes, and effectively addresses the gradient inversion problem caused by changes in target coating. Then, the Bag-of-Five (BOF) method is used to obtain a domain visual dictionary through feature clustering. Finally, using the dictionary and network local degree centrality, the local MS-PIIFD description vectors are weighted and projected to generate a global MS-PIIFD descriptor. Using degree centrality to weight the features of the evolved network graph effectively removes background noise and focuses on important feature point information.

[0128] S3, fusion of cognitive features and deep features. For example... Figure 2The remote sensing image rotation small target detection model shown uses the cognitive features Fr output by the physical cognitive feature extraction module. i Alignment feature Fd output by the feature alignment FAM module i The fusion process is performed to obtain the fusion feature F. i Input to the ODM orientation detection module.

[0129] Compared with deep learning features, the target point topology-cognitive features constructed in this invention can capture topological information that is difficult for deep learning features to represent, while having the advantages of strong interpretability and independence from labeled samples. Therefore, it is necessary to design a physical cognitive feature and deep learning feature fusion module to give full play to their respective advantages, achieve complementary advantages, improve the ability to express target characteristics, and thus improve target detection performance.

[0130] This invention introduces a model for fusing physical cognitive features and deep learning features, the specific structure of which is as follows: Figure 9 As shown, the model first maps the physical cognitive features of the target to a deep feature space through an embedding layer; secondly, it fuses the physical cognitive features and deep learning features through a multilayer perceptron (MLP) to obtain a fused feature representation based on cognitive perception, which makes up for the shortcomings of deep features and obtains more comprehensive information about the target.

[0131] F = MLP(f θ (Fr),Fd)

[0132] Among them, f θ (·) indicates an embedding layer mapping.

[0133] The second step is to construct the sample dataset (including training and validation datasets); there are different methods for this.

[0134] In one embodiment: The present invention selects data corresponding to small-sized vehicle targets from the existing publicly available DOTA remote sensing image target detection dataset to construct a remote sensing image land vehicle target detection dataset, which serves as the training sample dataset for the remote sensing image rotating small target detection network model that integrates cognitive features proposed in this invention.

[0135] In another embodiment: The present invention constructs a remote sensing image land vehicle target detection dataset, using satellite and airborne data obtained from Google Earth as the basic data source, and carries out image data preparation, target annotation and data augmentation.

[0136] 1) First, using satellite and airborne data obtained from Google Earth as the basic data source, select scene areas where specified land vehicle targets are parked, download the original resolution data corresponding to the sensors, and prepare a basic image dataset.

[0137] 2) Secondly, the prepared image dataset was labeled using Labelimg software, with the targets labeled using rotated bounding boxes;

[0138] 3) Finally, the number of samples in the dataset is increased through simulation augmentation. During the simulation augmentation process, techniques such as flipping, angle transformation, noise addition, changes in lighting intensity, and background transformation are used to greatly enrich the diversity of the samples and improve the generalization and robustness of the trained network model.

[0139] Following the above processing, this embodiment ultimately achieved the generation of a dataset of 44,450 target instances for four types of land vehicles.

[0140] Based on the remote sensing dataset constructed above, 60% was used as the training set for model training. The remaining 40% of the data was used as the validation set for model validation.

[0141] Step 3: Model training;

[0142] S1, Initialize the remote sensing image rotation small target detection network model that integrates cognitive features, including setting the initialization parameters and training parameters of the network model, and selecting the optimizer.

[0143] S2, Training the model: Following the initialization of the network model described above, train the entire network model and finally select the optimal network model.

[0144] In this embodiment, a random initialization method is used to initialize the weights of all convolutional layers in the entire detection network model;

[0145] The training parameters for the detection network model that need to be set in this embodiment mainly include: specifying the paths of the training dataset and the validation dataset, the batch size B, the training iteration period T, the learning rate strategy, and the optimizer strategy. In this embodiment, the batch size B is set to 32; the training iteration period T is set to 36; the learning rate strategy includes using a stepped learning rate adjustment strategy, adopting a linear warm-up strategy, warming up for 500 iterations, setting the initial warm-up learning rate to 1 / 3 of the final learning rate, and multiplying the learning rate by 0.1 in the 24th and 33rd iterations to reduce the learning rate; the optimizer strategy includes using a stochastic gradient descent (SGD) optimizer, setting the initial learning rate to 0.0025, a momentum factor of 0.9 to accelerate convergence, a weight decay coefficient of 0.0001, and using the L2 norm for gradient clipping with a maximum norm of 35.

[0146] This invention designs a loss function to iteratively optimize the proposed remote sensing image rotation small target detection network model that integrates cognitive features. Addressing the problem of low detection performance in current target detectors due to unreasonable positive and negative sample allocation using a fixed IoU threshold for anchor boxes, this invention employs a Dynamic Soft Label Allocation (DSLA) method. This method dynamically sets anchor box quality evaluation scores during model training, allowing the model itself to participate in the label allocation process. Based on this quality score, soft labels (instead of 0 or 1) are adaptively assigned to anchor boxes, improving the training loss function and thus enhancing detection performance.

[0147] Step 4: Model testing and verification;

[0148] For the remote sensing image rotation small target detection network model fused with cognitive features trained as described above, its performance was tested and verified using the validation set from the self-built detection dataset. Each set of validation data consists of one input image and its corresponding target annotation file. By inputting the input image from the validation set into the trained model, the output target detection result can be predicted.

[0149] The current evaluation metric for deep learning-based remote sensing image target detection algorithms is mAP. Therefore, this test verifies that mAP is used to evaluate the accuracy of a remote sensing image rotating small target detection network model that incorporates cognitive features.

[0150] Figure 10 shows the results of the comparative experiment 1: reducing the false alarm rate; where Figure 10(a) is the baseline algorithm and Figure 10(b) is the improved algorithm of this invention;

[0151] Figure 11 shows the results of comparative experiment 2: improved accuracy, where Figure 11(a) is the baseline algorithm and Figure 11(b) is the improved algorithm of this invention.

[0152] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to the embodiments, those skilled in the art should understand that modifications or equivalent substitutions to the technical solutions of the present invention do not depart from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for detecting rotating small targets in remote sensing images by incorporating cognitive features, comprising: The remote sensing image is input into a pre-established and trained target detection model to detect the location information of all small targets in the image and predict the category of the small targets, thus achieving small target detection; The target detection model uses a single-stage rotating target detection network S. 2 A-Net, as a basic target detection framework, enhances the ability to represent the topological characteristics of targets by introducing a physical cognitive feature extraction module and a cognitive feature and deep feature fusion module. This improves the detection accuracy of weak targets in complex scenes with few samples under low-resolution remote sensing observation conditions and reduces the false alarm rate. The input to the physical cognition feature extraction module is the anchor box position information reg_pre output by the classification branch of the ARN network. i Using remote sensing image I, anchor frame cognitive features are extracted to obtain high-quality physical cognitive features Fr of the rotated anchor frame. i , as input to the cognitive feature and deep feature fusion module; The input to the cognitive feature and deep feature fusion module is the physical cognitive feature Fr. i Alignment feature Fd output by the feature alignment module FAM i The fusion process is performed to obtain the fusion feature F. i Input the orientation detection module ODM; The processing steps of the physical cognitive feature extraction module include: Step S1) Use the FAST algorithm to perform point feature detection and obtain a set of feature points; Step S2) For the feature point set, perform point topological structure feature description to obtain the target global description vector; including: Step S2-1) Construct a complex network and use an autonomous dynamic evolution mechanism that combines edges and points for dynamic evolution; Step S2-2) Perform local MS-PIIFD description on the evolution network graphs obtained at each time step during the dynamic evolution process of the complex network; Steps S2-3) employ the Bag-of-Five (BOF) method to obtain a domain visual dictionary through feature clustering; Step S2-4) Utilize the dictionary and network locality centrality to perform weighted projection processing on the local MS-PIIFD descriptor vectors to generate the global MS-PIIFD descriptor; The processing steps of the cognitive feature and deep feature fusion module include: Target physical cognitive characteristics Fr i Mapped to a deep feature space through an embedding layer; Physical cognitive characteristics Fr i After embedding layer mapping, it is aligned with the feature Fd i By fusing the data using a multilayer perceptron, a fusion feature F based on cognitive perception is obtained. i : F i =MLP(f θ (Fr i ),Fd i ) Among them, f θ (·) represents the embedding layer mapping, and MLP(·) represents the multilayer perceptron.

2. The remote sensing image rotation small target detection method based on fused cognitive features according to claim 1, characterized in that, The S 2 A-Net comprises a backbone network, a feature pyramid network (FPN), a feature alignment module (FAM), and a direction detection module (ODM), all connected in sequence. The backbone network is used to extract basic convolutional features of the image; The Feature Pyramid Network (FPN) is used to extract multi-scale features; The Feature Alignment Module (FAM) is used to extract alignment features that are aligned with the high-quality rotating anchor frame. The directional detection module ODM is used to generate the final target detection result.

3. The remote sensing image rotation small target detection method based on fused cognitive features according to claim 2, characterized in that, The backbone network adopts a ResNet network to extract basic features from the input remote sensing image I and generate basic feature maps {C3, C4, C5}. The Feature Pyramid Network (FPN) performs multi-scale feature extraction on the basic feature maps {C3, C4, C5} of the image to generate multi-scale feature maps {P3, P4, P5, P6, P7}.

4. The remote sensing image rotation small target detection method based on fused cognitive features according to claim 3, characterized in that, The Feature Alignment (FAM) module includes an Anchor Box Optimization Network (ARN) and an Alignment Convolutional Layer (ACL). The ARN generates high-quality rotated anchor boxes, and the ACL adaptively aligns the convolutional features with these high-quality rotated anchor boxes. The anchor box optimization network (ARN) performs convolution operations on the multi-scale feature maps {P3, P4, P5, P6, P7} of the image, and outputs the classification and regression results for each feature map: cls_pre i =Conv(Conv(Conv(Pi)))i=3,4,...,7 reg_pre i =Conv(Conv(Conv(Pi)))i=3,4,...,7 In the formula, cls_pre i and reg_pre i These are the anchor box classification information and anchor box position information output by the classification branch of the ARN network for the i-th scale feature map, respectively; The aligned convolutional layer ACL is implemented through an aligned convolutional kernel. The input consists of multi-scale feature maps {P3, P4, P5, P6, P7} and the rotation anchor box position information reg_pre corresponding to each scale feature map. i For i = 3, 4, ..., 7, the output is the rotated convolutional feature, i.e., the aligned feature Fd. i : Fd i =AlignConv(Pi,reg_pre i ),i=3,4......,7 Here, AlignConv(·) represents the alignment convolution kernel.

5. The method for detecting rotating small targets in remote sensing images by fusing cognitive features according to claim 1, characterized in that, The processing steps of the orientation detection module ODM include: The input fused feature F is obtained by using an active rotating filter (ARF). i Encode directional information to generate directional-sensitive feature Z i ; The orientation-invariant feature Z′ is obtained by merging orientation-sensitive features. i ; The direction-sensitive feature Z i and direction-invariant feature Z′ i The bounding box regression branch and the bounding box classification branch are fed into the bounding box data respectively, and three convolutions are performed to obtain the bounding box regression result Reg. i and classification results Cls i .

6. The remote sensing image rotation small target detection method based on fused cognitive features according to claim 5, characterized in that, The target detection model also includes a post-processing module, the processing of which includes: Regression results of bounding boxes at 5 scale feature layers Reg i and classification results Cls i Decoding is performed, and bounding boxes with category confidence scores higher than the set threshold are selected. Non-maximum suppression is then applied to the bounding boxes, and the detection boxes with the highest confidence scores are retained to obtain category information.

7. The remote sensing image rotation small target detection method based on the fusion of cognitive features according to any one of claims 1-6, characterized in that, The method also includes a training step for the object detection model, comprising: A remote sensing image land vehicle target detection dataset can be constructed by selecting data corresponding to small-sized vehicle targets from the existing publicly available DOTA remote sensing image target detection dataset; or by using satellite and airborne data obtained from Google Earth as the basic data source, image data preparation, target annotation and data augmentation can be carried out to construct a remote sensing image land vehicle target detection dataset. A portion of the remote sensing image land vehicle target detection dataset was selected as the training set, and the other portion as the validation set. Set the initialization and training parameters for the object detection model; The training set data is sequentially input into the object detection model, the training loss function is calculated using the dynamic soft label assignment method, and the model is trained using the stochastic gradient descent (SGD) optimizer until the training requirements are met, resulting in a well-trained object detection model. The model is then validated using the mAP evaluation metric on the validation set.

Images