SAR image small sample detection method and system based on dynamic perception RPN and category knowledge decoding

Abstract

The invention discloses an SAR image small sample detection method and system based on dynamic perception RPN and category knowledge decoding, and mainly solves the problems of difficulty in foreground and background separation and difficulty in feature and label matching of an SAR image in the prior art. According to the implementation scheme, the method comprises the steps of obtaining multi-class target SAR images and dividing training sets; the small sample detection network comprises a backbone network, a dynamic perception RPN module, a category knowledge decoding module and a bidirectional constraint detection head module, the dynamic perception RPN module is used for outputting candidate areas of a query graph, and the category knowledge decoding module is used for effectively aggregating support features and query features and calculating decoding knowledge features; training the small sample detection network by using the training data set; and inputting the tested data into the trained small sample detection network to output a detection result of the SAR target. The method can effectively separate the foreground and the background, accurately match the features and the labels, realize high-precision robust classification under the condition of small samples, and can be used for target reconnaissance and situation awareness.

Description

Technical Field

[0001] The present invention belongs to the technical field of target detection, and further relates to a small sample detection method for SAR images, which can be used for target reconnaissance and situation awareness. Background Art

[0002] As an active microwave imaging sensor, synthetic aperture radar (SAR) plays an important role in reconnaissance and situational awareness due to its all-day, all-weather, and high-resolution capabilities. With the widespread availability of high-resolution SAR imagery, accurate detection of key targets such as aircraft has become a research hotspot. Current deep learning-based target detection methods are primarily oriented towards optical image optimization and are divided into two types of frameworks: two-stage, such as Faster RCNN, and single-stage, such as the YOLO series. The former improves accuracy through region proposals, while the latter enhances real-time performance through end-to-end regression. In recent years, detection technology has evolved towards a fully end-to-end approach. Anchor-free detection and Transformer architectures, such as DETR, reconstruct the detection process through an attention mechanism, demonstrating powerful feature modeling capabilities and providing a technical path for the next generation of detection systems.

[0003] However, deep learning methods rely on a large amount of training data and are prone to overfitting problems when samples are scarce. For this reason, research on small-sample target detection focuses on two types of methods: transfer learning and meta-learning. Although existing optical image detection methods have certain transfer capabilities in SAR scenarios, they still face essential challenges due to the complex electromagnetic scattering characteristics and imaging mechanisms of SAR: First, the inherent coherent speckle noise and low signal-to-noise ratio characteristics of SAR images lead to a nonlinear distribution of target and background scattering intensities, making it difficult to distinguish between targets and clutter; second, the differences in imaging principles between the optical domain and the microwave domain lead to limitations in cross-domain feature alignment. With limited samples, the model finds it difficult to capture the geometric deformation and azimuth-sensitive features of SAR targets, resulting in systematic deviations between feature representation and true labels; third, the intra-class differences of similar targets and the inter-class confusion of heterogeneous targets in SAR images significantly increase the difficulty of learning discriminative features for small-sample classifiers.

[0004] Patent application number CN 202411455746.1 discloses a "Small Sample SAR Image Target Detection System and Method Based on Context-Awareness and Gaussian Flow Representation." The system comprises five modules: an image acquisition module, a context-aware enhancement module, a manifold class distribution estimation module, a class-balanced differential aggregation module, and a cosine decoupling module. The system first acquires the query SAR image. The context-aware enhancement module then extracts supporting class prototypes and original query features. The manifold class distribution estimation module then converts the supporting features into class probability distributions. The class-balanced differential aggregation module then performs multi-class semantic fusion on the query features. Finally, target classification is achieved by comparing the original and enhanced query features using cosine similarity calculation and a learnable scaling factor. However, this method fails to fully account for the presence of speckle noise and background clutter in SAR images, and does not effectively separate foreground and background. Consequently, the recall rate of target detection in small sample conditions is relatively low.

[0005] The patent document with application number CN 202111489407.1 discloses a "small sample target detection system and method based on dynamic classifier". This method uses Faster RCNN as the basic detector. The model structure is not modified in the basic stage. The entire model is trained directly from random initialization on the base class training set, while the bounding box prediction module of the model uses a multi-classifier for the final target classification; in the fine-tuning stage, all model parameters except the bounding box prediction module are fixed, and the multi-classifier is replaced by a cosine classifier. However, this method does not fully consider the characteristics of SAR images. When training samples are extremely scarce, the model cannot effectively separate the foreground and background. It still has limitations in solving problems such as feature and label mismatch and low accuracy due to small inter-class differences and large intra-class variance of SAR images. Summary of the Invention

[0006] The purpose of the present invention is to address the shortcomings of the above-mentioned existing technologies and propose a small sample detection method and system for SAR images based on dynamic perception RPN and category knowledge decoding to avoid the mismatch between features and corresponding training labels, improve the target recall rate under small sample conditions, achieve high-precision robust classification under small sample conditions, and improve the SAR image target detection performance.

[0007] The technical approach to achieve the above goals is as follows: by designing a dynamic perception RPN module, the semantic information of supporting feature maps of different categories is assigned to different spatial positions of the query feature map, thereby alleviating the problem of low recall rate of detection targets due to insufficient data; by designing a category knowledge decoder module, the query feature vector is used as a hint to adaptively select appropriate supporting features for aggregation, thereby solving the mismatch problem between features and labels.

[0008] According to the above ideas, the technical solutions of the present invention include the following:

[0009] 1. A small sample detection method for SAR images based on dynamic perception RPN and category knowledge decoding, characterized by including:

[0010] (1) Obtain multi-class target SAR images as training sets and subdivide them into base class support training sets, base class query training sets, balanced support training sets, and balanced query training sets;

[0011] (2) Construct a small sample target detection network including a backbone network, a dynamic perception RPN module, a category knowledge decoding module, and a bidirectional constraint detection head module:

[0012] The dynamic perception RPN module includes a spatial mask generator and a dynamic kernel generator, which are used to output candidate regions of the query graph. The spatial mask generator includes a convolution layer, a global average pooling layer, a similarity map calculation layer and a regional mask generation layer; the dynamic kernel generator includes an adaptive average pooling layer, a convolution layer, a batch normalization layer and a ReLU activation function;

[0013] The category knowledge decoder module includes ROI Align operation, multi-head attention and FFN, convolution layer, global average pooling layer, and single-head attention, which is used to effectively aggregate support features and query features and calculate the feature f after category knowledge decoding. d ;

[0014] The bidirectional constraint detection head module consists of two parallel fully connected layers for bounding box regression and target category prediction, and calculates the large margin cosine loss and center calibration loss Achieve inter-class separation and intra-class compression of features;

[0015] (3) Use the training dataset to train the small sample target detection network:

[0016] (4) Input the test data into the trained small sample target detection network and output the detection results of the SAR target.

[0017] Preferably, the dynamic perception RPN module is used to output candidate regions of the query graph, and its implementation includes:

[0018] (2a) Generate a region-aware mask M through a spatial mask generator, where H×W represents the size of the query feature map;

[0019] (2b) Generate a dynamic convolution kernel W specifically for the nth category through the dynamic kernel generator n ,in c in、c out are the input and output channels of the convolution kernel respectively, and k is the size of the convolution kernel;

[0020] (2c) Based on the region-aware mask M, use N dynamic convolution kernels W n Calculate the query feature graph F q Feature map Y:

[0021] (2d) The obtained feature map Y is passed through two parallel 1×1 convolutions to obtain the foreground and background classification results and bounding box regression results. Candidate anchor boxes are screened according to the foreground classification scores, and the positions and sizes of these anchor boxes are fine-tuned using the bounding box regression parameters. Finally, redundant boxes are removed through non-maximum suppression to generate candidate regions of the query image.

[0022] Preferably, the category knowledge decoder module is used to effectively aggregate support features and query features and calculate the feature f after category knowledge decoding. d , whose implementation includes:

[0023] (2e) Through the ROI Align operation, the supporting feature map is obtained according to the real label to obtain the supporting RoI feature map The query feature map is obtained based on the candidate region output by RPN to obtain the query RoI feature map Where N is the number of categories and M is the number of candidate regions;

[0024] (2f) Through multi-head attention and FFN, and The global semantic information of the query ROI is used to perform semantic interaction between different RoI feature maps to obtain the pre-coded query ROI feature map. and supporting ROI feature maps

[0025] (2g) Through a 1×1 convolution linear mapping and a global average pooling layer, and Converted into query RoI feature vector And the supporting RoI feature vector in is the length of the ROI feature vector;

[0026] (2h) Through single-head attention, the support ROI feature vector f s Encoded as categorical knowledge

[0027] (2i) Query the RoI feature vector f q As a query token pair Decode to get f d .

[0028] 2. A SAR image small sample detection system based on dynamic perception RPN and category knowledge decoding, characterized by comprising:

[0029] Dataset generation module: used to obtain multi-class target SAR images as training sets, and subdivide them into base class support training set, base class query training set, balanced support training set, and balanced query training set;

[0030] Image preprocessing module: used to perform different image preprocessing operations on query images and support images to enhance the generalization ability of the model;

[0031] Network construction module: used to build a small-sample target detection network for SAR images based on dynamic perception RPN and category knowledge decoding;

[0032] Network training module: used to train the SAR image small sample target detection network based on dynamic perception RPN and category knowledge decoding;

[0033] Network inference module: used to extract and match features of the input image through the trained small sample target detection network to achieve target detection.

[0034] Compared with the prior art, the present invention has the following advantages:

[0035] 1. The present invention designs a dynamic perception RPN. This module adds a spatial mask generator and a dynamic kernel generator on the basis of the ordinary RPN. By assigning the semantics of supporting features of different categories to different spatial positions of the query feature map, it can effectively separate the foreground and background, thereby alleviating the problem of low recall rate of detection targets due to insufficient data.

[0036] 2. Due to the design of the category knowledge decoder in the present invention, the module no longer explicitly samples the supporting features of a specific category. Instead, it uses the query feature vector as a hint and adaptively selects appropriate supporting features for aggregation, thereby effectively solving the mismatch problem between features and labels. At the same time, it maintains the consistency of the model process in the training and inference stages, ensuring the stability of the model.

[0037] 3. Due to the design of a bidirectional constrained detection head, the present invention increases the distance between different categories in the high-dimensional feature embedding space through large marginal cosine loss, and reduces the distance between the same category in the high-dimensional feature space using center calibration loss, ultimately achieving inter-class separation and intra-class compression of features, thereby improving the accuracy of SAR image target classification under small sample conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is a flow chart of the SAR image small sample detection method based on dynamic perception RPN and category knowledge decoding of the present invention;

[0039] Figure 2 for Figure 1 Schematic diagram of the SAR image small sample detection network structure constructed in;

[0040] Figure 3 for Figure 2 The structural diagram of the dynamic perception RPN in;

[0041] Figure 4 for Figure 2 The structure diagram of the category knowledge decoder;

[0042] Figure 5 This is a structural block diagram of the SAR image small sample detection system based on dynamic perception RPN and category knowledge decoder of the present invention;

[0043] Figure 6 The following is a comparison chart of simulation results of detecting measured images using the present invention and the existing target detection and recognition method respectively. DETAILED DESCRIPTION

[0044] To help those skilled in the art better understand the present invention, the following will provide a clear and complete description of the technical solutions in the embodiments of the present invention, in conjunction with the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0045] Example 1: SAR image small sample detection method based on dynamic perception RPN and category knowledge decoder

[0046] Reference Figure 1 , the implementation steps of this embodiment include the following:

[0047] Step 1: Generate a training set.

[0048] 1.1) Obtain multiple SAR images and mark the location and category of the target in each SAR image as a training set;

[0049] 1.2) Divide the image labels in the training set into a base class training set and a new class training set according to their categories. Sample some training data for each category in the base class training set, and then merge this training data with the new class training set to form a balanced training set.

[0050] 1.3) The image data in the base training set and the balanced training set are used as query sets to form the base query training set and the balanced query training set; the target areas and background areas of some images in the base training set and the balanced training set are used as support sets to form the base support training set and the balanced support training set;

[0051] This example uses the SAR-Aircraft-1.0 dataset, acquired by the Gaofen-3 satellite with an image resolution of 1m×1m. After cropping and processing, the resulting images yield 4,368 images and 16,463 aircraft instances. The images are available in four sizes: 800×800, 1000×1000, 1200×1200, and 1500×1500. The dataset contains seven aircraft types: A220, A320 / 321, A330, ARJ21, Boeing 737, Boeing 787, and Other. This example uses the ARJ21 and Boeing 787 as new classes, and the remaining classes as base classes.

[0052] Step 2: Preprocess the images in the dataset.

[0053] 2.1) Perform channel normalization, scaling, random rotation, and random flipping on the images in the query set to obtain the preprocessed query images;

[0054] 2.2) Perform target sampling, channel normalization, random rotation, and random flipping on the image in the support set to obtain the preprocessed support image;

[0055] In this embodiment, the probability of random flipping is 0.5, the probability of random rotation is 0.5, the range is [-90°, 90°], and the images in the query set are uniformly scaled to 1024×1024.

[0056] Step 3: Build a small-sample target detection network based on dynamic perception RPN and category knowledge decoding.

[0057] Reference Figure 2 , the implementation of this step includes the following:

[0058] 3.1) The backbone network is selected as a Resnet-50 network consisting of one Stem layer and four residual layers. The query image and support image of the training set are input into two weight-sharing backbone networks in batches to obtain the query feature map F q and support feature maps Where n∈[1,N], N is the number of categories:

[0059] The structure of the Stem layer includes a convolution layer, a batch normalization layer, a ReLU activation function, and a maximum pooling layer. The convolution kernel size of the convolution layer is 7×7 and the stride is 2. The pooling kernel size of the pooling layer is 3×3 and the stride is 2.

[0060] The four residual layers are respectively composed of multiple residual blocks, wherein the first residual layer includes 3 residual blocks, the second residual layer includes 4 residual blocks, the third residual layer includes 6 residual blocks, and the fourth residual layer includes 3 residual blocks;

[0061] The residual block of each residual layer adopts a Bottleneck structure and is divided into two branches. The first branch first reduces the dimension through a 1×1 convolution, then processes the spatial features through a 3×3 convolution, and finally increases the dimension through a 1×1 convolution to restore the channel to the target value; the second branch obtains the output of this branch by passing the input through a 1×1 convolution, and finally adds the output results of the two branches to output the result of the residual block;

[0062] 3.2) Input the query feature map and the supporting feature map into the dynamic perception RPN, and output the candidate region of the query feature map:

[0063] Reference Figure 3 , the implementation of this step includes the following:

[0064] 3.2.1) Build a spatial mask generator to output a learnable region-aware mask M of the same size as the query feature map:

[0065] For the input query feature map and support feature maps First, linearly map the two through 3×3 convolution, and then flatten the query feature map after linear mapping along the spatial dimension to obtain the local features of the query feature map At the same time, the support feature map after linear mapping is globally averaged pooled to obtain the global feature of the support feature map of the nth category

[0066]

[0067] Where Flatten(·) represents flattening along the spatial dimension, GAP(·) represents global average pooling, and Conv 3×3 (·) represents a 3×3 convolution.

[0068] Afterwards, calculate and The inner product of the local query feature and the support feature of the nth category is obtained, and the similarity is readjusted to the shape of the query feature graph to obtain the similarity graph of the nth category.

[0069]

[0070] Finally, according to the similarity map, the element M(h,w) of the region-aware mask at the position (h,w) is calculated:

[0071] M(h,w)=argmax(S1(h,w),S2(h,w),...S n (h,w)...S N (h,w))

[0072] Where argmax(·) represents the index of the element with the maximum value, and N represents the number of categories;

[0073] 3.2.2) Construct a dynamic kernel generator to generate a dynamic convolution kernel W specifically for the nth category n :

[0074] First, the input support feature map First, average pooling is performed, and then the dimension is reduced to the original size through 1×1 convolution. And then pass through the batch normalization layer and Relu activation function in turn to obtain the reduced dimension features;

[0075] Then, for the reduced dimensionality features, the dimension is increased to c by 1×1 convolution and batch normalization. in ×c out ×k 2 After reshaping, we get a dynamic convolution kernel that is specific to a specific category. where c in 、c out are the input and output channels of the convolution kernel respectively, and k is the size of the convolution kernel;

[0076] 3.2.3) Based on the region-aware mask M, use N dynamic convolution kernels W n For the query feature graph F q Perform the following convolution operation to obtain the feature map Y:

[0077]

[0078] Where Y(h,w,o) is the eigenvalue of the oth channel of the feature map Y at the spatial coordinate (h,w), and F q (h,w,c) is the eigenvalue of the cth channel of the query feature map at the spatial coordinate (h,w), W M(h,w) (c, o, i, j) is the element value of the dynamic convolution kernel of the M(h, w) class at the c-th input channel and the o-th output channel at the (i, j) position, where M(h, w) is the value of the region-aware mask M at the spatial position (h, w);

[0079] 3.2.4) The resulting feature map Y is passed through two parallel 1×1 convolutions to obtain foreground and background classification results and bounding box regression results. Candidate anchor boxes are selected based on the foreground classification scores, and the positions and sizes of these anchor boxes are fine-tuned using the bounding box regression parameters. Finally, redundant boxes are removed through non-maximum suppression to generate candidate regions of the query image.

[0080] 3.3) Construct a category knowledge decoder and calculate the feature f after category knowledge decoding d :

[0081] Reference Figure 4 , the implementation of this step includes the following:

[0082] 3.3.1) Through the ROI Align operation, the supporting feature map is obtained according to the real label to obtain the supporting RoI feature map The query feature map is obtained based on the candidate region output by RPN to obtain the query RoI feature map Where N is the number of categories and M is the number of candidate regions;

[0083] 3.3.2) Through multi-head attention and FFN, and The global semantic information of the query ROI is used to perform semantic interaction between different RoI feature maps to obtain the pre-coded query ROI feature map. and supporting ROI feature maps

[0084]

[0085] Where FFN(·) is a feedforward network, MultiAtten(·) is multi-head self-attention, and GAP(·) is global average pooling;

[0086] 3.3.3) Through a 1×1 convolution linear mapping and a global average pooling layer, and Converted into query RoI feature vector And the supporting RoI feature vector in is the length of the ROI feature vector;

[0087] 3.3.4) Through single-head attention, the support ROI feature vector f s Encoded as categorical knowledge

[0088] First, the support ROI feature vector f s Transformed into the projection support vector of Q, K, V space

[0089]

[0090] Where W1 Q ,W1 K ,W1 V The variables in the model are The learnable weights of

[0091] Then, according to The supporting ROI feature vector f s Encoded as categorical knowledge

[0092]

[0093] In the formula, softmax is the activation function;

[0094] 3.3.5) Query the RoI feature vector f q As query tokens for category knowledge To decode:

[0095] First, the query RoI feature vector f q Project to the corresponding Q, K and V spaces to obtain the projection query vectors of the corresponding spaces and

[0096]

[0097] Then, the category knowledge Project to K and V space to obtain the category knowledge projection of the corresponding space and

[0098]

[0099] Next, calculate the attention weight A of the i-th candidate area i , and A i Perform channel splitting to obtain supporting attention weights and query attention weights

[0100]

[0101] Where, i∈[1,M], The query vector for the Q-space projection of the i-th candidate region is the K-space projection query vector of the i-th candidate region Concat(·) is a feature concatenation operation;

[0102] Finally, the decoding knowledge feature f of the i-th candidate region is calculated i d :

[0103]

[0104] Where, The V space projection query vector of the i-th candidate region

[0105] 3.4) Decode the knowledge feature f d Input to the bidirectional constraint detection head composed of two parallel fully connected layers to obtain the bounding box regression results and category prediction results, and calculate the large margin cosine loss and center calibration loss

[0106] First, using the decoded feature f d Calculate large margin cosine loss

[0107]

[0108] in, and is the i-th decoding knowledge vector, W j is the classifier weight corresponding to category j, y i is the target category of the i-th candidate region, m is the edge parameter preset to 30, and s is the scaling factor preset to 30;

[0109] Then, the support features f before decoding are used based on category knowledge s and query feature f q Calculating center calibration loss

[0110]

[0111] Where f i q is the query feature vector of the i-th candidate region, For category y i The supporting feature vector of , M is the number of RoIs in the query image.

[0112] Step 4: Train the small sample detection network.

[0113] 4.1) Input the base class training set into the small sample detection network for basic training;

[0114] 4.1.1) Set the batch size to 8 and the learning rate to 0.005;

[0115] 4.1.2) Input the base class support image and the base class query image into the small sample detection network, perform forward propagation, and calculate the loss function:

[0116]

[0117] in is the total loss of the candidate region for DP-RPN training, which has the same form as the ordinary RPN. is the large margin cosine loss, β is the corresponding weight factor and is set to 1, is the bounding box loss, is the center calibration loss, γ is the corresponding weight factor and is set to 10;

[0118] In this embodiment, the bounding box loss is the classification loss and positioning loss sum:

[0119] Classification loss Using cross entropy loss, it is expressed as:

[0120]

[0121] Among them, y i is the true category, The class predicted by the network;

[0122] Positioning loss Use Smooth L1 loss, expressed as:

[0123]

[0124] in, t i is the true target frame parameter, is the target box parameter predicted by the network, and the smoothing parameter δ is set to 1;

[0125] 4.1.3) Use stochastic gradient descent (SGD) optimizer to update the network parameters between layers;

[0126] 4.2) Repeat step 4.1) until the loss converges to obtain the basic network after preliminary training;

[0127] 4.3) Fine-tune the basic network with a small sample:

[0128] 4.3.1) Set the batch size to 8 and the learning rate to 0.0005;

[0129] 4.3.2) Input the balanced support set and the balanced query set into the base network. During training, freeze the parameters of the backbone network and the dynamic perception RPN, and keep the forward propagation consistent with the base training.

[0130] 4.3.3) Use SGD optimizer to update the network parameters between layers;

[0131] 4.4) Repeat step 4.3) until the loss converges and a fully trained network is obtained.

[0132] In step 5, the test data is input into the trained small sample detection network and the detection results of the SAR target are output.

[0133] Example 2: SAR Image Small Sample Detection System Based on Dynamic Perception RPN and Category Knowledge Decoder

[0134] Reference Figure 5 This embodiment includes a dataset generation module 1, an image preprocessing module 2, a network construction module 3, a network training module 4, and a network inference module 5. The network construction module 3 includes: a backbone network submodule 31, a dynamic perception RPN submodule 32, a category knowledge decoder submodule 33, and a bidirectional constraint detection head submodule 34. Its working principle is as follows:

[0135] The data set generation module 1 is used to obtain multiple types of target SAR images as training sets, and subdivide them into a base class support training set, a base class query training set, a balanced support training set and a balanced query training set;

[0136] The image preprocessing module 2 is used to perform different image preprocessing operations on the images of the divided base class query data set, base class query support data set, balanced query data set and balanced support data set to obtain preprocessed training images;

[0137] The network construction module 3 is used to construct a SAR image small sample detection network based on dynamic perception RPN and category knowledge decoder, wherein the backbone network submodule 31 is used to extract features of the input support image and the query image; the dynamic perception RPN submodule 32 is used to generate candidate regions of the query image, and improve the generation quality of the candidate regions by assigning semantics of support features of different categories to different spatial positions of the query feature map; the category knowledge decoder submodule 33 is used to adaptively select appropriate support features for aggregation based on the candidate regions by using the query feature vector as a hint, thereby calculating the decoded category knowledge features; the bidirectional constraint detection head submodule 34 is used to perform bounding box regression and target category prediction to achieve accurate target detection and recognition;

[0138] The network training module 4 is used to input the pre-processed training image into the small sample detection network for training to obtain a fully trained small sample detection network;

[0139] The network inference module 5 is used to perform feature extraction and matching on the input image through a fully trained small sample detection network to achieve target detection and recognition.

[0140] It should be noted that the above-mentioned functional modules can be implemented in whole or in part through software, hardware, firmware, or any combination thereof. When implemented using software, they can be implemented in whole or in part in the form of a program instruction product. The program instruction product includes one or a group of program instructions. When the program instructions are loaded and executed on a computer, the process or function described is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The program instructions can be stored in a computer-readable and writable storage medium, or transferred from a computer-readable and writable storage medium to another computer-readable and writable storage medium.

[0141] The direct coupling or communication connection between the modules shown or discussed in this embodiment can be achieved through indirect coupling or communication connection of some interfaces, devices or modules. The various functional modules and submodules in this embodiment can be dynamically located in a processing component, or each module can exist physically separately, or two or more modules can be dynamically located in a processing component. When the above-mentioned dynamic components are implemented in the form of software functional modules and sold or used as independent products, they can also be stored in a computer-readable and writable storage medium. The storage medium can be a memory, a magnetic disk, an optical disk, etc.

[0142] The flowchart representation or method representation of the above embodiments can be understood as representing a module, segment or portion of code including one or a group of executable instructions configured to implement the steps of a specific logical function or process. The present invention is not limited to the disclosed preferred embodiments, and its implementation can be performed in a different order than that shown or discussed, including performing functions in a substantially simultaneous manner according to the functions involved.

[0143] The effect of the present invention can be further illustrated by the following simulation experiments:

[0144] 1. Simulation conditions

[0145] The software platform of the simulation experiment of the present invention is the Ubuntu 20.04 operating system, the experimental environment is Python 3.8, torch 1.11.0 and CUDA 11.3, and the hardware configuration is an Intel Core i9-14900K processor and an NVIDIA GeForce RTX 4090 graphics card.

[0146] The simulation experiments used the SAR-Aircraft-1.0 dataset for training data. This dataset, acquired by the Gaofen-3 satellite, has an image resolution of 1m×1m. After cropping and processing, the resulting images yielded 4,368 images and 16,463 aircraft instances. The images were sized at four different sizes: 800×800, 1000×1000, 1200×1200, and 1500×1500. These images included seven aircraft classes: A220, A320 / 321, A330, ARJ21, Boeing 737, Boeing 787, and Other. The ARJ21 and Boeing 787 were treated as new classes, while the remaining classes were treated as base classes.

[0147] The test data for the simulation experiment uses two measured SAR images from the Gaofen-3 satellite.

[0148] 2. Simulation Content

[0149] Under the above simulation conditions, the present invention and the existing detection method are used to complete the training on the training data, and then the test data are input into the trained networks of the present method and the existing detection and recognition method respectively to obtain the detection results, such as Figure 6 As shown, the solid line indicates correct detection and recognition, the dotted line indicates detection or recognition errors, and the circle indicates missed detection.

[0150] from Figure 6 It can be seen that the existing technical solutions have more false alarms and missed detections, while the present invention has no false alarms and missed detections, indicating that the detection and recognition performance of the present invention is significantly better than the existing detection methods.

[0151] It should be noted that the step numbers in the specification and claims of the present invention are only for the purpose of clearly describing the embodiments of the present invention and facilitating understanding, and the order of the step numbers is not limited.

Claims

1. A small sample detection method for SAR images based on dynamic perception RPN and category knowledge decoding, characterized by: include: (1) Obtain multi-class target SAR images as training sets and subdivide them into base class support training sets, base class query training sets, balanced support training sets, and balanced query training sets; (2) Construct a small sample target detection network including a backbone network, a dynamic perception RPN module, a category knowledge decoding module, and a bidirectional constraint detection head module: The dynamic perception RPN module includes a spatial mask generator and a dynamic kernel generator, which are used to output candidate regions of the query graph. The spatial mask generator includes a convolution layer, a global average pooling layer, a similarity map calculation layer and a region mask generation layer; The dynamic kernel generator includes an adaptive average pooling layer, a convolutional layer, a batch normalization layer, and a ReLU activation function; The category knowledge decoder module includes ROI Align operation, multi-head attention and FFN, convolution layer, global average pooling layer, and single-head attention, which is used to effectively aggregate support features and query features and calculate the feature f after category knowledge decoding. d ; The bidirectional constraint detection head module consists of two parallel fully connected layers for bounding box regression and target category prediction, and calculates the large margin cosine loss and center calibration loss Achieve inter-class separation and intra-class compression of features; (3) Use the training dataset to train the small sample target detection network: (4) Input the test data into the trained small sample target detection network and output the detection results of the SAR target.

2. The method according to claim 1, characterized in that The implementation of step (1) includes the following steps: (1a) Obtain multiple SAR image data from the SAR-Aircraft-1.0 dataset, as well as the target location and target category in each SAR image, as a training set; (1b) Further subdivide the training set: (1b1) Construct base class training set and new class training set: The ARJ21 and Boing787 categories in the training set are taken as new classes, and the new class labels of each SAR image are taken to form a new class training set; The five categories of A220, A320 / 321, A330, Other, and Boeing737 in the training set are taken as base classes, and the base class labels of each SAR image are taken to form the base class training set; (1b2) Subdivide or merge the base class training set and the new class training set: Sample some training data from each category of the base class training set, and then merge these training data with the new class training set to form a balanced training set; The image data of the base class training set is used as the base class query training set; Sample the target area and part of the background area of ​​the image data from the base class training set as the base class support training set; The image data of the balanced training set is used as the balanced query training set; The target area and background area of ​​part of the image data are sampled from the balanced training set as the balanced support training set.

3. The method according to claim 1, characterized in that The backbone network in step (2) is a Resnet-50 network including one Stem layer and four residual layers, which is used to extract the features of the input image, wherein: The structure of the Stem layer, which includes a convolutional layer, a batch normalization layer, a ReLU activation function, and a maximum pooling layer. The convolution kernel size of the convolution layer is 7×7 and the stride is 2. The pooling kernel size of the pooling layer is 3×3 and the stride is 2. The four residual layers are composed of multiple residual blocks, where the first residual layer includes 3 residual blocks, the second residual layer includes 4 residual blocks, the third residual layer includes 6 residual blocks, and the fourth residual layer includes 3 residual blocks; The residual block of each residual layer adopts a Bottleneck structure and is divided into two branches. The first branch first reduces the dimension through a 1×1 convolution, then processes the spatial features through a 3×3 convolution, and finally increases the dimension through a 1×1 convolution to restore the channel to the target value; the second branch obtains the output of this branch by passing the input through a 1×1 convolution, and finally adds the output results of the two branches to output the result of the residual block; The query image and support image of the training set are input into two weight-sharing backbone networks in batches to obtain the query feature map F q and support feature maps Where n∈[1,N], N is the number of categories.

4. The method according to any one of claims 1 to 3, characterized in that The dynamic perception RPN module in step (2) is used to output candidate regions of the query graph, and its implementation includes: (2a) Generate a region-aware mask M through a spatial mask generator, where H×W represents the size of the query feature map; (2b) Generate a dynamic convolution kernel W specifically for the nth category through the dynamic kernel generator n ,in c in 、c out are the input and output channels of the convolution kernel respectively, and k is the size of the convolution kernel; (2c) Based on the region-aware mask M, use N dynamic convolution kernels W n For the query feature graph F q Perform the following convolution operation to obtain the feature map Y: Where Y(h,w,o) is the eigenvalue of the oth channel of the feature map Y at the spatial coordinate (h,w), and F q (h,w,c) is the eigenvalue of the cth channel of the query feature map at the spatial coordinate (h,w), W M(h,w) (c, o, i, j) is the element value of the dynamic convolution kernel of the M(h, w) class at the c-th input channel and the o-th output channel at the (i, j) position, where M(h, w) is the value of the region-aware mask M at the spatial position (h, w); (2d) The obtained feature map Y is passed through two parallel 1×1 convolutions to obtain the foreground and background classification results and bounding box regression results. Candidate anchor boxes are screened according to the foreground classification scores, and the positions and sizes of these anchor boxes are fine-tuned using the bounding box regression parameters. Finally, redundant boxes are removed through non-maximum suppression to generate candidate regions of the query image.

5. The method according to claim 4, characterized in that The step (2a) of generating the region-aware mask M by the spatial mask generator includes: (2a1) For the input query feature map and support feature maps First, linearly map the two through 3×3 convolution, and then flatten the query feature map after linear mapping along the spatial dimension to obtain the local features of the query feature map At the same time, the support feature map after linear mapping is globally averaged pooled to obtain the global feature of the support feature map of the nth category The calculation formula is as follows: Where Flatten(·) represents flattening along the spatial dimension, GAP(·) represents global average pooling, and Conv 3×3 (·) represents 3×3 convolution; (2a2) Through the similarity graph calculation layer, calculate and The inner product of the local query feature and the support feature of the nth category is obtained, and the similarity is readjusted to the shape of the query feature graph to obtain the similarity graph of the nth category. Where reshape(·) means adjusting the shape of the feature map; (2a3) Based on the similarity graph, the element M(h,w) of the region-aware mask at the position (h,w) is calculated through the region mask generation layer: M(h,w)=argmax(S1(h,w),S2(h,w),...S n (h,w)...S N (h,w)) Where argmax(·) represents the index of the element with the maximum value, and N represents the number of categories; (2a4) The elements M(h,w) at all positions constitute the region mask M.

6. The method according to claim 4, characterized in that In step (2b), a dynamic convolution kernel W is generated by a dynamic kernel generator specifically for the nth category. n , whose implementation includes: (2b1) Support feature map of the input First, average pooling is performed, and then the dimension is reduced to the original size through 1×1 convolution. And then pass through the batch normalization layer and Relu activation function in turn to obtain the reduced dimension features; (2b2) ​​For the reduced dimensionality features, the dimension is increased to c by 1×1 convolution and batch normalization. in ×c out ×k 2 After reshaping, we get a dynamic convolution kernel that is specific to a specific category. where c in 、c out are the input and output channels of the convolution kernel respectively, and k is the size of the convolution kernel.

7. The method according to claim 1, characterized in that The category knowledge decoder module in step (2) is used to effectively aggregate support features and query features and calculate the feature f after category knowledge decoding. d , whose implementation includes: (2e) Through the ROI Align operation, the supporting feature map is obtained according to the real label to obtain the supporting RoI feature map The query feature map is obtained based on the candidate region output by RPN to obtain the query RoI feature map Where N is the number of categories and M is the number of candidate regions; (2f) Through multi-head attention and FFN, and The global semantic information of the query ROI is used to perform semantic interaction between different RoI feature maps to obtain the pre-coded query ROI feature map. and supporting ROI feature maps Where FFN(·) is a feedforward network, MultiAtten(·) is multi-head self-attention, and GAP(·) is global average pooling; (2g) Through a 1×1 convolution linear mapping and a global average pooling layer, and Converted into query RoI feature vector And the supporting RoI feature vector in is the length of the ROI feature vector; (2h) Through single-head attention, the support ROI feature vector f s Encoded as categorical knowledge (2h1) will support ROI feature vector f s Transformed into the projection support vector of Q, K, V space Where W1 Q ,W1 K ,W1 V The variables in the model are The learnable weights of (2h2)Based on The supporting ROI feature vector f s Encoded as categorical knowledge In the formula, softmax is the activation function; (2i) Query the RoI feature vector f q As a query token pair Decode to get f d : (2i1) query the RoI feature vector f q Project to the corresponding Q, K and V spaces to obtain the projection query vectors of the corresponding spaces and (2i2) Classification knowledge Project to K and V space to obtain the category knowledge projection of the corresponding space and (2i3) Decode the category knowledge to obtain the decoded knowledge features First, calculate the attention weight A of the i-th candidate region i : Where, i∈[1,M], The query vector for the Q-space projection of the i-th candidate region is the K-space projection query vector of the i-th candidate region Concat(·) is a feature concatenation operation; Then, for A i Perform channel splitting to obtain supporting attention weights and query attention weights Finally, the decoding knowledge feature f of the i-th candidate region is calculated i d : Where, The V space projection query vector of the i-th candidate region 8. The method according to any one of claims 1 to 7, wherein: The bidirectional constraint detection head in step (2) calculates the large edge cosine loss and the center calibration loss, and its implementation includes the following: (2j) Using the decoded knowledge feature f d Calculate large margin cosine loss in In the above formula is the i-th decoding knowledge vector, W j is the classifier weight corresponding to category j, y i is the target category of the i-th candidate region, m is the pre-set edge parameter, and s is the pre-set scaling factor; (2k) Using the support feature f before decoding using category knowledge s and query feature f q Calculating center calibration loss Where f i q is the query feature vector f of the i-th candidate region q , For category y i The supporting eigenvector f s , M is the number of RoIs in the query image.

9. The method according to claim 1, characterized in that The training data set is used for training in (3), and its implementation includes: (3a) Image preprocessing: The query image is subjected to channel normalization, scaling, random rotation, and random flipping in sequence; The support image is randomly sampled, channel normalized, randomly rotated, and randomly flipped in sequence; (3b) The preprocessed base class dataset is input into the small sample target detection network for basic training, using the loss function Calculate the gradient of the network and use the SGD optimizer to update the parameters of each layer. The loss function is expressed as follows: In the formula The total loss of the candidate region for dynamic perception RPN training is the same as that of the ordinary RPN. is the large margin cosine loss, β is the corresponding weight factor, is the bounding box loss, is the center calibration loss, γ is the corresponding weight factor; (3c) Repeat step (3b) until the loss function converges and the basic network after preliminary training is obtained; (3d) Input the preprocessed balanced dataset into the basic network for small sample fine-tuning, freeze the parameters of the backbone network and dynamic perception RPN, and use the loss function Calculate the network gradient and use the SGD optimizer to update the parameters of each layer; (3e) Repeat step (3d) until the loss function converges to obtain the final trained small-sample object detection network.

10. A SAR image small sample detection system based on dynamic perception RPN and category knowledge decoding, characterized by: include: Dataset generation module: used to obtain multi-class target SAR images as training sets, and subdivide them into base class support training set, base class query training set, balanced support training set, and balanced query training set; Image preprocessing module: used to perform different image preprocessing operations on query images and support images to enhance the generalization ability of the model; Network construction module: used to build a SAR image small sample detection network based on dynamic perception RPN and category knowledge decoding; Network training module: used to train the SAR image small sample detection network based on dynamic perception RPN and category knowledge decoding; Network inference module: used to extract and match features of the input image through the trained small sample detection network to achieve target detection.

Images