A Small-Sample Object Detection Method and System for SAR Images Based on Meta-Learning

Abstract

The present invention discloses a small-sample target detection method and system for SAR images based on meta-learning, belonging to the technical field of SAR image target detection, which includes the following steps: S1: Data collection and production; S2: Model rough tuning; S3: Model fine tuning; S4: Target detection. The present invention can realize the fusion of multi-scale feature information between the features of the SAR image itself and the corresponding scattering characteristic map, enabling the model to obtain more effective features; the feature fusion part of the small-sample target detection algorithm is optimized. In particular, when calculating the M2 feature, the hybrid attention representation M1 of the support set and the query set is used, which can more fully explore the feature connection between the support set and the query set, screen out the most effective features and reduce the weight of inefficient features, making the algorithm performance index higher.

Description

Technical Field

[0001] The present invention relates to the technical field of SAR image target detection, and particularly relates to a small-sample target detection method and system for SAR images based on meta-learning. Background Art

[0002] China's aerospace reconnaissance equipment already has the capabilities of multi-source, multi-band, multi-mode, multi-application and high-resolution imaging of the ground. However, compared with the development of imaging equipment, the capabilities of real-time image interpretation and support for emergency reconnaissance and other intelligence applications on aircraft, missiles and satellites are relatively weak. Realizing the conversion of reconnaissance image data into intelligence is one of the ways to exert the combat effectiveness of reconnaissance equipment and improve the utilization rate of equipment. Target recognition technology is an important link in realizing the conversion of data into intelligence. At present, the technical indicators of the current image processing and intelligence application technology based on ground stations are good, but the development in the field of image interpretation is relatively slow, and it is impossible to quickly screen out effective and available intelligence information on aircraft, missile and satellite-borne platforms.

[0003] In the field of intelligent image interpretation for aircraft, missile and satellite-borne platforms, first of all, it is necessary to solve the recognition problem brought about by the lack of samples in the foreign military target database, that is, the small-sample recognition problem. For foreign military targets of interest, they are in a highly confidential or concealed state during non-war times, and it is extremely difficult to obtain their sample image data. And a small number of samples are difficult to support the training of deep neural networks, which greatly limits the performance of the target recognition model. Therefore, for the intelligent image interpretation of airborne platforms, it is an inevitable research trend to carry out target recognition technology under small-sample conditions.

[0004] At present, the target recognition algorithms based on small samples are mainly divided into two branches: one is to increase the sample size by expanding the data set and turn the small-sample problem into a target recognition problem with a normal sample size. However, the existing data expansion methods have various problems, including possible introduction of noise and inability to effectively increase effective features, etc.; and the other branch is to use methods such as model fine-tuning, metric learning or meta-learning, and use the similar features of existing large-sample data to assist in the recognition of small-sample targets.

[0005] At present, the SAR image small-sample recognition technology faces various challenges. In addition to the problems such as high background complexity and non-uniform resolution existing in SAR (Synthetic Aperture Radar) images themselves, the most important problem is the large intra-class deviation of targets, that is, some targets are numerous and easy to collect, but often another type of target similar to them is scarce and has strong concealment measures. For example, for ship targets, the number of warships is much larger than that of aircraft carriers. The above problems need to be solved urgently. For this reason, a small-sample target detection method and system for SAR images based on meta-learning are proposed. Summary of the Invention

[0006] The technical problem to be solved by the present invention is: how to solve the problem of sample imbalance in SAR image detection and recognition tasks, that is, the problem of low detection accuracy due to the small number of small sample classes, and a small sample target detection method for SAR images based on meta-learning is provided. The recognition of aircraft carriers can be assisted by means of the similar features of warships and aircraft carriers through meta-learning, so as to improve the detection ability of the model for aircraft carriers.

[0007] The present invention solves the above technical problems through the following technical solutions. The present invention includes the following steps:

[0008] S1: Data collection and production

[0009] Collect base class and small sample class data, and produce corresponding support set input and query set input;

[0010] S2: Coarse adjustment of the model

[0011] Input the base class data of the support set and the query set into the base class target detection model, extract three groups of features of different scales of the support set and the query set respectively, perform feature aggregation through cross-correlation operations, and input them into the target detection backend for regression of categories and detection frames to obtain the coarsely adjusted base class target detection model;

[0012] S3: Fine-tuning of the model

[0013] Take part of the base class data of the support set and the query set and all the small sample class data as input, and fine-tune the coarsely adjusted base class target detection model with a set learning rate to obtain the final base class-small sample class target detection model;

[0014] S4: Target detection

[0015] Input the SAR image to be detected into the final base class-small sample class target detection model for detection to obtain the detection result.

[0016] Furthermore, in the step S1, the base class is the class with a large number of samples, and the small sample class is the class with a small number of samples.

[0017] Furthermore, in the step S1, the support set input includes slices of different types of targets cropped from the SAR large image, binary mask images of the slices, and scattering characteristic maps obtained after the slices of different types of targets pass through the Harris-Laplace detector; the query set input includes slices of different types of targets cropped from the SAR large image and scattering characteristic maps obtained after the slices of different types of targets pass through the Harris-Laplace detector.

[0018] Furthermore, the binary mask image refers to a binary image with the same size as the support set input image, the target area is a white rectangular frame, and other areas are black.

[0019] Furthermore, in the step S2, the base-class object detection model is a siamese network model, including a support set feature extraction model and a query set feature extraction model. Among them, the support set feature extraction model and the query set feature extraction model share weights and form a siamese network structure.

[0020] Furthermore, in step S2, the specific process of the cross-correlation operation is as follows:

[0021] S21: First, after the feature map of W / 16*H / 16*C2 obtained by the support set feature extraction model passes through two parallel 1*1 convolutional layers and undergoes dimensional transformation, two feature maps are obtained: VQ2 (size WH / 256*C2 / 2), KQ2 (size WH / 256*C2 / 8); and the feature map of W / 16*H / 16*C2 obtained by the support set feature extraction model also passes through two parallel 1*1 convolutional layers and undergoes dimensional transformation, obtaining two feature maps: VS2 (size C2 / 8×NHW / 256), KS2 (size NHW / 256×C2 / 2), where N is the number of all categories (the sum of the base class and the few-shot classes);

[0022] S22: Perform matrix multiplication on KQ2 and VS2, and obtain the feature map M1 (size HW / 256×NHW / 256) through the softmax function as the mixed attention representation of the query set and the support set;

[0023] S23: Then perform matrix multiplication on M1 and KS2, and the obtained feature map M2 (size HW / 256×C2 / 2) is equivalent to using the attention representation to filter more useful support set features;

[0024] S24: Finally, perform a concatenation operation on M2 and VQ2 to obtain the final feature map M3 (size HW / 256×C2).

[0025] Furthermore, in the step S2, the object detection backend includes a detection head, a dimensional transformation module, and a prediction module. Input the three groups of feature maps obtained by the cross-correlation operation into the detection head, use the dimensional transformation module to perform dimensional transformation according to the 1*1 convolutional layer with the set number of channels, and then use the prediction module to predict the type, coordinates, length-width, and confidence of the three groups of features respectively.

[0026] The present invention also provides a few-shot object detection system for SAR images based on meta-learning, which uses the above method to perform the detection work of few-shot objects in SAR images, including:

[0027] A data collection and production module, used to collect base-class and few-shot class data and produce corresponding support set inputs and query set inputs;

[0028] The model rough - tuning module is used to input the base - class data of the support set and the query set into the base - class object detection model, extract three groups of features with different scales of the support set and the query set respectively, perform feature aggregation through cross - correlation operations, and input it into the object detection backend for regression of categories and detection boxes, so as to obtain the rough - tuned base - class object detection model;

[0029] The model fine - tuning module is used to take part of the base - class data of the support set and the query set and all the small - sample class data as input, and fine - tune the base - class object detection model obtained by rough - tuning with a set learning rate to obtain the final base - class - small - sample class object detection model;

[0030] The object detection module is used to input the SAR image to be detected into the final base - class - small - sample class object detection model for detection to obtain the detection result.

[0031] The present invention has the following advantages compared with the prior art:

[0032] 1. It can realize the fusion of multi - scale feature information between the features of the SAR image itself and the corresponding scattering characteristic map, enabling the model to obtain more effective features;

[0033] 2. It optimizes the feature fusion part of the small - sample object detection algorithm. Especially when calculating the M2 feature, it uses the hybrid attention representation M1 as the support set and the query set, which can more fully explore the feature connection between the support set and the query set, screen out the most effective features and reduce the weight of inefficient features, making the algorithm performance index higher. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 is a schematic flow chart of a SAR small - sample object detection method combining scattering features in an embodiment of the present invention;

[0035] Figure 2 is a schematic structural diagram of the final SAR image object detection model in an embodiment of the present invention;

[0036] Figure 3 is a schematic flow chart of the cross - correlation operation in an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0037] The following details the embodiments of the present invention. The embodiments are implemented on the premise of the technical solution of the present invention, and detailed implementation manners and specific operation processes are given. However, the protection scope of the present invention is not limited to the following embodiments.

[0038] Such as Figure 1As shown in the figure, this embodiment provides a technical solution: a meta-learning SAR few-shot target detection algorithm combined with scattering features. This method includes three steps: collection and production of the dataset, and coarse tuning and fine tuning of the model. The specific description is as follows:

[0039] (1) Collection and production of the dataset: The target categories of the dataset are divided into base classes and few-shot classes. The base classes are large-sample classes that can be collected. In terms of data production, considering that the model has two inputs, corresponding support set inputs and query set inputs need to be produced. The support set input includes slices of different types of targets cropped from the SAR large image, binary mask images of the slices, and scattering characteristic maps obtained after these slices pass through the Harris-Laplace detector; the query set input is slices containing various targets cropped from the SAR large image, and scattering characteristic maps obtained after these slices pass through the Harris-Laplace detector. The categories of the dataset can be divided into base classes and few-shot classes. The base classes refer to the categories with more samples, and the few-shot classes refer to the categories with fewer targets (usually less than 50 targets);

[0040] (2) Coarse tuning of the model: Input the base class data of the support set and the query set into the base class target detection model. This model is a siamese network model. The support set feature extraction model and the query set feature extraction model share weights to form a siamese network structure. These two models respectively extract three groups of features at different scales, perform feature aggregation through the cross-correlation operation proposed in this paper, and finally input them into the target detection backend for regression of categories and detection boxes to obtain the coarsely tuned base class-few-shot class target detection model (i.e., Figure 1 、 2 the base class / few-shot class target detection model in

[0041] (3) Fine tuning of the model: Use part of the base class data of the support set and the query set and all the few-shot class data as inputs, and use a lower learning rate to fine-tune the base class-few-shot class target detection model obtained by coarse tuning. The model obtained after fine tuning is the SAR image target detection model that takes both base classes and few-shot classes into account.

[0042] In this embodiment, the Harris-Laplace in step (1) is a common corner detector in image processing algorithms, generally used for feature matching. Its calculation process is simplified as follows:

[0043] S11: First, deblur and denoise the image;

[0044] S12: Take σ n = k n σ0 (k = 1.4, n = 1, 2, 3…N), σ D = sσ n(s = 0.7), where N is a positive integer, 1.4 and 0.7 are empirical values, σ D is the Gaussian weighted directional gradient in the x and y axis directions, σ n is the initial Harris response value of the image;

[0045] S13: Calculate the maximum corner degree of each initial Harris response value at different scales, and retain the points greater than the average value;

[0046] S14: Verify that the points obtained in the previous step calculate the normalized LoG operator in each scale space, and determine whether there are extreme points. If it is an extreme point, then this point is the Harris-Laplace corner point.

[0047] In this embodiment, the binary mask image in step (1) refers to a binary image with the same size as the support set input image. The target area is a white rectangular box (value is 1), and other areas are black (value is 0). The purpose of this mask image is to enable the support set feature extraction model to enhance the learning ability of the target.

[0048] In this embodiment, the image sizes of the support set and the query set in step (1) can be inconsistent, but the images trained in each batch will be uniformly normalized to fixed values W and H.

[0049] In this embodiment, the siamese network model involved in step (2) can be various common models, such as ResNet, MobielNet, EffcientNet series network models. The sizes of the three groups of feature maps finally output are W / 8*H / 8*C1, W / 16*H / 16*C2, W / 32*H / 32*C3. These three scales are determined according to the input image size, where W and H are the length and width of the input image, and C is the number of channels.

[0050] As Figure 3 shown, in this embodiment, a special cross-correlation operation is involved in step (2), and the cross-correlation operation needs to be performed on the three groups of feature maps of different scales. Taking the cross-correlation operation of the W / 16*H / 16*C2 scale as an example for illustration:

[0051] First, after the feature map of W / 16*H / 16*C2 obtained by the support set feature extraction model passes through two parallel 1*1 convolutional layers and undergoes dimensional transformation, two feature maps are obtained: VQ2 (with a size of WH / 256*C2 / 2), and KQ2 (with a size of WH / 256*C2 / 8). The feature map of W / 16*H / 16*C2 obtained by the support set feature extraction model also passes through two parallel 1*1 convolutional layers and undergoes dimensional transformation, resulting in two feature maps: VS2 (with a size of C2 / 8×NHW / 256), and KS2 (with a size of NHW / 256×C2 / 2), where N is the number of all categories (the sum of base classes and few-shot classes). First, perform matrix multiplication on KQ2 and VS2 and pass through the softmax function to obtain the feature map M1 (with a size of HW / 256×NHW / 256), which is the mixed attention representation of the query set and the support set; then perform matrix multiplication on M1 and KS2 to obtain the feature map M2 (with a size of HW / 256×C2 / 2), which is equivalent to using the attention representation to filter more useful support set features; finally, perform a concatenation operation on M2 and VQ2 to obtain the final feature map M3 (with a size of HW / 256×C2).

[0052] In this embodiment, in step (2), the final object detection backend adopts a design idea similar to that of the YOLO series network model. The three groups of feature maps with different scales obtained by the cross-correlation operation are input into the detection head, and dimensional transformation is performed according to the 1*1 convolutional layer with (N + 5)*3 channels (N is the object category, 5 represents the x and y coordinates of the object center point, the object length and width, and the object confidence, and 3 means that each point will regress three sizes of anchor boxes), and then the types, coordinates, length and width, and confidence of the three groups of features are predicted respectively.

[0053] In summary, the above-described few-shot object detection method for SAR images based on meta-learning can achieve the fusion of multi-scale feature information between the features of the SAR image itself and the corresponding scattering characteristic map, enabling the model to obtain more effective features; it optimizes the feature fusion part of the few-shot object detection algorithm. In particular, when calculating the M2 feature, the mixed attention representation M1 of the support set and the query set is used, which can more fully explore the feature connection between the support set and the query set, filter out the most effective features, and reduce the weight of inefficient features, making the algorithm performance indicators higher.

[0054] Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

Claims

1. A small-sample target detection method for SAR images based on meta-learning, characterized in that, It includes the following steps: S1: Data collection and production Collect base class and small sample class data, and produce corresponding support set input and query set input; S2: Model rough tuning Input the base class data of the support set and the query set into the base class object detection model, extract three groups of features with different scales of the support set and the query set respectively, perform feature aggregation through cross-correlation operation, and input it into the object detection backend for regression of categories and detection frames to obtain the roughly tuned base class object detection model; In step S2, the specific process of the cross-correlation operation is as follows: S21: First, the feature map of W / 16*H / 16*C2 obtained by the support set feature extraction model passes through two parallel 1*1 convolutional layers and undergoes dimensional transformation to obtain two feature maps: VQ2, with a size of WH / 256*C2 / 2; KQ2, with a size of WH / 256*C2 / 8; at the same time, the feature map of W / 16*H / 16*C2 obtained by the support set feature extraction model also passes through two parallel 1*1 convolutional layers and undergoes dimensional transformation to obtain two feature maps: VS2, with a size of C2 / 8×NHW / 256; KS2, with a size of NHW / 256×C2 / 2, where N is the number of all categories; S22: Perform matrix multiplication on KQ2 and VS2, and obtain the feature map M1 with a size of HW / 256×NHW / 256 through the softmax function. The feature map M1 is the mixed attention representation of the query set and the support set; S23: Perform matrix multiplication on M1 and KS2 to obtain the feature map M2 with a size of HW / 256×C2 / 2; S24: Finally, splice M2 and VQ2 to obtain the final feature map M3 with a size of HW / 256×C2; S3: Model fine tuning Use part of the base class data of the support set and the query set and all the small sample class data as input, and fine-tune the roughly tuned base class object detection model with a set learning rate to obtain the final base class - small sample class object detection model; S4: Object detection Input the SAR image to be detected into the final base class - small sample class object detection model for detection to obtain the detection result.

2. The method for small-sample target detection of SAR images based on meta-learning according to claim 1, wherein: In the said step S1, the base class is the category with a large number of samples, and the small sample class is the category with a small number of samples.

3. A small-sample target detection method for SAR images based on meta-learning according to claim 1, characterized in that: In the said step S1, the support set input includes slices of different types of targets cropped from the large SAR image, binary mask maps of the slices, and scattering characteristic maps obtained after the slices of different types of targets pass through the Harris-Laplace detector; the query set input includes slices of different types of targets cropped from the large SAR image and scattering characteristic maps obtained after the slices of different types of targets pass through the Harris-Laplace detector.

4. A small-sample target detection method for SAR images based on meta-learning according to claim 3, characterized in that: The binary mask map refers to a binary map with the same size as the support set input image, where the target area is a white rectangular frame and other areas are black.

5. A small-sample target detection method for SAR images based on meta-learning according to claim 1, characterized in that: In the step S2, the base class object detection model is a siamese network model, including a support set feature extraction model and a query set feature extraction model. Among them, the support set feature extraction model and the query set feature extraction model share weights and form a siamese network structure.

6. A small-sample target detection method for SAR images based on meta-learning according to claim 1, characterized in that: In the step S2, the object detection backend includes a detection head, a dimension transformation module, and a prediction module. The three groups of feature maps obtained by the cross-correlation operation are input into the detection head. The dimension transformation module performs dimension transformation using a 1*1 convolutional layer with a set number of channels, and then the prediction module predicts the type, coordinates, length-width, and confidence of the three groups of features respectively.

7. A small-sample target detection system for SAR images based on meta-learning, characterized in that, Using the method according to any one of claims 1 to 6 for the detection of small sample objects in SAR images, including: A data collection and production module, configured to collect base class and small sample class data and produce corresponding support set inputs and query set inputs; A model rough adjustment module, configured to input the base class data of the support set and the query set into the base class object detection model, extract three groups of features with different scales of the support set and the query set respectively, perform feature aggregation through cross-correlation operation, and input into the object detection backend for regression of categories and detection frames to obtain the roughly adjusted base class object detection model; A model fine-tuning module, configured to use part of the base class data and all small sample class data of the support set and the query set as inputs, and fine-tune the roughly adjusted base class object detection model at a set learning rate to obtain the final base class-small sample class object detection model; An object detection module, configured to input the SAR image to be detected into the final base class-small sample class object detection model for detection to obtain detection results.

Images