A SAR image small sample target recognition method based on a multi-scale network

Abstract

The application discloses a SAR image small sample target recognition method based on a multi-scale network, applies a machine learning framework of a twin network to a target recognition task of a SAR image, and comprises image pair sampling, training task sampling, test task sampling and the like; and a self-encoder and a multi-scale feature extraction network are used to enhance feature propagation in a model training process. The application can implement the target recognition task under the condition of the SAR image small sample, and relieves the target recognition effect that cannot be achieved by the machine learning model under the condition that the sample amount of the SAR image is extremely low.

Description

Technical Field

[0001] This invention relates to image recognition technology, specifically to a method for small-sample target recognition of SAR images based on multi-scale networks. Background Technology

[0002] Currently, synthetic aperture radar (SAR) is a radar system for Earth observation, and because it is active, it is often referred to as active imaging. Initially, this type of device was mainly installed on aircraft, spacecraft, and other flight platforms and high-altitude equipment for all-weather, all-time Earth monitoring. Later, with the development of SAR and related technologies, ground-based SAR radar, handheld SAR devices, and other equipment with various operating modes began to be widely used in various fields.

[0003] like Figure 1 As shown, image recognition requires three basic processes: image preprocessing, image feature extraction, and target classification.

[0004] In the image preprocessing stage, operations such as cropping, rotation, Gaussian filtering, median filtering, pixel grayscale conversion, and pixel binarization can be used to standardize the image input to the model, extract some useless information that may affect the training effect, and retain the target information to be recognized to the greatest extent, which facilitates the adjustment and alignment of parameters in the subsequent model training process. Image geometric transformations such as cropping and rotation can reduce useless background information in an image. Gaussian filtering uses a smoothing template and is a linear operation that performs a weighted average of the gray values ​​of the input image pixels to smooth the image signal. Median filtering is a non-linear smoothing technique based on ordinal statistics. It processes the neighborhood of a given pixel in the input image using a filtering template to obtain the median value within that neighborhood, replacing the original pixel value. This allows the gray value of the target pixel to better fit the real situation, eliminating isolated noise and reducing image noise. This denoising method is suitable for handling salt-and-pepper noise and preserves edge information. Grayscale conversion transforms the input color image from a three-channel image into a single-channel grayscale image. This reduces the number of parameters required for subsequent model calculations and speeds up model training. However, the grayscale image, like the color RGB image, still reflects the feature distribution of the entire image. The grayscale conversion formula for RGB images is as follows:

[0005]

[0006] Image features are the salient characteristics and properties that distinguish a class of objects in an image from other objects. Every image possesses features that differentiate it from others, including features naturally observable by the human eye such as brightness, edges, shape, and color, as well as features obtained through processing and transformation, such as histograms and moments. To distinguish image targets belonging to different categories, the image features extracted by the model not only need to comprehensively describe the image information, but also need to serve as a differentiator from other images. In the feature extraction module, various strategies can be employed to enhance the accuracy of feature extraction, such as attention mechanisms, multi-scale networks, dense connections, residual networks, and shared weights. However, traditional deep neural networks typically require a large amount of sample data, and the neural network model usually contains a large number of parameters to extract features from the input samples. While traditional deep neural networks can achieve relatively ideal performance in recognition tasks with a large number of samples, SAR image acquisition and labeling tasks can consume significant resources, making it impossible to obtain a large number of SAR image data samples. This leads to a significant drop in the model performance of deep learning techniques, which heavily rely on massive samples, for SAR image target recognition tasks with small sample sizes, thus giving rise to the problem of SAR image target recognition with small sample sizes. In the process of feature extraction from small samples, referencing the human ability to recognize new objects after learning from a very small number of samples, Few Shot Learning (FSL) was proposed. In the feature extraction process of FSL, by comparing various types of samples, the features between different types of samples can be extracted and differentiated, which facilitates the subsequent classification stage.

[0007] After extracting image features, the classifier calculates these features to build a classification model based on existing data. This model maps the extracted features and categorizes them into a given class. Commonly used classifiers include Linear Regression, Logistic Regression, Support Vector Machines (SVM), Decision Trees (DT), Naive Bayes Models (NBM), and Softmax. Classifiers are often divided into two categories: linear classifiers and nonlinear classifiers. Linear classifiers are fast but have poor fitting ability, while nonlinear classifiers can achieve better classification results, but their speed cannot compare with linear classifiers. Linear separability is a relatively ideal situation, but in the real world, cases that strictly follow a linear distribution are actually extremely rare. On the one hand, this is because the dimensions of real-world data are far greater than two dimensions, and as the dimensionality increases, the data distribution becomes more complex, and the success rate of achieving linear separability decreases. On the other hand, even if the original data is linearly separable, non-ideal conditions during the data acquisition process, such as non-linear factors introduced by noise, may render the originally linearly separable data unusable after acquisition.

[0008] A novel clustering objective has been proposed, which trains a neural network classifier from scratch, given only unlabeled data samples. This model achieves state-of-the-art (SOTA) results on eight unsupervised clustering benchmarks, including image classification and image segmentation, by discovering clusters that precisely match semantic categories. These benchmarks include STL10, an unsupervised learning variant of ImageNet, and CIFAR10, where the model outperforms the current state-of-the-art models by 6.6% and 9.5% absolute percentages, respectively. This method is not limited to computer vision but can learn from any dataset that can pair samples. The paper uses a random transformation to generate image pairs from all images. The trained network directly outputs semantic labels, rather than requiring external processing for high-dimensional representations used in semantic clustering. This maximizes the mutual information (MI) between each pair of sample type assignments. This information-theoretic approach avoids the degradation issues that other clustering methods are prone to.

[0009] Meta-learning is a framework for solving few-shot learning tasks under extremely scarce sample conditions. Its key step is to learn how to adapt the base learner to new tasks with only a very small number of samples, utilizing a large number of similar few-shot tasks. Because deep neural networks (DNNs) are prone to overfitting with only a few samples, meta-learning typically uses shallow neural networks (SNNs), thus limiting its model performance. Meta-Transfer Learning (MTL) has been proposed, which uses adaptive deep neural networks to perform few-shot learning tasks, learning the scaling and translation functions of the DNN weights for each task to achieve transfer learning. Furthermore, Hard-Task (HT) meta-batch learning is introduced as an effective few-shot learning objective in MTL. These two innovations contribute to faster model convergence and higher accuracy.

[0010] A new approach, Multi-scale Metric Learning (MSML), is proposed. This method extracts multi-scale features to learn multi-scale relationships between samples for the Few Shot classification task. It embeds multi-scale features into a feature pyramid structure, combining high-level features with strong semantic information with rich low-level visual features. Based on this, a Multi-scale Relation Generation Network (MRGN) is proposed for hierarchical metric learning. Features from higher levels are used for deep metric learning, while features from lower levels are applied to lightweight metric learning. Building upon MSML and MRGN, a novel loss function called Intra-class and Inter-class Relation Loss (IIRL) is designed to optimize the proposed deep network. This loss function enhances the correlation between similar sample groups and weakens the correlation between dissimilar sample groups, demonstrating good performance on the FSL problem in mini ImageNet and tiered ImageNet datasets.

[0011] SAR image labels are difficult to obtain in large quantities, resulting in a large number of small-sample SAR datasets. SAR images are full of speckle noise and unique geometric distortions, making it difficult to extract effective features by directly applying convolutional neural networks to small-sample SAR datasets. Deep learning-based methods require a large amount of training data to achieve good model performance, but labeling SAR images is a time-consuming and labor-intensive task, usually requiring expertise from the application domain. Obtaining a large amount of pre-labeled data to meet the needs of traditional machine learning in real-world applications is generally difficult. On the other hand, due to the lack of large-scale labeled SAR image datasets and the difficulty of learning from SAR images, directly applying traditional transfer learning-based methods...

[0012] Meta-learning (e.g.) is insufficient to solve the SAR target recognition problem. Summary of the Invention

[0013] The main objective of this invention is to provide a target recognition method for SAR images with small sample sizes based on multi-scale networks. This method can achieve target recognition tasks in the case of small sample sizes of SAR images, and alleviate the problem that machine learning models cannot achieve the target recognition effect when the sample size of SAR images is extremely low.

[0014] The technical solution adopted in this invention is: a method for small-sample target recognition in SAR images based on multi-scale networks, comprising:

[0015] During the input phase of the model, two images will be randomly selected and corresponding image pairs will be generated. If the selected image pairs belong to the same category, the corresponding supervision information Label = 1; if the selected image pairs belong to different categories, the corresponding supervision information Label = 0.

[0016] The selected image pairs are input into a multi-scale convolutional network. This network structure shares weight parameters and performs feature extraction on the images. After feature extraction of the dimensions of the original data input, a fully connected layer is formed to obtain the feature vector.

[0017] After obtaining the feature vectors of the corresponding dimensions, the cosine distance is used to calculate the similarity of the input images. When the similarity between the input images is high, the value of the calculated cosine similarity is close to 1, while when the similarity is low, the value is close to -1. For the input supervision information with the same category (Label=1) or different categories (Label=0), the cosine similarity is used to make the input feature vector independent of its input dimension. As the input dimension increases, the cosine similarity calculated can also maintain a similar value to that of the low-dimensional input.

[0018] The obtained cosine distance and the generated supervision information label are input into the binary classification cross-entropy loss calculation;

[0019] The model's overall loss function is used as a weighted sum of binary cross-entropy loss and mean squared error loss. Backpropagation is then performed to update the model's gradient.

[0020] Furthermore, the feature extraction network includes:

[0021] The convolutional layers used are 5×5 convolutional kernels and 3×3 convolutional kernels. Each convolutional layer includes batch normalization and unbiased convolution. The pooling layers all use 2×2 max pooling. This type of pooling layer learns features such as the edge and texture structure of the image by selecting local maxima.

[0022] By comparing each element within the 2×2 pooling template, the maximum value is extracted to obtain the local main features;

[0023] The batch normalization process is as follows: First, the mean of the input sample batch is calculated.

[0024]

[0025] m is the number of samples in the batch input to the feature extraction network, x i For each sample in a batch;

[0026] Calculate the variance of the samples in this batch:

[0027]

[0028] Where μ β This is the mean of the batches obtained in the steps above;

[0029] Normalize each sample in the batch to make it conform to a normal distribution:

[0030]

[0031] Where μ β and These are the mean and variance of the batch obtained in the previous step, respectively, and ε is a value greater than 0 for performance.

[0032] In the following convolution operations, each step will uniformly include convolution operations and batch normalization operations with corresponding parameters and weights. When using batch normalization operations to calculate the input information, the mean and variance of the input information are used to correct and normalize the parameters.

[0033] Before inputting the image into the multi-scale convolutional neural network, the image is first cropped to a uniform 100×100 grayscale image and then fed into the first convolutional layer. This first convolutional layer is a 5×5 convolution with 16 filters and a stride of 2, with whitespace padding. A 2×2 max-pooling layer is then used to downsample the image, resulting in a feature map with dimensions of 16×25×25. A 1×1 convolution then linearly transforms the channels of this feature map, yielding a 128-dimensional 25×25 feature map, which is used as the network output for the first scale. The second scale convolutional layer uses 32 filters. The first scale uses a 3×3 convolution with a stride of 1, padded with whitespace, followed by 2×2 max pooling to obtain a 32×12×12 feature map. This is then followed by a 1×1 convolution to obtain a 128-dimensional 12×12 feature map, which serves as the network output for the second scale. The third scale feature map uses three convolutional layers and one pooling layer. It first passes through a 3×3 convolution with 64 filters and a stride of 1, padded with whitespace, followed by 2×2 max pooling, and then through another 3×3 convolution with 128 filters and a 5×5 convolution, resulting in a 128×1×1 feature map, which is then output as the third scale feature map.

[0034] Before concatenating feature maps of multiple granularities, the feature maps of each granularity are set to 128×625, 128×144, and 128×1, respectively. Then, the feature maps are concatenated based on the channel dimension, resulting in a feature map dimension of 128×770.

[0035] Furthermore, the SAR image small sample target recognition method based on multi-scale networks further includes inputting the feature vector encoded by the multi-scale convolutional neural network into the decoder module of the autoencoder, specifically:

[0036] Given a training set S = {x i |x i ∈R d}, i∈[1,n], the autoencoder will perform the following operations on the input training set:

[0037] z = f(w) e ,b e ;x)

[0038] r = g(w d ,b d ;z)

[0039] Where f and g are typically encoder and decoder functions implemented by a neural network, w e and b e Let w be the weight matrix and bias vector of the encoder. d and b dHere are the weight matrix and bias vector of the decoder;

[0040] The loss function is a weighted loss function, and its expression is as follows:

[0041]

[0042] Where x1 and x2 represent the sample images input into the Siamese network, in the form of pixel matrices; label is the generated supervision information, 1 if the images belong to the same category, and 0 if they belong to different categories; α is a manually defined weight hyperparameter used to specify the weight ratio of the mean squared error loss, i.e., the image reconstruction loss; CosD(x1,x2) is the cosine distance, and its expression is as follows:

[0043]

[0044] BL(f(x),y) is the binary cross-entropy loss, and its expression is as follows:

[0045] BL(f(x),y)=-(ylogf(x)+(1-y)log(1-f(x)))

[0046] Where f(x) is the model prediction value, and y is the supervision information of the sample;

[0047] The mean squared error loss is expressed as follows:

[0048]

[0049] Where x and These are the original image and the reconstructed image after autoencoder reconstruction, respectively, and are pixel matrices; x i and x' i These are the gray values ​​of the corresponding pixels in the pixel matrix, where n is the number of pixels in the pixel matrix; the final output is the image reconstruction loss of the autoencoder.

[0050] After outputting the weighted loss function, backpropagation is performed on the model, and the ADAM adaptive optimizer is used for gradient updates and model optimization.

[0051] Furthermore, the SAR image small sample target recognition method based on multi-scale networks further includes the following steps: during training and testing, the model randomly selects images from the training and testing sets to generate image pairs, and combines the selected image pairs to generate the input format required by the Siamese network; during model training, the labeled input is used to perform gradient descent in the BCE loss function; when testing the trained model, the model calculates the similarity of the three input image pairs, and determines the image pair with the highest similarity. If the first image pair has the highest similarity, the model predicts accurately; if other image pairs have the highest similarity, the model predicts incorrectly.

[0052] Advantages of this invention:

[0053] This invention enables target recognition in SAR image scenarios with small sample sizes, alleviating the limitations imposed by SAR limitations.

[0054] The target recognition effect that machine learning models cannot achieve when the image sample size is extremely low.

[0055] In addition to the objectives, features, and advantages described above, the present invention has other objectives, features, and advantages. The invention will now be described in further detail with reference to the figures. Attached Figure Description

[0056] The accompanying drawings, which form part of this application, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0057] Figure 1 This is a diagram illustrating the image recognition process of the present invention;

[0058] Figure 2 This is the basic structure of the self-encoder of the present invention;

[0059] Figure 3 This is the overall structure of the SAR image small sample recognition model of the present invention;

[0060] Figure 4 This invention relates to a multi-scale convolutional neural network;

[0061] Figure 5 This is a schematic diagram of the maximum pooling method of the present invention;

[0062] Figure 6 This is a flowchart of the decoder for the asymmetric autoencoder of the present invention;

[0063] Figure 7 This is a diagram of the ReLU activation function of the present invention;

[0064] Figure 8This is a diagram of the Tanh activation function of the present invention;

[0065] Figure 9 This is the MSTAR dataset of this invention. Detailed Implementation

[0066] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0067] (I) Overall Structure of the Invention

[0068] The SAR image small sample recognition model proposed in this invention uses asymmetric autoencoders, Siamese networks, and multi-scale convolutional neural networks. Its overall network structure is as follows: Figure 3 As shown.

[0069] During the input phase of the model, two images will be randomly selected and corresponding image pairs will be generated. If the selected image pairs belong to the same category, the corresponding supervision information Label = 1; if the selected image pairs belong to different categories, the corresponding supervision information Label = 0.

[0070] The selected image pairs are input into a feature extraction network based on a Siamese structure, namely the multi-scale convolutional network selected in this invention. This network structure shares weight parameters and performs feature extraction on the images. After feature extraction of the dimensions of the original data input, a fully connected layer is formed to obtain the feature vector.

[0071] After obtaining the feature vectors of the corresponding dimensions, cosine distance is used to calculate the similarity of the input images. When the similarity between input image pairs is high, the calculated cosine similarity value is close to 1, while when the similarity is low, the value is close to -1. For the input supervision information with the same category (Label=1) or different categories (Label=0), using cosine similarity makes the input feature vector independent of its input dimension. As the input dimension increases, the calculated cosine similarity value can maintain a similarity to that of the low-dimensional input. This avoids the problem of excessively large distance values ​​for calculated image pairs due to excessively large input images and too many input features, which could lead to data overflow when calculating the loss function.

[0072] Next, the obtained cosine distance and the generated supervision information label are input into the binary cross entropy loss calculation (BCE Loss).

[0073] Finally, the weighted sum of the binary cross-entropy loss and the mean squared error loss is used as the overall loss function of the model for backpropagation to update the gradient of the model.

[0074] (II) Feature Extraction Network

[0075] like Figure 4 As shown in the multi-scale convolutional neural network, the feature extraction network used in this invention is a multi-scale convolutional neural network. The main convolutional layers used are 5×5 convolutional kernels and 3×3 convolutional kernels with relatively small receptive fields. Each convolutional layer includes batch normalization (BN) and unbiased convolution. The pooling layers all use 2×2 max pooling. This type of pooling layer learns the edge and texture structure features of the image by selecting the local maximum value. This feature can be regarded as the most important feature in the local feature map. It can reduce parameters and computation while retaining the main features, enhance the network's ability to learn image features, prevent overfitting, and is used for downsampling operations on images.

[0076] A schematic diagram of the max pooling process is shown below. Figure 5 As shown, taking 2×2 max pooling with stride=2 as an example, the maximum value is extracted by comparing each element in the 2×2 pooling template to obtain the local principal features.

[0077] The batch normalization process is as follows: First, the mean of the input sample batch is calculated.

[0078]

[0079] m is the number of samples in the batch input to the feature extraction network, x i These are the individual samples in the batch.

[0080] Calculate the variance of the samples in this batch:

[0081]

[0082] Where μ β This is the mean of the batch obtained in the above steps.

[0083] The third step is to normalize each sample in the batch to make it conform to a normal distribution:

[0084]

[0085] Where μ β and These are the mean and variance of the batch obtained in the previous steps, respectively. ε is a value greater than 0 for performance, which can prevent errors in the program calculation caused by the denominator being 0.

[0086] In the convolution operations described below, each step will uniformly include convolution operations with corresponding parameters and weights, as well as batch normalization operations. When using batch normalization to calculate the input information, the mean and variance of the input information will be used to correct and normalize the parameters in order to enhance the generalization performance of the model.

[0087] Before inputting the image into the multi-scale convolutional neural network, the image is first cropped to a uniform 100×100 grayscale image and then fed into the first convolutional layer. This first convolutional layer is a 5×5 convolution with 16 filters and a stride of 2, with whitespace padding. A 2×2 max-pooling layer is then used to downsample the image, resulting in a feature map with dimensions of 16×25×25. A 1×1 convolution then linearly transforms the channels of this feature map, yielding a 128-dimensional 25×25 feature map, which is used as the network output for the first scale. The second scale convolutional layer uses 32 filters. The first scale uses a 3×3 convolution with a stride of 1, padded with space, followed by 2×2 max pooling to obtain a 32×12×12 feature map. This is then followed by a 1×1 convolution to obtain a 128-dimensional 12×12 feature map, which serves as the network output for the second scale. The third scale feature map uses three convolutional layers and one pooling layer. First, it passes through a 3×3 convolution with 64 filters and a stride of 1, padded with space. After 2×2 max pooling, it passes through another 3×3 convolution with 128 filters and a 5×5 convolution, resulting in a 128×1×1 feature map, which is then output as the third scale feature map.

[0088] Before concatenating feature maps of multiple granularities, the feature maps of each granularity are set to 128×625, 128×144, and 128×1, respectively. Then, the feature maps are concatenated based on the channel dimension, resulting in a feature map dimension of 128×770.

[0089] While inputting the feature vector into the cosine distance calculation module, it is also necessary to input the feature vector into the decoder module in the autoencoder.

[0090] The encoder part of the autoencoder module in this network consists of a feature extraction network, namely a multi-scale convolutional network. The hidden layers are the feature vectors, which extract image features from the original input image and store them in the feature vectors, achieving data dimensionality reduction and feature encoding. The decoder part is... Figure 6The Decoder in the overall structure of the SAR image small sample recognition model, together with the multi-scale convolutional network and feature vectors, constitutes the autoencoder module proposed in this invention. An autoencoder is an unsupervised learning unit based on feature reconstruction. During model training, it reconstructs the input image and uses the mean squared error loss (MSE loss) function to update the model's gradient, thereby enhancing the features during model training and reducing the impact of image noise.

[0091] (III) Self-encoder

[0092] Given a training set S = {x i |x i ∈R d}, i∈[1,n], the autoencoder will perform the following operations on the input training set:

[0093] z = f(w) e ,b e ;x)

[0094] r = g(w d ,b d ;z)

[0095] Where f and g are typically encoder and decoder functions implemented by a neural network, w e and b e Let w be the weight matrix and bias vector of the encoder. d and b d Here are the weight matrix and bias vector of the decoder.

[0096] During the training of an autoencoder, its loss function is typically calculated using the following expression:

[0097]

[0098] Where θ=(w e ,b e ;w d ,b d ).

[0099] The autoencoder proposed in this paper is an asymmetric autoencoder structure, in which the encoder structure is a multi-scale convolutional neural network, which is a convolutional encoder; the hidden layer is the feature vector output by the convolutional encoder, which will not be elaborated here.

[0100] The decoder section of this patent primarily employs fully connected layers, the ReLU (Rectified Linear Unit) activation function, and the Tanh (Hyperbolic Tangent) activation function. This decoder structure differs from the encoder section, thus constituting an asymmetric self-encoder. The decoder structure is as follows: Figure 5The decoder section of the asymmetric autoencoder is shown.

[0101] In the decoder structure, the feature vectors from the input hidden layers are decoded and then fed into fully connected layers layer by layer. The number of parameters in each fully connected layer of the decoder increases progressively, amplifying the number of features in the input feature vectors from m to 10000. Nonlinear factors are introduced using the ReLU activation function and the Tanh hyperbolic tangent activation function. After obtaining a feature vector with a dimension of 10000, it is reconstructed into a 100×100 image, which serves as the decoder's output. The ReLU activation function expression is as follows:

[0102]

[0103] Where x is the input to the ReLU activation function, and in this patent, it refers to the feature map input into it, which is a feature matrix, and its function graph is as follows: Figure 7 As shown.

[0104] The Tanh activation function expression is as follows:

[0105]

[0106] Where x is the input to the Tanh activation function, which is the same as the ReLU activation function. In this paper, it refers to the feature map input into it, which is a feature matrix. Its function graph is as follows: Figure 8 As shown.

[0107] In the process of feature learning, a relatively deep model is used to extract features from the image and output a low-dimensional mapping of high-dimensional data. Due to the limitation of the dimension of the hidden layer, it can only extract the best features and output them first, so that the model structure can learn the best features of each layer.

[0108] (iv) Loss Function Calculation

[0109] The loss function used in this algorithm is a weighted loss function, the expression of which is as follows:

[0110]

[0111] Where x1 and x2 represent the sample images input into the Siamese network, in the form of pixel matrices; label is the generated supervision information, 1 if the images belong to the same category, and 0 if they belong to different categories; α is a manually defined weight hyperparameter used to specify the weight ratio of the mean squared error loss, i.e., the image reconstruction loss; CosD(x1,x2) is the cosine distance, and its expression is as follows:

[0112]

[0113] BL(f(x),y) is the binary cross-entropy loss, and its expression is as follows:

[0114] BL(f(x),y)=-(ylogf(x)+(1-y)log(1-f(x)))

[0115] Where f(x) is the model prediction value, and y is the supervision information of the sample.

[0116] The mean squared error loss is expressed as follows:

[0117]

[0118] Where x and These are the original image and the reconstructed image after autoencoder reconstruction, respectively, and are pixel matrices; x i and x' i These are the gray values ​​of the corresponding pixels in the pixel matrix, where n is the number of pixels in the pixel matrix; the final output is the image reconstruction loss of the autoencoder.

[0119] After outputting the weighted loss function, backpropagation is performed on the model, and the ADAM adaptive optimizer is used for gradient updates and model optimization.

[0120] (v) Sampling process for training and testing tasks

[0121] During training and testing, the model randomly selects images from the training and testing sets to generate image pairs. These image pairs are then combined to generate the input format required by the Siamese network. For example, the "3way-1shot" task will extract three sets of image pairs, totaling six images, to form either the training or testing task. Only the first image pair consists of images from the same category, while the other two pairs come from different categories. The generated label format is [1 0 0]. When training the model, these labels are used as input into the BCE loss function for gradient descent. When testing the trained model, the model calculates the similarity between the three input image pairs and determines the pair with the highest similarity. If the first image pair has the highest similarity, the model's prediction is accurate; otherwise, if any other pair has the highest similarity, the model's prediction is incorrect.

[0122] This invention applies the machine learning framework of Siamese networks to the target recognition task of SAR images, including image pair sampling, training task sampling, and testing task sampling;

[0123] Use autoencoders and multi-scale feature extraction networks to enhance feature propagation during model training;

[0124] A weighted loss function that combines classification loss and reconstruction loss.

[0125] This invention

[0126] Using Siamese networks, when training and predicting image inputs, two randomly selected images are used to form image pairs, which are then input into the network. This eliminates the need to retrain the model when new categories need to be added or removed from the data. Furthermore, in few-shot tasks, it can leverage the strong discriminative power learned on the initial sample set to generalize the network's predictive capabilities. This structure is applicable not only to new data but also to new categories from unknown distributions.

[0127] An autoencoder is an artificial neural network used to encode unlabeled data. It verifies and improves the encoder's performance by regenerating the input from the generated encoding. During network training, the input sample X is used as supervision to guide the network model to learn a mapping relationship, resulting in a reconstructed output XR. The structure of an autoencoder mainly consists of two parts: an encoder and a decoder. The encoder encodes the high-dimensional input X to obtain a low-dimensional latent variable h. This latent variable contains the features that best represent the original input X, allowing the neural network to learn only the most informative features from the input samples. The decoder restores the high-dimensional latent variable h obtained by the encoder back to the original input, rather than perfectly replicating the original input image. It discards irrelevant information from the original sample and retains the most relevant parts of the data in the original sample, such as... Figure 2 The structure of the self-encoder is shown.

[0128] Multi-scale feature extraction network. The feature extraction network proposed in this paper is a multi-scale convolutional neural network.

[0129] The image feature extraction network of this invention can be replaced by classic network models such as ResNet (Residual Network) and DenseNet (Dense Connection Network).

[0130] The algorithm of this invention is universal and robust. It can meet the requirements of small-sample SAR image recognition tasks across different sample categories.

[0131] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for small-sample target recognition in SAR images based on multi-scale networks, characterized in that, include: In the input stage of the multi-scale network, two images will be randomly selected and corresponding image pairs will be generated. If the selected images belong to the same category, the corresponding supervision information Label = 1; if the selected images belong to different categories, the corresponding supervision information Label = 0. The selected image pairs are input into a multi-scale network. This network structure shares weight parameters and performs feature extraction on the images. After feature extraction of the dimensions of the original data input, a fully connected layer is formed to obtain the feature vector. After obtaining the feature vectors of the corresponding dimensions, the cosine distance is used to calculate the similarity of the input images. When the similarity between the input images is high, the value of the calculated cosine similarity is close to 1, while when the similarity is low, the value is close to -1. For the input supervision information with the same category (Label=1) or different categories (Label=0), the cosine similarity is used to make the input feature vector independent of its input dimension. As the input dimension increases, the cosine similarity calculated can also maintain a similar value to that of the low-dimensional input. The obtained cosine distance and the generated supervision information label are input into the binary classification cross-entropy loss calculation; The weighted sum of binary cross-entropy loss and mean squared error loss is used as the overall loss function of the multi-scale network for backpropagation to update the gradient of the multi-scale network. Feature extraction networks include: The convolutional layers used are 5×5 convolutional kernels and 3×3 convolutional kernels. Each convolutional layer includes batch normalization and unbiased convolution. The pooling layers all use 2×2 max pooling. This type of pooling layer learns features such as the edge and texture structure of the image by selecting local maxima. By comparing each element within the 2×2 pooling template, the maximum value is extracted to obtain the local main features; The feature extraction network is a multi-scale convolutional neural network; The feature vector encoded by the multi-scale convolutional neural network is then input into the decoder module of the autoencoder.

2. The SAR image small sample target recognition method based on multi-scale networks according to claim 1, characterized in that, The feature extraction network includes: The batch normalization process is as follows: First, the mean of the input sample batch is calculated. m is the number of samples in the batch input to the feature extraction network, x i For each sample in a batch; Calculate the variance of the samples in this batch: Where μ β This is the mean of the batches obtained in the steps above; Normalize each sample in the batch to make it conform to a normal distribution: Where μ β and These are the mean and variance of the batch obtained in the previous step, respectively, and ε is a value greater than 0 for performance. In the following convolution operations, each step will uniformly include convolution operations and batch normalization operations with corresponding parameters and weights. When using batch normalization operations to calculate the input information, the mean and variance of the input information are used to correct and normalize the parameters. Before inputting the image into the multi-scale convolutional neural network, the image is first cropped to a uniform 100×100 grayscale image and then fed into the first convolutional layer. This first convolutional layer is a 5×5 convolution with 16 filters and a stride of 2, with whitespace padding. A 2×2 max-pooling layer is then used to downsample the image, resulting in a feature map with dimensions of 16×25×25. A 1×1 convolution then linearly transforms the channels of this feature map, yielding a 128-dimensional 25×25 feature map, which is used as the network output for the first scale. The second scale convolutional layer uses 32 filters. The first scale uses a 3×3 convolution with a stride of 1, padded with whitespace, followed by 2×2 max pooling to obtain a 32×12×12 feature map. This is then followed by a 1×1 convolution to obtain a 128-dimensional 12×12 feature map, which serves as the network output for the second scale. The third scale feature map uses three convolutional layers and one pooling layer. It first passes through a 3×3 convolution with 64 filters and a stride of 1, padded with whitespace, followed by 2×2 max pooling, and then through another 3×3 convolution with 128 filters and a 5×5 convolution, resulting in a 128×1×1 feature map, which is then output as the third scale feature map. Before concatenating feature maps of multiple granularities, the feature maps of each granularity are set to 128×625, 128×144, and 128×1, respectively. Then, the feature maps are concatenated based on the channel dimension, resulting in a feature map dimension of 128×770.

3. The SAR image small sample target recognition method based on multi-scale networks according to claim 1, characterized in that, The step of inputting the feature vector encoded by the multi-scale convolutional neural network into the decoder module of the autoencoder is specifically as follows: Given a training set S = {x i |x i ∈R d }, i∈[1,n], the autoencoder will perform the following operations on the input training set: z=f(w e ,b e ;x) r=g(w d ,b d ;z) Where f and g are typically encoder and decoder functions implemented by a multi-scale convolutional neural network, w e and b e Let w be the weight matrix and bias vector of the encoder. d and b d Here are the weight matrix and bias vector of the decoder; The loss function is a weighted loss function, and its expression is as follows: Where x1 and x2 represent the sample images input into the Siamese network, in the form of pixel matrices; label is the generated supervision information, 1 if the images belong to the same category, and 0 if they belong to different categories; α is a manually defined weight hyperparameter used to specify the weight ratio of the mean squared error loss, i.e., the image reconstruction loss; CosD(x1,x2) is the cosine distance, and its expression is as follows: BL(f(x),y) is the binary cross-entropy loss, and its expression is as follows: BL(f(x),y)=-(ylogf(x)+(1-y)log(1-f(x))) Where f(x) is the multi-scale network prediction value, and y is the supervision information of the sample; The mean squared error loss is expressed as follows: Where x and These are the original image and the reconstructed image after autoencoder reconstruction, respectively, and are pixel matrices; x i and x′ i These are the gray values ​​of the corresponding pixels in the pixel matrix, where n is the number of pixels in the pixel matrix; the final output is the image reconstruction loss of the autoencoder. After outputting the weighted loss function, backpropagation is performed on the multi-scale network, and the ADAM adaptive optimizer is used for gradient updates and multi-scale network optimization.

4. The SAR image small sample target recognition method based on multi-scale networks according to claim 1, characterized in that, This also includes the following steps during training and testing: the multi-scale network randomly selects images from the training and testing sets to generate image pairs, and combines these selected image pairs to generate the input format required by the Siamese network; during training, labeled inputs are used to perform gradient descent using the BCE loss function; when testing with the trained multi-scale network, the multi-scale network will calculate the similarity of the three input image pairs, and determine the image pair with the highest similarity. If the first image pair has the highest similarity, the multi-scale network predicts accurately; if other image pairs have the highest similarity, the multi-scale network predicts incorrectly.

Images