A synthetic aperture radar image target clustering method

Abstract

The application provides a synthetic aperture radar image target clustering method, and mainly solves the problem that the existing SAR automatic target recognition method has low utilization rate of unlabeled data and low timeliness of recognition task, and the problem that the degree of attention is insufficient. The scheme adopted by the application is to construct a multi-view SAR image sequence set based on the azimuth attribute, perform data enhancement on the sequences in the set to create positive and negative sample pairs. A pair of weight-shared feature extractors are responsible for learning sample features, a single-view feature extractor learns specific view features, a multi-view feature extractor is responsible for learning global features, and finally instance-level and class-level contrast learning is performed to generate soft labels as clustering allocation results; using the method of the application, each index is obviously improved in clustering evaluation.

Description

Technical Field

[0001] This invention belongs to the field of target clustering technology, specifically relating to a target clustering method for radar images. Background Technology

[0002] Synthetic Aperture Radar (SAR) is a microwave imaging radar with all-weather, all-day operation capabilities. Its captured images are widely used in military reconnaissance and scientific observation. In recent years, the rapid development of deep learning technology and the use of large amounts of labeled data have effectively improved the accuracy of supervised learning-based SAR automatic target recognition tasks. However, manually labeling SAR images requires specialized knowledge, and the significant time cost reduces the timeliness of target recognition tasks. Furthermore, compared to optical images, SAR images are affected by low resolution and unclear target characteristics, resulting in low inter-class image similarity and high intra-class image similarity. Directly applying contrastive learning to single-view images leads to insufficient accuracy and completeness of the captured unsupervised feature representations, resulting in low clustering performance. Existing SAR automatic target recognition methods do not adequately address the issues of low utilization of unlabeled data and low timeliness of recognition tasks, and they also pay little attention to unsupervised representation learning problems oriented towards target clustering. Summary of the Invention

[0003] To overcome the shortcomings of existing technologies and improve the timeliness of automatic target recognition tasks while effectively utilizing unlabeled data, this invention provides a synthetic aperture radar image target clustering method, comprising the following steps:

[0004] Step 1: Construct a multi-view sequence on the original image set using the azimuth attribute of SAR images;

[0005] Let the original image set X = {X 1 ,X 2 ,...,X R The sequence consists of R categories, with images in each category sorted by azimuth angle; the multi-view sequence contains k views.

[0006] The r-th category X in the original image set r The process of generating the k-view sequence is as follows: Set a window of length k+1, and place the window on X. r Slide the image from left to right with a step size of 1; combine the images in the window to obtain a k-view sequence containing k images; filter the obtained k-view sequences, retain the sequences in which the azimuth angle difference between any two images in the sequence is less than ω as network training samples, and discard duplicate sequences;

[0007] Perform the k-view sequence generation process on all categories to obtain the k-view sequence set X of the original image set X. total During network training, Xtotal The data is divided into several batches, each containing N k-view sequences, denoted as...

[0008] Step 2: Perform data augmentation to obtain two sets of augmented sequences for contrastive learning;

[0009] Based on the characteristics of SAR images, there are six image enhancement methods: cropping and scaling, grayscale conversion, horizontal flipping, color perturbation, rotation, and Gaussian filtering. Two enhancement strategies, A and B, are randomly selected from these six methods and applied to the image obtained in step 1. Obtain the enhanced sequence set and

[0010] Step 3: Feature extraction;

[0011] Single-view feature extraction methods and view feature fusion and multi-view feature extraction methods were used to extract features from the enhanced sequence set. and Extract features from;

[0012] Single-view feature extraction method:

[0013] The single-view feature extractor consists of L Transformer encoders and extracts k-view sequences. Features of each view, where This is the j-th view of the sequence;

[0014] Let the j-th view be... The size is w×h×3, where w is the width of the view and h is the height of the view. First, four convolutional blocks are used to... The convolutional blocks are reshaped into 2D patches, each consisting of a 3×3 convolutional kernel with a stride of 2, a batch normalization layer, and a ReLU activation function; then, a 1×1 convolutional kernel with a stride of 1 is used to convert the 2D patches into a patch sequence. in, Let M be a real number space of size M×D, where D = 192;

[0015] According to formula (1), in Add a learnable category tag before it. As a view The input sequence is represented by [a specific symbol] and a positional embedding E is introduced to preserve the relative positional information between the patch sequences. The single-view feature extractor is calculated using formulas (2) and (3), and will... Input into a single-view feature extractor to obtain the view Features

[0016]

[0017]

[0018]

[0019] MSA() represents multi-head self-attention, and MLP() represents multilayer perceptron.

[0020] View feature fusion and multi-view feature extraction methods:

[0021] Connect each view according to formula (4) Feature representation get And The input consists of a multi-view feature extractor composed of G Transformer encoders cascaded together. The k-view sequence is obtained by calculation according to formulas (5) and (6). Global feature representation Formula (7) classifies all views. Perform average pooling to obtain Features

[0022]

[0023]

[0024]

[0025]

[0026] Step 4: Instance-level comparative learning;

[0027] Step 3: Extract the enhanced sequence set and The features are used to obtain an enhanced sequence feature set. and Targeting features definition The set S of positive sequence feature pairs and negative sequence feature pairs is defined as follows:

[0028] S = S aa ∪S ab (8)

[0029]

[0030]

[0031] Using a three-layer fully connected neural network f I(·) Map the features to a low-dimensional subspace: Contrastive learning is performed according to the principle of maximizing the similarity of positive sequence feature pairs and minimizing the similarity of negative sequence feature pairs; Equation (11) represents the feature... Loss function:

[0032]

[0033] Where, τ S For sequence-level temperature parameters,

[0034] Calculate the enhanced sequence feature set and The loss of all features is used to obtain the sequence-level total loss function L. S :

[0035]

[0036] Step 5: Class-level comparative learning;

[0037] Using a three-layer fully connected neural network f C (·) Project the features into an R-dimensional feature vector: Where R is the total number of categories; and The soft labels generated by the softmax function represent the category to which the sequence belongs; after calculating the soft labels of 2N enhanced sequences, they are concatenated row-wise into a matrix M. A M B Columns of a matrix and Considered as the feature representation of category i under enhancement A and enhancement B; for definition The set C of positive and negative feature pairs is defined as follows:

[0038] C = C aa ∪C ab (13)

[0039]

[0040]

[0041] Contrastive learning is performed with the goal of maximizing the similarity of positive feature pairs and minimizing the similarity of negative feature pairs; Equation (16) is... Loss function:

[0042]

[0043] Where, τ C Class-level temperature parameters;

[0044] Calculate the loss function for all class feature representations to obtain the class-level total loss function L. C :

[0045]

[0046]

[0047] Where p(·) represents the clustering assignment probability function, This represents the probability that sample j belongs to category i under augmentation A;

[0048] Step 6: Jointly optimize sequence-level contrast loss L I Contrast loss L with class level C The formula for calculating the total loss L is as follows:

[0049] L = L I +L C

[0050] For X total All batches that are divided will execute steps 1-6 of the training process;

[0051] After the network training is completed, the soft labels output in the class-level contrastive learning are regarded as the clustering prediction results.

[0052] Furthermore, in step 1, ω = 45°.

[0053] Furthermore, in step 3, L = 4.

[0054] Furthermore, in step 3, G = 8.

[0055] Furthermore, in step 4, τ S Set it to 0.5.

[0056] Furthermore, in step 5, τ C Set to 1.0.

[0057] The beneficial effects of this invention are as follows: the data used in the experiment is MSTAR ten-category target data, and the target slice images are shown below. Figure 3 and Figure 4 Targets acquired at elevation angles of 15 degrees and 17 degrees were used as training samples, with azimuth angles covering 0 degrees to 360 degrees. Considering the time and space costs of training the network, multi-view sequences with 2 and 3 views were constructed. Table 1 shows the target categories, number of samples, and number of multi-view sequences constructed for different numbers of views in the original MSTAR data.

[0058] Table 1

[0059] 2S1 573 1130 1670 BMP2 428 831 1185 BRDM_2 572 1129 1668 BTR60 451 877 1263 BTR70 429 833 1188 D7 573 1131 1671 T62 571 1126 1665 T72 428 831 1197 ZIL131 573 1131 1671 ZSU_23_4 573 1131 1680

[0060] The performance of this invention was evaluated using commonly used metrics in cluster evaluation: Normalized Mutual Information (NMI), Adjusted Rand Index (ARI), and Clustering Accuracy (ACC). Higher metric values ​​indicate better clustering performance.

[0061] Table 2

[0062]

[0063] Table 2 illustrates the impact of different numbers of views on clustering performance, comparing the clustering performance of single-view, two-view, and three-view inputs. Compared to single-view input, all metrics show significant improvement when using multi-view input. The performance of the three metrics increased by 3.94%, 3.31%, and 3.90% for two-view input, respectively, while the performance of the metrics increased by 4.37%, 4.33%, and 4.95% for three-view input. As the number of views increases, the clustering performance gradually improves, demonstrating the promoting effect of the multi-view learning approach on clustering. Attached Figure Description

[0064] Figure 1 This is a flowchart of the present invention;

[0065] Figure 2 A schematic diagram of the Transformer encoder used for feature extraction;

[0066] Figure 3 SAR images of 2S1, BMP2, BRDM_2, BTR60, and BTR70 from the ten categories of MSTAR target data;

[0067] Figure 4 SAR images of D7, T62, T72, ZIL_131, and ZSU_23_4 from the MSTAR ten target data. Detailed Implementation

[0068] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0069] The main process of this invention is as follows: Figure 1 As shown, the implementation steps are as follows: First, a set of multi-view SAR image sequences based on azimuth attributes is constructed. Data augmentation is performed on the sequences within the set to create positive and negative sample pairs. A pair of weighted feature extractors is responsible for learning sample features: a single-view feature extractor learns specific view features, and a multi-view feature extractor learns global features. Finally, instance-level and class-level comparative learning is performed to generate soft labels as clustering assignment results.

[0070] A method for target clustering in synthetic aperture radar images includes the following steps:

[0071] Step 1: Construct a multi-view sequence on the original image set using the azimuth attribute of SAR images;

[0072] Let the original image set X = {X} 1 ,X 2 ,...,X R The sequence consists of R categories, with images in each category sorted by azimuth angle; the multi-view sequence contains k views.

[0073] The r-th category X in the original image set r The process of generating the k-view sequence is as follows: Set a window of length k+1, and place the window on X. r Slide the image from left to right with a step size of 1; combine the images within the window to obtain a k-view sequence containing k images; filter the obtained sequences, retaining sequences where the azimuth angle difference between any two images within the sequence is less than ω as network training samples, and discard duplicate sequences; ω = 45°

[0074] Perform the k-view sequence generation process on all categories to obtain the k-view sequence set X of the original image set X. total During network training, X total The data is divided into several batches, each containing N k-view sequences, denoted as... The following is based on The network training process is described using an example;

[0075] Step 2: Perform data augmentation to obtain two sets of augmented sequences for contrastive learning;

[0076] Based on the characteristics of SAR images, there are six image enhancement methods: cropping and scaling, grayscale conversion, horizontal flipping, color perturbation, rotation, and Gaussian filtering. Two enhancement strategies, A and B, are randomly selected from these six methods and applied to the images obtained in step 1. Obtain the enhanced sequence set and

[0077] Step 3: Feature extraction;

[0078] A pair of weight-shared feature extractors extract from the augmented sequence set respectively and Extracting features from [the data], the following example [is used]. The specific details are described using the feature extraction process as an example.

[0079] Single-view feature extraction method:

[0080] The single-view feature extractor consists of L Transformer encoders and is responsible for extracting k-view sequences. Features of each view, where This is the j-th view of the sequence; the Transformer encoder structure is as follows: Figure 2 As shown; where L = 4;

[0081] Let the j-th view be... The size is w×h×3, where w is the width of the view and h is the height of the view. First, four convolutional blocks are used to... The convolutional blocks are reshaped into 2D patches, each consisting of a 3×3 convolutional kernel with a stride of 2, a batch normalization layer, and a ReLU activation function; then, a 1×1 convolutional kernel with a stride of 1 is used to convert the 2D patches into a patch sequence. in, Let M be a real number space of size M×D, where D = 192;

[0082] As in formula (1), in Add a learnable category tag before it. As a view The input sequence is represented by [a specific symbol] and a positional embedding E is introduced to preserve the relative positional information between the patch sequences. The single-view feature extractor is calculated using formulas (2) and (3), and will... Input into a single-view feature extractor to obtain the view Features

[0083]

[0084]

[0085]

[0086] MSA() represents multi-head self-attention, and MLP() represents multilayer perceptron.

[0087] View feature fusion and multi-view feature extraction methods:

[0088] Connect each view according to formula (4) Feature representation get And The input consists of a multi-view feature extractor composed of G Transformer encoders cascaded together. The k-view sequence is obtained by calculation according to formulas (5) and (6). Global feature representation Formula (7) classifies all views. Perform average pooling to obtain Features Where G = 8;

[0089]

[0090]

[0091]

[0092]

[0093] Step 4: Instance-level comparative learning;

[0094] Step 3: Extract the augmented sequence set and The features are used to obtain an enhanced sequence feature set. and Targeting features definition The set S of positive sequence feature pairs and negative sequence feature pairs is defined as follows:

[0095] S = S aa ∪S ab (8)

[0096]

[0097]

[0098] Using a three-layer fully connected neural network f I (·) Map the features to a low-dimensional subspace: Contrastive learning is performed according to the principle of maximizing the similarity of positive sequence feature pairs and minimizing the similarity of negative sequence feature pairs; Equation (11) represents the feature... Loss function:

[0099]

[0100] Where, τ S The temperature parameter for the sequence level is set to 0.5;

[0101] Calculate the enhanced sequence feature set and The loss of all features is used to obtain the sequence-level total loss function L. S :

[0102]

[0103] Step 5: Class-level comparative learning;

[0104] Using a three-layer fully connected neural network f C (·) Project the features into an R-dimensional feature vector: Where R is the total number of categories; and The soft labels generated by the softmax function represent the category to which the sequence belongs; after calculating the soft labels of 2N enhanced sequences, they are concatenated row-wise into a matrix M. A M B Columns of a matrix and It can be regarded as the feature representation of category i under enhancement A and enhancement B; for definition The set C of positive and negative feature pairs is defined as follows:

[0105] C = C aa ∪C ab (13)

[0106]

[0107]

[0108] Contrastive learning is performed with the goal of maximizing the similarity of positive feature pairs and minimizing the similarity of negative feature pairs; Equation (16) is... Loss function:

[0109]

[0110] Where, τ C Set the class-level temperature parameter to 1.0;

[0111] Calculate the loss function for all class feature representations to obtain the class-level total loss function L. C :

[0112]

[0113]

[0114] Where p(·) represents the clustering assignment probability function, This represents the probability that sample j belongs to category i under augmentation A;

[0115] Step 6: Jointly optimize sequence-level contrast loss L I Comparison loss with class level L C The formula for calculating the total loss L is as follows:

[0116] L = L I +L C

[0117] For X total All batches that are divided will execute steps 1-6 of the training process;

[0118] After the network training is completed, the soft labels output in the class-level contrastive learning are regarded as the clustering prediction results.

Claims

1. A method for target clustering in synthetic aperture radar images, characterized in that, Includes the following steps: Step 1: Construct a multi-view sequence on the original image set using the azimuth attribute of the SAR image; Let the original image set X = {X} 1 ,X 2 ,...,X R The sequence consists of R categories, with images in each category sorted by azimuth angle; the multi-view sequence contains k views. The r-th category X in the original image set r The process of generating the k-view sequence is as follows: Set a window of length k+1, and place the window on the X... r Slide up from left to right with a step size of 1; The images within the window are combined to obtain a k-view sequence containing k images; the obtained k-view sequences are filtered, and sequences in which the azimuth angle difference between any two images is less than ω are retained as network training samples, while duplicate sequences are discarded. Perform the k-view sequence generation process on all categories to obtain the k-view sequence set X of the original image set X. total During network training, X total The data is divided into several batches, each containing N k-view sequences, denoted as... Step 2: Perform data augmentation to obtain two sets of augmented sequences for contrastive learning; Based on the characteristics of SAR images, there are six image enhancement methods: cropping and scaling, grayscale conversion, horizontal flipping, color perturbation, rotation, and Gaussian filtering. Two enhancement strategies, A and B, are randomly selected from these six methods and applied to the image obtained in step 1. Obtain the enhanced sequence set and Step 3: Feature extraction; Single-view feature extraction methods and view feature fusion and multi-view feature extraction methods were used to extract features from the enhanced sequence set. and Extract features from; Step 4: Instance-level comparative learning; Step 3: Extract the augmented sequence set and The features are used to obtain an enhanced sequence feature set. and Targeting features definition The set S of positive sequence feature pairs and negative sequence feature pairs is defined as follows: S=S aa ∪S ab (8) Using a three-layer fully connected neural network f I (·) Map the features to a low-dimensional subspace: Contrastive learning is performed based on the principle of maximizing the similarity of positive sequence feature pairs and minimizing the similarity of negative sequence feature pairs. Formula (11) represents the characteristic Loss function: Where, τ S For sequence-level temperature parameters, Calculate the enhanced sequence feature set and The loss of all features is used to obtain the sequence-level total loss function L. S : Step 5: Class-level comparative learning; Using a three-layer fully connected neural network f C (·) Project the features into an R-dimensional feature vector: Where R is the total number of categories; and The soft labels generated by the softmax function represent the category to which the sequence belongs; after calculating the soft labels of 2N enhanced sequences, they are concatenated row-wise into a matrix M. A M B Columns of a matrix and Considered as the feature representation of category i under enhancement A and enhancement B; for Definition The set C of positive and negative feature pairs is defined as follows: C=C aa ∪C ab (13) Contrastive learning is performed with the goal of maximizing the similarity of positive feature pairs and minimizing the similarity of negative feature pairs; Equation (16) is... Loss function: Where, τ C Class-level temperature parameters; Calculate the loss function for all class feature representations to obtain the class-level total loss function L. C : Where p(·) represents the clustering assignment probability function, This represents the probability that sample j belongs to category i under augmentation A; Step 6: Jointly optimize sequence-level contrast loss L I Comparison loss with class level L C The formula for calculating the total loss L is as follows: L=L I +L C For X total All batches that are divided will execute steps 1-6 of the training process; After the network training is completed, the soft labels output in the class-level contrastive learning are regarded as the clustering prediction results.

2. The synthetic aperture radar image target clustering method according to claim 1, characterized in that: In step 3, the single-view feature extraction method is as follows: The single-view feature extractor consists of L Transformer encoders and extracts k-view sequences. Features of each view, where This is the j-th view of the sequence; Let the j-th view be... The size is w×h×3, where w is the width of the view and h is the height of the view. First, four convolutional blocks are used to... The convolutional blocks are reshaped into 2D patches, each consisting of a 3×3 convolutional kernel with a stride of 2, a batch normalization layer, and a ReLU activation function; then, a 1×1 convolutional kernel with a stride of 1 is used to convert the 2D patches into a patch sequence. in, Let M be a real number space of size M×D, where D = 192; According to formula (1), in Add a learnable category tag before it. As a view The input sequence is represented by [a specific symbol] and a positional embedding E is introduced to preserve the relative positional information between the patch sequences. The single-view feature extractor is calculated using formulas (2) and (3), and will... Input into a single-view feature extractor to obtain the view Features MSA() represents multi-head self-attention, and MLP() represents multilayer perceptron.

3. The synthetic aperture radar image target clustering method according to claim 1, characterized in that: In step 3, the method for view feature fusion and multi-view feature extraction is as follows: Connect each view according to formula (4) Feature representation get And The input consists of a multi-view feature extractor composed of G Transformer encoders cascaded together. The k-view sequence is obtained by calculation according to formulas (5) and (6). Global feature representation Formula (7) provides a classification label for all views. Perform average pooling to obtain Features 4. The synthetic aperture radar image target clustering method according to claim 1, characterized in that: In step 1, ω = 45°.

5. The synthetic aperture radar image target clustering method according to claim 2, characterized in that: The value of L is 4.

6. The synthetic aperture radar image target clustering method according to claim 3, characterized in that, The value of G is 8.

7. The synthetic aperture radar image target clustering method according to claim 1, characterized in that: In step 4, τ S Set it to 0.

5.

8. The synthetic aperture radar image target clustering method according to claim 1, characterized in that: In step 5, τ C Set to 1.0.

Images