Multi-band SAR image pixel-level ground feature classification method

Abstract

The invention provides a multi-band SAR image pixel-level ground feature classification method. The method comprises the following implementation steps: acquiring a training and testing sample set; constructing a multi-band SAR image pixel-level ground feature classification network model and carrying out iterative training on the multi-band SAR image pixel-level ground feature classification network model; and obtaining a pixel-level ground feature classification result of the multi-band SAR image. According to the invention, each feature weighting fusion module performs weighting on a pixel-level feature cube, and CNN performs depth feature extraction on the weighted feature cube, so that feature vectors with enhanced useful features and suppressed interference features can be obtained; the encoder carries out multi-scale full-band semantic information extraction on a sequence composed of all feature vectors and a classification vector so as to capture the global long-distance dependency relationship of all bands, and organic combination of local features and full-band joint features of each band is realized. The defect that in the prior art, the capacity of capturing the global semantic relation and the cross-band long-distance dependency relation is poor is overcome, and the classification precision is effectively improved.

Description

A pixel-level ground feature classification method for multi-band SAR images Technical Field

[0001] This invention belongs to the field of radar remote sensing image processing technology, and specifically relates to a pixel-level land feature classification method for multi-band SAR images, which can be used for land resource surveys, environmental monitoring, and disaster emergency assessment. Background Technology

[0002] SAR image land cover classification is a core component of intelligent processing and refined interpretation of remote sensing images. This task, by extracting key features such as polarimetric scattering and spatial texture, accurately identifies and labels land cover types, and has been widely applied in various fields such as natural resource monitoring, urban planning, crop yield estimation, and disaster emergency response. With the maturity of multi-band detection technology, combining data from different bands such as S, C, and L has become an important means to improve classification reliability. Utilizing the differences in penetration and scattering mechanisms between bands, the physical properties of complex land cover can be more comprehensively characterized.

[0003] For example, patent application with publication number "CN 114758238 A" and titled "A Method for Classifying Ground Features in Airborne Multi-Dimensional Synthetic Aperture Radar Images" discloses a deep learning-based method for airborne multi-dimensional...

[0004] This invention presents a method for land cover classification in synthetic aperture radar (SAR) images. It constructs separate land cover classification models based on the HRNet semantic segmentation network for each dimension of the image, utilizing multi-scale feature fusion for training and feature extraction. The sensitivity of each model to different land cover categories is evaluated using a test set. Finally, the model results are fused by weighted stacking of test probabilities, using the optimal result of a single band as a benchmark, combined with a category priority strategy based on the intersection-union ratio (IUGR). This approach resolves pixel category conflicts and improves land cover classification accuracy. However, this invention employs a pure CNN network architecture, which is relatively weak in capturing global semantic connections and long-distance dependencies across bands, hindering further improvements in classification accuracy. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the existing technology and propose a pixel-level land cover classification method for multi-band SAR images to solve the technical problem of low classification accuracy in the existing technology.

[0006] To achieve the above objectives, the technical solution adopted by the present invention includes the following steps:

[0007] (1) Obtain the training sample set and the test sample set:

[0008] For the acquisition of Each land cover category The pseudo-color three-channel SAR images of each band were preprocessed, and the pixels of the multiple grayscale SAR image slices obtained after preprocessing in each band were labeled. Then, a training sample set including feature cubes of multiple pixels and their labels, and a test sample set including multiple feature cubes were constructed. , ;

[0009] (2) Construct a multi-band SAR image pixel-level ground feature classification network model based on CNN-Transformer:

[0010] Construct a multi-band SAR image pixel-level ground feature classification network model that includes cascaded feature embedding modules, a Transformer encoder, and an MLP classification head. The feature embedding module includes parallel arrangements of Each branch includes a cascaded feature weighted fusion module and a convolutional CNN module.

[0011] (3) Iteratively train the pixel-level land cover classification network model for multi-band SAR images:

[0012] A pixel-level ground cover classification network model for multi-band SAR images was trained using a sample set. Iterative training is performed to obtain a well-trained land cover classification network model. ;

[0013] (4) Obtain pixel-level land cover classification results from multi-band SAR images:

[0014] The test sample set is used as the trained ground cover classification network model. The input is forward propagated to obtain the pixel-level classification results of ground features in the multi-band SAR image corresponding to each test sample.

[0015] Compared with the prior art, the present invention has the following advantages:

[0016] In the land cover classification network model of this invention, each feature weighting and fusion module weights the pixel-level feature cubes. The CNN performs deep feature extraction and downsampling on the weighted feature cubes, which can obtain feature vectors with enhanced useful features and suppressed interfering features. The Transformer encoder extracts multi-scale full-band semantic information from the sequence containing all feature vectors and a classification vector to capture the global long-distance dependencies of all bands. This achieves the organic combination of local features of each band and joint features of the whole band, avoiding the shortcomings of existing technologies in capturing global semantic connections and long-distance dependencies across bands, and effectively improving classification accuracy. Attached Figure Description

[0017] Figure 1 is a flowchart of the implementation of the present invention.

[0018] Figure 2 is a schematic diagram of the classification network model of the present invention.

[0019] Figure 3 is a schematic diagram of the feature weighted fusion module of the present invention. Detailed Implementation

[0020] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0021] Referring to Figure 1, the present invention includes the following steps:

[0022] Step 1) Obtain the training sample set and the test sample set:

[0023] The included high-resolution airborne multidimensional SAR ground feature classification dataset Each land cover category The pseudo-color three-channel SAR images of each band are preprocessed, and the pixels of the multiple grayscale SAR image slices obtained after preprocessing of each band are labeled. At the same time, the feature cube of each pixel is obtained, and then... Feature cubes for each category in grayscale SAR image slices of each band, according to their proportion of all feature cubes in that category. Sampling is performed, and the sampled feature cubes and their labels are combined to form a training sample set; for the remaining feature cubes under each category, they are sorted according to their proportion of all remaining feature cubes in that category. Sampling is performed, and the sampled feature cubes are used to form a test sample set. In this embodiment... =0.5%, =0.2%, =3, =9.

[0024] For the acquisition of Each land cover category The steps for preprocessing pseudo-color three-channel SAR images in each band are as follows:

[0025] The pseudo-color three-channel SAR images corresponding to the C, S, and L bands in the dataset are segmented, with the C band as the baseline, and the slice size is set to [value missing]. For segments not divisible by a factor of 50, black borders are used to pad them. The size of the segments in other bands is scaled proportionally to the C-band segment size, and segments not divisible by a factor of 50 are also padded with black borders. Each pseudo-color three-channel SAR image segment is then converted into a single-channel grayscale SAR image segment. The effective pixel percentage of all grayscale SAR image segments is statistically analyzed, and segments with more than 50% black borders are removed to prevent interference with subsequent training. This results in 82 grayscale SAR image segments with less than 50% black borders for each band. Except for the C-band, all grayscale SAR image segments in other bands are upsampled using bicubic interpolation to unify their size. All grayscale SAR image slices were quantized to a uniform 16 gray levels. This resulted in 82 grayscale SAR image slices for each band, after being resized and quantized.

[0026] The method for obtaining the feature cube of each pixel is as follows:

[0027] (1a) Construct a gray-level co-occurrence matrix for each pixel in each gray-level SAR image slice after preprocessing:

[0028] Set the neighborhood window size to Then the radius Use mirror filling to fill the size of Original grayscale SAR image slice Expand to size is of .for any pixel in In extended image Extract from the middle Centered A local window of a certain size, denoted as Initial size is All-zero matrix Define a set of offsets in four directions. Then, the following nested traversal is performed to update the matrix. :

[0029] First-level traversal: Traversal window Each pixel within ;

[0030] Second level traversal: targeting the current pixel traverse the set Each offset in Calculate the corresponding neighborhood coordinates If the neighborhood coordinates Still located Internally, the grayscale value of the current pixel is obtained respectively. grayscale values ​​of neighboring pixels and the matrix Perform an incremental update to obtain pixels. The corresponding gray-level co-occurrence matrix :

[0031] ;

[0032] ;

[0033] (1b) Normalization yields the probability distribution matrix. :

[0034] ;

[0035] in This indicates the operation of taking the 1-norm.

[0036] (1c) Probability distribution matrix of each pixel based on the original grayscale SAR image slice Calculate the texture features of this pixel, including the mean. ,variance Contrast Heterogeneity Homogeneity Second moment Correlation Entropy The eight texture features are then arranged sequentially to obtain an 8-dimensional feature vector. The feature vector of each pixel is then arranged according to its position within the original grayscale SAR image slice, resulting in an 8-channel feature map of the same size as the original grayscale SAR image slice.

[0037] ;

[0038]

[0039]

[0040]

[0041]

[0042]

[0043]

[0044]

[0045] (1d) Set the neighborhood window size to The feature map obtained in step (1c) is filled using the same method as in step (1a) to obtain the filled feature map. For each pixel to be classified in the original grayscale SAR image slice... In the filled feature map, locate the pixel at the same position, and open up a new area centered on that pixel. The size of the neighborhood yields a... The feature cube, which is essentially the pixel to be classified. High-dimensional feature representation.

[0046] Step 2) Construct a multi-band SAR image pixel-level ground feature classification network model based on CNN-Transformer, the structure of which is shown in Figure 2:

[0047] Construct a multi-band SAR image pixel-level ground feature classification network model that includes cascaded feature embedding modules, a Transformer encoder, and an MLP classification head. The feature embedding module includes parallel arrangements of Each branch includes a cascaded feature weighted fusion module and a convolutional CNN module.

[0048] The feature embedding module, whose structure is shown in Figure 3, includes stacked global average pooling layers, The CNN consists of convolutional layers, ReLU activation function layers, and softmax activation function layers, with the input and output of the feature weighted fusion module connected by multiplication. The convolutional module comprises three cascaded sub-modules, the first two of which consist of two stacked layers. A convolutional layer and a pooling layer, the last submodule consisting of stacked layers. Convolutional layers and pooling layers.

[0049] The Transformer encoder includes four cascaded self-attention coding modules. Each self-attention coding module contains a cascaded multi-head self-attention module and a feedforward neural network module, as well as a first normalization module and a second normalization module connected to the outputs of the two modules respectively. The input and output of the multi-head self-attention module are residually connected, and the input and output of the feedforward neural network module are residually connected.

[0050] Step 3) Iteratively train the pixel-level ground feature classification network model for multi-band SAR images:

[0051] A pixel-level ground cover classification network model for multi-band SAR images was trained using a sample set. Iterative training is performed to obtain a well-trained land cover classification network model. The specific steps are as follows:

[0052] (3a) Initialize the number of iterations to be The maximum number of iterations is , , No. Sub-iteration land cover classification network model The weights and bias parameters are respectively , and order ;

[0053] (3b) The feature embedding module performs weighted summation and deep feature extraction on the feature cubes of each band to obtain three feature vectors; the Transformer encoder performs multi-scale full-band semantic information extraction on the vector sequence of length 4 composed of the three feature vectors extracted by the feature embedding module and one classification vector; the MLP classification head performs linear projection and Softmax normalization on the extracted feature vectors containing full-band semantic information to obtain the probability distribution of each sample belonging to the nine land cover categories. The specific method is as follows:

[0054] (3b1) The feature embedding module performs weighted summation and deep feature extraction on the feature cubes of each band:

[0055] (3b11) The feature weighted fusion module encodes the feature cube of each band to obtain the weight vector. and will With size Feature cube Multiplying them together yields a weighted feature cube. :

[0056] ;

[0057] ;

[0058] ;

[0059] ;

[0060] in, For element-wise multiplication, For example, ReLU represents the activation function. This represents the convolution operation. This is the global average pooling function. For feature cube The elements in .

[0061] (3b12) The convolutional module CNN processes the weighted feature cube Perform deep feature extraction:

[0062] After two kernels of size Furthermore, the convolutional layers with a stride of 1 have 16 and 32 kernels respectively, followed by a... The pooling layer is downsampled to obtain the feature map. ; After two kernels of size Furthermore, the convolutional layers with a stride of 1 have 64 and 128 kernels respectively, followed by a... The pooling layer yields the feature map. ; After a kernel size of Furthermore, the convolutional layer with a stride of 1 has 16 kernels, followed by a... The pooling layer yields the feature map. Ultimately Flatten and output eigenvectors.

[0063] (3b2) The Transformer encoder performs multi-scale, full-band semantic information extraction on a vector sequence of length 4 composed of 3 feature vectors and 1 classification vector extracted by the feature embedding module:

[0064] Each of the three independent branches outputs one. The feature vectors, each representing the feature of its respective band, are concatenated with a randomly initialized classification vector of the same dimension to obtain a... A vector sequence, and a vector sequence with a Position encoding sequence Adding them together yields a vector sequence that incorporates location information. ,Will As input to the Transformer encoder, the vector sequence is processed sequentially through four self-attention encoding modules. Within each module, a multi-head self-attention module extracts features from the input vector sequence. The first layer normalization module performs hierarchical standardization on the residuals between the input and output of the multi-head self-attention module. A feedforward neural network module performs a nonlinear transformation on the vector sequence processed by the first layer normalization module. The second layer normalization module performs hierarchical standardization on the residuals between the input and output of the feedforward neural network module, resulting in a vector sequence containing full-band semantic information. The output sequence of the final self-attention encoding module is then processed. The classification vectors are extracted and used as feature vectors containing full-band semantic information; their dimension remains the same. .

[0065] Each feature weighting fusion module weights the pixel-level feature cubes. The CNN then performs deep feature extraction and downsampling on the weighted feature cubes, which can obtain feature vectors that enhance useful features while suppressing interfering features. The Transformer encoder extracts multi-scale, full-band semantic information from the sequence containing all feature vectors and a classification vector to capture the global long-distance dependencies of all bands. This achieves an organic combination of local features of each band and joint features of the entire band, avoiding the shortcomings of existing technologies in capturing global semantic connections and long-distance dependencies across bands, and effectively improving classification accuracy.

[0066] (3b3) The MLP classification head performs linear projection and Softmax normalization on the extracted feature vector containing full-band semantic information to obtain the probability distribution of each sample belonging to 9 land cover categories. Each element ( All samples belong to a category. The probability of.

[0067] (3c) Employ the cross-entropy loss function, based on the category of each sample... Probability distribution of each land cover category and their corresponding real tags Calculate the loss value of the network model Then, the gradient descent method is used to... For weights and bias parameters , The model is updated to obtain the land cover classification network model for this iteration. ;

[0068] The network model loss value The calculation formula is:

[0069] ;

[0070] in, It is the first The sample belongs to the first The probability of a class, when =1 indicates the first The sample belongs to the first Class, when =0 indicates the first The sample does not belong to the first kind, , Indicates the total number of training samples; yes The corresponding real tags, Is with the first The number of training samples corresponding to each land cover category has an inversely proportional weight;

[0071] Calculate using the chain rule For weight parameters and bias parameters partial derivatives and and according to , right , Update:

[0072] ;

[0073] ;

[0074] in, Indicates bias parameters The update results express The update results Indicates the learning rate. This indicates the partial derivative operation.

[0075] (3d) judgment If true, then a well-trained land cover classification network model is obtained. Otherwise, let , Then proceed with step (3b).

[0076] Step 4) Obtain pixel-level land cover classification results from multi-band SAR images:

[0077] The test sample set is used as the trained ground cover classification network model. The input is forward propagated to obtain the pixel-level classification results of ground features in the multi-band SAR image corresponding to each test sample.

Claims

1. A pixel-level ground feature classification method for multi-band SAR images, characterized in that, Includes the following steps: (1) Obtain the training sample set and the test sample set: For the obtained sample set containing Each land cover category The pseudo-color three-channel SAR images of each band were preprocessed, and the pixels of the multiple grayscale SAR image slices obtained after preprocessing in each band were labeled. Then, a training sample set including feature cubes of multiple pixels and their labels, and a test sample set including multiple feature cubes were constructed. ， (2) Construct a multi-band SAR image pixel-level ground feature classification network model based on CNN-Transformer: Construct a multi-band SAR image pixel-level ground feature classification network model including cascaded feature embedding modules, Transformer encoders and MLP classification heads. The feature embedding module includes parallel-arranged... Each branch includes a cascaded feature weighted fusion module and a convolutional CNN module; (3) Iteratively train the multi-band SAR image pixel-level ground object classification network model: train the multi-band SAR image pixel-level ground object classification network model through the training sample set. Iterative training is performed to obtain a well-trained land cover classification network model. (4) Obtain pixel-level land cover classification results of multi-band SAR images: use the test sample set as the trained land cover classification network model. The input is forward propagated to obtain the pixel-level classification results of ground features in the multi-band SAR image corresponding to each test sample.

2. The method according to claim 1, characterized in that, The preprocessing described in step (1) involves the following steps: segmenting each pseudo-color three-channel SAR image, converting the resulting multiple pseudo-color three-channel SAR image slices into single-channel grayscale image slices, and then filtering and upsampling the single-channel grayscale image slices to obtain multiple grayscale SAR image slices with uniform scale after preprocessing.

3. The method according to claim 1, characterized in that, The training sample set and test sample set mentioned in step (1) are constructed as follows: (1a) The gray-level co-occurrence matrix of each pixel in each gray-level SAR image slice is normalized, and multiple texture features corresponding to multiple statistics in the normalized probability distribution matrix are combined into a multi-channel feature map. Then, a feature cube is constructed with the number of channels of the multi-channel feature map as the length and the length and width of the neighborhood window of each pixel in the feature map as the height and width; (1b) ... training sample set and test sample set are combined into a multi-channel feature map. Feature cubes of each category in a grayscale SAR image slice for each band, scaled proportionally. The training sample set is composed of multiple feature cubes obtained through sampling and their labels, and the remaining feature cubes are then proportionally... The test sample set is composed of multiple feature cubes obtained through sampling.

4. The method according to claim 1, characterized in that, The land cover classification network model described in step (2) includes: a feature embedding module, wherein the feature weighted fusion module includes stacked global average pooling layers, convolutional layers, ReLU activation function layers and Softmax activation function layers, and the input and output of the feature weighted fusion module are multiplicatively connected; a convolutional module CNN includes multiple cascaded sub-modules consisting of one or more convolutional layers and a pooling layer; a Transformer encoder includes multiple cascaded self-attention encoding modules, each self-attention encoding module including a cascaded multi-head self-attention module and a feedforward neural network module, and a first-layer normalization module and a second-layer normalization module respectively connected to the output of these two modules, and the input and output of the multi-head self-attention module are residually connected, and the input and output of the feedforward neural network module are residually connected.

5. The method according to claim 4, characterized in that, Step (3) involves iteratively training the pixel-level land cover classification network model for multi-band SAR images. The steps are as follows: (3a) Initialize the number of iterations to be... The maximum number of iterations is ， , the Sub-iteration land cover classification network model The weights and bias parameters are respectively 、 and order ； (3b) The feature embedding module performs weighted summation and deep feature extraction on the feature cubes in each band grayscale SAR image slice to obtain... Each feature vector; extracted by the Transformer encoder from the feature embedding module. Composed of one feature vector and one classification vector Multi-scale, full-band semantic information extraction is performed on a vector sequence of length; the MLP classification head performs linear projection and Softmax normalization on the extracted feature vectors containing full-band semantic information to obtain the classification of each sample. Probability distribution of each land cover category (3c) Employs the cross-entropy loss function, based on the category of each sample... Probability distribution of each land cover category and their corresponding real tags Calculate the loss value of the network model Then, the gradient descent method is used to... For weights and bias parameters 、 The model is updated to obtain the land cover classification network model for this iteration. (3d) Judgment If true, then a well-trained land cover classification network model is obtained. Otherwise, let ， Then proceed with step (3b).

6. The method according to claim 5, characterized in that, The deep feature extraction described in step (3b) is implemented as follows: the feature weighted fusion module of each branch in the feature embedding module encodes the feature cube and weights the encoded feature weight vector with the corresponding feature cube to achieve weighted reconstruction of the feature cube; the CNN performs deep feature extraction and downsampling on the weighted reconstructed feature cube to obtain the feature vector output by each branch.

7. The method according to claim 5, characterized in that, The multi-scale, full-band semantic information extraction described in step (3b) is implemented as follows: the multi-head self-attention module in each coding module... Feature extraction is performed on a vector sequence of length, and the first layer normalization module performs hierarchical standardization on the residual operation results of the input and output of the multi-head self-attention module. The feedforward neural network module performs a nonlinear transformation on the vector sequence processed by the first layer normalization module, and the second layer normalization module performs hierarchical normalization on the residual operation results of the input and output of the feedforward neural network module to obtain a feature vector containing semantic information of the entire band.

8. The method according to claim 5, characterized in that, Each sample described in step (3b) belongs to Probability distribution of each land cover category The acquisition method is as follows: the MLP classification head performs a multi-scale linear transformation on the extracted feature vector containing global semantic information, and the Softmax activation function normalizes the feature vector with the same dimension as the number of target land cover categories to obtain the category of each sample. Probability distribution of each land cover category 。 9. The method according to claim 5, characterized in that, The loss value described in step (3c) The calculation formula is: ;in, It is the first The sample belongs to the first The probability of a class, when =1 indicates the first The sample belongs to the first Class, when =0 indicates the first The sample does not belong to the first kind, ， Indicates the total number of training samples; yes The corresponding real tags, Is with the first The number of training samples corresponding to each land cover category has an inversely proportional weight.

10. The method according to claim 5, characterized in that, Step (3c) describes the adjustment of weights and bias parameters. 、 The update is performed using the chain rule, and the update formulas are as follows: ； ;in, Indicates bias parameters The update results express The update results Indicates the learning rate. This indicates the partial derivative operation.

Images