PolSAR ground feature classification method based on cross-channel attention mechanism
By combining the Unet and ViT models, the cross-channel attention mechanism and multi-scale Transformer module are introduced, which solves the problem of insufficient feature extraction capabilities in PolSAR geographic classification, and realizes high-precision classification in complex scenarios.
Patent Information
- Application Number
- CN202510364154.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-26
- Publication Date
- 2025-07-18
AI Technical Summary
The existing PolSAR geographic classification method has limited feature extraction capabilities and weak generalization performance under the condition of scarcity of samples, making it difficult to fully utilize phase information and global features, resulting in low classification accuracy.
Combining the Unet and ViT models, a cross-channel attention mechanism and multi-scale Transformer module are introduced to build a hybrid network model, and the local, global and multi-scale features of PolSAR images are fully utilized through the cross-channel attention mechanism and multi-scale Transformer module.
It significantly improves the accuracy of PolSAR landform classification, especially in complex scenarios, and effectively captures the local, global and multi-scale features of the image.
Smart Images
Figure CN120339687A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of image processing, and further relates to the technology of PolSAR ground object classification. Specifically, it is a ground object classification method for polarimetric synthetic aperture radar (PolSAR) based on a cross-channel attention mechanism, which can be used to achieve ground object classification in complex scenarios such as urban planning and agricultural monitoring. Background Art
[0002] In the remote sensing satellite technology, the polarimetric synthetic aperture radar (PolSAR) technology has become an important technology in the remote sensing field because it can provide data of multiple polarizations, multiple frequencies and multiple perspectives. The PolSAR technology can capture more comprehensive and detailed surface features, which is of great significance and practical application value to related fields such as urban planning, agricultural monitoring and military detection. Therefore, by deeply studying the application of PolSAR in ground object classification, technical support can be provided for improving the interpretation accuracy of remote sensing data.
[0003] With the development of PolSAR technology and the remarkable achievements of deep learning in fields such as computer vision and autonomous driving, the combination of the two has become a hot issue. Through the powerful feature extraction and pattern recognition capabilities of deep learning, PolSAR data can be analyzed and processed more effectively, thereby improving the accuracy and efficiency of ground object classification. The application of convolutional neural networks alleviates the problem of dependence on manually designed features in traditional PolSAR ground object classification methods and improves the ground object classification performance in complex scenarios, but it is still limited by the loss of phase information and receptive fields. To solve the above problems and considering that PolSAR ground object classification belongs to a pixel-level classification task, fully convolutional neural networks have been introduced into the field of PolSAR ground object classification. However, the existing methods do not consider the potential relationships between different feature maps. Its variant Unet retains more low-level semantic information and reduces the loss of spatial information by connecting the feature maps of the encoder and decoder parts, which is very important for the PolSAR ground object classification task. However, the convolutional operations in it are still local and are easily limited by the size of the receptive field. The self-attention mechanism in the Transformer model can effectively capture the global dependencies of each position in the input sequence, which means that the Transformer can better retain the information far from the central pixel, thereby improving the ability to understand the global image structure.
[0004] Dosovitskiy et al. proposed the Vision Transformer (ViT) model in DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale [C] / / International Conference on Learning Representations. 2021. By virtue of the self-attention mechanism, it can obtain global features, effectively making up for the locality of CNN and being able to well learn the dependencies of global pixels in images. However, the above method cannot fully utilize the phase information of PolSAR data. Moreover, the scattering mechanism of complex ground objects is also complex, and its phase information is unique and important, containing rich ground object features, such as the geometric structure and roughness of ground objects. If not fully utilized, it is impossible to comprehensively characterize the properties of ground objects, resulting in difficulty in accurately distinguishing different ground objects during classification and reducing the classification accuracy. Zhang et al. first proposed the Complex-valued Convolutional Neural Network (CV-CNN) by using the complex domain information of PolSAR data as input data in ZHANG Z, WANG H, XU F, et al. Complex-valued convolutional neural network and its application in polarimetric SAR image classification [J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(12): 7177-7188. It verified the importance of the phase information contained in the complex domain of PolSAR data and further illustrated the broad development prospects of PolSAR ground object classification in the field of deep learning. However, although this scheme improved the classification accuracy to a certain extent, due to the insufficient extraction of global information, the classification effect when facing complex ground objects was not ideal. Summary of the Invention
[0005] The object of the present invention is to address the deficiencies of the above-mentioned prior art and propose a PolSAR ground object classification method based on a cross-channel attention mechanism to solve the problems of limited feature extraction ability and weak generalization performance of the PolSAR image classification model under the condition of scarce samples. The present invention introduces ViT into the complex domain and combines it with the Unet model. By establishing a multi-scale Transformer module and a multi-scale cross-channel attention mechanism module, a classification model that can make full use of phase information is obtained, effectively capturing the local, global, and multi-scale features of PolSAR images, and significantly improving the accuracy of PolSAR ground object classification in complex scenarios.
[0006] The specific steps for the present invention to achieve the above object are as follows:
[0007] (1) Obtain PolSAR image data from a public dataset and preprocess it;
[0008] (2) Use the preprocessed images to construct a sample set, and select 5-10% of the data as training samples, and the remaining data as test samples;
[0009] (3) Construct a hybrid network model based on Unet and ViT, with the structure: input layer → encoder → cross-channel attention mechanism module → multi-scale Transformer module → decoder → output layer, and there are two structures from the encoder to the decoder, that is, the model has two branches; the encoders in the two branches are both combinations of ViT encoders and Unet encoders; the input layer divides the sample set data into real and imaginary parts according to the number of channels and sends them into the two branches respectively; the output layer is used to fuse the feature maps obtained from the two branches and restore the image size through a reshaping operation;
[0010] (4) Optimize and train the hybrid network model based on Unet and ViT using the training samples to obtain a trained hybrid network model;
[0011] (5) Input the test samples into the trained hybrid network model to obtain the predicted classification results of the data to be measured;
[0012] (6) Use the pre-set colors to color the pixels at each position according to the pixel values in the predicted classification results to obtain the final classification result map and output it.
[0013] The present invention has the following advantages compared with the prior art:
[0014] First, since the present invention selects to combine Unet and ViT encoders in the complex domain to jointly achieve feature extraction, it can make full use of the local, global features and phase information of the existing PolSAR data, thereby effectively extracting the features of complex ground objects.
[0015] Second, since the present invention adopts a cross-channel attention mechanism, it makes up for the original limitation of ViT in utilizing channel information, further improving the feature extraction ability of the encoder, and thus effectively enhancing the ground object classification ability of the model.
[0016] Aiming at the problems of limited feature extraction ability and poor model generalization ability in the current PolSAR ground object classification method in deep learning, the present invention improves the application of the deep learning method in PolSAR ground object classification, providing a new solution idea for PolSAR ground object classification. Experimental results show that the PolSAR ground object classification method combining the cross-channel attention mechanism Transformer-Unet model proposed by the present invention effectively improves the PolSAR ground object classification accuracy. Brief Description of the Drawings
[0017] Figure 1 is the overall implementation flowchart of the present invention;
[0018] Figure 2 is the image of Xi'an area used in the experiment in the embodiment of the present invention;
[0019] Figure 3 is the schematic diagram of the true ground object marking of the Xi'an area image in the embodiment of the present invention;
[0020] Figure 4 is for using the Unet network model in the embodiment of the present invention for Figure 2 the schematic diagram of the classification result;
[0021] Figure 5 is for using the CV-Unet network model in the embodiment of the present invention for Figure 2 the schematic diagram of the classification result;
[0022] Figure 6 is for using the method of the present invention for Figure 2 the schematic diagram of the classification result. Detailed Embodiment
[0023] The technical solution of the present invention will be further described below with reference to the drawings.
[0024] Embodiment 1: Referring to the attached Figure 1 , the PolSAR ground object classification method based on the cross-channel attention mechanism proposed by the present invention includes the following steps:
[0025] Step 1) Obtain PolSAR image data from a public dataset and perform preprocessing on it; the preprocessing operations adopted in this embodiment include Pauli decomposition and sliding window operation; among them, Pauli decomposition represents the measured scattering matrix [S] using the so-called Pauli basis; the sliding window operation specifically sets a corresponding sliding window and stride size according to the characteristics of the area where the PolSAR image belongs, and performs sliding windowing to finally obtain the preprocessed PolSAR image.
[0026] Step 2) Construct a sample set using the preprocessed image, and select 5 - 10% of the data as training samples, and the remaining data as test samples;
[0027] Step 3) Construct a hybrid network model based on Unet and ViT, with the structure: input layer → encoder → cross-channel attention mechanism module → multi-scale Transformer module → decoder → output layer, and there are two encoder-to-decoder structures, that is, the model has two branches; the encoders in the two branches are both combinations of ViT encoders and Unet encoders; the input layer divides the sample set data into real and imaginary parts according to the number of channels and sends them into the two branches respectively; the output layer is used to fuse the feature maps obtained from the two branches and restore the image size through a reshaping operation.
[0028] The above encoder contains four Unet encoding layers with the same structure, and this encoding layer extracts features through convolution and pooling; the above decoder consists of four decoding layers corresponding to the encoding layers, fuses shallow and deep features through skip connections, and outputs the reconstructed image features. The above multi-scale cross-channel attention mechanism module is used to compress spatial information; the above multi-scale Transformer module consists of ViT encoders.
[0029] Step 4) Optimize and train the hybrid network model based on Unet and ViT using the training samples to obtain a trained hybrid network model; for the training of the model in this embodiment, specifically, the training samples are used as the model input, the features of each pixel point are extracted, and then the result of the forward propagation of the model is used as the output of the model. By solving the error between this output and the input image and performing backpropagation on the error, parameter optimization is achieved.
[0030] Step 5) Input the test samples into the trained hybrid network model to obtain the predicted classification results of the data to be measured;
[0031] Step 6) Use the pre-set colors to color each position in the prediction classification result according to the pixel value at that position, obtain the final classification result map and output it. In this embodiment, each position in the prediction classification result is colored according to the pixel value at that position. The classification includes at least three terrain categories: grassland, city, and river. The number of pre-set color types is the same as the number of terrain categories. Preferably, colors that are easy to distinguish, such as red, blue, and green, are used.
[0032] Embodiment 2: The overall implementation steps of the ground object classification method proposed in this embodiment are the same as those in Embodiment 1. Now, taking the image of Xi'an area as an example, specific parameter settings are given to describe the implementation process of the present invention in detail:
[0033] Step 1. Preprocess the PolSAR image. In this embodiment, for the image of Xi'an area, the sliding window and stride are set to 64×64 and 6 respectively, and a sliding window operation is performed to obtain a sample set.
[0034] Step 2. According to the real ground object map, select 5% of the data in the image of Xi'an area after sliding window as training samples, and the remaining data as test samples, and respectively form a training data set and a test data set.
[0035] Step 3. Construct a hybrid network model based on Unet and ViT, with the structure: input layer → encoder → cross-channel attention mechanism module → multi-scale Transformer module → decoder → output layer. Among them, there are two structures of encoder → cross-channel attention mechanism module → multi-scale Transformer module → decoder, and the encoders of both branches are a combination of a ViT encoder and a Unet encoder. The input layer divides the original input information into real and imaginary parts according to the number of channels and sends them into the encoders of the two branches of the network model respectively. The output layer is used to fuse the feature maps obtained from the two branches and restore the image size through a reshaping operation.
[0036] Specifically, the encoder contains four Unet encoding layers with the same structure, which continuously perform convolution and pooling operations for feature extraction. After each encoding layer, the obtained features are input into the cross-channel attention mechanism module, and the output of this module is then input into the multi-scale Transformer module, that is, the module composed of the ViT encoder. The cross-channel attention mechanism module enables the model to compress the spatial information carried by the original scale downsampling module, making the model pay more attention to channel information, while improving the efficiency and utilization rate of feature transmission. Since the average pooling layer compresses the spatial dimension of the downsampled feature map, the key channel information is retained, enabling the multi-scale Transformer module to process these features more efficiently, thereby achieving the purpose of improving the classification effect. The decoder consists of four decoding layers corresponding to the Unet encoding layers and is connected by skip connections to fuse and interact the shallow features and deep features, and finally the output is the reconstructed image features.
[0037] Step 4. Use the training dataset as the input of the model constructed in Step 3, extract the features of each pixel point of the training samples using the model, and use the result of the forward propagation of the model as the output of the model. By solving the error between the output and the original input image and performing backpropagation on the error, parameter optimization is achieved, and the trained hybrid network model Transformer-Unet is obtained.
[0038] Step 5. Input the test dataset into the trained model Transformer-Unet to obtain the predicted classification results of the test data.
[0039] Step 6. Compare the output predicted results with the true labels, and calculate the overall classification accuracy of the PolSAR image. Specifically, the overall accuracy, Kappa coefficient, and average intersection over union are used to measure the classification accuracy.
[0040] Step 7. Color according to the preset color based on the pixel values at each position during output to obtain the final classification result map and output it.
[0041] Embodiment 3: The overall implementation steps of the classification method proposed in this embodiment are the same as those in Embodiment 1 or 2. Now, the hybrid network model constructed by the present invention will be further described in detail:
[0042] The hybrid network model based on Unet and ViT constructed by the present invention has a structure that sequentially includes an input layer, an encoder, a cross-channel attention mechanism module, a multi-scale Transformer module, a decoder, and an output layer. Among them, the encoder is formed by fusing a ViT encoder and a Unet encoder. The input layer divides the original input information into a real part and an imaginary part according to the number of channels and sends them into two branch structures respectively. The two branch structures are exactly the same and include an encoder, a cross-channel attention mechanism module, a multi-scale Transformer module, and a decoder. The role of the output layer is to fuse the feature maps obtained from the two branches and restore the image size through a reshaping operation. Specifically, the encoder contains four Unet encoding layers with the same structure. These encoding layers extract features through continuous convolution and pooling operations. After each encoding layer, the extracted features are sent into the cross-channel attention mechanism module, and then the output of this module is passed to the multi-scale Transformer module, that is, the module composed of the ViT encoder. The cross-channel attention mechanism module can compress the spatial information carried by the downsampling module, making the model more focused on channel information, while improving the efficiency and utilization rate of feature transmission. Since the average pooling layer compresses the spatial dimension of the downsampled feature map, thereby retaining key channel information, this enables the multi-scale Transformer module to process these features more efficiently, thereby improving the classification effect. The decoder consists of four decoding layers corresponding to the Unet encoding layers and is connected by skip connections to achieve the fusion and interaction of shallow features and deep features, and finally outputs the reconstructed image features.
[0043] The following preferred parameter settings are given in this embodiment:
[0044] Each of the four encoding layers in the encoder is composed of a complex convolutional layer with a convolutional kernel size of 3*3 and a sliding step of 1, and the number of channels of the convolutional layers in the four encoding layers is sequentially set to 30, 60, 120, and 240.
[0045] The multi-scale cross-channel attention mechanism module is divided into two parts. The first part stacks six multi-head self-attention mechanism modules, and the number of self-attention heads is set to four. The second part stacks two multi-head self-attention mechanism modules, and the number of self-attention heads is set to four.
[0046] Each of the four decoding layers in the decoder is composed of a complex transposed convolutional layer with a convolutional kernel size of 3*3 and a sliding step of 1. The number of channels after splicing the underlying features transmitted through the skip connections is sequentially 480, 240, 120, and 60.
[0047] The following further illustrates the effect of the present invention in combination with simulation experiments.
[0048] 1. Simulation conditions:
[0049] The simulation experiment of the present invention was carried out in the hardware environment of Nvidia GTX 4070 GPU and the software environment of pycharm with python3.6 GPU version.
[0050] 2. Simulation content:
[0051] It includes 1) the deep learning PolSAR ground object classification method; 2) the method of the present invention, that is, the PolSAR ground object classification method combining the cross-channel attention mechanism Transformer-Unet model. Among them, the deep learning AE anomaly detection methods are all classical methods widely cited in the PolSAR ground object classification task.
[0052] Figure 2 It is the false color image of the Xi'an area image used in the simulation of this embodiment, and the scene size is 512×512. Figure 3 It is the real ground object marking of the PolSAR image used in this embodiment. The methods and results of the simulation experiment are shown in Table 1.
[0053] Simulation 1. Use the Unet model to Figure 2 perform ground object classification, and the classification results can be seen in Figure 4 .
[0054] Simulation 2. Use the CV-Unet model to Figure 2 perform ground object classification, and the classification results can be seen in Figure 5 .
[0055] Simulation 3. Use the present invention to Figure 2 perform ground object classification, and the classification results can be seen in Figure 6 .
[0056] Table 1 Classification result accuracy of existing methods and the method of the present invention
[0057]
[0058] 3. Result analysis:
[0059] As can be seen from Table 1 and Figures 4 - 6 , the accuracy rate of ground object classification by the Unet model is 96.65%; the accuracy rate of ground object classification by the CV-Unet model is 96.86%; the accuracy rate of the method of the present invention is 98.70%. It can be seen that the present invention has obtained the highest accuracy rate. From the classification result diagrams obtained by the Unet network model and the CV-Unet network model, it can be seen that there are a large number of misclassification phenomena in the grassland area represented by green, and the classification boundary in the urban area is more obvious in the shape of sawteeth. However, from the effect diagram of the MCT-Unet with the multi-scale cross-channel network structure added, it can be seen that there are significantly fewer misclassification phenomena than the other several images, and the classification accuracy is further improved.
[0060] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all information and data that have been authorized by the user or fully authorized by all parties. Moreover, the collection, use, and processing of relevant data need to comply with the laws, regulations, and standards of relevant countries and regions, and corresponding operation entrances are provided for users to choose to authorize or refuse.
[0061] The above simulation analysis proves the effectiveness of the method proposed by the present invention.
[0062] The parts not detailed in the present invention belong to the common general knowledge of those skilled in the art.
[0063] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Obviously, for professionals in the field, after understanding the content and principle of the present invention, various modifications and changes in form and details may be made without departing from the principle and structure of the present invention. However, these modifications and changes based on the idea of the present invention are still within the scope of protection of the claims of the present invention.
Claims
1. A PolSAR ground object classification method based on cross-channel attention mechanism, characterized in that It includes the following steps: (1) Obtain PolSAR image data from a public dataset and preprocess it; (2) Use the preprocessed images to construct a sample set, and select 5-10% of the data as training samples, and the remaining data as test samples; (3) Construct a hybrid network model based on Unet and ViT, with the structure: input layer → encoder → cross-channel attention mechanism module → multi-scale Transformer module → decoder → output layer, and there are two encoder-to-decoder structures, that is, the model has two branches; the encoders in the two branches are both combinations of ViT encoders and Unet encoders; the input layer divides the sample set data into real and imaginary parts according to the number of channels, and sends them into the two branches respectively; The output layer is used to fuse the feature maps obtained from the two branches and restore the image size through a reshaping operation; (4) Use the training samples to optimize and train the hybrid network model based on Unet and ViT to obtain a trained hybrid network model; (5) Input the test samples into the trained hybrid network model to obtain the predicted classification results of the data to be measured; (6) Use the preset colors to color the pixels at each position according to the pixel values in the predicted classification results to obtain the final classification result map and output it.
2. The method according to claim 1, wherein: The preprocessing in step (1) includes Pauli decomposition and sliding window operation; specifically, the sliding window operation is to set the corresponding sliding window and stride size according to the characteristics of the region where the PolSAR image belongs, and finally obtain the preprocessed PolSAR image.
3. The method according to claim 1, wherein: The encoder in step (3) includes four Unet encoding layers with the same structure, and this encoding layer extracts features through convolution and pooling; the decoder consists of four decoding layers corresponding to the encoding layers, and fuses shallow and deep features through skip connections to output the reconstructed image features.
4. The method according to claim 3, wherein: The Unet encoding layer is composed of complex convolutional layers with a kernel size of 3*3 and a sliding stride of 1; and the number of channels of the convolutional layers of the four Unet encoding layers with the same structure is 30, 60, 120, and 240 in sequence.
5. The method according to claim 3, wherein: The decoding layer is composed of complex transposed convolutional layers with a kernel size of 3*3 and a sliding stride of 1; and the number of channels of the four decoding layers is 480, 240, 120, and 60 in sequence.
6. The method according to claim 1, characterized in that: The multi-scale cross-channel attention mechanism module in step (3) is used to compress spatial information; this module is divided into two parts, where the first part is composed of six stacked multi-head self-attention mechanism modules, and the number of self-attention heads is set to four; the second part is composed of two stacked multi-head self-attention mechanism modules, and the number of self-attention heads is set to four.
7. The method according to claim 1, characterized in that: The multi-scale Transformer module in step (3) is composed of ViT encoders.
8. The method according to claim 1, wherein: The optimization training of the hybrid network model based on Unet and ViT in step (4) is specifically to use the training samples as the model input, extract the features of each pixel point, and then use the result of the forward propagation of the model as the output of the model. By solving the error between the output and the input image and performing backpropagation on the error, parameter optimization is achieved.
9. The method according to claim 1, wherein: In step (6), coloring is performed according to the pixel values at each position in the predicted classification result. The classification includes at least three terrain categories: grassland, city, and river; the number of preset color types is the same as the number of terrain categories.