Polarimetric sar image classification method based on physical driven dual-flow mamba network

Abstract

This invention discloses a polarimetric SAR image classification method based on a physically driven dual-stream Mamba network, primarily addressing the problem of poor classification performance in complex scenes using existing methods. The method includes: 1) preprocessing the input polarimetric synthetic aperture radar data and adaptively compensating for polarimetric azimuth offset using a learnable polarimetric calibration layer; 2) constructing a physical-semantic dual-stream feature extraction network to simultaneously acquire scattering mechanism features and deep semantic features, and using a physically guided attention mechanism to achieve dual-stream feature fusion; 3) performing physically-aware sequential rearrangement of the fused features based on feature similarity and spatial distance, modeling the sequence features using the Mamba module, and obtaining the category prediction result for the center pixel; 4) using the trained classification model to obtain the classification result of the image under test. This invention effectively improves the accuracy and robustness of polarimetric SAR image classification in complex scenes by combining physical driving and sequence modeling.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing image processing and intelligent interpretation technology, and further relates to polarimetric synthetic aperture radar image classification technology. Specifically, it is a polarimetric SAR image classification method based on a physically driven dual-stream Mamba network, which can be applied to tasks such as land cover classification, agricultural monitoring, and regional land cover interpretation. Background Technology

[0002] Polarimetric Synthetic Aperture Radar (PolSAR) can acquire polarimetric scattering information of ground objects under all-weather and all-day conditions, and has important application value in fields such as land cover classification, resource surveys, environmental monitoring, and military reconnaissance. Compared with single-polarimetric SAR, polarimetric SAR can provide richer information on target scattering mechanisms, and is therefore more suitable for refined ground object interpretation in complex scenarios.

[0003] Existing polarimetric SAR image classification methods mainly include traditional methods based on scattering mechanism analysis and intelligent classification methods based on deep learning. Traditional methods typically rely on polarization decomposition parameters, statistical features, and manually designed features for classification, offering a degree of physical interpretability. However, under conditions of complex terrain, noise interference, and complex category distributions, their feature representation capabilities are limited, thus constraining classification accuracy. In recent years, deep learning methods such as convolutional neural networks and Transformers have been widely applied to polarimetric SAR image classification tasks, achieving certain results in automatic feature extraction and high-level semantic representation.

[0004] In existing technologies, some studies have attempted to improve key issues in polarimetric SAR image classification. Fang et al. proposed achieving target orientation unification through polarimetric coherence matrix sequence construction, implicit POA learning, and channel alignment, which can mitigate the scattering representation differences caused by polarimetric azimuth offset to some extent. However, this processing mainly relies on subsequent sequence alignment and feature modeling. Since polarimetric azimuth offset will first cause distortion of the input polarimetric representation, if it mainly relies on subsequent alignment and feature modeling compensation, the problem of difficulty in suppressing input representation fluctuations at the front end still exists. Kuang et al. proposed introducing the state-space model Mamba into the PolSAR image classification task and proposed a spiral scanning strategy and tail category labeling to improve the sequence modeling effect in pixel-level classification tasks. This scheme has noticed the impact of sequence organization on the adaptability of the central pixel classification task and enhances the continuity of central neighborhood information through spiral scanning. However, its sequence construction is still mainly based on preset scanning rules, and the way local spatial neighborhood relationships are maintained is still relatively fixed. It has not yet explicitly combined the physical correlation and spatial proximity relationship between pixels and the central pixel. Therefore, there is still room for further improvement in balancing local spatial consistency and the classification requirements of the central pixel.

[0005] Therefore, although existing technologies have explored aspects such as polarization azimuth processing, physical feature utilization, and sequence modeling, there is still room for improvement in the stability of input polarization representation and the ability to co-express physical and semantic information in polarimetric SAR image classification tasks. Summary of the Invention

[0006] The purpose of this invention is to address the shortcomings of existing technologies by proposing a polarimetric SAR image classification method based on a physically driven dual-stream Mamba network. This method solves problems such as insufficient classification accuracy in complex scenes, distortion of scattering features due to polarimetric azimuth offset, insufficient utilization of physical prior information, and insufficient matching between local spatial structure and the central pixel classification task during sequence modeling. First, this invention reduces scattering feature distortion caused by polarimetric azimuth offset by adaptively calibrating the input polarimetric representation. Then, it enhances the joint representation capability of physical prior information and deep semantic information through physical-semantic dual-stream feature extraction and guided fusion. Furthermore, it strengthens the matching between local spatial structure and the central pixel classification task by serializing and rearranging the fused features and combining them with the Mamba module for sequence modeling. This invention improves the accuracy and robustness of polarimetric SAR image classification in complex scenes.

[0007] To achieve the above objectives, the technical solution of the present invention includes the following:

[0008] (1) Input the polarimetric synthetic aperture radar data to be classified and preprocess it to construct training samples with local image patches as input and the central pixel category label as supervision information, and construct the corresponding input samples to be classified in the prediction stage;

[0009] (2) Construct a learnable polarization calibration layer to adaptively calibrate the input polarization characterization, obtain the calibrated polarization characterization and output it;

[0010] (3) Construct a physical-semantic dual-stream feature extraction network including physical flow branches and semantic flow branches, and use a physical guided attention mechanism to perform physical scattering mechanism modeling and deep semantic feature extraction on the calibrated polarization representation to obtain the corresponding physical features and semantic features; use the physical features to generate spatial attention weights, and guide the semantic features to enhance them to obtain fused features. ;

[0011] (4) Fusion characteristics Physical perception serialization rearrangement is performed. The center pixel of the local image block is used as the anchor point. The comprehensive relevance score is calculated by combining the feature similarity and spatial distance between each pixel in the image block and the center pixel. Based on the score, the corresponding features of each pixel in the image block are rearranged to obtain a one-dimensional input sequence that matches the center pixel classification task.

[0012] (5) Input the one-dimensional input sequence into the Mamba-based sequence modeling module to perform state space modeling on the input sequence and obtain a sequence feature representation containing context dependencies;

[0013] (6) Extract classification features to characterize the center pixel based on the sequence feature representation, and output the category prediction result of the center pixel through the classification layer;

[0014] (7) The classification model is trained using training samples to obtain a trained polarimetric synthetic aperture radar image classification model. The model is then used to predict the test samples to obtain the classification results of the polarimetric synthetic aperture radar images to be classified.

[0015] Compared with the prior art, the present invention has the following advantages:

[0016] First, by setting a learnable polarization calibration layer, this invention can adaptively compensate for the polarization azimuth offset problem that is common in polarization synthetic aperture radar images, effectively reducing its interference on polarization characterization, and significantly improving the physical consistency and numerical stability of polarization features, thereby providing a more reliable feature basis for subsequent classification tasks.

[0017] Secondly, because the present invention constructs a physical-semantic dual-stream feature extraction network and introduces a physical-guided attention mechanism, it efficiently and collaboratively integrates scattering information with clear physical meaning and deep semantic information, which not only ensures the interpretability of features but also enhances the ability to distinguish complex ground objects, effectively solving the problem of insufficient single-stream feature representation capability of traditional methods.

[0018] Third, this invention combines a physical perception sequence rearrangement strategy with a Mamba-based sequence modeling module. On the one hand, it optimizes the organization of pixel sequences, making them highly compatible with the central pixel classification task. On the other hand, it leverages Mamba's linear complexity and dynamic selective modeling advantages to efficiently capture local neighborhood relationships and contextual dependencies, thereby improving classification accuracy while reducing computational costs, thus balancing performance and efficiency. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments will be briefly described below. Obviously, the drawings described below are merely some embodiments of the present invention, and those skilled in the art can obtain other drawings based on these drawings without any creative effort.

[0020] Figure 1 This is a flowchart illustrating the implementation of the method of the present invention;

[0021] Figure 2 This is a schematic diagram illustrating the implementation process of polarimetric SAR image classification in this invention;

[0022] Figure 3 This is a schematic diagram of the learnable polarization calibration layer in an embodiment of the present invention;

[0023] Figure 4 This is a schematic diagram comparing the structure of the sequence modeling module based on Mamba for long-distance dependency modeling of rearranged sequences in an embodiment of the present invention with the traditional raster scanning method;

[0024] Figure 5 This is a schematic diagram of the Mamba block architecture in an embodiment of the present invention.

[0025] Figure 6 Here are the Pauli RGB images and ground truth maps from the San Francisco II dataset, where (a) is the Pauli RGB image and (b) is the ground truth map;

[0026] Figure 7 Here are the Pauli RGB images and ground truth maps from the Flevoland II dataset, where (a) is the Pauli RGB image and (b) is the ground truth map;

[0027] Figure 8 Here are the Pauli RGB images and ground truth maps for the Oberpfaffenhofen dataset, where (a) is the Pauli RGB image and (b) is the ground truth map;

[0028] Figure 9 This is a schematic diagram illustrating the classification comparison of the San Francisco II dataset in an embodiment of the present invention;

[0029] Figure 10 This is a schematic diagram illustrating the classification comparison of the Flevoland II dataset according to an embodiment of the present invention;

[0030] Figure 11 This is a schematic diagram illustrating the classification comparison of the Oberpfaffenhofen dataset according to an embodiment of the present invention; Detailed Implementation

[0031] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the following embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit the scope of protection of the present invention. All equivalent substitutions, improvements and modifications made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0032] This invention uses polarimetric synthetic aperture radar (PSAR) data as input. It adaptively corrects the polarimetric azimuth offset through a learnable polarimetric calibration layer, then extracts features in parallel using physical and semantic flows, and achieves feature fusion through a physically guided attention mechanism. Next, a physically-aware sequential rearrangement strategy is employed to convert two-dimensional image patches into a one-dimensional rearranged sequence suitable for state-space model processing. Finally, a Mamba-based sequence modeling module completes long-range dependency modeling and outputs the center pixel classification result. This method can be used for PSAR image classification tasks in complex scenes.

[0033] Example 1: Refer to Figure 1 This invention proposes a polarimetric SAR image classification method based on a physically driven dual-stream Mamba network, with the following specific steps:

[0034] Step 1) Input the polarimetric synthetic aperture radar (PSAR) data to be classified and preprocess it to construct training samples with local image patches as input and the class label of the center pixel as supervision information. In the prediction stage, construct the corresponding input samples to be classified. In this embodiment, the preprocessing process includes acquiring the PSAR data to be classified, constructing the corresponding polarimetric coherence matrix, and extracting real-valued polarimetric features from the polarimetric coherence matrix; normalizing the polarimetric features, and extracting fixed-size local image patches centered on pixels with class labels as input samples.

[0035] Step 2) Construct a learnable polarization calibration layer to adaptively calibrate the input polarization representation, obtaining and outputting the calibrated polarization representation. In this embodiment, the learnable polarization calibration layer constructed in this step is specifically set at the network front end. This layer introduces learnable parameters to simulate polarization basis rotation and adaptively redistributes and scales the relevant scattering components in the input polarization data according to the learnable parameters to correct feature distortion caused by polarization azimuth offset. The learnable parameters are updated by the classification loss through backpropagation to achieve joint optimization of polarization calibration and classification tasks.

[0036] Step 3) Construct a physical-semantic dual-stream feature extraction network including physical flow and semantic flow branches. Utilize a physical-guided attention mechanism to model the physical scattering mechanism and extract deep semantic features from the calibrated polarization representation, obtaining the corresponding physical and semantic features. Use the physical features to generate spatial attention weights and enhance the semantic features to obtain fused features. The aforementioned physical-guided attention mechanism includes: inputting the physical features into an attention generation module, generating a physical-guided spatial attention map through convolutional mapping and activation operations; using the spatial attention map to perform position-wise weighted enhancement of the semantic features, and performing residual fusion between the enhanced semantic features and the original semantic features to obtain the fused features. express:

[0037] 3.1) Input the calibrated polarization characterization into the physical flow branch, perform scattering mechanism encoding and feature mapping, and obtain the physical feature representation. In this embodiment, the physical flow branch constructs physical features based on Freeman decomposition results. These physical features include scattering mechanism information corresponding to surface scattering components, secondary scattering components, and volume scattering components. Feature mapping is performed through a physical branch encoder to obtain physical feature representations. The implementation steps include:

[0038] 3.1.1) The physical flow branch encodes the scattering mechanism of the calibrated polarization characterization and constructs physical feature results based on the main scattering component. :

[0039] ;

[0040] in, These represent surface scattering, secondary scattering, and volume scattering components, respectively. This represents the total scattered power, which is the sum of the three scattered components.

[0041] 3.1.2) will Input the physical branch encoder for feature mapping to obtain the physical feature representation. .

[0042] 3.2) Generate a spatial attention map using a physical-guided attention weight generation module. ;

[0043] 3.3) Input the calibrated polarization representation into the semantic flow branch, perform convolutional encoding and extraction to obtain the semantic feature representation. ;

[0044] 3.4) Utilizing spatial attention maps semantic feature representation Weighted enhancement is performed, and residual connections are used to fuse physical and semantic features to obtain fused features. :

[0045] ;

[0046] in, This indicates element-wise multiplication.

[0047] Step 4) Analyze the fusion features Physically perceptual sequential rearrangement is performed. Using the center pixel of a local image patch as the anchor point, a comprehensive relevance score is calculated by combining the feature similarity and spatial distance between each pixel within the image patch and the center pixel. Based on this score, the corresponding features of each pixel within the image patch are rearranged to obtain a one-dimensional input sequence that matches the center pixel classification task. This embodiment specifically obtains the one-dimensional input sequence according to the following steps:

[0048] 4.1) Using the center pixel of a local image patch as the classification anchor point, calculate the feature similarity between each pixel within the image patch and the center pixel according to the following formula:

[0049] ;

[0050] in, Represents the i-th pixel With the center pixel Similarity in feature space; for Corresponding feature representation; The feature representation corresponding to the center pixel;

[0051] 4.2) Based on pixels With the center pixel The spatial distance is used to obtain the spatial decay weight of the pixel relative to the center pixel, combined with the exponential decay function. :

[0052] ;

[0053] in, For pixels spatial coordinates, Center pixel Spatial coordinates; express and The square of the spatial distance between the two Hyperparameters for controlling the spatial attenuation range;

[0054] 4.3) Using the two-dimensional coordinates of each spatial location in the local image patch, the corresponding positional feature vector is generated through learnable coordinate mapping, and then fused with the feature vector. By adding the features position by position, we obtain the position-enhanced feature representation. ;

[0055] 4.4) Jointly model feature similarity and spatial decay weights to obtain a comprehensive correlation score of each pixel within an image patch relative to the center pixel, and then use this score to represent the features of all pixels within the image patch. The pixels are sorted in ascending order so that the pixel features with higher overall relevance scores are located at the end of the one-dimensional input sequence, forming a one-dimensional input sequence for center pixel classification; wherein, pixels Regarding the center pixel The overall relevance score is expressed as .

[0056] Step 5) Input the one-dimensional input sequence into the Mamba-based sequence modeling module to perform state-space modeling on the input sequence, obtaining a sequence feature representation containing context dependencies. In this embodiment, the Mamba-based sequence modeling module is composed of one or more stacked Mamba blocks, used for long-distance dependency modeling of the one-dimensional input sequence; the Mamba block includes a feature mapping unit, a local feature extraction unit, a nonlinear activation unit, and a gated modulation unit.

[0057] Step 6) Extract classification features to characterize the center pixel based on the sequence feature representation, and output the category prediction result of the center pixel through the classification layer. In this embodiment, the classification features to characterize the center pixel are specifically the features corresponding to the last position extracted from the sequence feature representation, which are then input into the linear classification layer to finally obtain the category prediction result of the center pixel.

[0058] Step 7) Train the classification model using training samples to obtain a trained polarimetric synthetic aperture radar (PSRaD) image classification model. Use this model to predict the test samples to obtain the classification result of the PSRaD image to be classified. In this embodiment, during the testing phase, the PSRaD data samples to be tested are sequentially input into the trained learnable polarimetric calibration layer, physical-semantic dual-stream feature extraction network, physical guided attention mechanism, physical-aware sequence rearrangement module, and Mamba-based sequence modeling module. The classification layer outputs the category label corresponding to the test sample, and the above processing process is repeated for each sample in the test image to obtain the pixel-level classification result image of the entire PSRaD image.

[0059] Example 2: Refer to Appendix Figure 2-5 The classification method proposed in this embodiment is implemented in the same way as in Embodiment 1. The specific implementation steps of the method of the present invention will be further described in detail with examples:

[0060] Step 1: Data Acquisition and Preprocessing

[0061] (1.1) Obtain the polarimetric synthetic aperture radar data to be classified, construct the corresponding polarimetric coherence matrix T, and extract 9-channel real-valued features from the polarimetric coherence matrix T as polarimetric representations for network input.

[0062] (1.2) Normalize the 9-channel real-valued features, and extract a fixed-size local image patch as input sample from the preprocessed multi-channel polarization representation centered on the pixel with class label. At the same time, use the corresponding class label as supervision information for subsequent classification model training.

[0063] Step 2: Construct a learnable polarization calibration layer to adaptively calibrate the input polarization representation;

[0064] (2.1) Input the polarization characterization obtained after preprocessing in step 1 into the learnable polarization calibration layer to compensate for the scattering feature distortion caused by polarization azimuth offset.

[0065] (2.2) The learnable polarization calibration layer introduces trainable parameters. And generate the weighting coefficients required for rotation calibration based on the parameters. :

[0066] ;

[0067] ;

[0068] ;

[0069] in, For learnable parameters, and These represent the weighting coefficients corresponding to the rotation calibration process.

[0070] (2.3) The learnable polarization calibration layer performs a calibration transformation on the input polarization representation using the weight coefficients; when the input is a polarization coherence matrix T, where remain unchanged. and Reassignment is performed based on learnable weights, with off-diagonal correlated components. According to weight Scaling is performed to obtain the calibrated output tensor. Its expression can be represented as:

[0071] ;

[0072] ;

[0073] ;

[0074] ;

[0075] in, and These represent the corresponding components after calibration.

[0076] Step 3: Construct a physical-semantic dual-stream feature extraction network to obtain fused features;

[0077] (3.1) Encode the scattering mechanism of the calibrated polarization characterization output in step (2) and construct physical feature results based on the main scattering components. :

[0078] ;

[0079] in, These represent surface scattering, secondary scattering, and volume scattering components, respectively. This represents the total scattering power, which is the sum of the three scattering components.

[0080] (3.2) Input the physical feature results into the physical branch encoder for feature mapping to obtain the physical feature representation. Based on the physical characteristics, a spatial attention map is generated through a physical-guided attention weight generation module.

[0081] (3.3) Input the calibrated polarization representation output from step 2 into the semantic branch encoder to obtain the semantic feature representation. .

[0082] (3.4) Using the aforementioned spatial attention map semantic features Weighted enhancement is performed, and residual connections are used to fuse physical and semantic features to obtain fused features. :

[0083] ;

[0084] in, This indicates element-wise multiplication.

[0085] Step 4: Utilize a physically-aware serialization rearrangement strategy to fuse features. Perform rearrangement;

[0086] Let any pixel in the image patch be Its corresponding feature is represented as The center pixel is Its corresponding feature is represented as .

[0087] (4.1) Calculate the feature similarity between each pixel in the image patch and the center pixel to characterize their proximity in the feature space. The expression is as follows:

[0088] ;

[0089] in, Represents pixels With the center pixel Similarity in the feature space.

[0090] (4.2) Based on pixels With the center pixel Spatial distance is used to construct spatial attenuation weights to characterize spatial correlation within the local neighborhood. Let the pixel... Spatial coordinates are center pixel Spatial coordinates are The spatial attenuation weight is then defined as:

[0091] ;

[0092] in, Represents pixels Relative to the center pixel Spatial neighborhood weight, This represents the square of the spatial distance between the two. This is a hyperparameter used to control the spatial attenuation range.

[0093] (4.3) Introducing position embedding The feature representation is augmented, and the location-enhanced features can be represented as follows:

[0094] ;

[0095] (4.4) The feature similarity and spatial attenuation weight are jointly modeled to obtain the comprehensive correlation score of each pixel within the image patch relative to the center pixel. Its expression is:

[0096] ;

[0097] in, Represents pixels Regarding the center pixel The overall relevance score.

[0098] Furthermore, based on right Sort the data in ascending order to obtain the physical perception sequence representation of the input sequence modeling module:

[0099] ;

[0100] in, This indicates the score based on the overall relevance. The index mapping obtained after ascending sorting, where N represents the total number of pixels in the image patch. This represents the features of the rearranged physical perception sequence.

[0101] Step 5: Input the physical perception sequence features into the Mamba-based sequence modeling module to complete the classification output;

[0102] (5.1) Input the physical sensing sequence features obtained in step 4 into the Mamba-based sequence modeling module to obtain the sequence output features H. Its expression can be represented as:

[0103] ;

[0104] Where Mamba(⋅) represents the sequence modeling module, and H represents the output feature after sequence dependency modeling.

[0105] (5.2) Extract the feature vector corresponding to the last position from the sequence output feature H as the anchor feature. The anchor point features are input into a linear classification layer to obtain prediction outputs for each category. Its expression is:

[0106] ;

[0107] ;

[0108] in, Let represent the feature vector corresponding to the last position in the sequence output feature H, and N represent the sequence length. W and b represent the weight parameters and bias parameters of the classification layer, respectively, and z represents the predicted output of each category.

[0109] (5.3) During the training process, the objective function is constructed by combining the real labels of the training samples, and the network parameters are optimized end-to-end.

[0110] Step 6: After completing the model training, input the polarimetric synthetic aperture radar data sample to be tested into the trained classification model to obtain the corresponding category prediction results and generate a pixel-level classification result map of the entire polarimetric synthetic aperture radar image.

[0111] Example 3: The overall implementation steps of the image classification method proposed in this example are the same as in Example 1 or 2. Now, in conjunction with the appendix... Figure 1-5 The specific implementation process of obtaining the final classification result using the method of the present invention will be further explained.

[0112] Reference Figure 2-5 This invention mainly comprises three parts: learnable polarization calibration, physical-semantic dual-stream feature extraction and guided fusion, and physical-aware serialization rearrangement and Mamba-based sequence modeling. Learnable polarization calibration is used to adaptively calibrate the original polarimetric synthetic aperture radar data, reducing the impact of polarimetric azimuth offset on scattering features. Physical-semantic dual-stream feature extraction and guided fusion are used to extract physical scattering features and semantic features separately, and to achieve synergistic fusion of the two types of features. Physical-aware serialization rearrangement and Mamba-based sequence modeling are used to reorder the fused features and perform long-range dependency modeling on the rearranged sequence, thereby outputting the final classification prediction result.

[0113] Step S1: Input polarimetric SAR data and the corresponding label data from the training phase:

[0114] S101. Obtain the polarimetric SAR data to be classified and input the label data corresponding to the training samples; wherein, the label data is used to indicate the land cover category to which each training sample belongs;

[0115] S102. The original polarimetric SAR data is preprocessed and divided into image block samples according to a preset window size, which are used as network input data; preferably, local image blocks are constructed with the pixel to be classified as the center.

[0116] Step S2: Construct a learnable polarization calibration layer to adaptively calibrate the preprocessed polarization characterization.

[0117] S201. Set up a learnable polarization calibration layer (LPCL) at the network front end and introduce learnable parameters. , used to simulate polarization base rotation.

[0118] S202, Based on the learnable parameters Adaptive redistribution of the relevant scattering components in the input polarization data is performed to correct the feature distortion caused by polarization azimuth offset.

[0119] S203. Use the calibrated output tensor as the input to the subsequent feature extraction module; wherein, the learnable parameters The classification loss is updated via backpropagation, thereby achieving joint optimization of polarization calibration and classification tasks. The structure of the learnable polarization calibration layer is as follows: Figure 3 As shown.

[0120] Step S3: Construct a physical-semantic dual-stream feature extraction network to process the calibrated data in parallel and extract physical scattering features and semantic features respectively.

[0121] S301. Construct a physical flow branch, perform Freeman decomposition on the calibrated polarization data, and extract physical scattering features using a physical branch encoder. The physical flow branch is used to characterize the scattering mechanism information of ground objects.

[0122] S302. Construct a semantic flow branch, input the calibrated multi-channel polarization data into the semantic branch encoder, and extract semantic features. The semantic flow branch is used to characterize spatial structure and high-dimensional texture information.

[0123] S303. The physical flow branch and the semantic flow branch work in parallel to obtain two complementary types of feature representations, providing input for subsequent feature enhancement and fusion.

[0124] Step S4: Generate a physical guidance attention map based on physical scattering features, and perform weighted enhancement on semantic features to obtain physical-semantic fusion features:

[0125] S401, Physical scattering characteristics The input attention generation module generates a physically guided attention map through convolutional mapping and activation operations. .

[0126] S402, Using the physical-guided attention map to apply semantic features Perform weighted summation on the corresponding elements.

[0127] S403. Perform residual fusion between the weighted semantic features and the original semantic features to obtain the physical-semantic fusion features. ; Figure 2 The process of dual-stream feature extraction and attention enhancement is illustrated.

[0128] Step S5: Perform physical-aware serialization rearrangement on the fused features, and input the rearranged sequence into the Mamba-based sequence modeling module for long-distance dependency modeling. Output the category prediction results and calculate the classification loss.

[0129] S501. Using the center pixel of the image block as the classification anchor point, calculate the comprehensive correlation score between each pixel in the fused feature and the center pixel; the comprehensive correlation score considers both feature similarity and spatial distance.

[0130] S502. Generate corresponding position embeddings using the two-dimensional coordinates of each spatial position in the local image block, and add the position embeddings and fused features position by position to obtain the position-enhanced feature representation;

[0131] S503. Based on the comprehensive correlation score, the position-enhanced features are rearranged and serialized so that pixels with lower comprehensive correlation scores with the center pixel are placed at the beginning of the sequence, and pixels with higher comprehensive correlation scores with the center pixel are placed at the end of the sequence, thereby forming a physically-perceptual rearranged sequence. Figure 4 The rearrangement method is compared with the traditional raster scanning method; and it is input into a Mamba-based sequence modeling module for long-distance dependency modeling.

[0132] S504, the Mamba-based sequence modeling module is composed of one or more stacked Mamba blocks; such as Figure 5 As shown, the Mamba block includes a feature mapping unit, a local feature extraction unit, a nonlinear activation unit, and a gated modulation unit, which are used to extract features from the input sequence and model context dependencies.

[0133] S505. Input the sequence features output by the Mamba-based sequence modeling module into the classifier to obtain the category prediction results, and calculate the classification loss by combining the real labels to complete the model training.

[0134] Step S6: Use the trained classification model to classify and predict the test samples to obtain the final classification result.

[0135] S601. Input the polarimetric SAR samples input during the testing phase into the trained classification model to obtain the corresponding feature representations;

[0136] S602. The classifier outputs the category label corresponding to the test sample to obtain the classification result of a single sample;

[0137] S603. Repeat the above steps for each sample in the image to be tested to obtain the final classification result of the entire polarimetric SAR image.

[0138] The effects of the present invention will be further explained below with reference to experiments.

[0139] 1. Simulation conditions

[0140] The simulation experiments of this invention were conducted on an NVIDIA RTX 4090D (24GB) hardware platform and implemented using a Python deep learning framework. The experiments selected three publicly available polarimetric synthetic aperture radar datasets—San Francisco II, Flevoland II, and Oberpfaffenhofen—for validation, and used overall classification accuracy (OA), average classification accuracy (AA), and Kappa coefficient as evaluation metrics. Under the following experimental conditions, the method of this invention was tested, and compared with methods such as U-Net, STL-Net, 3D-CNN, LSTCNN, MCDiff, PolMPCNN, and Vmamba. The above experimental settings, comparison methods, and evaluation methods are used to illustrate the implementation of the method of this invention.

[0141] 2. Simulation Dataset

[0142] This embodiment uses three polarimetric SAR images to illustrate the method of the present invention.

[0143] The first set of data is the San Francisco II dataset. This dataset was acquired by NASA / JPL's AIRSAR platform. It is L-band fully polarimetric data, with a file size of 900×1024 pixels. After removing the background, it contains 802,302 valid pixels, including five types of land features: bare soil, mountains, ocean, urban areas, and vegetation. Its Pauli RGB image and ground truth map are shown below. Figure 6 As shown.

[0144] The second dataset is the Flevoland II dataset. Acquired by NASA / JPL's AIRSAR platform, this L-band fully polarimetric dataset is 1020×1024 pixels in size, containing 135,350 valid pixels after background removal. It includes 14 land cover types: potatoes, fruits, oats, beets, barley, onions, wheat, beans, peas, corn, flax, rapeseed, grassland, and alfalfa. Its Pauli RGB image and ground truth map are shown below. Figure 7 As shown.

[0145] The third dataset is the Oberpfaffenhofen dataset. This dataset was acquired by the E-SAR airborne platform of the German Aerospace Center (DLR). It is L-band fully polarimetric data, with a file size of 1300×1200 pixels. After removing the background, it contains 1,311,618 valid pixels and includes three types of land features: built-up areas, woodlands, and open areas. Its Pauli RGB image and ground truth map are shown below. Figure 8 As shown.

[0146] 3. Comparison and analysis of experimental results

[0147] (1) Experimental results of the San Francisco II dataset

[0148] like Figure 9 As shown in Table 1, the method of this invention achieves high classification accuracy on the San Francisco II dataset. Specific classification results are shown in Table 1. Experimental results demonstrate that the method of this invention is suitable for polarimetric synthetic aperture radar image classification tasks under complex terrain conditions.

[0149] Table 1 Comparative experimental results of different methods on the San Francisco II dataset

[0150]

[0151] (2) Experimental results of Flevoland II dataset

[0152] like Figure 10 As shown in Table 2, the method of this invention achieved high classification accuracy on the Flevoland II dataset. Specific classification results are shown in Table 2. Experimental results demonstrate that the method of this invention is suitable for polarimetric synthetic aperture radar image classification tasks with a large number of land cover categories and fragmented land parcel distribution.

[0153] Table 2 Comparative experimental results of different methods on the Flevoland II dataset

[0154]

[0155] (3) Experimental results of the Oberpfaffenhofen dataset

[0156] like Figure 11 As shown in Table 3, the method of this invention achieved high classification accuracy on the Oberpfaffenhofen dataset. Specific classification results are shown in Table 3. Experimental results indicate that the method of this invention is suitable for polarimetric synthetic aperture radar image classification tasks involving complex boundaries of land cover categories and differences in scattering responses, such as built-up areas, woodlands, and open areas.

[0157] Table 3 Comparative experimental results of different methods on the Oberpfaffenhofen dataset

[0158]

[0159] Based on the simulation results above, the polarization synthetic aperture radar image classification method based on a physical-driven dual-stream Mamba network proposed in this invention has achieved good classification results on multiple public datasets, demonstrating the feasibility of the technical solution of this invention.

[0160] The parts of this invention not described in detail are common knowledge to those skilled in the art.

[0161] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Obviously, those skilled in the art, after understanding the content and principle of the present invention, may make various modifications and changes in form and detail without departing from the principle and structure of the present invention. However, these modifications and changes based on the concept of the present invention are still within the scope of protection of the claims of the present invention.

Claims

1. A polarimetric SAR image classification method based on physically driven double-flow Mamba network, characterized in that, Includes the following steps: (1) Input the polarimetric synthetic aperture radar data to be classified and preprocess it to construct training samples with local image patches as input and the central pixel category label as supervision information, and construct the corresponding input samples to be classified in the prediction stage; (2) Construct a learnable polarization calibration layer to adaptively calibrate the input polarization characterization, obtain the calibrated polarization characterization and output it; (3) Construct a physical-semantic dual-stream feature extraction network including physical flow branch and semantic flow branch, and use physical guided attention mechanism to model the physical scattering mechanism and extract deep semantic features on the calibrated polarization representation to obtain the corresponding physical features and semantic features; The spatial attention weight is generated by using physical features, and semantic features are guided and enhanced to obtain fusion features ; (4) for the fusion feature Physical perception sequence rearrangement is performed, taking the center pixel of a local image block as an anchor point, combining the feature similarity and spatial distance between each pixel in the image block and the center pixel to calculate a comprehensive correlation score, and rearranging the corresponding features of each pixel in the image block according to the score to obtain a one-dimensional input sequence matched with the center pixel classification task; (5) Input the one-dimensional input sequence into the Mamba-based sequence modeling module to perform state space modeling on the input sequence and obtain a sequence feature representation containing context dependencies; (6) Extract classification features to characterize the center pixel based on the sequence feature representation, and output the category prediction result of the center pixel through the classification layer; (7) The classification model is trained using training samples to obtain a trained polarimetric synthetic aperture radar image classification model. The model is then used to predict the test samples to obtain the classification results of the polarimetric synthetic aperture radar images to be classified.

2. The method according to claim 1, characterized in that: The preprocessing in step (1) includes: acquiring polarimetric synthetic aperture radar data to be classified, constructing the corresponding polarimetric coherence matrix, and extracting real-valued polarimetric features from the polarimetric coherence matrix; normalizing the polarimetric features, and extracting local image patches of a fixed size as input samples centered on pixels with category labels.

3. The method according to claim 1, characterized in that: The learnable polarization calibration layer in step (2) is set at the front end of the network. This layer introduces learnable parameters to simulate polarization base rotation and adaptively redistributes and scales the relevant scattering components in the input polarization data according to the learnable parameters in order to correct the feature distortion caused by polarization azimuth offset. The learnable parameters are updated by backpropagation of the classification loss to achieve joint optimization of polarization calibration and classification tasks.

4. The method according to claim 1, characterized in that: The physical guidance attention mechanism in step (3) includes: inputting the physical features into the attention generation module, generating a physical guidance spatial attention map through convolutional mapping and activation operations; using the spatial attention map to perform position-wise weighted enhancement of the semantic features, and performing residual fusion of the enhanced semantic features with the original semantic features to obtain the fused features. express.

5. The method according to claim 4, characterized in that: The fusion feature described in step (3) Specifically, it can be obtained through the following steps: (3.1) Input the calibrated polarization characterization into the physical flow branch, encode the scattering mechanism and perform feature mapping to obtain the physical feature representation. ; (3.2) Generate a spatial attention map using a physical-guided attention weight generation module. ; (3.3) Input the calibrated polarization representation into the semantic flow branch, perform convolutional encoding and extract the semantic feature representation to obtain the semantic feature representation. ; (3.4) Using spatial attention maps semantic feature representation Weighted enhancement is performed, and residual connections are used to fuse physical and semantic features to obtain fused features. : ； in, This indicates element-wise multiplication.

6. The method according to claim 5, characterized in that: The physical flow branch constructs physical features based on Freeman decomposition results. These physical features include scattering mechanism information corresponding to surface scattering components, secondary scattering components, and volume scattering components. Feature mapping is performed through a physical branch encoder to obtain physical feature representations. The implementation steps include: (3.1.1) The physical flow branch encodes the scattering mechanism of the polarization characterization after calibration, and constructs physical feature results based on the main scattering component. : ； in, These represent surface scattering, secondary scattering, and volume scattering components, respectively. This represents the total scattered power, which is the sum of the three scattered components. (3.1.2) will Input the physical branch encoder for feature mapping to obtain the physical feature representation. .

7. The method according to claim 1, characterized in that: The one-dimensional input sequence mentioned in step (4) is obtained specifically according to the following steps: (4.1) Using the center pixel of a local image patch as the classification anchor point, calculate the feature similarity between each pixel within the image patch and the center pixel according to the following formula: ； in, Represents the i-th pixel With the center pixel Similarity in feature space; for Corresponding feature representation; The feature representation corresponding to the center pixel; (4.2) Based on pixels With the center pixel The spatial distance is used to obtain the spatial decay weight of the pixel relative to the center pixel, combined with the exponential decay function. : ； in, For pixels spatial coordinates, Center pixel Spatial coordinates; express and The square of the spatial distance between the two Hyperparameters for controlling the spatial attenuation range; (4.3) Using the two-dimensional coordinates of each spatial location in the local image patch, the corresponding position feature vector is generated through learnable coordinate mapping, and then fused with the feature vector. By adding the features position by position, we obtain the position-enhanced feature representation. ; (4.4) The feature similarity and spatial decay weight are jointly modeled to obtain the comprehensive correlation score of each pixel in the image patch relative to the center pixel, and the feature representation of all pixels in the image patch is based on this score. The pixels are sorted in ascending order to form a one-dimensional input sequence for center pixel classification, with pixel features that have higher overall correlation scores located at the end of the one-dimensional input sequence; wherein, pixels Regarding the center pixel The overall relevance score is expressed as .

8. The method according to claim 1, characterized in that: The sequence modeling module based on Mamba in step (5) consists of one or more stacked Mamba blocks, used for long-distance dependency modeling of one-dimensional input sequences; the Mamba block includes a feature mapping unit, a local feature extraction unit, a nonlinear activation unit, and a gated modulation unit.

9. The method according to claim 1, characterized in that: In step (6), the feature vector corresponding to the last position is extracted from the sequence feature representation as the classification feature representing the center pixel, and the classification feature is input into the linear classification layer to obtain the category prediction result of the center pixel.

10. The method according to claim 1, characterized in that: In step (7), during the testing phase, the polarimetric synthetic aperture radar data samples to be tested are sequentially input into the trained learnable polarimetric calibration layer, physical-semantic dual-stream feature extraction network, physical guided attention mechanism, physical perception sequence rearrangement module and Mamba-based sequence modeling module. The classification layer outputs the category label corresponding to the test sample, and the above processing process is repeated for each sample in the image to be tested to obtain the pixel-level classification result map of the entire polarimetric synthetic aperture radar image.

Images