Heterogenous polarimetric SAR image classification method based on improved U-Net and capsule network

Abstract

The embodiment of the invention provides a heterogeneous polarized SAR image classification method and device based on an improved U-Net and a capsule network. The method comprises the following steps: firstly, collecting a heterogenous polarimetric SAR image and preprocessing the heterogenous polarimetric SAR image; then, constructing a deep learning model based on the improved U-Net; secondly, integrating a capsule network in the deep learning model, and providing object recognition which is more accurate than that of a traditional convolutional neural network by more accurately representing each part of an object and a spatial relationship between the parts; then establishing a multi-scale jump connection structure in the capsule network; and finally, feature domain adaptation of a source domain and a target domain is realized through a weight adversarial neural network, and heterogeneous sample classification is completed in combination with polarization prior constraint similarity analysis. The method effectively solves the problem of heterogeneous polarized SAR image data fusion, and has significant advantages in three-dimensional super-resolution imaging and complex recognition scenes.

Description

Technical Field

[0001] This invention relates to the field of SAR image processing technology, and in particular to a heterogeneous polarization SAR image classification method and apparatus based on an improved U-Net and capsule network. Background Technology

[0002] Synthetic Aperture Radar (SAR), as an advanced microwave imaging system, detects targets by actively emitting electromagnetic waves and receiving the scattered echoes from the targets. It possesses all-weather, all-day, and high-resolution remote sensing capabilities for Earth observation. SAR image classification is a crucial research area in SAR image interpretation, widely used in geological exploration, topographic mapping, vegetation growth assessment, urban planning, and marine monitoring. Compared to single-polarization SAR images, polarimetric SAR systems can obtain data from multiple polarization channels, providing a more complete characterization of ground object scattering characteristics. Classification using polarimetric SAR system data has become one of the main research directions in SAR image processing.

[0003] However, due to factors such as sensor and imaging mode, heterogeneous polarimetric SAR images suffer from differences in feature probability distribution and inconsistent classification tasks. Some existing methods indiscriminately stack and combine the features derived from polarimetric decomposition before inputting them into the network, ignoring the inherent limitations of the polarimetric decomposition method itself. During polarimetric feature normalization, some methods perform only simple normalization without considering data distribution characteristics, while others disregard all scattering information and various polarimetric scattering features, using only a single, incomplete polarimetric data input scheme as network input. These problems all contribute to difficulties in achieving high-precision cross-data classification. Therefore, how to utilize artificial intelligence technology to provide an algorithmic model and operating device for accurate classification of heterogeneous polarimetric SAR images is the technical problem this invention aims to solve. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies by providing a heterogeneous polarimetric SAR image classification method and apparatus based on an improved U-Net and capsule network, thereby solving the problems mentioned in the background section. The specific technical solution is as follows:

[0006] According to a first aspect of this application, a heterogeneous polarimetric SAR image classification method based on an improved U-Net and capsule network is provided, the method comprising the following specific steps:

[0007] Step 1: Collect heterogeneous polarimetric SAR images and perform preprocessing. The heterogeneous polarimetric SAR images are mainly sourced from domestic high-resolution image MASR and mainstream international SAR datasets.

[0008] Step 2: Construct a deep learning model based on the improved U-Net;

[0009] Step 3: Integrate capsule networks into deep learning models to provide more accurate object recognition than traditional convolutional neural networks by more accurately representing the parts of an object and the spatial relationships between them;

[0010] Step 4: Establish a multi-scale skip connection structure in the capsule network;

[0011] Step 5: Use the deep learning model to classify heterogeneous polarimetric SAR images.

[0012] In one possible embodiment, as a preferred technical solution, the specific method for collecting and preprocessing heterogeneous polarimetric SAR images in step one includes the following steps:

[0013] Preferably, the SAR image acquired from the data source is subjected to source range calibration, wherein the source range includes:

[0014] Polarization methods, specifically including: full polarization and dual polarization; parameter characteristics, specifically including: spatial resolution, coverage, time series requirements, incident angle range, and frequency band requirements; heterogeneous source combination, specifically including: comparison of different sensors, different time variations, the influence of different incident angles, and fusion of different frequency bands;

[0015] Preferably, the SAR images acquired from the data source are subjected to data management and preprocessing, and the preprocessing method includes:

[0016] The original digital values ​​are converted into radar backscattering coefficients to address crosstalk and channel imbalance introduced by the fully polarimetric data correction system described in S1-1, ensuring the accuracy of the polarization scattering matrix. Slant range / ground distance images are converted into maps with latitude and longitude using a digital elevation model (DEM). Orthorectification is performed using a high-precision DEM, based on amplitude cross-correlation, phase coherence, and feature point matching. Control points are found between image pairs, and high-order polynomial transformations are applied to precisely align images from different sources and time phases to the same geographic coordinate system, achieving sub-pixel accuracy. The registered images are resampled to the same pixel size and grid, and speckle noise filtering is applied to single-time-phase images to suppress speckle noise. The DEM is used to correct geometric deformation and radiation distortion caused by terrain, yielding normalized backscattering coefficients.

[0017] In one possible embodiment, as a preferred technical solution, the specific structure of the deep learning model based on the improved U-Net in step two includes:

[0018] Input: Receives pre-processed heterogeneous polarimetric SAR images;

[0019] Feature Extraction Layer: Combining the data characteristics of heterogeneous polarimetric SAR images, discriminative features are extracted from the images based on a non-negative sparse coding method. The discriminative features are solved through an optimization problem. The specific method of the non-negative sparse coding method includes the following calculation formula:

[0020]

[0021] Where Y is the observation matrix, Z is the basis matrix, and μ and ν are regularization parameters;

[0022] Feature fusion layer: This layer utilizes visual cognitive mechanisms to mine prior information from heteropolarimetric SAR images. It combines sparse representation theory and a classifier-based learning method to extract discriminative features from heteropolarimetric SAR images, thereby improving the accuracy of heteropolarimetric SAR image change detection. The specific methods of this learning method include the following calculation formulas:

[0023] P(f|I)∝P(I|f)P(f);

[0024] Where f is the prior feature distribution and I is the input image;

[0025] Feature fusion at different resolutions is achieved through bilinear interpolation. Specific methods include the following calculation formulas:

[0026]

[0027] Where K(s) is the scale factor kernel function;

[0028] Output network: Based on the Leweenberg-Marquardt algorithm, single-pulse forward-looking super-resolution imaging of heterogeneous polarization SAR images is achieved, enabling three-dimensional super-resolution imaging and accurate extraction of target parameters such as ship hulls and base stations within the sea area; the specific method of the Leweenberg-Marquardt algorithm includes the following calculation formula:

[0029]

[0030] Where θ is the parameter to be estimated, and R(θ) is the regularization term.

[0031] Preferably, the connection relationships between the structures are as follows: the input end is directly connected to the feature extraction layer; the feature extraction layer is directly connected to the feature fusion layer; and the feature fusion layer is directly connected to the output network.

[0032] In one possible embodiment, as a preferred technical solution, the specific structure of the capsule network integrated into the deep learning model in step three includes:

[0033] Input layer: processes raw input data, including the output network heterogeneous polarimetric SAR image single-pulse forward-looking super-resolution imaging described in S2-4;

[0034] Initial capsule layer: Initial features are extracted using convolution operations, feature weights are calculated through the following steps, and they are represented as vectors;

[0035] Master Capsule Layer: The initial features are vectorized to generate multiple master capsules, each capturing different types of features. A spatial attention gating mechanism is introduced into the deep learning model described in S3, with specific methods and calculation formulas as follows:

[0036]

[0037] Where z ij Let σ be the feature vector of the j-th channel of the i-th feature map, and σ be the Sigmoid function.

[0038] Dynamic routing: The output features of the main capsule are optimized for inter-class separation through a dynamic routing mechanism. The dynamic routes are passed to higher-level capsules, and the connection strength is determined based on consistency. Specific methods include the following calculation formula:

[0039]

[0040] Where r ij Let s be the routing weight from capsule i to neuron j. i The capsule input vector;

[0041] Classification capsule layer: The higher-level capsules output the final classification results.

[0042] Preferably, the capsule network is better able to capture and encode complex patterns and hierarchical information.

[0043] In one possible embodiment, as a preferred technical solution, the specific method for establishing the multi-scale skip connection structure in the capsule network in step four includes the following steps:

[0044] Preferably, a polarization response constraint layer is constructed in the multi-scale skip connection structure described in step four to force the network to learn polarization scattering characteristics. The specific method includes the following calculation formula:

[0045]

[0046] in The polarization matrix is ​​to be predicted, where P is the true polarization matrix and α is the regularization coefficient.

[0047] Preferably, a polarization potential energy function calculation unit is provided in the polarization response constraint layer. The specific method includes the following calculation formula:

[0048]

[0049] Where v i Let u be the input vector. j Let β and γ be the output vector, and β and γ be the potential energy parameters.

[0050] Preferably, a space-frequency joint domain scattering model building unit is set in the polarization response constraint layer. The space-frequency joint domain scattering model building unit establishes a three-dimensional electromagnetic scattering model of a typical structure. The specific method for establishing the building unit model includes the following calculation formula:

[0051] S=∫ V J(r)e -jk·r dV;

[0052] Where J(r) is the volume current density and k is the wave vector.

[0053] The specific method for setting the building unit includes the following calculation formula:

[0054] S(f, k)=∫E(r)e -j2π(fr+kx) dr;

[0055] Where E(r) is the electric field distribution;

[0056] Preferably, a position estimation accuracy CRLB non-uniform aperture division motion compensation unit is set in the polarization response constraint layer. The motion compensation unit performs non-uniform aperture division to overcome the influence of irregular motion trajectories on imaging. The specific method of non-uniform aperture division by the motion compensation unit includes the following calculation formula:

[0057]

[0058] in For estimating parameters.

[0059] Preferably, the multi-scale jump connection structure can achieve accurate modeling of fully polarized forward-looking SAR three-dimensional imaging and reduce wavelet disturbances under static interference preset conditions.

[0060] In one possible embodiment, as a preferred technical solution, the specific method for classifying heterogeneous polarimetric SAR images using the deep learning model in step five includes the following steps:

[0061] Preferably, the deep learning model, based on the probability distribution characteristics of heterogeneous polarimetric SAR image features, combines polarimetric information to constrain similarity, and uses polarimetric priors to determine the similarity between heterogeneous samples in the target domain and various categories in the source domain, thereby achieving the classification of heterogeneous samples in the target domain.

[0062] Preferably, the deep learning model is combined with a weighted adversarial neural network, which utilizes an adversarial learning strategy to achieve feature domain adaptation between the source and target domains, thereby improving the classification accuracy of the target domain polarimetric SAR image. The weighted adversarial neural network performs heterogeneous polarimetric SAR image classification based on the heterogeneous polarimetric SAR image classification task, and the specific method includes the following calculation formula:

[0063]

[0064] Where L GAN To generate the adversarial loss function, L Polar λ is the polarization constraint term, and λ is the equilibrium coefficient.

[0065] According to a second aspect of this application, a heterogeneous polarimetric SAR image classification device based on an improved U-Net and capsule network is provided, the device comprising:

[0066] The preprocessing module is used to input heterogeneous polarimetric SAR images into the image classification model. The heterogeneous polarimetric SAR images are mainly sourced from domestic high-resolution image MASR and mainstream international SAR datasets.

[0067] An image classification module is used to input the image to be processed into the image classification model and obtain the classification result of the image to be processed output by the image classification model.

[0068] An image determination module is used to determine initial detection images with the same classification category from multiple images to be processed based on the classification results;

[0069] Preferably, the image classification model is the heterogeneous polarization SAR image classification model based on the improved U-Net and capsule network.

[0070] Preferably, the classification result includes: category probability, and the image to be processed is multiple;

[0071] According to a third aspect of this application, an electronic device is provided, comprising:

[0072] Memory, used to store computer programs;

[0073] A processor, when executing a program stored in memory, implements any of the methods described in the first aspect.

[0074] According to a fourth aspect of this application, a computer-readable storage medium is provided, characterized in that the computer-readable storage medium stores computer instructions, which, when executed by a computer, cause the computer to connect to the device via a technology such as wireless communication and perform the method described in any one of claims 1-6.

[0075] This application also provides a computer program product containing instructions that, when run on a computer, cause the computer to perform any of the methods described in the first aspect above.

[0076] Compared with the prior art, the embodiments of this application have the following main advantages:

[0077] First, this invention proposes an improved U-Net deep learning model. Combining the data characteristics of heteropolar SAR images, it extracts discriminative features from heteropolar SAR images based on a non-negative sparse coding method. The discriminative features are solved by an optimization problem. The learning method of extracting discriminative features from heteropolar SAR images by combining sparse representation theory and classifier improves the accuracy of heteropolar SAR image change detection.

[0078] Second, this invention proposes an integrated capsule network in the improved U-Net deep learning model. The initial features are vectorized to generate multiple master capsules. Each capsule captures different types of features. The output features of the master capsules are optimized for inter-class separation through a dynamic routing mechanism. The dynamic routing is passed to higher-level capsules, and the higher-level capsules output the final classification results. The capsule network can better capture and encode complex patterns and hierarchical information.

[0079] Third, this invention proposes to establish a multi-scale jump connection structure in the capsule network and construct a polarization response constraint layer to force the network to learn polarization scattering characteristics. By setting up a polarization potential energy function calculation unit, a space-frequency joint domain scattering model construction unit, and a non-uniform aperture division motion compensation unit for position estimation accuracy, it can achieve accurate modeling of full polarization forward-looking SAR three-dimensional imaging and reduce wavelet disturbances under static interference preset conditions.

[0080] Fourth, based on the characteristics of the probability distribution of heterogeneous polarimetric SAR images, this invention combines polarimetric information to constrain similarity, uses polarimetric priors to assess the similarity between heterogeneous samples in the target domain and various categories in the source domain, thereby achieving the classification of heterogeneous samples in the target domain. Combined with a weighted adversarial neural network, an adversarial learning strategy is used to achieve feature domain adaptation between the source and target domains, thereby improving the classification accuracy of polarimetric SAR images in the target domain.

[0081] This method effectively solves the challenge of heterogeneous polarimetric SAR image data fusion and has significant advantages in 3D super-resolution imaging and complex recognition scenarios. Of course, implementing any product or method of this application does not necessarily require achieving all of the advantages described above simultaneously.

[0082] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0083] Figure 1A flowchart illustrating a heterogeneous polarimetric SAR image classification method based on an improved U-Net and capsule network, provided for embodiments of this application;

[0084] Figure 2 A schematic diagram of an improved U-Net model structure integrating capsule networks is provided for an embodiment of this application;

[0085] Figure 3 A schematic diagram of a multi-scale skip connection structure of an improved U-Net model provided in this application embodiment;

[0086] Figure 4 A schematic diagram of the structure of a heterogeneous polarization SAR image classification device based on an improved U-Net and capsule network provided in this application embodiment; Detailed Implementation

[0087] The method of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0088] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0089] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0090] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0091] For ease of description, spatial relative terms such as "above," "on top of," "on the upper surface of," "above," etc., are used herein to describe the spatial positional relationship of a device or feature as shown in the figures to other devices or features. It should be understood that spatial relative terms are intended to encompass different orientations in use or operation beyond the orientation of the device as described in the figures. For example, if the device in the figures were inverted, a device described as "above" or "on top of" other devices or structures would subsequently be positioned as "below" or "under" other devices or structures. Thus, the exemplary term "above" can include both "above" and "below." The device may also be positioned in other different ways (rotated 90 degrees or in other orientations), and the spatial relative descriptions used herein will be interpreted accordingly.

[0092] Figure 1 This paper presents a flowchart illustrating a heterogeneous polarimetric SAR image classification method based on an improved U-Net and capsule network, according to an embodiment of this application. The specific method includes the following steps:

[0093] S1: Data Preparation and Preprocessing: Collect heterogeneous polarimetric SAR images and perform preprocessing. The heterogeneous polarimetric SAR images are mainly sourced from domestic high-resolution imagery MASR and mainstream international SAR datasets. The method includes the following steps:

[0094] S1-1: Determine the source range of the acquired SAR image. The source range includes:

[0095] Polarization methods, specifically including: full polarization and dual polarization; parameter characteristics, specifically including: spatial resolution, coverage, time series requirements, incident angle range, and frequency band requirements; heterogeneous source combination, specifically including: comparison of different sensors, different time variations, the influence of different incident angles, and fusion of different frequency bands;

[0096] S1-2: Perform data management and preprocessing on the SAR images acquired from the data source. The preprocessing method includes:

[0097] The original digital values ​​are converted into radar backscattering coefficients to address crosstalk and channel imbalance introduced by the fully polarimetric data correction system described in S1-1, ensuring the accuracy of the polarization scattering matrix. Slant range / ground distance images are converted into maps with latitude and longitude using a digital elevation model (DEM). Orthorectification is performed using a high-precision DEM, based on amplitude cross-correlation, phase coherence, and feature point matching. Control points are found between image pairs, and high-order polynomial transformations are applied to precisely align images from different sources and time phases to the same geographic coordinate system, achieving sub-pixel accuracy. The registered images are resampled to the same pixel size and grid, and speckle noise filtering is applied to single-time-phase images to suppress speckle noise. The DEM is used to correct geometric deformation and radiation distortion caused by terrain, yielding normalized backscattering coefficients.

[0098] S2: Model Construction: Constructing a deep learning model based on the improved U-Net; the method includes the following steps:

[0099] S2-1: Input: Receives heterogeneous polarimetric SAR images after data preprocessing;

[0100] S2-2: Feature Extraction Layer: Combining the data characteristics of heterogeneous polarimetric SAR images, discriminative features are extracted from the heterogeneous polarimetric SAR images based on a non-negative sparse coding method. The discriminative features are solved through an optimization problem. The non-negative sparse coding method includes the following calculation formula:

[0101]

[0102] Where Y is the observation matrix, Z is the basis matrix, and μ and ν are regularization parameters;

[0103] S2-3: Feature Fusion Layer: This layer utilizes visual cognitive mechanisms to mine prior information from heteropolarimetric SAR images. It combines sparse representation theory and a classifier-based learning method to extract discriminative features from heteropolarimetric SAR images, improving the accuracy of heteropolarimetric SAR image change detection. The learning method includes the following calculation formula:

[0104] P(f|I)∝P(I|f)P(f);

[0105] Where f is the prior feature distribution and I is the input image;

[0106] The method for fusing features at different resolutions using bilinear interpolation includes the following calculation formula:

[0107]

[0108] Where K(s) is the scale factor kernel function;

[0109] S2-4: Output Network: Based on the Levenberg-Marquardt algorithm, single-pulse forward-looking super-resolution imaging of heterogeneous polarization SAR images is achieved, realizing three-dimensional super-resolution imaging and accurate extraction of target parameters such as ship hulls and base stations within the sea area; the method of the Levenberg-Marquardt algorithm includes the following calculation formula:

[0110]

[0111] Where θ is the parameter to be estimated, and R(θ) is the regularization term.

[0112] The connections between the structures are as follows: the input layer is directly connected to the feature extraction layer; the feature extraction layer is directly connected to the feature fusion layer; and the feature fusion layer is directly connected to the output network.

[0113] S3: Integrating a capsule network into the deep learning model provides more accurate object recognition than traditional convolutional neural networks by more precisely representing the various parts of an object and the spatial relationships between them; the method includes the following steps:

[0114] S3-1: Input layer: processes raw input data, including the heterogeneous polarimetric SAR image single-pulse forward-looking super-resolution imaging described in S2-4;

[0115] S3-2: Initial capsule layer: Initial features are extracted using convolution operations, feature weights are calculated through the following steps, and they are represented as vectors;

[0116] S3-3: Master Capsule Layer: The initial features are vectorized to generate multiple master capsules, each capturing different types of features. A spatial attention gating mechanism is introduced into the deep learning model described in S3, and the method includes the following calculation formula:

[0117]

[0118] Where z ij Let σ be the feature vector of the j-th channel of the i-th feature map, and σ be the Sigmoid function.

[0119] S3-4: Dynamic Routing: The output features of the main capsule are optimized for inter-class separation through a dynamic routing mechanism. The dynamic route is passed to higher-level capsules, and the connection strength is determined based on consistency. The method includes the following calculation formula:

[0120]

[0121] Where r ij Let s be the routing weight from capsule i to neuron j. i The capsule input vector;

[0122] S3-5: Classification Capsule Layer: The higher-level capsules output the final classification results.

[0123] The capsule network is better able to capture and encode complex patterns and hierarchical information.

[0124] S4: Establish a multi-scale skip connection structure in the capsule network; the method includes the following steps:

[0125] S4-1: Construct a polarization response constraint layer in the multi-scale skip connection structure described in S4 to force the network to learn polarization scattering characteristics. The method includes the following calculation formula:

[0126]

[0127] in The polarization matrix is ​​to be predicted, where P is the true polarization matrix and α is the regularization coefficient.

[0128] S4-2: A polarization potential energy function calculation unit is set in the polarization response constraint layer described in S4-1. The method includes the following calculation formula:

[0129]

[0130] Where v i Let u be the input vector. j Let β and γ be the output vector, and β and γ be the potential energy parameters.

[0131] S4-3: A space-frequency joint domain scattering model building unit is set in the polarization response constraint layer described in S4-1. The space-frequency joint domain scattering model building unit establishes a three-dimensional electromagnetic scattering model of a typical structure. The method for establishing the building unit model includes the following calculation formula:

[0132] S=∫ V J(r)e -jk·r dV;

[0133] Where J(r) is the volume current density and k is the wave vector.

[0134] The method set by the building unit includes the following calculation formula:

[0135] S(f, k)=∫E(r)e -j2π(fr+kx) dr;

[0136] Where E(r) is the electric field distribution;

[0137] S4-4: A position estimation accuracy CRLB non-uniform aperture division motion compensation unit is set in the polarization response constraint layer described in S4-1. The motion compensation unit performs non-uniform aperture division to overcome the influence of irregular motion trajectories on imaging. The method of non-uniform aperture division by the motion compensation unit includes the following calculation formula:

[0138]

[0139] in For estimating parameters.

[0140] The multi-scale jump connection structure enables accurate modeling of fully polarized forward-looking SAR three-dimensional imaging and reduces wavelet disturbances under static interference preset conditions.

[0141] S5: Model Application: Using the deep learning model for heterogeneous polarimetric SAR image classification; the method includes the following steps:

[0142] S5-1: The deep learning model, based on the characteristics of the probability distribution of heterogeneous polarimetric SAR image features, combines polarization information to constrain similarity, and uses polarization priors to determine the similarity between heterogeneous samples in the target domain and various categories in the source domain, thereby achieving the classification of heterogeneous samples in the target domain.

[0143] S5-2: The deep learning model, combined with a weighted adversarial neural network, utilizes an adversarial learning strategy to achieve feature domain adaptation between the source and target domains, thereby improving the classification accuracy of polarimetric SAR images in the target domain. The weighted adversarial neural network performs heterogeneous polarimetric SAR image classification based on the heterogeneous polarimetric SAR image classification task. The method includes the following calculation formula:

[0144]

[0145] Where L GAN To generate the adversarial loss function, L Polar λ is the polarization constraint term, and λ is the equilibrium coefficient.

[0146] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0147] Figure 2 This illustration shows a schematic diagram of an improved U-Net model structure integrating capsule networks according to an embodiment of this application. The specific method includes the following steps:

[0148] The U-Net model's workflow mainly includes downsampling and upsampling, i.e., an encoder-decoder structure. After convolution, the number of channels in the feature map of the input image continuously increases. Then, max pooling is used to increase the receptive field and extract image features, while the feature map size gradually decreases. The decoder part uses convolution to double the image size. The U-Net model uses convolution and the ReLU activation function to perform image sampling and output feature maps.

[0149] The following provides a preferred technical solution of this embodiment, wherein the method mainly includes three parts: encoding, capsule network and decoding.

[0150] The encoding part of U-net is designed with the first fourteen layers of a VGG16 architecture, comprising ten convolutional layers and four pooling layers, which can be divided into four modules. The first and second modules each contain two convolutional layers and one max-pooling layer; the third and fourth modules each contain three convolutional layers and one max-pooling layer. The convolutional kernels of the convolutional layers are 3×3 with a stride of 1; ReLU activation is applied after convolution, followed by batch normalization (BN); the max-pooling layers have a pooling window size of 2×2 with a stride of 2.

[0151] The capsule network mainly consists of three convolutional layers, one primary capsule layer, and one classification capsule layer. The first convolutional layer has a 5×5 kernel size, while the second and third convolutional layers have 1×1 kernel sizes, with a stride of 1. All convolutional layers are activated using the ReLU function after convolution. The primary capsule layer uses a 5×5 kernel size with a stride of 1. The classification capsule layer is an improvement on the digit capsule layer, with a locally connected dynamic routing between it and the primary capsules; it also has a 1×1 kernel size and a stride of 1.

[0152] The decoding section is symmetrical to the encoding section, mainly consisting of ten convolutional layers and four upsampling-convolutional layers, which can also be divided into four modules. The first and second modules each contain one upsampling-convolutional layer and three convolutional layers; the third and fourth modules each contain one upsampling-convolutional layer and two convolutional layers. Each upsampling-convolutional layer has an upsampling factor of 2×2 and a kernel size of 2×2. After the upsampling-convolution operation, the resulting feature map is concatenated with the feature map obtained from the encoding section. The convolutional layers have a kernel size of 3×3 and a stride of 1. Each convolution is followed by ReLU activation and batch normalization (BN). Furthermore, the decoding section ends with a 1×1 convolutional layer activated by the Sigmoid function.

[0153] Figure 3 This illustration shows a schematic diagram of a multi-scale skip connection structure for an improved U-Net model provided in an embodiment of this application. The specific method includes the following steps:

[0154] The U-Net model extracts information from images in a multi-scale manner. Low-level detail feature maps contain richer spatial information, such as boundaries, while high-level feature maps contain more high-level semantic features, such as object locations. However, as the network downsamples and upsamples, this high- and low-level information is not fully utilized. By introducing full-scale skip connections, low-level details and high-level semantics are fused in the full-scale feature maps, making full use of multi-scale features while requiring fewer parameters.

[0155] The multi-scale skip connection structure includes skip connections from the encoder to the decoder, as well as connections between encoders at different levels. Its information comes from two sources: shallower encoders (including those at the same level) and deeper decoders. Features from different levels are scaled uniformly using maxpooling and bilinear upsampling.

[0156] The following provides a preferred technical solution of this embodiment, wherein the method includes: using... Figure 3 Taking the nodes in the code as an example, the convolution of the decoding layer has two steps. The first step is to perform convolution on the information from each node. When performing the first convolutional layer, the data from each layer is convolved to the same number of feature maps (here, it is N / 5, where N is the number of feature maps in the layer). The second step is to fuse and extract information from the stacked features through convolution.

[0157] Figure 4 This illustration shows a schematic diagram of a heterogeneous polarimetric SAR image classification device based on an improved U-Net and capsule network according to an embodiment of this application; the specific structure includes:

[0158] The preprocessing module 400, the image classification module 410, and the image determination module 420, wherein:

[0159] The preprocessing module is used to input heterogeneous polarimetric SAR images into the image classification model. The heterogeneous polarimetric SAR images are mainly sourced from domestic high-resolution image MASR and mainstream international SAR datasets.

[0160] An image classification module is used to input the image to be processed into the image classification model and obtain the classification result of the image to be processed output by the image classification model.

[0161] An image determination module is used to determine initial detection images with the same classification category from multiple images to be processed based on the classification results;

[0162] Preferably, the image classification model is the heterogeneous polarization SAR image classification model based on the improved U-Net and capsule network.

[0163] Preferably, the classification result includes: category probability, and the image to be processed is multiple;

[0164] Specific limitations regarding the heterogeneous polarimetric SAR image classification device based on the improved U-Net and capsule networks can be found in the limitations of the heterogeneous polarimetric SAR image classification method based on the improved U-Net and capsule networks mentioned above, and will not be repeated here. Each module in the aforementioned heterogeneous polarimetric SAR image classification device based on the improved U-Net and capsule networks can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0165] This embodiment also provides preferred experimental conditions: the experimental environment is a 64-bit Windows 11 operating system, using an NVIDIA GeForce RTX 3090 Ti graphics card for accelerated image processing, the programming language is Python (version 3.13), and the deep learning platform is PyTorch (version 2.4.1). During network model training, the training image size is 256×256, the batch size is set to 16, the initial learning rate is 0.001, and the number of iterations is 400.

[0166] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented in hardware, software modules executed by a processor, or a combination of both. The software modules can be housed in random access memory, main memory, read-only memory, electrically erasable programmable memory, registers, hard disks, or any other form of storage medium known in the art. The hardware modules can be housed within the apparatus disclosed herein and installed in any other form of device known in the field of SAR image processing, but such implementation should not be considered beyond the scope of this invention.

[0167] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented by means of apparatus, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of the invention.

[0168] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A heterogeneous polarimetric SAR image classification method based on an improved U-Net and capsule network, characterized in that: The specific steps are as follows: S1: Data preparation and preprocessing: Collect heterogeneous polarimetric SAR images and perform preprocessing. The heterogeneous polarimetric SAR images are mainly sourced from domestic high-resolution image MASR and mainstream international SAR datasets. S2: Model Building: Building a deep learning model based on the improved U-Net; S3: Integrating capsule networks into the deep learning model provides more accurate object recognition than traditional convolutional neural networks by more accurately representing the various parts of an object and the spatial relationships between them; S4: Establish a multi-scale skip connection structure in the capsule network; S5: Model Application: Use the deep learning model to classify heterogeneous polarization SAR images.

2. The heterogeneous polarimetric SAR image classification method based on improved U-Net and capsule network according to claim 1, characterized in that, The specific method for collecting heterogeneous polarization SAR images and performing preprocessing in S1 The steps are as follows: S1-1: Determine the source range of the SAR image acquired from the data source, wherein the source range includes: Polarization methods, specifically including: full polarization and dual polarization; parameter characteristics, specifically including: spatial resolution, coverage, time series requirements, incident angle range, and frequency band requirements; heterogeneous source combination, specifically including: comparison of different sensors, different time variations, the influence of different incident angles, and fusion of different frequency bands; S1-2: Perform data management and preprocessing on the SAR images acquired from the data source. The preprocessing method includes: The original digital values ​​are converted into radar backscattering coefficients to address crosstalk and channel imbalance introduced by the fully polarimetric data correction system described in S1-1, ensuring the accuracy of the polarization scattering matrix. Slant range / ground distance images are converted into maps with latitude and longitude using a digital elevation model (DEM). Orthorectification is performed using a high-precision DEM, based on amplitude cross-correlation, phase coherence, and feature point matching. Control points are found between image pairs, and high-order polynomial transformations are applied to precisely align images from different sources and time phases to the same geographic coordinate system, achieving sub-pixel accuracy. The registered images are resampled to the same pixel size and grid, and speckle noise filtering is applied to single-time-phase images to suppress speckle noise. The DEM is used to correct geometric deformation and radiation distortion caused by terrain, yielding normalized backscattering coefficients.

3. The heterogeneous polarimetric SAR image classification method based on improved U-Net and capsule network according to claim 1, characterized in that, The specific structure of the deep learning model based on the improved U-Net in S2 includes: S2-1: Input: Receives heterogeneous polarimetric SAR images after data preprocessing; S2-2: Feature Extraction Layer: Combining the data characteristics of heterogeneous polarimetric SAR images, discriminative features are extracted from the heterogeneous polarimetric SAR images based on a non-negative sparse coding method. The discriminative features are solved through an optimization problem. The specific method of the non-negative sparse coding method includes the following calculation formula: Where Y is the observation matrix, Z is the basis matrix, and μ and ν are regularization parameters; S2-3: Feature Fusion Layer: This layer utilizes visual cognitive mechanisms to mine prior information from heteropolarimetric SAR images. It combines sparse representation theory and a classifier-based learning method to extract discriminative features from heteropolarimetric SAR images, improving the accuracy of heteropolarimetric SAR image change detection. The specific methods of this learning method include the following calculation formulas: P(f|I)∝P(I|f)P(f); Where f is the prior feature distribution and I is the input image; Feature fusion at different resolutions is achieved through bilinear interpolation. Specific methods include the following calculation formulas: Where K(s) is the scale factor kernel function; S2-4: Output Network: Based on the Levenburg-Marquardt algorithm, single-pulse forward-looking super-resolution imaging of heterogeneous polarization SAR images is achieved, realizing three-dimensional super-resolution imaging and accurate extraction of target parameters such as ship hulls and base stations within the sea area; the specific method of the Levenburg-Marquardt algorithm includes the following calculation formula: Where θ is the parameter to be estimated, and R(θ) is the regularization term. The connections between the structures are as follows: the input layer is directly connected to the feature extraction layer; the feature extraction layer is directly connected to the feature fusion layer; and the feature fusion layer is directly connected to the output network.

4. The heterogeneous polarimetric SAR image classification method based on improved U-Net and capsule network according to claim 1, characterized in that, The specific structure of the capsule network integrated in the deep learning model in S3 includes: S3-1: Input layer: processes raw input data, including the heterogeneous polarimetric SAR image single-pulse forward-looking super-resolution imaging described in S2-4; S3-2: Initial capsule layer: Initial features are extracted using convolution operations, feature weights are calculated through the following steps, and they are represented as vectors; S3-3: Master Capsule Layer: The initial features are vectorized to generate multiple master capsules, each capturing different types of features. A spatial attention gating mechanism is introduced into the deep learning model described in S3. The specific method and calculation formula are as follows: Where z ij Let σ be the feature vector of the j-th channel of the i-th feature map, and σ be the Sigmoid function. S3-4: Dynamic Routing: The output features of the main capsule are optimized for inter-class separation through a dynamic routing mechanism. The dynamic route is passed to higher-level capsules, and the connection strength is determined based on consistency. Specific methods include the following calculation formula: Where r ij Let s be the routing weight from capsule i to neuron j. i The capsule input vector; S3-5: Classification Capsule Layer: The higher-level capsules output the final classification results. The capsule network is better able to capture and encode complex patterns and hierarchical information.

5. The heterogeneous polarimetric SAR image classification method based on improved U-Net and capsule network according to claim 1, characterized in that, The specific method for establishing the multi-scale skip connection structure in the capsule network in S4 The steps are as follows: S4-1: Construct a polarization response constraint layer in the multi-scale skip connection structure described in S4 to force the network to learn polarization scattering characteristics. The specific method includes the following calculation formula: in The polarization matrix is ​​to be predicted, where P is the true polarization matrix and α is the regularization coefficient. S4-2: A polarization potential energy function calculation unit is set in the polarization response constraint layer described in S4-1. The specific method includes the following calculation formula: Where v i Let u be the input vector. j Let β and γ be the output vector, and β and γ be the potential energy parameters. S4-3: A space-frequency joint domain scattering model building unit is set in the polarization response constraint layer described in S4-1. The space-frequency joint domain scattering model building unit establishes a three-dimensional electromagnetic scattering model of a typical structure. The specific method for establishing the building unit model includes the following calculation formula: S=∫ V J(r)e -jk·r dV; Where J(r) is the volume current density and k is the wave vector. The specific method for setting the building unit includes the following calculation formula: S(f,k) = ∫E(r)e -j2π(fr+kx) dr; Where E(r) is the electric field distribution; S4-4: In the polarization response constraint layer described in S4-1, a position estimation accuracy CRLB non-uniform aperture division motion compensation unit is set. The motion compensation unit performs non-uniform aperture division to overcome the influence of irregular motion trajectories on imaging. The specific method for non-uniform aperture division of the motion compensation unit includes the following calculation formula: in For estimating parameters. The multi-scale jump connection structure enables accurate modeling of fully polarized forward-looking SAR three-dimensional imaging and reduces wavelet disturbances under static interference preset conditions.

6. The heterogeneous polarimetric SAR image classification method based on improved U-Net and capsule network according to claim 1, characterized in that, The specific method for heterogeneous polarization SAR image classification using the deep learning model described in S5 The steps are as follows: S5-1: The deep learning model, based on the characteristics of the probability distribution of heterogeneous polarimetric SAR image features, combines polarization information to constrain similarity, and uses polarization priors to determine the similarity between heterogeneous samples in the target domain and various categories in the source domain, thereby achieving the classification of heterogeneous samples in the target domain. S5-2: The deep learning model, combined with a weighted adversarial neural network, utilizes an adversarial learning strategy to achieve feature domain adaptation between the source and target domains, thereby improving the classification accuracy of polarimetric SAR images in the target domain. The weighted adversarial neural network performs heterogeneous polarimetric SAR image classification based on the heterogeneous polarimetric SAR image classification task. The specific method includes the following calculation formula: Where L GAN To generate the adversarial loss function, L Polar λ is the polarization constraint term, and λ is the equilibrium coefficient.

7. An apparatus for performing the heterogeneous polarimetric SAR image classification method based on an improved U-Net and capsule network as described in any one of claims 1-6, characterized in that, The device includes: The preprocessing module is used to input heterogeneous polarimetric SAR images into the image classification model. The heterogeneous polarimetric SAR images are mainly sourced from domestic high-resolution imagery MASR and mainstream international SAR datasets. The image classification model is a heterogeneous polarimetric SAR image classification model based on an improved U-Net and capsule network as described in any one of claims 1-6. An image classification module is used to input the image to be processed into the image classification model and obtain the classification result of the image to be processed output by the image classification model; the classification result includes: category probability; the image to be processed is multiple; the device further includes: An image determination module is used to determine, based on the classification results, an initial detection image that shares the same classification category from multiple images to be processed.

8. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, when executing a program stored in memory, implements the method of any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a computer, cause the computer to connect to the device via wireless communication or other technologies and perform the method described in any one of claims 1-6.

Images