An Interactive CNN Classification Method Based on Scattering Mechanism for Compact Polarimetric SAR

Abstract

The present invention discloses an interactive CNN classification method based on a scattering mechanism for compact polarimetric SAR, which includes extracting the Stokes vector: extracting the scattering matrix [S] of the compact polarimetric SAR data and extracting the Stokes vector to obtain the scattering mechanisms of different ground object targets; obtaining the scattering mechanisms: calculating the polarization degree m and the relative phase difference δ as the basic parameters for target decomposition, and distinguishing the scattering mechanisms of different ground object targets through m-δ decomposition; classifying the test samples, calculating the confusion matrix of the classification results, classifying the image information of the compact polarimetric SAR through the established neural network model, calculating the confusion matrix of each ground object, and obtaining the classification accuracy of each ground object. The present invention uses a multi-channel convolutional neural network method with weight sharing and an interactive channel mechanism in the field of compact polarimetric SAR, improving the classification accuracy.

Description

Technical Field

[0001] The present invention relates to an interactive CNN classification method based on a scattering mechanism for compact polarimetric SAR, belonging to the field of polarimetric synthetic aperture radar image processing. Background Art

[0002] Synthetic Aperture Radar (SAR) is an active microwave remote sensing imaging system with the ability to work all day and all weather. By changing the operating frequency, polarization mode, and irradiation direction of the transmitted wave, etc., it can reflect the scattering characteristics of the observed ground objects and targets in an intuitive high-resolution image manner, and meet a wide range of application requirements at different levels.

[0003] With the successive development of comprehensive microwave imaging sensors, the foreign research on microwave imaging technology has gradually shifted from single-band, single-polarization, single-angle to multi-band, multi-polarization, multi-angle, and the detection, extraction, and application of comprehensive microwave imaging information. The single-polarization SAR radar system uses an antenna with a single polarization mode to transmit and receive radar signals. Since radar signals are electromagnetic waves with vector properties, this is equivalent to scalar processing of the vector electromagnetic waves, resulting in loss of the information carried by the radar signals. In order to comprehensively measure radar signals, the full-polarization radar technology has been developed, which uses two mutually perpendicular antennas to transmit and receive the components of radar signals in two mutually perpendicular directions to obtain all the information about scattering carried by the radar signals.

[0004] The idea of compact polarization is proposed in view of the design problems of the existing spaceborne polarimetric SAR system and is a new system proposed after the development of full polarization. It has advantages that cannot be compared with full-polarization technology, that is, there is no difference between co-polarization and cross-polarization. This will endow it with good inter-channel consistency inherently, thus reducing the complexity of the compensation and calibration system, and largely avoiding the amplification of the cross-polarization channel system noise during the compensation process, improving the polarimetric imaging quality of weak reflection targets. Since compact polarization only has two polarization channels, under the condition of maintaining the data downlink rate, it can process data with a larger mapping bandwidth. Due to the similarity of the receiving channel power, it makes on-board data compression possible. The compact polarimetric SAR system is a compromise between the dual-polarization SAR and the full-polarization SAR systems. It emits a special polarization wave and receives a pair of orthogonal polarization waves, and is a specific subset of the full-polarization SAR polarization synthesized into data under any transmitting and receiving polarization bases. Compared with the dual-polarization SAR system, the compact polarimetric SAR system has a richer combination of transmitted and received signals, can obtain the relative phase of the echo signal, and has more abundant information.

[0005] With the increasing demand for more accurate targets in radar image acquisition, many radar image classification methods have emerged, which can generally be divided into extracting polarization features and selecting classifiers. Improving the classification accuracy of radar images by extracting polarization features does not have universality in the division between different ground objects. Currently in the industry, machine learning methods are generally used to classify radar images, such as support vector machines, BP neural networks, convolutional neural networks, etc. Machine learning methods not only improve the classification accuracy but also save a large amount of manual calculation, with accurate results and convenient application.

[0006] The interactive CNN classification method based on the scattering mechanism for compact polarimetric SAR belongs to the classification method in the field of deep learning and has a high classification accuracy. It uses the polarization features of compact polarimetry and classifies ground objects based on the differences in scattering mechanisms between different ground object targets. In order to better interpret polarimetric synthetic aperture radar (POLSAR) images, researchers often classify POLSAR images using the extracted polarization features. The polarization features obtained in the compact polarimetric m-δ decomposition method correspond to odd-order scattering, even-order scattering, and volume scattering respectively. This method divides the polarization features into three channels according to the scattering mechanism and inputs them into the network, avoiding the mutual interference between polarization features. At the same time, for some complex ground object types, the scattering features often contain more than one scattering component, and some polarization features in compact polarimetry cannot be decomposed and are divided into multiple channels. Through the network structure with weight sharing, the image information in each channel is subjected to feature extraction and splicing through 3 groups of convolutional kernels with shared weights, and then enters its respective channel for training. Finally, the classification result is obtained, and the same features as those in the other two channels can be extracted in a certain scattering channel, avoiding the problem of loss of polarization features within the channel. Summary of the Invention

[0007] The main purpose of the present invention is to provide an interactive convolutional neural network classification method (Cross CNN) based on the scattering mechanism, which is applied to the classification of compact polarimetric SAR ground object images.

[0008] By obtaining more polarization features through multiple attention channels, different ground object targets can be better separated, the scattering mechanism of the target can be more comprehensively described, the differences in scattering mechanisms between different targets are more obvious, and the target classification accuracy is higher. On the premise of fully researching convolutional neural network classification methods at home and abroad, the present invention proposes an interactive CNN classification method based on the scattering mechanism for compact polarimetric SAR, which improves the classification accuracy by obtaining the polarization features of compact polarimetric SAR data and using the mechanism of weight sharing and interactive channels.

[0009] Specifically, the technical solution of the present invention mainly includes the following content:

[0010] Step (1) Extract the Stokes vector: Extract the scattering matrix [S] of the compact polarimetric SAR data, and extract the Stokes vector to obtain the scattering mechanisms of different ground targets.

[0011] Step (2) Obtain the scattering mechanism: Calculate the polarization degree m and relative phase difference δ of different ground objects in the SAR classification process using the Stokes vector extracted in step (1), and use them as the basic parameters for target decomposition to distinguish the scattering mechanisms of different ground targets through m-δ decomposition.

[0012] Step (3) Classify the test samples: Extract the polarization features from the scattering mechanisms of the targets obtained in step (2) and introduce them into a 17-layer convolutional neural network (Cross CNN) with weight sharing and channel interaction, and obtain the classification results after multiple rounds of training.

[0013] Step (4) Calculate the confusion matrix of the classification results: Calculate the confusion matrix of each ground object based on the classification results obtained in step (3) to obtain the classification accuracy of each ground object.

[0014] In step (1), the Stokes vector is extracted from the compact polarimetric SAR image. The method for defining the Stokes vector feature is a method for defining the polarization state of electromagnetic waves using power measurement values. The Jones vector E and its conjugate transpose vector E *T The outer product of can obtain a 2×2 Hermitian matrix:

[0015]

[0016] where x represents the horizontal polarization direction, y represents the vertical polarization direction, and * represents the conjugate matrix. Substitute the Pauli matrix group {σ 0 ,σ 1 ,σ 2 ,σ 3} into Equation 1 and decompose Equation 1 into:

[0017]

[0018] where j represents the imaginary unit, and the Pauli matrix group {g 0 ,g 1 ,g 2 ,g 3} are the Stokes parameters. According to Equation (2), we can get:

[0019]

[0020] where g is the Stokes vector.

[0021] For the compact polarimetric SAR mode, it is the CTLR mode that transmits right-handed circularly polarized waves and receives horizontally / vertically polarized waves. In this mode, to calculate the Stokes vector, the polarimetric scattering matrix needs to be obtained first:

[0022] The polarimetric scattering matrix is also called the Sinclair matrix, which is used to describe the electromagnetic scattering phenomenon of the target. Let

[0023]

[0024] where S represents the polarimetric scattering matrix, X and Y represent different polarization directions of the scattered waves, and S ij in the matrix is its complex scattering parameter. The diagonal elements of the matrix are the co-polarization components, representing that the polarization directions of the incident wave and the scattered wave are the same. The anti-diagonal elements of the matrix become the cross-polarization components, representing that the polarization directions of the incident wave and the scattered wave are orthogonal.

[0025] In the compact polarimetric SAR mode, equation (4) can be transformed into:

[0026]

[0027] where S (h,v) represents the horizontal / vertical polarization basis scattering matrix, H represents the horizontal polarization direction of the scattered wave, and V represents the vertical polarization direction of the scattered wave.

[0028] Currently, the full polarimetric systems are generally based on the horizontal / vertical polarization basis. As shown in equation (6), by transforming the polarization basis, the polarimetric scattering matrix of any transmit-receive polarization combination can be obtained from the full polarimetric scattering matrix. Since the polarization mode at the receiving end remains unchanged compared with the full polarization, the polarization basis at the receiving end remains unchanged, and the polarization basis at the transmitting end is the polarization basis in the CTLR mode:

[0029]

[0030]

[0031]

[0032] where U 1 and U 2 are the polarization basis transformation matrices at the transmitting end and the receiving end respectively.

[0033] The full polarimetric scattering matrix of the transmitted circular polarization can be expressed as:

[0034]

[0035] And the compact polarimetric scattering vector S RCL transmitted with right-handed circular polarization can be expressed as:

[0036]

[0037] Therefore, the horizontal potential vector E can be obtained. H , the horizontal potential vector E V :

[0038]

[0039]

[0040] Substituting equations (10) and (11) into equation (3) gives:

[0041]

[0042] where Re represents taking the real part of the complex scattering parameter, and Im represents taking the imaginary part of the complex scattering parameter.

[0043] In step (2), the polarization degree m and the relative phase difference δ of different ground objects in the SAR classification process are calculated using the Stokes vector extracted in step (1). They are used as the basic parameters for target decomposition, and different ground object scattering mechanisms are distinguished through m-δ decomposition.

[0044] 1) Calculate the polarization degree m:

[0045] The polarization degree m reflects the randomness of ground object scattering and is one of the most important characteristics of partially polarized waves. The higher the randomness, the lower the polarization degree, and vice versa.

[0046]

[0047] where {g 0 , g 1 , g 2 , g 3} are the Stokes parameters.

[0048] 2) Decompose the ground object through the polarization degree m and the phase difference δ:

[0049] The polarization degree m and the relative phase difference δ can be used as two basic parameters for target decomposition. The polarization degree m reflects the randomness of ground object scattering, and the phase difference δ can distinguish even-order scattering and odd-order scattering mechanisms. For ground objects with volume scattering, their scattering process is usually very complex and highly random, resulting in a significant reduction in the polarization degree of the echo. Therefore, it is assumed that the completely depolarized component corresponds to the volume scattering component. The weights of even-order scattering, volume scattering, and odd-order scattering can be obtained from m and δ.

[0050]

[0051]

[0052]

[0053] Among them, P dbl refers to the even-order scattering mechanism, P odd refers to the odd-order scattering mechanism, P dep refers to the volume scattering component.

[0054] In step (3), the polarization features are extracted from the scattering mechanisms of the targets obtained in step (2) and introduced into a 17-layer convolutional neural network (Cross CNN) with weight sharing and channel interaction. After multiple rounds of training, the classification results are obtained. In the present invention, first, a multi-channel wide neural network (Widen CNN) based on multiple scattering mechanisms as shown in Figure 1 is constructed, and the training feature samples are divided and input into three channels according to the scattering mechanisms (surface scattering, dihedral angle scattering, and volume scattering), avoiding mutual interference between features. Further considering that, first, the scattering characteristics of some complex ground object types often contain more than one scattering component; second, some polarization features in the compact polarization cannot be decomposed and are divided into multiple channels. The weights of the convolutional kernels in the first layer are shared, and information interaction is performed with all channels, resulting in an interactive CNN classification method (Cross CNN) for compact polarization SAR based on scattering mechanisms as shown in Figure 2 .

[0055] The interactive convolutional neural network is composed of an interactive layer and three identically structured convolutional neural networks spliced together, which can ensure that each channel makes the same contribution to the final classification. Each channel contains blocks of n×n×1. The weight sharing layer is composed of 3 convolutional layers of 3×3×64 with a stride of 1. In each subsequent channel, there is 1 convolutional layer of 3×3×128 with a stride of 1, 1 convolutional layer of 3×3×256 with a stride of 1, and a max pooling layer of 2×2 size with a stride of 2. A ReLU activation function follows each convolutional layer.

[0056] For the convolutional layer of a single-channel convolutional neural network (CNN), the relationship between the i-th input polarization feature and the j-th output feature map can be defined as:

[0057]

[0058] Among them, m is the number of input polarization features, is the weight matrix connecting the i-th polarization feature and the j-th feature map. represents the bias of the j-th feature map output by the convolutional layer.

[0059] Assume the number of output feature maps is n, then the feature output of a certain convolutional layer can be expressed as:

[0060]

[0061] In the convolutional layer of Cross CNN, each channel inputs only one feature, so m = 1. Therefore, the input feature F in and the output feature map of the j-th layer The relationship between them can be defined as:

[0062]

[0063] Let the number of feature maps output by one channel be m, then the feature maps output by this channel can be defined as:

[0064]

[0065] Then the feature maps output by n channels can be defined as:

[0066] F out =[F out1 ,F out2 ,…,F outn (22)

[0067] The overall experimental network structure is shown in Table 1. We cascade the feature maps of each channel to construct a hybrid scattering model and use the feature maps of each channel as the final feature output to cascade the hybrid scattering model. Then, these feature maps will pass through two fully connected layers of 1024 and 512, and a dropout of 0.2 is performed on the second fully connected layer to prevent the network from overfitting. A Softmax classifier is used at the end of the network to obtain the classification results.

[0068] Table 1 Interactive CNN structure table for compact polarimetric SAR based on scattering mechanism

[0069]

[0070] In step (4), through the established neural network model, the image information of the compact polarimetric SAR is classified, the confusion matrix of each ground object is calculated, and the classification accuracy of each ground object is obtained. In the classification results and labeled samples of the test samples, the number of sample points of each category in the classification results and the number of sample points of each category in the labeled samples are counted, and the confusion matrix C is calculated:

[0071]

[0072] The rows of the confusion matrix represent the actual categories, and the columns represent the divided categories. Therefore, the diagonal elements (C 11 、C22 …C ii ) The number of rows and columns is equal, representing the number of sample points correctly classified. The non-diagonal element C ij represents the number of samples of the i-th type of ground object misclassified into the j-th type. The sum of each row in the confusion matrix is the total number of samples in that class. After normalizing the above confusion matrix in the row direction, the elements in the normalized confusion matrix represent the proportion of the classification result in the actual class.

[0073]

[0074] For the decomposition method of compact polarization, the odd-order scattering and even-order scattering are distinguished by the phase angle, and the completely depolarized wave is identified as the volume scattering component by the polarization degree. This gives rise to two problems: First, some polarization characteristics in compact polarization cannot be decomposed and are divided into multiple channels. For this, a network structure with cross channels and shared weights is proposed. The image information in each channel is subjected to feature extraction through 3 groups of convolution kernels with shared weights, and then enters its respective channel for training, and finally the classification result is obtained. The advantage of sharing weights is that the same features as the other two channels can be extracted within a certain scattering channel, avoiding the problem of feature loss within the channel. Second, for some complex ground object types, the scattering characteristics often contain more than one scattering component. Through the cascading of multiple feature maps, a mixed scattering model is constructed, which can classify complex ground object types. Description of the Drawings

[0075] Figure 1 Multi-channel wide-width neural network (Widen CNN) based on multiple scattering mechanisms.

[0076] Figure 2 Interactive CNN (Cross CNN) for compact polarization SAR based on scattering mechanisms.

[0077] Figure 3 Classification decision tree of Flevoland.

[0078] Figure 4 Flowchart of the implementation of this method. Detailed Implementation Manner

[0079] The present invention will be described in detail below in conjunction with the drawings and embodiments.

[0080] The method proposed by the present invention is verified on the classic AIRSAR Flevoland dataset. At the same time, several other representative classification methods (1D-CNN, 2D-CNN) are used for experiments and the experimental results are compared. Combining Figure 3Analyze the experimental results using the classification decision tree of Flevoland in

[0081] The Flevoland data was obtained by AIRSAR on August 16, 1989. The size of the image is 750×1024, and its true color map contains 177,018 samples of a total of 15 types of ground objects. In its Pauli false color map and true color map: 5% of all the labeled samples were randomly selected as training samples, and the remaining labeled samples were used as test samples. The information of each specific type of sample is shown in Table 2 as follows:

[0082] Table 2 Specific information of Flevoland experimental data

[0083]

[0084] In the comparative experiment, all the comparative methods used the same training and test data and ensured the same parameter settings, where the learning rate was 0.1, the input batch size was 15, and the total number of iterations was 100. The classification accuracy of each specific type is shown in Table 3 as follows:

[0085] Table 3 Comparison of classification results of different methods for Flevoland

[0086]

[0087]

[0088] Combined with Figure 3 By comparing the classification decision tree of Flevoland in and the classification accuracy of each category in this experiment, it can be found that our method has achieved better classification results than dual polarization, and is quite close to the full polarization classification results; compared with other classification methods under the reduced polarization data, our method has achieved better classification accuracy for difficult-to-separate variables such as rapeseed, wheat, peas, and sugar beets, proving that weight sharing can extract the same features as the other two channels within a certain scattering channel, avoiding the problem of polarization feature loss within the channel. Similarly, for ground object targets with mixed scattering mechanisms such as forests, better classification accuracy has also been achieved, proving that our model can classify complex ground object types. According to the method in step (4), the confusion matrix for classifying Flevoland ground objects using Cross CNN can be obtained.

[0089] Table 4 Confusion matrix for classifying Flevoland ground objects using Cross CNN

[0090]

Claims

1. An interactive CNN classification method based on scattering mechanism for compact polarimetric SAR, characterized in that: The steps of this method include, Step (1): Extract the Stokes vector: Extract the scattering matrix [S] of the compact polarimetric SAR data, and extract the Stokes vector to obtain the scattering mechanisms of different ground targets; Step (2): Obtain the scattering mechanism: Calculate the polarization degree m and the relative phase difference δ as the basic parameters for target decomposition, and distinguish the scattering mechanisms of different ground targets through m-δ decomposition; Step (3): Classify the test samples: Introduce a 17-layer convolutional neural network CrossCNN with weight sharing and channel interaction, and extract polarization features from the scattering mechanism of the target for ground object classification; Step (4): Calculate the confusion matrix of the classification results: Through the established neural network model, classify the image information of the compact polarimetric SAR, calculate the confusion matrix of each ground object, and obtain the classification accuracy of each ground object; In step (1), the Stokes vector is extracted from the compact polarimetric SAR image; the Stokes vector feature definition method is a method of defining the polarization state of electromagnetic waves using power measurement values, and the Jones vector E The outer product of it and its conjugate transpose vector yields a 2×2 Hermitian matrix: where x and y represent the horizontal and vertical directions, and * represents the conjugate matrix; substituting the Pauli matrix group {σ 0 , σ 1 , σ 2 , σ 3} into Equation 1, Equation 1 is decomposed into: Among them, In formula (2), {g 0 , g 1 , g 2 , g 3} are Stokes parameters. According to formula (2), it can be obtained that: where, g is the Stokes vector; For the compact polarimetric SAR mode, it is the CTLR mode of transmitting right-handed circularly polarized waves and receiving horizontal / vertical polarized waves. In this CTLR mode, to calculate the Stokes vector, it is necessary to first obtain the polarization scattering matrix: The polarization scattering matrix is also called the Sinclair matrix, which is used to describe the electromagnetic scattering phenomenon of the target. Let Among them, S represents the polarization scattering matrix, and S in the matrix ij is its complex scattering parameter. The diagonal elements of the matrix become the co-polarization components, representing that the polarization states of the incident wave and the scattered wave are the same. The anti-diagonal elements of the matrix become the cross-polarization components, representing that the polarization states of the incident wave and the scattered wave are orthogonal; In the compact polarimetric SAR mode, transform Equation (4) into: Among them, S (h,v) represents the horizontal / vertical polarization basis scattering matrix, where H and V represent the horizontal polarization mode and the vertical polarization mode respectively; As shown in Equation (6), by transforming the polarization basis, the polarization scattering matrix of any transceiver polarization combination is obtained from the full polarimetric scattering matrix; because compared with the full polarization, the polarization mode at the receiving end remains unchanged, so the polarization basis at the receiving end remains unchanged, and the polarization basis at the transmitting end is the polarization basis in the CTLR mode: Among them, U 1 and U 2 are the polarization basis change matrices of the transmitting end and the receiving end, respectively; The full polarimetric scattering matrix of the transmitted circular polarization is expressed as: while the compact polarization scattering vector S for transmitting right-handed circular polarization RCL is expressed as: Obtain the horizontal potential vector E H and the horizontal potential vector E V : Substitute Equations (10) and (11) into Equation (3) to get: In Step (2), to obtain the scattering mechanism of the compact polarimetric ground object, calculate the polarization degree m and the relative phase difference δ, and use them as the basic parameters for target decomposition, and distinguish the scattering mechanisms of different ground targets through m-δ decomposition; The polarization degree m reflects the randomness of ground object scattering, and it is one of the most important characteristics of partially polarized waves. The higher the randomness, the lower the polarization degree, and vice versa, the higher the polarization degree; Among them, {g 0 , g 1 , g 2 , g 3} are Stokes parameters; The polarization degree m and the relative phase difference δ are two basic parameters for target decomposition. The polarization degree m reflects the randomness of ground object scattering, and the phase difference δ distinguishes the even-order scattering and odd-order scattering mechanisms; the weights of even-order scattering, volume scattering, and odd-order scattering are obtained from m and δ; Among them, P dbl refers to the even-order scattering mechanism, P odd refers to the odd-order scattering mechanism, P dep refers to the volume scattering component; In Step (3), classify the test samples: Introduce a 17-layer convolutional neural network Cross CNN with weight sharing and channel interaction, and extract polarization features from the scattering mechanism of the target for ground object classification; First, construct a multi-channel wide neural network WidenCNN based on multiple scattering mechanisms, and divide the training feature samples into three channels according to the scattering mechanism. Share the weights of the convolutional kernels in the first layer and perform information interaction with all channels to obtain an interactive convolutional neural network CrossCNN for compact polarimetric SAR classification based on the scattering mechanism; The interactive convolutional neural network is composed of an interactive layer and three identically structured convolutional neural networks spliced together to ensure that each channel makes the same contribution to the final classification; each channel contains blocks of n×n×1, and the weight sharing layer consists of 3 convolutional layers of 3×3×64 with a stride of 1. In each subsequent channel, there is a convolutional layer of 3×3×128 with a stride of 1, a convolutional layer of 3×3×256 with a stride of 1, and a max pooling layer of size 2×2 with a stride of 2; a ReLU activation function follows each convolutional layer; The feature maps of each channel are cascaded to construct a hybrid scattering model, and the hybrid scattering models are cascaded using the feature maps of each channel as the final feature outputs; the feature maps pass through two fully connected layers of 1024 and 512, and dropout of 0.2 is performed on the second fully connected layer to prevent overfitting of the network; a Softmax classifier is used at the end of the network to obtain the classification results; In step (4), the image information of the compact polarimetric SAR is classified through the established neural network model, the confusion matrix of each ground object is calculated, and the classification accuracy of each ground object is obtained; in the classification results of the test samples and the labeled samples, the number of sample points of each category in the classification results and the number of sample points of each category in the labeled samples are statistically counted, and the confusion matrix C is calculated: The rows of the confusion matrix represent the actual classes, and the columns represent the classified classes; the diagonal elements (C 11 , C 22 …C ii ) where the row and column are equal, represent the number of correctly classified sample points. The non-diagonal elements c ij represent the number of samples of the i-th class of features misclassified into the j-th class; the sum of each row in the confusion matrix is the total number of samples in that class. After normalizing the confusion matrix in the row direction, each element in the normalized confusion matrix represents the proportion of the classification result in that actual class;

Images