Polarimetric SAR image classification method, device and equipment based on complex and real numbers

Abstract

The present application relates to a method, device and equipment for polarization SAR image classification based on complex and real numbers, which preprocesses the full polarization SAR sample data, extracts multiple polarization features by using the reflection symmetry decomposition method and performs normalization processing, and divides them into real polarization features and complex polarization features, and constructs corresponding training data, and inputs the training data into an image classification composite network for training, wherein the image classification composite network includes a real branch network and a complex branch network that process real polarization features and complex polarization features respectively, and a fusion classification network that fuses the output features of the real branch network and the complex branch network and classifies according to the fused features, and inputs the real polarization features and the complex polarization features corresponding to the full polarization data into the trained image classification composite network to realize image classification. The present method can effectively improve the classification accuracy of polarization images.

Description

Technical Field

[0001] The present application relates to the technical field of polarization SAR image classification, and in particular to a polarization SAR image classification method, device and equipment based on complex numbers and real numbers. Background Art

[0002] Synthetic aperture radar, due to its ability to penetrate clouds and fog and smog and its all-day advantage, makes the use of SAR for earth observation increasingly important. The electromagnetic waves in the microwave frequency band used by SAR also have a certain ability to penetrate natural vegetation, artificial camouflage and land surface, and can obtain information hidden under the shallow surface and vegetation. Polarimetric synthetic aperture radar has received widespread attention in many application fields because it can obtain polarization backscattering characteristics of resolvable ground observation targets. In these observation tasks, image classification plays a very important role in the interpretation of full polarimetric SAR. It not only affects the credibility of subsequent applications, but also determines the breadth of application fields.

[0003] Compared with traditional SAR, polarimetric SAR transmits and receives electromagnetic waves of different polarization modes through two orthogonal channels. The four sets of data measured can form a complete polarization basis, and any electromagnetic wave can be represented in this polarization basis. Therefore, the radar cross-sectional area obtained by full polarimetric SAR can comprehensively obtain the scattering characteristics of the target to any electromagnetic wave in the observation direction. Deep learning is currently the mainstream method for polarimetric SAR image classification. Compared with traditional classification methods, this method can extract the features of the objects in the image more deeply, and has better performance in speed and accuracy.

[0004] However, the existing methods for classifying polarimetric SAR images mostly use complex neural networks, which only consider complex scenarios and have many defects, such as incomplete information utilization, loss of features carried by the real part, and reduced adaptability to actual scenarios. Summary of the invention

[0005] Based on this, it is necessary to provide a polarization SAR image classification method, device and equipment based on complex numbers and real numbers that can perform accurate classification in order to solve the above technical problems.

[0006] A polarization SAR image classification method based on complex numbers and real numbers, the method comprising:

[0007] Acquire full polarization SAR sample data, and extract multiple polarization features by using a reflection symmetry decomposition method after preprocessing the full polarization SAR sample data;

[0008] After normalizing the polarization features, the polarization features are divided into real polarization features and complex polarization features, and corresponding real polarization feature training data and complex polarization feature training data are constructed;

[0009] Inputting the real polarization feature training data and the complex polarization feature training data into an image classification composite network for training to obtain a trained image classification composite network, wherein the image classification composite network includes a real branch network and a complex branch network for processing the real polarization feature and the complex polarization feature respectively, and a fusion classification network for fusing the output features of the real branch network and the complex branch network, and performing classification according to the fused features;

[0010] Fully polarized SAR data is obtained, and real polarization features and complex polarization features corresponding to the fully polarized SAR data are input into the trained image classification composite network to obtain an image classification result.

[0011] In one embodiment, preprocessing the fully polarimetric SAR sample data includes: radiation correction and polarization filtering.

[0012] In one embodiment, when constructing the real polarization feature training data and the complex polarization feature training data:

[0013] Randomly select a plurality of pixel points on the fully polarized SAR sample data;

[0014] The selected pixel point is taken as the center according to the preset size to generate the corresponding slice sample;

[0015] The real polarization feature training data and the complex polarization feature training data are constructed according to the real polarization feature and the complex polarization feature corresponding to each of the slice samples.

[0016] In one embodiment, the real branch network is built based on the ResNet50 residual structure and the CBAM attention mechanism.

[0017] In one embodiment, the complex branch network includes three parallel complex vector blocks with different convolution kernel sizes, and each of the complex vector blocks includes a complex convolution layer, a complex batch normalization layer, a complex activation function layer and a complex pooling layer.

[0018] In one embodiment, a reflection symmetric decomposition method is used to extract multiple polarization features including 15 real polarization features and 6 complex polarization features.

[0019] The present application also provides a polarization SAR image classification device based on complex numbers and real numbers, the device comprising:

[0020] A polarization feature extraction module is used to obtain full polarization SAR sample data, and after preprocessing the full polarization SAR sample data, a reflection symmetry decomposition method is used to extract multiple polarization features;

[0021] A training data set construction module is used to normalize the polarization features, separate them into real polarization features and complex polarization features, and construct corresponding real polarization feature training data and complex polarization feature training data;

[0022] An image classification composite network training module, used for inputting the real polarization feature training data and the complex polarization feature training data into the image classification composite network for training, so as to obtain a trained image classification composite network, wherein the image classification composite network includes a real branch network and a complex branch network for processing the real polarization feature and the complex polarization feature respectively, and a fusion classification network for fusing the output features of the real branch network and the complex branch network, and performing classification according to the fused features;

[0023] The polarization SAR image classification module based on complex and real numbers is used to obtain full polarization SAR data, and input the real polarization features and complex polarization features corresponding to the full polarization SAR data into the trained image classification composite network to obtain image classification results.

[0024] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0025] Acquire full polarization SAR sample data, and extract multiple polarization features by using a reflection symmetry decomposition method after preprocessing the full polarization SAR sample data;

[0026] After normalizing the polarization features, the polarization features are divided into real polarization features and complex polarization features, and corresponding real polarization feature training data and complex polarization feature training data are constructed;

[0027] Inputting the real polarization feature training data and the complex polarization feature training data into an image classification composite network for training to obtain a trained image classification composite network, wherein the image classification composite network includes a real branch network and a complex branch network for processing the real polarization feature and the complex polarization feature respectively, and a fusion classification network for fusing the output features of the real branch network and the complex branch network, and performing classification according to the fused features;

[0028] Fully polarized SAR data is obtained, and real polarization features and complex polarization features corresponding to the fully polarized SAR data are input into the trained image classification composite network to obtain an image classification result.

[0029] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0030] Acquire full polarization SAR sample data, and extract multiple polarization features by using a reflection symmetry decomposition method after preprocessing the full polarization SAR sample data;

[0031] After normalizing the polarization features, the polarization features are divided into real polarization features and complex polarization features, and corresponding real polarization feature training data and complex polarization feature training data are constructed;

[0032] Inputting the real polarization feature training data and the complex polarization feature training data into an image classification composite network for training to obtain a trained image classification composite network, wherein the image classification composite network includes a real branch network and a complex branch network for processing the real polarization feature and the complex polarization feature respectively, and a fusion classification network for fusing the output features of the real branch network and the complex branch network, and performing classification according to the fused features;

[0033] Fully polarized SAR data is obtained, and real polarization features and complex polarization features corresponding to the fully polarized SAR data are input into the trained image classification composite network to obtain an image classification result.

[0034] The above-mentioned polarization SAR image classification method, device and equipment based on complex and real numbers, after preprocessing the full polarization SAR sample data, uses the reflection symmetry decomposition method to extract multiple polarization features, and after normalization, divides them into real polarization features and complex polarization features, and constructs corresponding training data, and inputs the training data into the image classification composite network for training, wherein the image classification composite network includes a real branch network and a complex branch network that process real polarization features and complex polarization features respectively, and a fusion classification network that fuses the output features of the real branch network and the complex branch network and classifies according to the fused features, and inputs the real polarization features and complex polarization features corresponding to the full polarization SAR data into the trained image classification composite network to realize image classification. The method can effectively improve the classification accuracy of polarization SAR images. Through this framework, the idea of ​​"divide and conquer" is adopted to improve the information extraction efficiency of real and complex polarization features, and the use of a neural network suitable for data features is conducive to fully mining the high-order polarization information of ground objects. In addition, for the real branch network, the residual network integrated with the CBAM attention mechanism can effectively mine deep polarization information in both the channel and spatial dimensions of real polarization features. The complex branch network can extract information about the amplitude and phase features of polarization SAR images. These extracted high-order polarization features have a larger amount of information than a single neural network. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 1 is a flow chart of a polarization SAR image classification method based on complex numbers and real numbers in one embodiment;

[0036] Figure 2 A schematic diagram of the structure of a real number branch network in one embodiment;

[0037] Figure 3 A schematic diagram of the structure of a plurality of branch networks in one embodiment;

[0038] Figure 4 A schematic diagram of a data processing flow of the method in an embodiment;

[0039] Figure 5 This is a pseudo-color synthetic image of fully polarized SAR in area A and the corresponding true value map in an experiment, where: Figure 5 (a) shows the pseudo-color composite image of the full polarimetric SAR in area A. Figure 5 (b) represents the corresponding truth map;

[0040] Figure 6 This is a pseudo-color synthetic image of the full polarimetric SAR in area B in an experiment and the corresponding true value map, where: Figure 6 (a) shows the pseudo-color composite image of the full polarimetric SAR in area B. Figure 6 (b) represents the corresponding truth map;

[0041] Figure 7 This is a pseudo-color synthetic image of the fully polarized SAR in area C in an experiment and the corresponding true value map, where: Figure 7 (a) shows the pseudo-color composite image of the full polarimetric SAR in region C. Figure 7 (b) represents the corresponding truth map;

[0042] Figure 8 The following is a schematic diagram of the results of classifying the actual fully polarized SAR images of region A using this method and multiple existing methods in an experiment, where: Figure 8 (a) shows the pseudo-color image of region A. Figure 8 (b) represents the corresponding truth map, Figure 8 (c) is a schematic diagram showing the classification results using the AlexNet+CVNet method. Figure 8 (d) is a schematic diagram showing the classification results using the VGG13+CVNet method. Figure 8 (e) is a schematic diagram showing the results of using the VGG16+CVNet classification method. Figure 8 (f) is a schematic diagram showing the classification results using the VGG19+CVNet method. Figure 8(g) is a schematic diagram showing the classification results using the ResNet18+CVNet method. Figure 8 (h) is a schematic diagram of the classification results using the ResNet34+CVNet method. Figure 8 (i) is a schematic diagram showing the classification results using the ResNet50+CVNet method. Figure 8 (j) is a schematic diagram of the classification results using the ResNet101+CVNet method. Figure 8 (k) is a schematic diagram of the classification results using the ResNet152+CVNet method. Figure 8 (l) A schematic diagram showing the classification results of the method;

[0043] Fig. 9 The schematic diagram shows the results of classifying the actual fully polarized SAR images of region B using this method and multiple existing methods in an experiment, where: Fig. 9 (a) shows the pseudo-color image of region B. Fig. 9 (b) represents the corresponding truth map, Fig. 9 (c) is a schematic diagram showing the classification results using the AlexNet+CVNet method. Fig. 9 (d) is a schematic diagram showing the classification results using the VGG13+CVNet method. Fig. 9 (e) is a schematic diagram showing the results of using the VGG16+CVNet classification method. Fig. 9 (f) is a schematic diagram showing the classification results using the VGG19+CVNet method. Fig. 9 (g) is a schematic diagram showing the classification results using the ResNet19+CVNet method. Fig. 9 (h) is a schematic diagram of the classification results using the ResNet34+CVNet method. Fig. 9 (i) is a schematic diagram showing the classification results using the ResNet50+CVNet method. Fig. 9 (j) is a schematic diagram showing the classification results using the ResNet101+CVNet method. Fig. 9 (k) is a schematic diagram of the classification results using the ResNet152+CVNet method. Fig. 9 (l) A schematic diagram showing the classification results of the method;

[0044] Fig.10 The schematic diagram shows the results of classifying the actual polarimetric SAR images of region C using this method and multiple existing methods in an experiment. Fig.10 (a) shows the pseudo-color image of region C. Fig.10 (b) represents the corresponding truth map, Fig.10 (c) is a schematic diagram showing the classification results using the AlexNet+CVNet method. Fig.10 (d) is a schematic diagram showing the classification results using the VGG13+CVNet method. Fig.10 (e) is a schematic diagram showing the results of using the VGG16+CVNet classification method. Fig.10 (f) is a schematic diagram showing the classification results using the VGG19+CVNet method. Fig.10 (g) is a schematic diagram showing the classification results using the ResNet110+CVNet method. Fig.10 (h) is a schematic diagram of the classification results using the ResNet34+CVNet method. Fig.10 (i) is a schematic diagram showing the classification results using the ResNet50+CVNet method. Fig.10 (j) is a schematic diagram showing the classification results using the ResNet101+CVNet method. Fig.10 (k) is a schematic diagram of the classification results using the ResNet152+CVNet method. Fig.10 (l) A schematic diagram showing the classification results of the method;

[0045] Fig.11 is a structural block diagram of a polarization SAR image classification device based on complex numbers and real numbers in one embodiment;

[0046] Fig.12 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0048] In the existing methods for classifying polarimetric SAR images, most of the complex neural networks used only consider complex scenes, resulting in incomplete information utilization, loss of features carried by the real part, and reduced adaptability to actual scenes. Figure 1 As shown, a polarization SAR image classification method based on complex numbers and real numbers is provided, comprising the following steps:

[0049] Step S100: acquiring full polarization SAR sample data, preprocessing the full polarization SAR sample data, and extracting multiple polarization features using a reflection symmetric decomposition method.

[0050] Step S110, after normalizing the polarization feature, the polarization feature is divided into a real polarization feature and a complex polarization feature, and corresponding real polarization feature training data and complex polarization feature training data are constructed.

[0051] Step S120, input the real polarization feature training data and the complex polarization feature training data into the image classification composite network for training, and obtain a trained image classification composite network, wherein the image classification composite network includes a real branch network and a complex branch network for processing real polarization features and complex polarization features respectively, and a fusion classification network for fusing the output features of the real branch network and the complex branch network, and performing classification according to the fused features.

[0052] Step S130, obtaining full polarization SAR data, and inputting the real polarization features and complex polarization features corresponding to the full polarization SAR data into the trained image classification composite network to obtain an image classification result.

[0053] In this application, in order to simultaneously consider the real and complex features of polarimetric SAR, a composite network for image classification is proposed. Two sub-networks are designed in the network to process complex features and real features respectively, so as to further extract the high-latitude features of the two types of data. After fusing them, classification is performed based on the fused data, thereby improving the classification accuracy.

[0054] This method includes two parts. The first part is the part of training the image classification composite network, including steps S100 to S120. The second part is the part of applying the trained image classification composite network, which is step S130.

[0055] In step S100, the fully polarized SAR sample data is preprocessed including radiation correction and polarization filtering. The radiation calibration formula needs to refer to the satellite data manual, and the method used for polarization filtering is non-local mean filtering. These two steps are to improve the quality of the image and facilitate subsequent image interpretation. It should be noted here that the method in this application is to classify the types of ground objects, that is, the fully polarized SAR sample data is obtained by detecting from a high altitude with the ground as the target.

[0056] Furthermore, after preprocessing the image, a reflection symmetric decomposition method capable of complete decomposition is used to perform polarization decomposition on the polarization SAR image, and multiple polarization features capable of representing the scattering information of the ground objects are extracted.

[0057] In this embodiment, the extracted polarization feature data and type can be selected according to the specific application scenario. For example, when the extracted polarization features exist simultaneously with real polarization features and complex polarization features, the composite classification network mentioned in this article can be selected; when there are only real polarization features, the complex branch network can be ignored; when there are only complex polarization features, the real branch network can be ignored.

[0058] According to the reflection symmetry decomposition algorithm, 21 polarization features that can characterize the scattering information of ground objects are extracted, including 15 real polarization features and 6 complex polarization features. The specific feature information is shown in the following table:

[0059]

[0060] In step S110, in order to meet the requirements of the neural network input, the polarization characteristic parameters need to be normalized. The main method is to take the form of ratio or maximum and minimum normalization with the total polarization power to increase the differentiation of the polarization characteristic parameters of different ground objects. The distribution interval of the polarization characteristics is normalized to 0~1. Among them, the total polarization power .

[0061] In this embodiment, when constructing real polarization feature training data and complex polarization feature training data: on the fully polarized SAR sample data, a plurality of pixel points are randomly selected, and the selected pixel points are used as the center according to a preset size to generate corresponding slice samples, and finally, according to the real polarization features and complex polarization features corresponding to each slice sample, real polarization feature training data and complex polarization feature training data are constructed.

[0062] In step S120, the real branch network is built based on the ResNet50 residual structure and the CBAM attention mechanism.

[0063] like Figure 2 and Figure 4 As shown in the figure, in the real number branch network, the branch network consists of a convolutional module (ConvBlock) and three residual modules (RCBlock) that integrate the CBAM attention mechanism. ConvBlock first extracts the input real number parameter features, and the features extracted by the ConvBlock convolution block are recorded as , , and then after the first RCBlock convolution block, the feature , Similarly, the second feature after the RCBlock convolution block is, , , and finally the features extracted by the real branch network (Features extracted by the 4th RCBlock). Among them, Indicates element-wise multiplication. During the multiplication process, the channel attention mechanism broadcasts along the spatial dimension, It is the feature after the CBAM attention mechanism.

[0064] like Figure 3 and Figure 4As shown in Figure 2, the complex branch network includes three parallel complex blocks with different convolution kernel sizes. Figure 3 Represents a complex block. This module includes two complex convolution layers (CCD) with convolution kernel sizes of 3, 7, and 11, a complex batch normalization layer (CBN), a complex activation function layer (CRL), and a complex pooling layer (CMP). Specifically, the extracted feature map is input into three complex layers with convolution kernels of different sizes, and then the extracted features are additively summed as the input features of the next complex block.

[0065] In this embodiment, the complex branch network uses complex neural network modules such as complex convolutional layers and complex pooling layers to extract features of 6 complex polarization features.

[0066] First, for the 2D complex domain filter matrix With complex vector The multiplication operation is as follows:

[0067] (1)

[0068] The above operation is expressed in matrix form as follows:

[0069] (2)

[0070] In formula (2), A and B are real matrices, and x and y are real vectors.

[0071] In the The first layer (that is, the layer of complex operations) The polarization characteristics can be expressed as follows:

[0072] (3)

[0073] In formula (3), and represent the real and imaginary parts of a complex number respectively.

[0074] Furthermore, the CReLU function is used in the complex branch network to process the real and imaginary components. The CReLU function is expressed as:

[0075] (4)

[0076] For the final output layer of the network, As the final extracted complex features.

[0077] In the complex branch network, the convolution kernel sizes are set to , , Three CVBlocks are set in the network. The three modules contain three layers of convolution kernels of different sizes. After the convolution kernels of different sizes are extracted, the first Layer Features .

[0078] Finally, a fusion classification network is used to fuse the features extracted by the complex branch network with the features extracted by the real branch network, and then a fully connected layer is used to classify objects based on the fused features. Among them, the size adjustment is achieved by using convolution kernels of different sizes in the pooling layer.

[0079] In this embodiment, when updating the network parameters of the image classification complex network, the real and imaginary parts of the output of the fully connected layer are calculated using the following formula:

[0080] (5)

[0081] In the above formula, Y is the complex output of the fully connected layer. Assume that the output The node output of the layer is ,but:

[0082] (6)

[0083] (7)

[0084] In the formula, O Indicated in Layer nodes. and They are Layer Node and Layer Node The weights and biases between them.

[0085] In this embodiment, the stochastic gradient descent algorithm is used to optimize Formula (7), and the hyperparameter optimization process is expressed as:

[0086] (8)

[0087] (9)

[0088] In formula (8) and formula (9), and They represent the weight and bias updated at the tth iteration respectively. is the learning rate of the SGD optimization algorithm. According to the chain propagation criterion, the following partial derivative formula can be obtained:

[0089] (10)

[0090] (11)

[0091] By substituting the above partial derivative results into formula (10) and formula (11), the complex value update of the network is completed.

[0092] In this embodiment, when training the composite network for image classification, a batch form is adopted to randomly select a certain number of data from the training set into the model for training and testing. During the training process, the polarization feature sizes of the two branch networks are calibrated and fused at the front and back ends, that is, the extracted polarization feature sizes are calibrated to facilitate the final feature fusion, and the category to which the training sample belongs is determined according to the convolution processing results. Whenever the accuracy on the validation set is improved, the weight at this time is retained, and when the training is completed, the weight that makes the validation set most accurate is stored. The model with the highest accuracy of the validation set is saved, and finally the data with the highest accuracy is obtained. This model is used for testing subsequent data, and the cross entropy loss value is calculated at the same time. After comparing the accuracy of the training and validation sets, if the model is not overfitting, the trained model is saved by comparing the training and validation accuracy sizes.

[0093] In step S130, after radiation correction and polarization filtering are performed on a whole frame of fully polarized SAR data, 21 polarization features are extracted using the reflection symmetry decomposition method, and after the polarization features are normalized, they are input into the image classification composite network to obtain the final ground object classification result.

[0094] like Figure 4 The data processing flow chart of the method in this paper is shown in FIG.

[0095] In this paper, experiments are also provided to prove the effectiveness of this method. Figures 5 to 7 As shown in the figure, it is a schematic diagram of the region used in the experiment and the corresponding truth map, where: Figure 5 This is the pseudo-color synthetic image of the full polarimetric SAR in area A and the corresponding true value map. Figure 6 This is the pseudo-color synthetic image of the full polarimetric SAR in area B and the corresponding true value map. Figure 7 This is the full polarimetric SAR pseudo-color composite image of area C and the corresponding true value map.

[0096] like Figure 8 As shown in Figure 1, it is a schematic diagram of the classification results of the actual full-polarization SAR images of area A using this method and multiple existing methods. From this figure, it can be seen that whether from the overall classification effect or from the junction of the red frame and the black frame, the composite network proposed in this paper achieves the best effect.

[0097] like Fig. 9As shown in Figure 1, it is a schematic diagram of the classification results of the actual full-polarization SAR images of area B using this method and multiple existing methods. From this figure, it can be seen that whether from the overall classification effect or from the junction of the purple frame and the pink frame, the composite network proposed in this paper achieves the best effect.

[0098] like Fig.10 As shown in Figure 1, it is a schematic diagram of the classification results of the actual full-polarization SAR images of region C using this method and multiple existing methods. From this figure, it can be seen that from the perspective of overall classification effect, the composite network proposed in this paper has the best effect.

[0099] exist Figures 8 to 10 middle, Figure 8 (a) Fig. 9 (a) Fig.10 (a) represents a pseudo-color image of a certain area. Figure 8 (b) Fig. 9 (b) Fig.10 (b) represent the corresponding truth graphs, Figure 8 (c) Fig. 9 (c) Fig.10 (c) are schematic diagrams showing the classification results using the AlexNet+CVNet method. Figure 8 (d) Fig. 9 (d) Fig.10 (d) are schematic diagrams of classification results using the VGG13+CVNet method. Figure 8 (e) Fig. 9 (e) Fig.10 (e) are schematic diagrams showing the results of using the VGG16+CVNet classification method. Figure 8 (f) Fig. 9 (f) Fig.10 (f) are schematic diagrams of classification results using the VGG19+CVNet method. Figure 8 (g) Fig. 9 (g) Fig.10 (g) are schematic diagrams of classification results using the ResNet18+CVNet method. Figure 8 (h) Fig. 9 (h) Fig.10 (h) are schematic diagrams of classification results using the ResNet34+CVNet method. Figure 8 (i) Fig. 9 (i) Fig.10 (i) are schematic diagrams of classification results using the ResNet50+CVNet method. Figure 8 (j) Fig. 9 (j) Fig.10 (j) are schematic diagrams of classification results using the ResNet101+CVNet method. Figure 8 (k) Fig. 9 (k) Fig.10 (k) are schematic diagrams of classification results using the ResNet152+CVNet method. Figure 8 (l)、 Fig. 9 (l)、 Fig.10 (l) are schematic diagrams showing the classification results of the method.

[0100] In the above-mentioned polarimetric SAR image classification method based on complex and real numbers, after obtaining the full polarimetric SAR data, the original data is firstly subjected to radiation correction and polarimetric filtering. Then, the reflection symmetry decomposition method is used to extract 21 polarimetric features, which characterize the backscattering information of the ground object from different angles. Then, the polarimetric features are normalized by different methods and divided into two groups, one group is 15 real polarimetric features, and the other group is the real and imaginary parts of 6 complex polarimetric features, and a training set and a validation set are prepared. After that, a real and complex composite network is constructed, and the real and complex parts of the collected data set are trained and validated respectively by using the network, and the extracted features are fused and input into the classifier, and the parameters such as weights and biases are saved at the same time. Finally, the trained model is used to classify the entire image. In the neural network, the size of the feature value can be obtained by real and complex numbers and convolution. The fully connected layer and the Softmax function are used to determine the category to which the ground object belongs. Then fill the judgment result into the empty matrix of the same size as the predicted image to obtain the classification result of the entire image. The composite network proposed in this method solves the polarization SAR classification task when real and complex numbers exist at the same time. The reflection symmetry decomposition algorithm is used to extract 21 polarization features, realizing the extraction of complete polarization scattering information of the ground objects. In this method, ResNet50 is used as the backbone network, and the real polarization network feature branch of the CBAM attention mechanism is integrated. Compared with the single residual network structure, the amount of information of the ground object features is increased. For the complex polarization features, the real and imaginary features of the complex number are simultaneously input into the complex network, realizing the information extraction of the complex features after polarization decomposition. At the same time, the experimental data sets and empirical results on three typical data sets show that under small sample sizes, the method proposed in this paper is competitive in terms of accuracy and stability.

[0101] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0102] In one embodiment, Fig.11 As shown, a polarization SAR image classification device based on complex numbers and real numbers is provided, comprising: a polarization feature extraction module 200, a training data set construction module 210, an image classification composite network training module 220 and a polarization SAR image classification module 230 based on complex numbers and real numbers, wherein:

[0103] The polarization feature extraction module 200 is used to obtain full polarization SAR sample data, and extract multiple polarization features by using a reflection symmetric decomposition method after preprocessing the full polarization SAR sample data;

[0104] A training data set construction module 210 is used to normalize the polarization features, separate them into real polarization features and complex polarization features, and construct corresponding real polarization feature training data and complex polarization feature training data;

[0105] An image classification composite network training module 220 is used to input the real polarization feature training data and the complex polarization feature training data into the image classification composite network for training to obtain a trained image classification composite network, wherein the image classification composite network includes a real branch network and a complex branch network for processing the real polarization feature and the complex polarization feature respectively, and a fusion classification network for fusing the output features of the real branch network and the complex branch network, and performing classification according to the fused features;

[0106] The complex and real number based polarization SAR image classification module 230 is used to obtain full polarization SAR data, input the real number polarization features and complex number polarization features corresponding to the full polarization SAR data into the trained image classification composite network to obtain image classification results.

[0107] For the specific definition of the polarization SAR image classification device based on complex numbers and real numbers, please refer to the definition of the polarization SAR image classification method based on complex numbers and real numbers in the above text, which will not be repeated here. Each module in the above-mentioned polarization SAR image classification device based on complex numbers and real numbers can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0108] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Fig.12 As shown. The computer device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a polarization SAR image classification method based on complex numbers and real numbers is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0109] Those skilled in the art will understand that Fig.12 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0110] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0111] Acquire full polarization SAR sample data, and extract multiple polarization features by using a reflection symmetry decomposition method after preprocessing the full polarization SAR sample data;

[0112] After normalizing the polarization features, the polarization features are divided into real polarization features and complex polarization features, and corresponding real polarization feature training data and complex polarization feature training data are constructed;

[0113] Inputting the real polarization feature training data and the complex polarization feature training data into an image classification composite network for training to obtain a trained image classification composite network, wherein the image classification composite network includes a real branch network and a complex branch network for processing the real polarization feature and the complex polarization feature respectively, and a fusion classification network for fusing the output features of the real branch network and the complex branch network, and performing classification according to the fused features;

[0114] Fully polarized SAR data is obtained, and real polarization features and complex polarization features corresponding to the fully polarized SAR data are input into the trained image classification composite network to obtain an image classification result.

[0115] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0116] Acquire full polarization SAR sample data, and extract multiple polarization features by using a reflection symmetry decomposition method after preprocessing the full polarization SAR sample data;

[0117] After normalizing the polarization features, the polarization features are divided into real polarization features and complex polarization features, and corresponding real polarization feature training data and complex polarization feature training data are constructed;

[0118] Inputting the real polarization feature training data and the complex polarization feature training data into an image classification composite network for training to obtain a trained image classification composite network, wherein the image classification composite network includes a real branch network and a complex branch network for processing the real polarization feature and the complex polarization feature respectively, and a fusion classification network for fusing the output features of the real branch network and the complex branch network, and performing classification according to the fused features;

[0119] Fully polarized SAR data is obtained, and real polarization features and complex polarization features corresponding to the fully polarized SAR data are input into the trained image classification composite network to obtain an image classification result.

[0120] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0121] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0122] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the method of the present application shall be subject to the attached claims.

Claims

1. A polarimetric SAR image classification method based on complex numbers and real numbers, characterized in that: The method comprises: Acquire full polarization SAR sample data, and extract multiple polarization features by using a reflection symmetry decomposition method after preprocessing the full polarization SAR sample data; After normalizing the polarization features, the polarization features are divided into real polarization features and complex polarization features, and corresponding real polarization feature training data and complex polarization feature training data are constructed; Input the real polarization feature training data and the complex polarization feature training data into the image classification composite network for training to obtain a trained image classification composite network, wherein the image classification composite network includes a real branch network and a complex branch network for processing the real polarization feature and the complex polarization feature respectively, and a fusion classification network for fusing the output features of the real branch network and the complex branch network, and classifying according to the fused features, wherein the real branch network is built based on the ResNet50 residual structure and the CBAM attention mechanism, and the complex branch network includes three parallel complex vector blocks with different convolution kernel sizes, and each of the complex vector blocks includes a complex convolution layer, a complex batch normalization layer, a complex activation function layer, and a complex pooling layer; Fully polarized SAR data is obtained, and real polarization features and complex polarization features corresponding to the fully polarized SAR data are input into the trained image classification composite network to obtain an image classification result.

2. The polarization SAR image classification method based on complex numbers and real numbers according to claim 1, characterized in that: The preprocessing of the fully polarized SAR sample data includes: radiation correction and polarization filtering.

3. The polarization SAR image classification method based on complex numbers and real numbers according to claim 2, characterized in that: When constructing the real polarization feature training data and the complex polarization feature training data: On the fully polarized SAR sample data, a plurality of pixel points are randomly selected according to different ground object categories; The selected pixel point is taken as the center according to the preset size to generate the corresponding slice sample; The real polarization feature training data and the complex polarization feature training data are constructed according to the real polarization feature and the complex polarization feature corresponding to each of the slice samples.

4. The method for polarization SAR image classification based on complex numbers and real numbers according to any one of claims 1 to 3, characterized in that: The reflection symmetry decomposition method is used to extract multiple polarization features including 15 real polarization features and 6 complex polarization features.

5. A polarimetric SAR image classification device based on complex numbers and real numbers, characterized in that: The device comprises: A polarization feature extraction module is used to obtain full polarization SAR sample data, and after preprocessing the full polarization SAR sample data, a reflection symmetry decomposition method is used to extract multiple polarization features; A training data set construction module is used to normalize the polarization features, separate them into real polarization features and complex polarization features, and construct corresponding real polarization feature training data and complex polarization feature training data; An image classification composite network training module is used to input the real polarization feature training data and the complex polarization feature training data into the image classification composite network for training to obtain a trained image classification composite network, wherein the image classification composite network includes a real branch network and a complex branch network for processing the real polarization feature and the complex polarization feature respectively, and a fusion classification network for fusing the output features of the real branch network and the complex branch network and performing classification according to the fused features, wherein the real branch network is constructed based on the ResNet50 residual structure and the CBAM attention mechanism, and the complex branch network includes three parallel complex vector blocks with different convolution kernel sizes, and each of the complex vector blocks includes a complex convolution layer, a complex batch normalization layer, a complex activation function layer, and a complex pooling layer; The polarization SAR image classification module based on complex and real numbers is used to obtain full polarization SAR data, and input the real polarization features and complex polarization features corresponding to the full polarization SAR data into the trained image classification composite network to obtain image classification results.

6. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

Images