Polarimetric sar image classification method based on deep semantic topology fusion network

Abstract

The application discloses a polarimetric SAR image classification method based on a deep semantic topology fusion network, and comprises the following steps: acquiring a polarimetric SAR image to be classified and a corresponding ground object real label image; preprocessing the polarimetric SAR image to be classified, and normalizing a polarimetric coherence matrix of each pixel point of the polarimetric SAR image after preprocessing; extracting a feature vector of each pixel point from the normalized polarimetric coherence matrix, and constructing a feature matrix of the polarimetric SAR image to be classified according to a feature vector set of all pixel points; constructing a training data set and a test data set according to the ground object real label image and the feature matrix of the polarimetric SAR image to be classified; constructing a deep semantic topology fusion network model; training the deep semantic topology fusion network model by using the training data set; and classifying the polarimetric SAR image to be classified in the test data set by using the trained deep semantic topology fusion network model. The application can effectively improve the classification accuracy of the polarimetric SAR image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of polarimetric synthetic aperture radar image processing, and particularly relates to a polarimetric SAR image classification method based on a deep semantic topological fusion network. BACKGROUND

[0002] At present, in the technical field of polarimetric synthetic aperture radar (Polarimetric Synthetic Aperture Rader, referred to as polarimetric SAR) image classification based on deep feature learning, deep learning models based on CNN framework are widely studied. However, since CNN uses a fixed-size convolution kernel to perform convolution on a regular region, CNN can only model short-distance local spatial relationships and cannot capture the overall characteristics of the class reflected by the long-term non-local spatial relationships, which limits the further improvement of the classification performance of polarimetric SAR images.

[0003] In order to solve the above problems, scholars have proposed a graph convolutional neural network GCN based on a graph structure for polarimetric SAR image classification. Since the graph structure belongs to a topological structure, the neighborhood of the nodes in the graph is not limited to the local, and the graph convolutional neural network GCN based on the graph structure can break the spatial limitation caused by the fixed convolution kernel in CNN, model and describe the long-term non-local spatial relationships between samples (nodes), and therefore can capture the overall characteristics of the classes in the polarimetric SAR image, which helps to improve the classification performance. For example, Cheng et al. proposed a multiscale superpixel-based graph convolutional network SP-GCN for polarimetric SAR image classification (see document: J. Cheng et al, “PolSAR image classification with multiscale superpixel-based graph convolutional network,” IEEE Transactions on Geoscience and Remote Sensing, 2021, 60:5209314).

[0004] However, the graph convolutional neural network GCN for polarimetric SAR image classification currently available has the following problems:

[0005] First, the existing graph convolutional neural network model needs to establish an accurate graph structure for the entire polarimetric SAR image and perform full-image training and learning, so the calculation complexity and amount are high.

[0006] Second, the existing graph convolutional neural network model is difficult to aggregate and predict new graph nodes, so the flexibility and generalization are poor.

[0007] Thirdly, the discriminative features extracted by the existing graph convolutional neural network model are single, and only part of the information of the ground object target can be captured, which is insufficient to accurately classify similar ground object targets in complex polarimetric SAR images. SUMMARY

[0008] To solve the technical problems in the background art, the application provides a polarimetric SAR image classification method based on a deep semantic topology fusion network.

[0009] The polarimetric SAR image classification method based on the deep semantic topology fusion network provided by the application comprises the following steps:

[0010] The polarimetric SAR image to be classified and the corresponding ground object real label image are obtained, the polarimetric SAR image to be classified is preprocessed, the polarimetric coherence matrix of each pixel point of the preprocessed polarimetric SAR image is calculated, the polarimetric coherence matrix is normalized to obtain the normalized polarimetric coherence matrix, the feature vector of each pixel point is extracted from the normalized polarimetric coherence matrix, and the feature matrix of the polarimetric SAR image to be classified is constructed according to the feature vector set of all pixel points; the training data set and the test data set are constructed according to the ground object real label image and the feature matrix of the polarimetric SAR image to be classified; the deep semantic topology fusion network model is constructed; the deep semantic topology fusion network model is trained by using the training data set to obtain the trained deep semantic topology fusion network model; the polarimetric SAR image to be classified in the test data set is classified by using the trained deep semantic topology fusion network model to obtain the classification result of the polarimetric SAR image to be classified.

[0011] Preferably, the deep semantic topology fusion network model comprises an input layer, a semantic information extraction module, a topology information extraction module, a feature fusion module and a Softmax classifier output layer.

[0012] The output end of the input layer is connected to the input end of the semantic information extraction module, the output end of the semantic information extraction module is connected to the input end of the topology information extraction module, the output end of the semantic information extraction module and the output end of the topology information extraction module are respectively connected to the input end of the feature fusion module, and the output end of the feature fusion module is connected to the Softmax classifier output layer.

[0013] Preferably, the semantic information extraction module comprises a CNN subnetwork and a global pooling layer, the output end of the CNN subnetwork is connected to the global average pooling layer, and the CNN subnetwork comprises a first convolutional layer, a first activation layer, a second convolutional layer, a second activation layer, a third convolutional layer and a third activation layer connected in sequence.

[0014] Preferably, the topology information extraction module comprises a GraphSAGE subnetwork; wherein the GraphSAGE subnetwork comprises a first GraphSAGE layer and a second GraphSAGE layer cascaded in sequence.

[0015] Preferably, the feature fusion module comprises a first parallel layer and a first fully connected layer cascaded in sequence.

[0016] Preferably, the deep semantic topology fusion network model is trained by using a training data set to obtain a trained deep semantic topology fusion network model, specifically including:

[0017] According to the weights of all filters in each convolutional layer, the weights of each GraphSAGE layer, and the weights of each fully connected layer, a weight parameter set is established, and the weight parameter set is initialized by using a He weight initialization method; K groups of labeled data are randomly and with replacement selected from the training data set to obtain a batch training sample label set; a batch set is established according to the training sample features of the batch training sample label set, and the batch set is input to the input layer; the semantic feature extraction module is used to extract semantic features from the batch set to obtain a semantic feature set; the topology information extraction module is used to extract topology features from the semantic feature set to obtain a topology feature set; the feature fusion module is used to extract fusion features from the semantic feature set and the topology feature set to obtain a fusion feature set; the Softmax classifier output layer is used to classify and predict the fusion feature set to obtain a predicted label set; the weight parameter set is adjusted according to the batch training sample label set corresponding to the batch set and the predicted label set to obtain a trained deep semantic topology fusion network model.

[0018] Preferably, the weight parameter set is adjusted according to the batch training sample label set corresponding to the batch set and the predicted label set to obtain a trained deep semantic topology fusion network model, specifically including:

[0019] According to the batch training sample label set corresponding to the batch set and the predicted label set, a cross-entropy loss function is used to calculate a loss function value; according to the loss function value, it is judged whether the cross-entropy loss function converges; if yes, the training is stopped; otherwise, the weight parameter set is adjusted, and the next training is performed until the cross-entropy loss function converges or the number of training reaches a preset training number threshold.

[0020] Preferably, the weight parameter set is adjusted, specifically including:

[0021] The partial derivative of the loss function value to each weight parameter in the weight parameter set is calculated; the weight parameters are updated in a back propagation manner according to the gradient descent method and the partial derivative to obtain updated weight parameters; and an updated weight parameter set is established according to all the updated weight parameters.

[0022] Preferably, the trained deep semantic topological fusion network model is used to classify the polarimetric SAR image in the test data set, and the classification result of the polarimetric SAR image is obtained, which specifically includes:

[0023] The test sample features in the test data set are input into the input layer of the trained deep semantic topological fusion network model; the semantic feature extraction module is used to extract semantic features from the test sample features to obtain the semantic features; the topological feature extraction module is used to extract topological features from the semantic features to obtain the topological features; the fusion module is used to extract fusion features from the semantic features and the topological features; and the Softmax classifier output layer is used to classify and predict the fusion features to obtain the classification result of the polarimetric SAR image to be classified.

[0024] Preferably, the polarimetric SAR image to be classified is preprocessed, and the polarimetric coherence matrix of each pixel point of the preprocessed polarimetric SAR image is calculated, and the polarimetric coherence matrix is normalized to obtain the normalized polarimetric coherence matrix, which specifically includes:

[0025] Each pixel point in the preprocessed polarimetric SAR image to be classified is represented by a 3x3 polarimetric coherence matrix; the average value and the variance value of each element in the polarimetric coherence matrix of each pixel point are calculated respectively according to the polarimetric coherence matrix of each pixel point; the normalized value of each element in the polarimetric coherence matrix of each pixel point is calculated according to each element in the polarimetric coherence matrix of each pixel point and the average value and the variance value of each element; the normalized value of the polarimetric coherence matrix of each pixel point is calculated according to the normalized value of each element in the polarimetric coherence matrix of each pixel point; and the normalized polarimetric coherence matrix of the polarimetric SAR image to be classified is obtained according to the normalized value of the polarimetric coherence matrix of each pixel point.

[0026] Preferably, the feature vector of each pixel point is extracted from the normalized polarimetric coherence matrix, and the feature matrix of the polarimetric SAR image to be classified is established according to the feature vector set of all pixel points, which specifically includes:

[0027] According to the normalized polarimetric coherence matrix of each pixel point, the upper triangular 6 elements thereof are selected to form a 9-dimensional feature vector of the pixel point; and the feature matrix of the polarimetric SAR image to be classified is constructed according to the 9-dimensional feature vector of each pixel point.

[0028] Preferably, the training data set and the test data set are constructed according to the ground object real label image and the feature matrix of the polarimetric SAR image to be classified, which specifically includes:

[0029] According to the real ground object marking map, 10% of the marked data in each ground object category is randomly selected as a training sample marking set, and the remaining 90% of the marked data is used as a test sample marking set;

[0030] According to the coordinate value of each training sample in the ground object real marking map in the training sample marking set, a first feature value vector corresponding to the coordinate position in the feature matrix is found; a plurality of second feature value vectors are selected in the upper, lower, left and right four directions with the first feature value vector as the center point, and the first feature value vector and the second feature value vectors selected in the four directions form a first feature matrix block, which is used as the training sample feature of the corresponding training sample; the training sample features of all training samples are combined to form a training sample feature set, and the elements of the training sample marking set and the training sample feature set are combined to form a training data set;

[0031] According to the coordinate position of each test sample in the ground object real marking map in the test sample marking set, a third feature value vector corresponding to the coordinate position in the feature matrix is found; a plurality of fourth feature value vectors are selected in the upper, lower, left and right four directions with the third feature value vector as the center point, and the third feature value vector and the fourth feature value vectors selected in the four directions form a second feature matrix block, which is used as the test sample feature of the corresponding test sample; the test sample features of all test samples are combined to form a test sample feature set, and the elements of the test sample marking set and the test sample feature set are combined to form a test data set.

[0032] In the present application, the polarimetric SAR image classification method based on the deep semantic topology fusion network can flexibly and efficiently gather new node information through the constructed deep semantic topology fusion network model, avoiding the problems of high computational complexity and large amount of calculation caused by using full image training learning in the prior art; and can simultaneously capture deep semantic information and non-local topological information, obtain a more comprehensive description of the ground object target, avoid the problem of insufficient capture of ground object target information caused by the use of a single structure in the prior art, and obtain more compact intra-class and more separated inter-class robust features, effectively improving the accuracy of polarimetric SAR image classification. BRIEF DESCRIPTION OF DRAWINGS

[0033] Figure 1 The flowchart of the polarimetric SAR image classification method based on the deep semantic topology fusion network in an embodiment of the present application is shown.

[0034] Figure 2 The polarimetric SAR data synthesis map in an embodiment of the present application is shown.

[0035] Figure 3 The real ground object marking map of Figure 2 .

[0036] Figure 4 A classification result map classified by the polarimetric SAR classification method based on the SP-GCN.

[0037] Figure 5 A classification result map classified by the method. DETAILED DESCRIPTION

[0038] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in combination with embodiments.

[0039] Reference Figure 1 The polarimetric SAR image classification method based on the deep semantic topology fusion network provided by the present application comprises:

[0040] Obtaining a polarimetric SAR image to be classified and a corresponding ground object real label map;

[0041] Pretreating the polarimetric SAR image to be classified, calculating a polarimetric coherence matrix of each pixel point of the pretreated polarimetric SAR image, and normalizing the polarimetric coherence matrix to obtain a normalized polarimetric coherence matrix;

[0042] Extracting a feature vector of each pixel point from the normalized polarimetric coherence matrix, and constructing a feature matrix of the polarimetric SAR image to be classified according to a feature vector set of all pixel points;

[0043] According to the ground object real label map and the feature matrix of the polarimetric SAR image to be classified, constructing a training data set and a test data set;

[0044] Building a deep semantic topology fusion network model;

[0045] Training the deep semantic topology fusion network model by using the training data set to obtain a trained deep semantic topology fusion network model;

[0046] Classifying the polarimetric SAR image to be classified in the test data set by using the trained deep semantic topology fusion network model to obtain a classification result of the polarimetric SAR image to be classified.

[0047] In the present application, the proposed polarimetric SAR image classification method based on deep semantic topology fusion network can flexibly and efficiently aggregate new node information through the constructed deep semantic topology fusion network model, avoiding the problems of high computational complexity and large amount of calculation caused by using full image training learning in the prior art; and can simultaneously capture deep semantic information and non-local topology information, obtain a more comprehensive description of the ground object target, and avoid the problem of insufficient capture of ground object target information caused by the use of a single structure in the prior art; and can obtain more compact intra-class and more separated inter-class robust features, effectively improving the precision of polarimetric SAR image classification.

[0048] In the present embodiment, the polarimetric SAR image to be classified is preprocessed, and the polarimetric coherence matrix of each pixel point of the preprocessed polarimetric SAR image is calculated, and the polarimetric coherence matrix is normalized to obtain the normalized polarimetric coherence matrix, specifically including:

[0049] Each pixel point in the preprocessed polarimetric SAR image to be classified is represented by a 3x3 polarimetric coherence matrix;

[0050] Wherein,

[0051] In the formula, T (ij) represents the polarimetric coherence matrix of the i-th row and j-th column pixel point in the polarimetric SAR image to be classified, i=1, 2,..., a, j=1, 2,..., b, and a and b respectively represent the length and width of the polarimetric SAR image to be classified, represents the element of the u-th row and v-th column of the polarimetric coherence matrix, u, v∈{1, 2, 3};

[0052] According to the polarimetric coherence matrix of each pixel point, the average value of each element in the polarimetric coherence matrix is calculated;

[0053] Wherein,

[0054] In the formula, represents the average value of each element in the polarimetric coherence matrix, and · represents the product operation;

[0055] According to the polarimetric coherence matrix of each pixel point, the variance value of each element in the polarimetric coherence matrix is calculated;

[0056] Wherein,

[0057] In the formula, represents the variance value of each element in the polarimetric coherence matrix, represents the conjugate operation;

[0058] According to each element in the polarimetric coherence matrix of each pixel point and the average value and variance value of each element, a normalized value of each element in the polarimetric coherence matrix of each pixel point is calculated;

[0059] wherein, In the formula, The normalized value of each element in the polarimetric coherence matrix of each pixel point is represented.

[0060] According to the normalized value of each element in the polarimetric coherence matrix of each pixel point, a normalized value of the polarimetric coherence matrix of each pixel point is calculated.

[0061] wherein,

[0062] In the formula, T (ij) The normalized value of the polarimetric coherence matrix of each pixel point is represented.

[0063] According to the normalized value of the polarimetric coherence matrix of each pixel point, a normalized polarimetric coherence matrix of the polarimetric SAR image to be classified is obtained.

[0064] In the embodiment, a feature vector of each pixel point is extracted from the normalized polarimetric coherence matrix, and a feature matrix of the polarimetric SAR image to be classified is established according to a feature vector set of all pixel points, specifically including:

[0065] According to the normalized polarimetric coherence matrix of each pixel point, six elements in the upper triangular matrix thereof are selected to form a 9-dimensional feature vector of the pixel point.

[0066] wherein,

[0067] In the formula, I (ij) The 9-dimensional feature vector of each pixel point is represented. T The transposition operation is represented, The real part operation is represented, The imaginary part operation is represented, i=1, 2,..., a, j=1, 2,..., b.

[0068] According to the 9-dimensional feature vector of each pixel point, a feature matrix of the polarimetric SAR image to be classified is constructed.

[0069] wherein, F=(I (ij) ) a×b ; In the formula, F represents the feature matrix of the polarimetric SAR image to be classified.

[0070] In the embodiment, according to the ground object real label image and the feature matrix of the polarimetric SAR image to be classified, a training data set and a test data set are constructed, specifically including:

[0071] According to the ground object real label map, 10% of the labeled data in each ground object category is randomly selected as a training sample label set, and the remaining 90% of the labeled data is used as a test sample label set;

[0072] According to the coordinate value of each training sample in the ground object real label map in the training sample label set, a first feature value vector corresponding to the coordinate position in the feature matrix is found. A plurality of second feature value vectors are selected in the upper, lower, left and right four directions respectively with the first feature value vector as the center point, and the first feature value vector and the second feature value vectors selected in the four directions form a first feature matrix block, which is used as the training sample feature of the corresponding training sample. The training sample features of all training samples are combined to form a training sample feature set, and the elements of the training sample label set and the training sample feature set are paired to form a training data set;

[0073] According to the coordinate position of each test sample in the ground object real label map in the test sample label set, a third feature value vector corresponding to the coordinate position in the feature matrix is found. A plurality of fourth feature value vectors are selected in the upper, lower, left and right four directions respectively with the third feature value vector as the center point, and the third feature value vector and the fourth feature value vectors selected in the four directions form a second feature matrix block, which is used as the test sample feature of the corresponding test sample. The test sample features of all test samples are combined to form a test sample feature set, and the elements of the test sample label set and the test sample feature set are paired to form a test data set.

[0074] Specifically, for the nth training sample in the test sample label set, according to the coordinate position of the training sample in the ground object real label map, a first feature value vector corresponding to the coordinate position in the feature matrix is found, and six second feature value vectors are selected in the upper, lower, left and right four directions respectively with the first feature value vector as the center point. The first feature value vector and the second feature value vectors selected in the four directions form a 13x13 first feature matrix block, and the first feature matrix block is used as the training sample feature of the corresponding training sample. The training sample features of all training samples are combined to form a training sample feature set, which contains the same number of training sample features as the number of training samples in the training sample label set. The training sample label set and the elements in the training sample feature set are paired in order to form a training data set; wherein, In the formula, D train represents the training data set, represents the nth training sample, represents the training sample feature corresponding to the nth training sample.

[0075] Specifically, for the mth test sample in the test sample label set, according to the coordinate position of the test sample in the ground object true label map, a third feature vector corresponding to the coordinate position in the feature matrix is found, and a second feature matrix block of 13*13 size is formed with the third feature vector as the center point and 6 fourth feature vectors selected in the upper, lower, left and right four directions, and the second feature matrix block is taken as the test sample feature corresponding to the test sample. The test sample feature set of all test samples is composed of the test sample feature set, and the number of test sample features contained in the test sample feature set is the same as the number of test samples contained in the test sample label set. The test sample label set and the test sample feature set are sequentially paired to form a test data set; wherein, D = 1 test represents the test data set, represents the mth test sample, represents the test sample feature corresponding to the mth test sample.

[0076] In this embodiment, the deep semantic topology fusion network model comprises an input layer, a semantic information extraction module, a topology information extraction module, a feature fusion module and a Softmax classifier output layer.

[0077] The output end of the input layer is connected with the input end of the semantic information extraction module, the output end of the semantic information extraction module is connected with the input end of the topology information extraction module, the output end of the semantic information extraction module and the output end of the topology information extraction module are respectively connected with the input end of the feature fusion module, and the output end of the feature fusion module is connected with the Softmax classifier output layer.

[0078] In this way, the deep semantic topology fusion network model containing the topology information extraction module can flexibly and efficiently gather new node information, avoiding the problems of high computational complexity and large amount of calculation caused by using full map training and learning in the prior art. Moreover, the deep semantic topology fusion network model further contains the semantic information extraction module, which jointly captures deep semantic information and non-local topology information with the topology information extraction module, obtains a more comprehensive description of the ground object target, avoids the problem of insufficient capture of ground object target information caused by the single structure in the prior art, and can obtain more compact intra-class and more separated inter-class robust features, effectively improving the accuracy of polarimetric SAR image classification.

[0079] In order to capture the deep semantic information and non-local topological information at the same time, and obtain a more comprehensive description of the ground object, in a further embodiment, the semantic information extraction module comprises a CNN subnetwork and a global pooling layer, and the output end of the CNN subnetwork is cascaded with the global average pooling layer; wherein the CNN subnetwork comprises a first convolutional layer, a first activation layer, a second convolutional layer, a second activation layer, a third convolutional layer and a third activation layer which are sequentially cascaded.

[0080] In a further embodiment, the parameters of the semantic information extraction module are set as follows:

[0081] The first convolutional layer has 16 feature map numbers of output feature data, a convolutional filter size of 3x3, a moving step of 1 and a padding of 1; the first activation layer has 16 feature map numbers of output feature data; the second convolutional layer has 32 feature map numbers of output feature data, a convolutional filter size of 3x3, a moving step of 1 and a padding of 1; the second activation layer has 32 feature map numbers of output feature data; the third convolutional layer has 64 feature map numbers of output feature data, a convolutional filter size of 3x3, a moving step of 1 and a padding of 1; the third activation layer has 64 feature map numbers of output feature data; and the first global average pooling layer has 64 feature map numbers of output feature data.

[0082] In order to flexibly and efficiently aggregate new node information, in a further embodiment, the topological information extraction module comprises a GraphSAGE subnetwork; wherein the GraphSAGE subnetwork comprises a first GraphSAGE layer and a second GraphSAGE layer which are sequentially cascaded.

[0083] In a further embodiment, the parameters of the topological information extraction module are set as follows:

[0084] The first GraphSAGE layer has 64 feature map numbers of output feature data; and the second GraphSAGE layer has 32 feature map numbers of output feature data.

[0085] In a further embodiment, the feature fusion module comprises a first parallel layer and a first fully connected layer which are sequentially cascaded.

[0086] In a further embodiment, the parameters of the feature fusion module are set as follows:

[0087] The first cascaded layer has 96 feature map numbers of output feature data; and the first fully connected layer has 5 feature map numbers of output feature data.

[0088] In a further embodiment, the input layer has 9 feature map numbers of output feature data.

[0089] In a further embodiment, the Softmax classifier output layer outputs a number of feature maps of the feature data as a number of classification categories 5.

[0090] In the embodiment, the deep semantic topology fusion network model is trained by using the training data set to obtain the trained deep semantic topology fusion network model, specifically including:

[0091] According to the weights of all filters in each convolutional layer, the weights of each GraphSAGE layer, and the weights of each fully connected layer, a weight parameter set is established, and the weight parameter set is initialized by using the He weight initialization method;

[0092] K groups of data are randomly and with replacement selected from the training data set; wherein the K groups of data are wherein k = 1, 2, …, K, and K < N, N is the total number of training samples; K training sample features in the K groups of data selected are grouped into a batch set, and the batch set is input into the input layer of the deep semantic topology fusion network model; wherein the batch set is

[0093] The semantic feature set is obtained by using the semantic information extraction module to perform semantic feature extraction on the batch set; the topological feature set is obtained by using the topological information extraction module to perform topological feature extraction on the semantic feature set; the fusion feature set is obtained by using the feature fusion module to perform fusion feature extraction on the semantic feature set and the topological feature set; the prediction label set is obtained by using the Softmax classifier output layer to perform classification prediction on the fusion feature set; and the weight parameter set is adjusted according to the batch training sample label set corresponding to the batch set and the prediction label set, to obtain the trained deep semantic topology fusion network model.

[0094] In a further embodiment, the weight parameter set is adjusted according to the batch training sample label set corresponding to the batch set and the prediction label set to obtain the trained deep semantic topology fusion network model, specifically including:

[0095] According to the batch training sample label set corresponding to the batch set and the prediction label set, a loss function value is calculated using a cross-entropy loss function; whether the cross-entropy loss function converges is determined according to the loss function value; if yes, the training is stopped; otherwise, the weight parameter set is adjusted, and the next training is performed until the cross-entropy loss function converges or the number of training reaches a preset training number threshold.

[0096] In a further embodiment, the weight parameter set is adjusted, specifically including:

[0097] Derivatives of the loss function value with respect to each weight parameter in the weight parameter set are obtained; the weight parameters are updated in a back propagation manner according to the gradient descent method and the derivatives to obtain updated weight parameters; and the updated weight parameter set is established according to all the updated weight parameters.

[0098] In a further embodiment, the weight parameter set is initialized by using a He weight initialization method, specifically including:

[0099] The weight parameters of the deep semantic topology fusion network model are initialized by using a normal distribution of (0, 1).

[0100] In a further embodiment, the semantic feature set is obtained by performing semantic feature extraction on the batch set by using the semantic information extraction module.

[0101] The batch set is taken as input feature data of the first convolutional layer, first convolutional output feature data of each training sample feature in the batch set obtained through the first convolutional layer is calculated, and first convolutional output features of all training sample features in the batch set are grouped to form a first convolutional output feature set.

[0102] Wherein,

[0103] In the formula, X c1 represents the first convolutional output feature set, represents first convolutional output feature data of the kth training sample feature, W c1 is a convolutional filter matrix in the first convolutional layer, is the kth training sample feature, and * represents a convolutional operation.

[0104] The first convolutional output feature set is taken as input feature data of the first activation layer, first activation output feature data of each training sample feature in the batch set obtained through the first activation layer is calculated, and first activation output features of all training sample features in the batch set are grouped to form a first activation output feature set.

[0105] Wherein,

[0106] In the formula, Z r1 represents the first activation output feature set, represents first activation output feature data of the kth training sample feature, and ReLU(·) is a rectified linear unit (ReLU) activation operation.

[0107] The first activation output feature set is taken as input feature data of the second convolutional layer, second convolutional output feature data of each training sample feature in the batch set obtained through the second convolutional layer is calculated, and second convolutional output features of all training sample features in the batch set are grouped to form a second convolutional output feature set.

[0108] wherein,

[0109] wherein, X c2 denotes the second convolutional output feature set, denotes the second convolutional output feature data of the k-th training sample feature, W c2 is a convolutional filter matrix in the second convolutional layer;

[0110] taking the second convolutional output feature set as input feature data of the second activation layer, calculating second activation output feature data of each training sample feature in the batch set obtained through the second activation layer, and grouping second activation output features of all training sample features in the batch set into a second activation output feature set;

[0111] wherein,

[0112] wherein, Z r2 denotes the second activation output feature set, denotes the second activation output feature data of the k-th training sample feature, and ReLU(·) is a rectified linear unit (ReLU) activation operation;

[0113] taking the second activation output feature set as input feature data of the third convolutional layer, calculating third convolutional output feature data of each training sample feature in the batch set obtained through the third convolutional layer, and grouping third convolutional output features of all training sample features in the batch set into a third convolutional output feature set;

[0114] wherein,

[0115] wherein, X c3 denotes the third convolutional output feature set, x c k 3 denotes the third convolutional output feature data of the k-th training sample feature, W c3 is a convolutional filter matrix in the third convolutional layer;

[0116] taking the third convolutional output feature set as input feature data of the third activation layer, calculating third activation output feature data of each training sample feature in the batch set obtained through the third activation layer, and grouping third activation output features of all training sample features in the batch set into a third activation output feature set;

[0117] wherein,

[0118] wherein, Z r3 denotes the third activation output feature set, a third activation output feature data representing a k-th training sample feature; ReLU(·) is a rectified linear unit (ReLU) activation operation;

[0119] The third activation output feature set is taken as input feature data of the first global average pooling layer, first global average pooling output feature data of each training sample feature in the batch set obtained through the layer is calculated, and the first global average pooling output features of all training sample features in the batch set form a semantic feature set;

[0120] wherein,

[0121] In the formula, F m1 represents a semantic feature set, f k m1 represents first global average pooling output feature data of a k-th training sample feature, G ave (·) represents a global average pooling operation.

[0122] It should be noted that the elements in the convolution filter matrix in the first convolution layer, the second convolution layer and the third convolution layer are all weight parameters in the weight parameter set.

[0123] In further embodiments, topology information extraction module is used to extract topology features from the semantic feature set to obtain a topology feature set, specifically including:

[0124] All training sample features in the batch set are randomly taken as graph nodes in a 15x15 size topology graph structure to complete topology graph initialization; wherein the feature of each graph node is the first global average pooling output feature of the corresponding training sample feature;

[0125] First GraphSAGE output feature data of each training sample feature in the batch set obtained through the first GraphSAGE layer is calculated according to the initialized topology graph;

[0126] wherein,

[0127] In the formula, represents first GraphSAGE output feature data of a k-th training sample feature, υ is a neighborhood graph node of the k-th training sample feature in the initialized topology graph, and υ∈L(k), L(·) represents an upper, lower, left and right neighborhood function, W g1 is a weight parameter in the first GraphSAGE layer, σ g1 (·) represents a first GraphSAGE layer nonlinear activation function, F Mean (·) represents an average aggregation function, and ∪ represents a set difference operation.

[0128] According to the first GraphSAGE output feature data corresponding to the feature of all training samples in the batch set, the feature of the graph node of the initialized topological graph is updated, and an updated topological graph is obtained.

[0129] In a further embodiment, the semantic feature set and the topological feature set are fused and feature extracted by using a feature fusion module to obtain a fused feature set, specifically including:

[0130] The first global average pooling output feature data and the first GraphSAGE output feature data of each training sample feature in the batch set are taken as the input feature data of the first concatenation layer, the first concatenation output feature data of each training sample feature in the batch set obtained through the first concatenation layer is calculated, and the first concatenation output feature of all training samples in the batch set is taken as the first concatenation output feature set.

[0131] wherein, In the formula, denotes the first concatenation output feature set, denotes the first concatenation output feature data of the kth training sample feature, and [·,·] denotes a concatenation operation.

[0132] The first concatenation output feature set is taken as the input feature data of the first full connection layer, and the first full connection output feature data of each training sample feature in the batch set obtained through the first full connection layer is calculated.

[0133] wherein, In the formula, denotes the first full connection output feature data of the kth training sample feature, and w m3 is the weight vector of the first full connection layer. It should be known that the elements of the weight vector of the first full connection layer are weight parameters in the weight parameter set.

[0134] In a further embodiment, the fused feature set is classified and predicted by using a Softmax classifier output layer to obtain a predicted label set, specifically including:

[0135] The first full connection output feature data of each training sample feature in the batch set is taken as the input feature data of the Softmax classifier output layer, the output prediction of each training sample feature in the batch set obtained through the Softmax classifier output layer is calculated, and the output prediction of all training samples in the batch set is taken as the first output prediction set.

[0136] wherein, In the formula, Y denotes the first output prediction set, denotes the output prediction of the kth training sample feature, and exp(·) denotes an exponential operation, a first fully connected output feature data representing a kth training sample feature a cth feature map of the first fully connected output feature data.

[0137] In a further embodiment, the loss function value is calculated by

[0138] wherein Loss represents the loss function value, represents a kth training sample, represents an output prediction of a kth training sample feature.

[0139] In the embodiment, the trained deep semantic topology fusion network model is used to classify the polarimetric SAR image to be classified in the test data set, and the classification result of the polarimetric SAR image to be classified is obtained, which specifically includes:

[0140] The test sample feature in the test data set is input into the input layer of the trained deep semantic topology fusion network model; the semantic feature extraction module is used to extract the semantic feature of the test sample feature to obtain the semantic feature; the topology information extraction module is used to extract the topology feature of the semantic feature to obtain the topology feature; the fusion module is used to extract the fusion feature of the semantic feature and the topology feature; the Softmax classifier output layer is used to classify and predict the fusion feature to obtain the output prediction, that is, the classification result of the polarimetric SAR image to be classified.

[0141] The effects of the present application can be further illustrated by the following simulation:

[0142] 1. Simulation conditions

[0143] The hardware platform is: Intel(R) Core(TM) i7-7700 @ 3.20 GHz, 64.0 GB RAM;

[0144] The software platform is: Pycharm, under the PyTorch framework, with TensorFlow as the backend.

[0145] 2. Simulation method

[0146] (1) The existing technology of SP-GCN based polarimetric SAR classification method; (2) The method of the present application, that is, the polarimetric SAR image classification method based on deep semantic topology fusion network.

[0147] 3. Simulation content and simulation results

[0148] The polarimetric SAR image selected in the simulation experiment is a polarimetric SAR image with a real ground object label map, such as Figure 2As shown in the polarimetric SAR image synthesis map of the San Francisco Bay area acquired by the Chinese spaceborne C-band Gaofen-3 system in 2008, the image mainly includes five land cover types of forest, water, high-density urban area, low-density urban area and developing urban area, and the land cover type numbers of the five land cover types are represented by {1, 2, 3, 4, 5} respectively. Figure 2 As shown in the real land cover label map of the polarimetric SAR image in Figure 3 , the size of the real land cover label map is the same as that of the polarimetric SAR image to be classified. The value at a pixel point in the real land cover label map where the land cover class has been determined is the land cover class number, and such a pixel point is called a labeled pixel point. The value at a pixel point where the land cover class is not determined is 0.

[0149] In simulation experiment 1, the polarimetric SAR image in Figure 2 is classified by using the prior art polarimetric SAR classification method based on SP-GCN, and the classification result is as shown in Figure 4 . Among them, the land cover classification accuracy and the overall classification accuracy are as shown in Table 1.

[0150] In simulation experiment 2, the polarimetric SAR image in Figure 2 is classified by using the method of the present application, and the classification result is as shown in Figure 5 . Among them, the land cover classification accuracy and the overall classification accuracy are as shown in Table 1.

[0151] Table 1 Land cover classification accuracy A c and overall classification accuracy OA

[0152]

[0153] As can be seen from Table 1, compared with the prior art polarimetric SAR classification method SP-GCN based on multi-scale superpixel segmentation and graph convolutional neural network, the present application not only has a significant improvement in the overall accuracy OA, but also has a large improvement in the classification accuracy of the land cover types other than the low-density urban area, which verifies that the present application can extract more comprehensive and more discriminative features to reflect the properties of polarimetric SAR data, and is conducive to improving the classification accuracy of polarimetric SAR images.

[0154] As can be seen from Figure 3 , Figure 4 and Figure 5 , Figure 5 is closer to the real land cover label map of the classified polarimetric SAR image in Figure 3 , and Figure 5The area consistency of the classification result is better, mainly because the polarimetric SAR image classification method based on the deep semantic topology fusion network considers more spatial information, greatly reduces the influence of speckle noise on the classification result, and obtains more compact in-class and more separable in-class robust features, thereby helping to distinguish similar ground cover types and improving the polarimetric SAR image classification accuracy.

[0155] The above merely describes preferred specific embodiments of the present application, but the protection scope of the present application is not limited thereto, and any person skilled in the art, within the technical range disclosed by the present application, according to the technical solution and inventive concept of the present application, makes equivalent replacements or changes, should be covered within the protection scope of the present application.

Claims

1. A method for polarimetric SAR image classification based on deep semantic topology fusion network, characterized in that, The method comprises the following steps: acquiring a polarimetric SAR image to be classified and a corresponding ground object real label image; preprocessing the polarimetric SAR image to be classified, and calculating a polarimetric coherence matrix of each pixel point of the preprocessed polarimetric SAR image; normalizing the polarimetric coherence matrix to obtain a normalized polarimetric coherence matrix; extracting a feature vector of each pixel point from the normalized polarimetric coherence matrix, and constructing a feature matrix of the polarimetric SAR image to be classified according to a feature vector set of all pixel points; constructing a training data set and a test data set according to the ground object real label image and the feature matrix of the polarimetric SAR image to be classified; constructing a deep semantic topological fusion network model; training the deep semantic topological fusion network model by using the training data set to obtain a trained deep semantic topological fusion network model; classifying the polarimetric SAR image to be classified in the test data set by using the trained deep semantic topological fusion network model to obtain a classification result of the polarimetric SAR image to be classified; The deep semantic topological fusion network model comprises an input layer, a semantic information extraction module, a topological information extraction module, a feature fusion module and a Softmax classifier output layer; the output end of the input layer is connected with the input end of the semantic information extraction module; the output end of the semantic information extraction module is connected with the input end of the topological information extraction module; the output end of the semantic information extraction module and the output end of the topological information extraction module are respectively connected with the input end of the feature fusion module; and the output end of the feature fusion module is connected with the Softmax classifier output layer. The semantic information extraction module comprises a CNN subnetwork and a global pooling layer; the output end of the CNN subnetwork is connected with the global average pooling layer; the CNN subnetwork comprises a first convolutional layer, a first activation layer, a second convolutional layer, a second activation layer, a third convolutional layer and a third activation layer which are sequentially connected. The topological information extraction module comprises a GraphSAGE subnetwork; the GraphSAGE subnetwork comprises a first GraphSAGE layer and a second GraphSAGE layer which are sequentially connected.

2. The polarimetric SAR image classification method based on deep semantic topology fusion network according to claim 1, characterized in that, The feature fusion module comprises a first parallel layer and a first full connection layer which are sequentially connected.

3. The polarimetric SAR image classification method based on deep semantic topology fusion network according to claim 2, characterized in that, The training of the deep semantic topological fusion network model by using the training data set to obtain the trained deep semantic topological fusion network model comprises the following steps: establishing a weight parameter set according to the weights of all filters in each convolutional layer, the weights of each GraphSAGE layer and the weights of each full connection layer, and initializing the weight parameter set by using a He weight initialization method; randomly and with replacement from the training data set K The group label data is obtained, a batch training sample label set is obtained, a batch set is established according to the training sample features of the batch training sample label set, and the batch set is input into an input layer; semantic feature extraction is performed on the batch set by using a semantic information extraction module to obtain a semantic feature set; topological feature extraction is performed on the semantic feature set by using a topological information extraction module to obtain a topological feature set; fusion feature extraction is performed on the semantic feature set and the topological feature set by using a feature fusion module to obtain a fusion feature set; and classification prediction is performed on the fusion feature set by using a Softmax classifier output layer to obtain a predicted label set; adjusting the weight parameter set according to a batch training sample label set and a predicted label set corresponding to a batch set to obtain the trained deep semantic topological fusion network model.

4. The polarimetric SAR image classification method based on deep semantic topology fusion network according to claim 3, characterized in that, The adjustment of the weight parameter set according to the batch training sample label set and the predicted label set corresponding to the batch set to obtain the trained deep semantic topological fusion network model comprises the following steps: calculating a loss function value by using a cross-entropy loss function according to the batch training sample label set and the predicted label set corresponding to the batch set; and determining whether the cross-entropy loss function converges according to the loss function value. If yes, stop training; otherwise, adjust the weight parameter set, and perform the next training until the cross-entropy loss function converges or the number of training reaches a preset training number threshold.

5. The polarimetric SAR image classification method based on deep semantic topology fusion network according to claim 4, characterized in that, The adjusting the weight parameter set specifically comprises: Deriving a partial derivative of the loss function value with respect to each weight parameter in the weight parameter set; updating the weight parameter in a back propagation manner according to the gradient descent method and the partial derivative to obtain an updated weight parameter; and establishing an updated weight parameter set according to all the updated weight parameters.

6. The polarimetric SAR image classification method based on deep semantic topology fusion network according to any one of claims 1-5, characterized in that, The trained deep semantic topology fusion network model is used to classify the to-be-classified polarimetric SAR image in the test data set to obtain a classification result of the to-be-classified polarimetric SAR image, specifically comprising: The test sample features in the test data set are input into the input layer of the trained deep semantic topology fusion network model; the semantic feature extraction module is used to extract semantic features from the test sample features to obtain the semantic features; the topology information extraction module is used to extract topology features from the semantic features to obtain the topology features; the fusion module is used to extract fusion features from the semantic features and the topology features; and the Softmax classifier output layer is used to classify and predict the fusion features to obtain the classification result of the to-be-classified polarimetric SAR image.

7. The polarimetric SAR image classification method based on deep semantic topology fusion network according to claim 1, characterized in that, The to-be-classified polarimetric SAR image is preprocessed, and the polarimetric coherence matrix of each pixel point of the preprocessed polarimetric SAR image is calculated, and the polarimetric coherence matrix is normalized to obtain a normalized polarimetric coherence matrix, specifically comprising: Each pixel point in the preprocessed to-be-classified polarimetric SAR image is represented by a 3*3 polarimetric coherence matrix; the average value and the variance value of each element in the polarimetric coherence matrix of each pixel point are calculated according to the polarimetric coherence matrix of each pixel point; the normalized value of each element in the polarimetric coherence matrix of each pixel point is calculated according to each element in the polarimetric coherence matrix of each pixel point and the average value and the variance value of each element; the normalized value of the polarimetric coherence matrix of each pixel point is calculated according to the normalized value of each element in the polarimetric coherence matrix of each pixel point; and the normalized polarimetric coherence matrix of the to-be-classified polarimetric SAR image is obtained according to the normalized value of the polarimetric coherence matrix of each pixel point.

8. The polarimetric SAR image classification method based on deep semantic topology fusion network according to claim 1, characterized in that, The feature vector of each pixel point is extracted from the normalized polarimetric coherence matrix, and a feature matrix of the to-be-classified polarimetric SAR image is established according to a feature vector set of all pixel points, specifically comprising: The upper triangular 6 elements of each pixel point are selected to form a 9-dimensional feature vector of the pixel point according to the normalized polarimetric coherence matrix of each pixel point; and the feature matrix of the to-be-classified polarimetric SAR image is constructed according to the 9-dimensional feature vector of each pixel point.

9. The polarimetric SAR image classification method based on deep semantic topology fusion network according to claim 1, characterized in that, The training data set and the test data set are constructed according to the real landmark map and the feature matrix of the to-be-classified polarimetric SAR image, specifically comprising: 10% of the labeled data in each feature category is randomly selected as a training sample label set, and the remaining 90% of the labeled data is used as a test sample label set according to the real landmark map. According to the coordinate value of each training sample in the ground object real label map in the training sample label set, a first feature value vector corresponding to the coordinate position in the feature matrix is found; a plurality of second feature value vectors are selected in the upper, lower, left and right four directions respectively with the first feature value vector as the center point, and the first feature value vector and the second feature value vectors selected in the four directions form a first feature matrix block, which is used as the training sample feature of the corresponding training sample; the training sample features of all training samples are combined to form a training sample feature set, and the elements of the training sample label set and the training sample feature set are combined in pairs to form a training data set; According to the coordinate position of each test sample in the ground object real label map in the test sample label set, a third feature value vector corresponding to the coordinate position in the feature matrix is found; a plurality of fourth feature value vectors are selected in the upper, lower, left and right four directions respectively with the third feature value vector as the center point, and the third feature value vector and the fourth feature value vectors selected in the four directions form a second feature matrix block, which is used as the test sample feature of the corresponding test sample; the test sample features of all test samples are combined to form a test sample feature set, and the elements of the test sample label set and the test sample feature set are combined in pairs to form a test data set.

Images