Multi-time-sequence PolSAR image labeling method and system based on local-to-global time polarization relation reasoning network

Abstract

The invention discloses a multi-time-sequence PolSAR image labeling method and system based on a local-to-global time polarization relation inference network. The method comprises the steps that a multi-time-sequence PolSAR image is acquired and preprocessed; constructing an inference network model based on the local-to-global time polarization relationship; training a constructed reasoning network model based on a local-to-global time polarization relationship by using the preprocessed multi-time-sequence PolSAR image; and inputting an original multi-time-sequence PolSAR image into the trained reasoning network model based on the local-to-global time polarization relationship, and labeling the multi-time-sequence PolSAR image to obtain a classification result. The multi-time-sequence PolSAR image classification method can be used for achieving multi-time-sequence PolSAR image classification, and meanwhile, compared with a traditional model, the multi-time-sequence PolSAR image classification method can better capture change features of multi-time-sequence PolSAR data so as to improve classification precision.

Description

Technical Field

[0001] The present invention relates to a multi-time series PolSAR image annotation method and system based on a local to global temporal polarization relationship reasoning network, and belongs to the technical field of multi-time series polarization SAR image processing in computer vision. Background Art

[0002] High-resolution crop type automatic classification and labeling is essential for various applications related to agriculture and food security, including monitoring crop yields, assessing the potential impact of natural disasters on agricultural production, and analyzing the potential impact of climate change on agriculture. The electromagnetic waves used by synthetic aperture radar (SAR) are penetrating and not easily affected by weather and sampling time, so they are widely used in agriculture, geology, surveying and mapping and other fields.

[0003] Traditional SAR systems work in a single-polarization mode and can only transmit and receive electromagnetic waves in a single polarization mode to obtain the scattering characteristics of the surface. This is not conducive to obtaining information such as the orientation, geometric size, surface roughness, etc. of surface targets, nor is it conducive to analyzing the scattering mechanism of surface targets. Polarimetric synthetic aperture radar (PolSAR) is an extended system of single-polarization SAR. It uses vector measurement to obtain radar polarization information with more complete and richer target information by alternately transmitting and receiving radar signals in horizontal and vertical polarizations, and can provide a more comprehensive description of the target. However, due to the limitations of satellite orbit control technology and the difficulty of multi-time series PolSAR data acquisition, traditional methods usually work in a single time series form and can only capture regional information at one point in time, which is not conducive to capturing the dynamic changes of surface features over time.

[0004] Multi-time series PolSAR can perform multiple SAR imaging of the same area at different time points in a short period of time. By collecting data at different time points, multi-time series PolSAR can reveal the dynamic changes of the surface or target in time. This time series data provides valuable information for studying environmental changes and the evolution of surface characteristics, and helps to identify and quantify seasonal and cyclical changes in surface characteristics. More complete and richer target information such as seasonal changes in surface cover, growth trends, and other vegetation evolution characteristics can be used to provide a more comprehensive description of the target.

[0005] In view of the progress of multi-time series PolSAR technology and the generation of a large amount of research data, some researchers currently tend to use multi-time series PolSAR to study the impact of polarization scattering characteristics on land cover classification, but they encounter many difficulties in the process. The land cover scattering characteristics are affected by climate, time, and both internal and external factors, but researchers currently do not know the law of the dynamic change of polarization scattering characteristics over time. In addition, with the continuous expansion of PolSAR data in the time dimension, the number of changing features has also increased exponentially. This surge in number not only puts higher requirements on data processing and analysis algorithms, but also increases the complexity of expressing polarization relationships in the time dimension, which in turn poses a challenge to the improvement of land cover classification performance.

[0006] Therefore, the present invention is proposed to solve the following key problems:

[0007] 1. How to learn the dynamic changes of target scattering characteristics over time;

[0008] The target polarization feature information obtained by multi-time series PolSAR has more time dimension information than that of single-time series PolSAR. Because the scattering characteristics of the target are directly related to the description of the polarization characteristics, for single-time series PolSAR, we only need to pay attention to the relationship between the spatial structure and scattering characteristics of the target. For multi-time series PolSAR, the change of the spatial structure of the target (such as vegetation phenology characteristics) over time will directly lead to the change of the scattering information of the target, thereby affecting the performance of the classification network. Therefore, studying the law of dynamic changes of target scattering characteristics over time has become a primary issue.

[0009] 2. How to solve the problem of geometric growth of multi-time series PolSAR data?

[0010] Multi-time series PolSAR can obtain surface coverage information of the same area at different time points in a short period of time. For example, Canada's RADARSAT satellite group can re-cover the same location every four days. After the launch of the Land Exploration No. 1B satellite, the revisit period of the SAR system in China will be shortened from 8 days to 4 days. The shortening of the revisit period has contributed a data foundation to the applied research of multi-time series PolSAR and interferometric SAR. With the continuous expansion of PolSAR data in the time dimension, the number of changing features has also increased exponentially, which has put pressure on the expression of time-polarization relationships and made the improvement of classification performance unsatisfactory.

[0011] 3. How to alleviate the problem of extreme imbalance in multi-time series PolSAR data categories;

[0012] Multi-time series PolSAR data are all real data collected by satellites. Since the distribution of different types of targets on the ground in the natural environment is inherently uneven, the sizes of different targets may vary greatly, and they are limited by many factors such as satellite or sensor orbits, imaging time, and imaging angle. As a result, in the data set, the majority class has a large number of samples, while the minority class has very few samples. This imbalance may reach an order of magnitude difference. Such differences will lead to network prediction bias, so how to alleviate the extreme imbalance of data categories is also a problem that needs to be solved. Summary of the invention

[0013] In order to solve the above problems, the present invention provides a multi-time series PolSAR image annotation method based on a local to global time-polarization relationship reasoning network, which is used to construct a local to global time-polarization relationship reasoning network, and further annotate the multi-time series polarization SAR images based on the constructed local to global time-polarization relationship reasoning network; the present invention improves the classification performance, accuracy and reliability for the surface coverage classification scenario of multi-time series PolSAR data.

[0014] The technical solution of the present invention is: according to the first aspect of the present invention, a multi-time series PolSAR image annotation method based on a local to global time polarization relationship reasoning network includes: obtaining multi-time series PolSAR images and preprocessing them; constructing a local to global time polarization relationship reasoning network model; using the preprocessed multi-time series PolSAR images to train the constructed local to global time polarization relationship reasoning network model; inputting the original multi-time series PolSAR images into the trained local to global time polarization relationship reasoning network model, annotating the multi-time series PolSAR images, and obtaining classification results.

[0015] The method of acquiring and preprocessing multi-time series PolSAR images includes: after preprocessing the multi-time series PolSAR images, forming coherence matrices of different periods [T3], and then using the intensity of the coherence matrix elements as the initial features of the neural network; after preprocessing, the pixel-level features of each PolSAR image are composed of a set of 1×6-dimensional feature vectors.

[0016] The specific steps of obtaining multi-time series PolSAR images and preprocessing them are as follows:

[0017] SAR sensors characterize the scattering characteristics of a target by emitting electromagnetic waves and receiving scattered echoes. The data characteristics of the echo signal obtained will vary depending on the material, geometry, and conductivity of the target. The relationship between the incident electromagnetic wave and the scattered electromagnetic wave is expressed by the scattering matrix S as follows:

[0018]

[0019] Among them, S in the scattering matrix ij Indicates the polarization information of the transmitted and received signals, (i, j = H, V) indicates the polarization form of the transmitted and received signals, H and V represent horizontal polarization and vertical polarization respectively; when S HV =S VH When , the corresponding 3×3 coherence matrix [T3] is expressed as:

[0020]

[0021] in:

[0022]

[0023] Where <·> represents multi-view processing, and the window size of multi-view processing is set to 7×7 to avoid the interference of strong coherent multiplicative noise on the classifier;

[0024] Then obtain the intensity information of the elements in the [T3] matrix (log|T 11 |,log|T 12 |,log|T 13 |,log|T 22 |,log|T 23 | and log|T 33 |).

[0025] The local-to-global temporal polarization relationship reasoning network model includes a self-attention module, a GRU and a Transformer encoder;

[0026] In the self-attention module, the feature vectors at different times are comprehensively compared and analyzed to analyze the speech characteristics of the target and enhance the network's ability to focus on the input sequence at the critical period. Through different self-attention weights, the polarization features at different periods are given different importance.

[0027] In GRU, the weighted feature vector will be processed through a gating mechanism to capture the long-term and short-term dependencies in the sequence and learn important polarization information in the time series feature vector;

[0028] Finally, the features output by the GRU and self-attention modules are summarized by the Transformer encoder for global information analysis, thereby generating effective contextual representations and achieving classification.

[0029] The self-attention module is used to learn the key information of the polarization feature. The processing process in the self-attention module is as follows:

[0030] For the input preprocessed multi-time series PolSAR image, the feature sequence is expressed as [F t p], p∈[1,P] is the sampling time of the multi-time series PolSAR image, t∈[1,6], is the intensity information of the coherence matrix [T3] element in the pth sampling period, that is, log|T 11 |,log|T 12 |,log|T 13 |,log|T 22 |,log|T 23 |,log|T 33 |; In order to capture the change information of the t-th polarization feature, the query matrix Q, key matrix K, and value matrix V of the self-attention module are set to The similarity calculation formula is:

[0031] Similarity(Q,K)=Softmax(MLP(Q,K))

[0032] Among them, MLP(·) is a multi-layer perceptron, which replaces the original dot product operation of the traditional self-attention module, aiming to capture the complex temporal polarization relationship, so that the attention mechanism can learn more change information; then V is multiplied by the attention weight to calculate the weighted feature

[0033]

[0034] The output features of the self-attention module are recorded as: It represents the weighted feature obtained after the t-th polarization feature passes through the self-attention module at the P-th sampling period.

[0035] The GRU processes the output from the self-attention module and the hidden state of the previous time step to infer the temporal relationship; the operation of GRU at the pth time step is summarized as:

[0036]

[0037] Where R p and Z p They are the reset gate and update gate that control the information flow; is the candidate hidden state, is the current input and is represented by R p A combination of modified past information; H p is the final hidden state at time step p, combined with the old state H p-1 and new information H p ; σ(·) and tanh(·) are the S-shaped and hyperbolic tangent activation functions respectively; ⊙ is the dot product operation, W * and b * Denotes weights and biases.

[0038] The Transformer encoder aims to focus on the features of each element in the input sequence and its contextual relationship in the sequence; the specific operations in the Transformer encoder include:

[0039] The output of the self-attention module and the output H of GRU p Merge and use as input C of Transformer encoder p :

[0040]

[0041] The two-layer four-head attention encoder converts the weighted features of the self-attention module and the features provided by GRU into rich contextual representations:

[0042] E attn =MultiHeadAttention(C p )

[0043] The probability is then calculated as follows:

[0044] P(class|input)=Softmax(E attn W+b)

[0045] Where P(class|input) represents the probability of different categories for classification, and W and b represent weights and biases.

[0046] When the training constructed local-to-global temporal polarization relationship inference network model is performed, a label smoothing loss function is used to calculate the difference between the prediction result and the ground truth, and the network parameters are optimized by minimizing the difference.

[0047] According to a second aspect of the present invention, a multi-time series PolSAR image annotation system based on a local-to-global time polarization relationship reasoning network is provided, comprising: an acquisition and preprocessing module, used to acquire and preprocess multi-time series PolSAR images; a construction module, used to construct a local-to-global time polarization relationship reasoning network model; a training module, used to train the constructed local-to-global time polarization relationship reasoning network model using the preprocessed multi-time series PolSAR images; an acquisition module, used to input the original multi-time series PolSAR images into the trained local-to-global time polarization relationship reasoning network model, annotate the multi-time series PolSAR images, and obtain classification results.

[0048] According to a third aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium comprising a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the multi-time series PolSAR image annotation methods based on the local to global time-polarization relationship inference network described above.

[0049] The beneficial effects of the present invention are:

[0050] 1. The present invention uses a customized Self-Attention module to enable the network to focus on different parts of the input sequence regardless of their location. This capability enables the network to more effectively capture long-distance dependencies and calculate weights for different parts of the input sequence, thereby significantly enhancing the network's ability to process complex sequence data and its interpretability;

[0051] 2. The present invention combines GRU (Gated Recurrent Unit) with Transformer decoder, taking advantage of their advantages in extracting features from the time dimension of the sequence. This combination provides a powerful tool for processing specific modalities of pixel-level multi-time series PolSAR data sequences, especially in extracting discriminative information that is critical for classification both locally and globally.

[0052] 3. This invention aims at the problem of class imbalance in small sample multi-time series PolSAR data and uses label smoothing cross entropy loss function to optimize the network. This method improves the generalization ability of the network by preventing the network from being overconfident in prediction, that is, avoiding the network from assigning all probabilities to a single category, which is of great significance for improving classification performance and stability.

[0053] 4. The present invention has better classification results, and the labels in the region are smoother. The method proposed in the present invention is significantly superior to other popular classification methods in terms of overall accuracy and Kappa coefficient, and can better capture the changing characteristics of multi-time series PolSAR data to improve classification accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is a flowchart of the method of the present invention;

[0055] Figure 2 It is the overall framework structure diagram of the model of the present invention;

[0056] Figure 3 This is the schematic diagram of the self-attention module;

[0057] Figure 4 These are the classification results of GRU, iTransformer, and HAG-Former in the two datasets. DETAILED DESCRIPTION

[0058] The invention will be further described below in conjunction with the accompanying drawings and embodiments, but the content of the invention is not limited to the scope of the embodiments.

[0059] Example 1: Figure 1-Figure 4 As shown, according to the first aspect of an embodiment of the present invention, a multi-time series PolSAR image annotation method based on a local to global time polarization relationship reasoning network is provided, including: acquiring multi-time series PolSAR images and preprocessing them; constructing a local to global time polarization relationship reasoning network model; using the preprocessed multi-time series PolSAR images to train the constructed local to global time polarization relationship reasoning network model; inputting the original multi-time series PolSAR images into the trained local to global time polarization relationship reasoning network model, annotating the multi-time series PolSAR images, and obtaining classification results.

[0060] Optionally, the acquiring of multi-time series PolSAR images and preprocessing includes: after preprocessing the multi-time series PolSAR images, forming coherence matrices of different periods [T3], and then using the intensity of the coherence matrix elements as the initial features of the neural network; after preprocessing, the pixel-level features of each PolSAR image are composed of a set of 1×6-dimensional feature vectors.

[0061] Optionally, the specific steps of acquiring multi-time series PolSAR images and performing preprocessing are as follows:

[0062] SAR sensors characterize the scattering characteristics of a target by emitting electromagnetic waves and receiving scattered echoes. The data characteristics of the echo signal obtained will vary depending on the material, geometry, and conductivity of the target. The relationship between the incident electromagnetic wave and the scattered electromagnetic wave is expressed by the scattering matrix S as follows:

[0063]

[0064] Among them, S in the scattering matrix ij Indicates the polarization information of the transmitted and received signals, (i, j = H, V) indicates the polarization form of the transmitted and received signals, H and V represent horizontal polarization and vertical polarization respectively; when S HV =S VH When , the corresponding 3×3 coherence matrix [T3] is expressed as:

[0065]

[0066] in:

[0067]

[0068] Where <·> represents multi-view processing, and the window size of multi-view processing is set to 7×7 to avoid the interference of strong coherent multiplicative noise on the classifier;

[0069] Then obtain the intensity information of the elements in the [T3] matrix (log|T 11 |,log|T 12 |,log|T 13 |,log|T 22 |,log|T 23 | and log|T 33 |).

[0070] Optionally, the local-to-global temporal polarization relationship reasoning network model includes a self-attention module, a GRU (Gated Recurrent Unit) and a Transformer encoder; the overall architecture is as follows Figure 2 As shown;

[0071] In the self-attention module, the feature vectors at different times are comprehensively compared and analyzed to analyze the speech characteristics of the target and enhance the network's ability to focus on the input sequence at the critical period. Through different self-attention weights, the polarization features at different periods are given different importance.

[0072] In GRU, the weighted feature vector will be processed through a gating mechanism to capture the long-term and short-term dependencies in the sequence and learn important polarization information in the time series feature vector;

[0073] Finally, the features output by the GRU and self-attention modules are summarized by the Transformer encoder for global information analysis, thereby generating effective contextual representations and achieving classification.

[0074] Alternatively, in the process of sampling the time series features of PolSAR data, as the sampling time increases, it will become more difficult for the classification network to learn effective temporal polarization features over a long period of time. Therefore, it is particularly important to highlight important polarization features in some key periods. For this purpose, a targeted self-attention module is established to learn the key information of polarization features, such as Figure 3 As shown, Figure 3 This is the principle diagram of the self-attention module of the present invention. The processing process in the self-attention module is as follows:

[0075] For the input preprocessed multi-time series PolSAR image, the feature sequence is expressed as [F t p ], p∈[1,P] is the sampling time of the multi-time series PolSAR image, t∈[1,6], F t p is the intensity information of the coherence matrix [T3] element in the pth sampling period, that is, log|T11 |,log|T 12 |,log|T 13 |,log|T 22 |,log|T 23 |,log|T 33 |; In order to capture the change information of the t-th polarization feature, the query matrix Q, key matrix K, and value matrix V of the self-attention module are set to The similarity calculation formula is:

[0076] Similarity(Q,K)=Softmax(MLP(Q,K))

[0077] Among them, MLP(·) is a multi-layer perceptron, which replaces the original dot product operation of the traditional self-attention module, aiming to capture the complex temporal polarization relationship, so that the attention mechanism can learn more change information; then V is multiplied by the attention weight to calculate the weighted feature

[0078]

[0079] The output features of the self-attention module are recorded as: It represents the weighted feature obtained after the t-th polarization feature passes through the self-attention module at the P-th sampling period.

[0080] Optionally, the GRU processes the output from the self-attention module and the hidden state of the previous time step to infer the temporal relationship; the operation of GRU at the pth time step is summarized as:

[0081]

[0082] Where R p and Z p They are the reset gate and update gate that control the information flow; is the candidate hidden state, is the current input and is represented by R p A combination of modified past information; H p is the final hidden state at time step p, combined with the old state H p-1 and new information H p ; σ(·) and tanh(·) are the S-shaped and hyperbolic tangent activation functions respectively; ⊙ is the dot product operation, W * and b * Represents weights and biases, tailored for each gate and candidate state. This wrapper exploits gates to selectively integrate new inputs with past information, enabling efficient modeling of temporal dependencies. It is particularly well suited for tasks that require understanding of temporal dynamics, such as multi-time series PolSAR images.

[0083] Optionally, the Transformer encoder aims to focus on the features of each element in the input sequence and its contextual relationship in the sequence. The multi-head self-attention mechanism enables the encoder to obtain information from different representation subspaces at the same time, enhancing the network's ability to capture different types of features. The specific operations in the Transformer encoder include:

[0084] The output of the self-attention module and the output H of GRU p Merge and use as input C of Transformer encoder p :

[0085]

[0086] The two-layer four-head attention encoder converts the weighted features of the self-attention module and the features provided by GRU into rich contextual representations:

[0087] E attn =MultiHeadAttention(C p )

[0088] The probability is then calculated as follows:

[0089] P(class|input)=Softmax(E attn W+b)

[0090] Where P(class|input) represents the probability of different categories for classification, and W and b represent weights and biases.

[0091] Optionally, when the training constructed local-to-global temporal polarization relationship inference network model, a label smoothing loss function is used to calculate the difference between the prediction result and the ground truth, and the network parameters are optimized by minimizing the difference.

[0092] Traditional one-hot encoded labels lead to an infinitely increasing difference between correct and incorrectly predicted logits. Overfitting and poor generalization occur when the training data is insufficient to cover all scenarios. Label smoothing involves converting the traditional one-hot encoded label vectors into Replaced with the updated label vector y hot .

[0093]

[0094] Where K represents the total number of classes in multi-class classification, and α is a hyperparameter (set to 0.1 in this invention), expressed as:

[0095]

[0096] Label smoothing leads to more compact clustering within classes, which improves generalization and increases the consistency between the network's prediction confidence and prediction accuracy.

[0097] According to a second aspect of the present invention, a multi-time series PolSAR image annotation system based on a local-to-global time polarization relationship reasoning network is provided, comprising: an acquisition and preprocessing module, used to acquire and preprocess multi-time series PolSAR images; a construction module, used to construct a local-to-global time polarization relationship reasoning network model; a training module, used to train the constructed local-to-global time polarization relationship reasoning network model using the preprocessed multi-time series PolSAR images; an acquisition module, used to input the original multi-time series PolSAR images into the trained local-to-global time polarization relationship reasoning network model, annotate the multi-time series PolSAR images, and obtain classification results.

[0098] According to a third aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium comprising a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute any one of the multi-time series PolSAR image annotation methods based on the local to global time-polarization relationship inference network described above.

[0099] The present invention makes full use of multi-time series PolSAR data, deeply understands the physical basis and polarization scattering mechanism of PolSAR data, combines the temporal variation characteristics of multi-time series analysis and the unique advantages of PolSAR data, and designs a local-to-global temporal polarization relationship inference network model for multi-time series PolSAR image classification. The model aims to find the differences between different targets from the temporal variation characteristics of multi-time series PolSAR data, further improve the accuracy of surface cover classification in complex surface environments, and more effectively monitor and analyze the surface cover conditions that change over time.

[0100] The new network architecture designed by the present invention for land cover classification based on the time dimension sequence of pixel-level multi-time series PolSAR data, HAG-Former, is a network model based on the local to global temporal polarization relationship inference. The multi-module combination architecture is adopted to capture the change information of the polarization characteristics of the target in the key time dimension of multi-time series PolSAR by integrating the different focuses and advantages of different modules, thereby improving performance, accuracy and reliability.

[0101] Secondly, in terms of vegetation phenology, the changes in polarization characteristics are not significant in a short period of time, but the changes in a certain key period will be particularly obvious. Therefore, the present invention customizes the self-attention module to focus on the key changes in polarization characteristics, so that the network can focus on different parts of the input sequence regardless of their location, thereby enhancing its ability to capture dependencies, calculating weights for different parts of the input sequence, and enhancing the interpretability of the network. Adaptive weights are applied to different stages to help the neural network focus on important changes in polarization characteristics.

[0102] Next. The present invention uses GRU to infer the polarized feature relationship of local time. GRU has good performance in time series data classification stability and accuracy. The feature vector weighted by the customized self-attention module is input into GRU. GRU uses the gating mechanism to capture the dependency relationship in the sequence and learn the time dimension information in the time series feature vector.

[0103] In order to prevent the information decay that may be caused by polarizing feature relations during long-term relation reasoning of GRU, the local temporal dependencies are merged with the global weighted features from the self-attention module, which are then input into the Transformer encoder module to extract global information.

[0104] Finally, label smoothing cross entropy loss is used to alleviate the problem of extremely unbalanced data distribution in the sample time PolSAR dataset. This effectively reduces the tendency of the network to be overconfident in its predictions. By distributing the probabilities across multiple categories instead of focusing on a single category, the generalization ability of the network is significantly enhanced.

[0105] In order to verify the effectiveness of the proposed method, experiments were finally conducted on two datasets. The first dataset is the UAVSAR dataset, which contains 5 multi-time series PolSAR images. The 15000×9900 pixel area is divided into 16 categories, 9047044 pixels. 0.1% of the pixels are randomly selected as training samples. The second dataset is the Radarsat-2 dataset, which has 4 multi-time series PolSAR images. In the 5300×3100 pixel area, 5281383 pixels are labeled into 21 categories, and 0.1% of the pixels are randomly selected as training samples.

[0106] Furthermore, the training samples of the UAVSAR dataset are used to train each model first to obtain a trained model, and then the trained model is tested with the test samples from the UAVSAR dataset. The results are shown in Table 1; for the Radarsat-2 dataset, the training samples of the Radarsat-2 dataset are also used to train each model first to obtain a trained model, and then the trained model is tested with the test samples from the Radarsat-2 dataset. The results are shown in Table 2. That is, the results in the table show that compared with the existing method, the classification performance of OA on each data set is improved by 2.21% and 1.79%, respectively, proving the superiority of the method.

[0107] Table 1 Experimental results of UAVSAR dataset

[0108] method Overall accuracy (%) Kappa HAG-Former (the present invention) 90.32 0.8929 GRU 87.97 0.8669 LSTM 87.97 0.8661 RNN 86.65 0.8534 N-BEATS 88.01 0.8673 Transformer 88.02 0.8680 PP-LCNet 83.39 0.8163 MDOAU-Net 85.42 0.8387 iTransformer 88.11 0.8685

[0109] Table 2 Experimental results of Radarsat-2 dataset

[0110] method Overall accuracy (%) Kappa HAG-Former (the present invention) 89.98 0.8892 GRU 88.19 0.8626 LSTM 88.05 0.8611 RNN 87.08 0.8513 N-BEATS 87.27 0.8520 Transformer 87.88 0.8592 PP-LCNet 83.41 0.8165 MDOAU-Net 85.42 0.8387 iTransformer 87.78 0.8648

[0111] In the results, the method proposed in the present invention is obviously superior to other comparative experiments in classification accuracy. On the Radarsat-2 dataset, the OA and Kappa classification accuracy of the HAG-Former of the present invention can reach 89.98% and 88.92%. In contrast, only GRU performed best in the comparative experiment, but its OA and Kappa accuracy were only 88.19% and 86.26%. Although PP-LCNet can achieve good classification results in image classification, it seems to perform poorly in pixel-level classification tasks, so its classification performance is the worst. In the UAVSAR dataset, the classification method proposed in the present invention still has the best classification performance, and its OA and Kappa classification accuracy can reach 90.32% and 89.29%. Among other classification methods, iTransformer showed the best classification performance, but its OA and Kappa classification accuracy were 2.21% and 2.44% lower than our method, respectively. In order to show the differences between different methods in multi-time series PolSAR image classification in more detail, Figure 4The classification results of three methods, HAG-Former, GRU and iTransformer, are shown. It can be seen from the classification results that these three methods have certain classification capabilities in unlabeled areas, which shows that they have certain generalization and scalability. At the same time, our method has better classification results, and the labels in the area are smoother. This shows that the method proposed in the present invention is significantly better than other popular classification methods in terms of overall accuracy and Kappa coefficient. It can better capture the changing characteristics of multi-time series PolSAR data to improve classification accuracy.

[0112] The present invention has wide application and potential in the field of surface type detection and classification. First, in the known technical application fields, the present invention can be used for surface cover classification. By analyzing multi-time series PolSAR data, different surface cover types, such as farmland, forest, wetland, etc., can be accurately identified and distinguished, which is of great significance for resource management, ecological monitoring and land use planning. Secondly, the present invention shows unique advantages in monitoring surface changes. By comparing multiple time series PolSAR images, surface changes such as urban expansion, deforestation, and lake changes can be effectively monitored, providing accurate data support for urban planning, environmental protection and natural resource management.

[0113] In addition, the present invention can also be applied in environmental monitoring and management. Because it can obtain data under different weather conditions and is sensitive to specific environmental features such as water bodies and wetlands, it can be used to monitor coastline changes, lake water quality, ecosystem health, etc., providing important information and technical means for environmental protection and ecological restoration. Finally, PolSAR has important application value in disaster monitoring and response. Its all-weather, high-precision monitoring capabilities enable it to quickly obtain surface information after a disaster occurs, support disaster assessment, rescue response and subsequent reconstruction work, and protect people’s lives and property.

[0114] In terms of potential technical application areas, the present invention is expected to play a greater role in urban planning and development. Through multi-time series PolSAR data, it is possible to achieve detailed monitoring and evaluation of urban expansion, land use changes and their ecological impacts, providing a scientific basis for sustainable urban development. In addition, the present invention can also be applied to agricultural and forestry resource management, including farmland health assessment, forest cover change monitoring, etc., to improve agricultural production efficiency and sustainable use of forest resources.

[0115] In summary, the present invention not only provides important technical support and decision-making basis in the currently known application fields, but also will continue to exert its unique advantages in future technological innovation and application expansion, bringing more innovation and application possibilities to the fields of earth observation, resource management and environmental monitoring.

[0116] The specific implementation modes of the present invention are described in detail above in conjunction with the accompanying drawings, but the present invention is not limited to the above implementation modes, and various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention.

Claims

1. A multi-time series PolSAR image annotation method based on a local-to-global temporal polarization relationship inference network, characterized by: include: Acquire multi-time series PolSAR images and perform preprocessing; Construct a network model based on local-to-global temporal polarization relationship reasoning; The pre-processed multi-time series PolSAR images are used to train and construct a local-to-global temporal polarization relationship inference network model; The original multi-time series PolSAR images are input into the trained local-to-global temporal polarization relationship inference network model, and the multi-time series PolSAR images are annotated to obtain the classification results.

2. The multi-time series PolSAR image annotation method based on the local to global temporal polarization relationship inference network according to claim 1 is characterized in that: The method of acquiring and preprocessing multi-time series PolSAR images includes: after preprocessing the multi-time series PolSAR images, forming coherence matrices of different periods [T3], and then using the intensity of the coherence matrix elements as the initial features of the neural network; after preprocessing, the pixel-level features of each PolSAR image are composed of a set of 1×6-dimensional feature vectors.

3. The multi-time series PolSAR image annotation method based on the local to global temporal polarization relationship inference network according to claim 2 is characterized in that: The specific steps of obtaining multi-time series PolSAR images and preprocessing them are as follows: SAR sensors characterize the scattering characteristics of a target by emitting electromagnetic waves and receiving scattered echoes. The data characteristics of the echo signal obtained will vary depending on the material, geometry, and conductivity of the target. The relationship between the incident electromagnetic wave and the scattered electromagnetic wave is expressed by the scattering matrix S as follows: Among them, S in the scattering matrix ij Indicates the polarization information of the transmitted and received signals, (i, j = H, V) indicates the polarization form of the transmitted and received signals, H and V indicate horizontal polarization and vertical polarization respectively; when S HV =S VH When , the corresponding 3×3 coherence matrix [T3] is expressed as: in: Where <·> represents multi-view processing, and the window size of multi-view processing is set to 7×7 to avoid the interference of strong coherent multiplicative noise on the classifier; Then obtain the intensity information of the elements in the [T3] matrix (log|T 11 |,log|T 12 |,log|T 13 |,log|T 22 |,log|T 23 | and log|T 33 |).

4. The multi-time series PolSAR image annotation method based on the local to global temporal polarization relationship inference network according to claim 1 is characterized in that: The local-to-global temporal polarization relationship reasoning network model includes a self-attention module, a GRU and a Transformer encoder; In the self-attention module, the feature vectors at different times are comprehensively compared and analyzed to analyze the speech characteristics of the target and enhance the network's ability to focus on the input sequence at the critical period. Through different self-attention weights, the polarization features at different periods are given different importance. In GRU, the weighted feature vector will be processed through a gating mechanism to capture the long-term and short-term dependencies in the sequence and learn important polarization information in the time series feature vector; Finally, the features output by the GRU and self-attention modules are summarized by the Transformer encoder for global information analysis, thereby generating effective contextual representations and achieving classification.

5. The multi-time series PolSAR image annotation method based on the local to global temporal polarization relationship inference network according to claim 4 is characterized in that: The self-attention module is used to learn the key information of the polarization feature. The processing process in the self-attention module is as follows: For the input preprocessed multi-time series PolSAR image, the feature sequence is expressed as p∈[1,P] is the sampling time of the multi-time series PolSAR image, t∈[1,6], is the intensity information of the coherence matrix [T3] element in the pth sampling period, that is, log|T 11 |,log|T 12 |,log|T 13 |,log|T 22 |,log|T 23 |,log|T 33 |; In order to capture the change information of the t-th polarization feature, the query matrix Q, key matrix K, and value matrix V of the self-attention module are set to The similarity calculation formula is: Similarity(Q,K)=Softmax(MLP(Q,K)) Among them, MLP(·) is a multi-layer perceptron, which replaces the original dot product operation of the traditional self-attention module, aiming to capture the complex temporal polarization relationship, so that the attention mechanism can learn more change information; then V is multiplied by the attention weight to calculate the weighted feature The output features of the self-attention module are recorded as: It represents the weighted feature obtained after the t-th polarization feature passes through the self-attention module at the P-th sampling period.

6. The multi-time series PolSAR image annotation method based on the local to global temporal polarization relationship inference network according to claim 4 is characterized in that: The GRU processes the output from the self-attention module and the hidden state of the previous time step to infer the temporal relationship; the operation of GRU at the pth time step is summarized as: Where R p and Z p They are the reset gate and update gate that control the information flow; is the candidate hidden state, is the current input and is composed of R p A combination of modified past information; H p is the final hidden state at time step p, combined with the old state H p-1 and new information H p ; σ(·) and tanh(·) are the S-shaped and hyperbolic tangent activation functions respectively; ⊙ is the dot product operation, W * and b * Denotes weights and biases.

7. The multi-time series PolSAR image annotation method based on the local to global temporal polarization relationship inference network according to claim 4 is characterized in that: The Transformer encoder aims to focus on the features of each element in the input sequence and its contextual relationship in the sequence; the specific operations in the Transformer encoder include: The output of the self-attention module and the output H of GRU p Merge and use as input C of Transformer encoder p : The two-layer four-head attention encoder converts the weighted features of the self-attention module and the features provided by GRU into rich contextual representations: E attn =MultiHeadAttention(C p ) The probability is then calculated as follows: P(class|input)=Softmax(E attn W+b) Where P(class|input) represents the probability of different categories for classification, and W and b represent weights and biases.

8. The multi-time series PolSAR image annotation method based on the local to global temporal polarization relationship inference network according to claim 1 is characterized in that: When the training constructed local-to-global temporal polarization relationship inference network model is performed, a label smoothing loss function is used to calculate the difference between the prediction result and the ground truth, and the network parameters are optimized by minimizing the difference.

9. A multi-time series PolSAR image annotation system based on a local-to-global temporal polarization relationship inference network, characterized in that: include: The acquisition and preprocessing module is used to acquire multi-time series PolSAR images and perform preprocessing; A building module for constructing a network model for reasoning about temporal polarization relationships from local to global; The training module is used to train the constructed local-to-global temporal polarization relationship inference network model using pre-processed multi-time series PolSAR images; The acquisition module is used to input the original multi-time series PolSAR images into the trained local-to-global temporal polarization relationship inference network model, annotate the multi-time series PolSAR images, and obtain classification results.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the multi-time series PolSAR image annotation method based on the local to global time-polarization relationship inference network described in any one of claims 1 to 8.

Images