Unsupervised synthetic aperture radar image change detection method based on deep feature map

Abstract

The application discloses a kind of unsupervised synthetic aperture radar image change detection methods based on deep feature map, its implementation steps are: first, generate mean ratio difference map.Then using fuzzy C-means clustering algorithm, difference map is processed by three classification, and the initial result map containing change area, non-change area and uncertain area is generated, and in the change area and non-change area of initial result map, pixel is selected as pseudo-label respectively;With pseudo-label as center, image block is divided on two time SAR image, and training sample set is formed.Next, the training sample set is trained to convolutional neural network model, and after training, change detection analysis is carried out, and the prediction result map containing change and non-change category probability is obtained.Again, construct deep feature map energy function, and use minimum cut algorithm to optimize it, to generate the final change result map.The application has clear advantage in noise resistance and detection precision.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of synthetic aperture radar image detection methods, and particularly relates to an unsupervised synthetic aperture radar image change detection method based on deep feature maps. BACKGROUND

[0002] SAR sensors use microwave imaging principles and can perform all-weather, all-day imaging processing under any atmospheric conditions. Due to this characteristic, SAR image change detection technology has become a research hotspot in the current remote sensing field. This technology aims to obtain change information of ground objects in a region by analyzing SAR images collected at different times in the region. Such information can provide important data guidance for military and civilian applications.

[0003] Generally, SAR image change detection technology is mainly divided into two categories: supervised change detection methods and unsupervised change detection methods. Among them, the supervised change detection method is to first use a large amount of labeled data to train a classifier, and then use the trained classifier to predict the change detection result. The unsupervised method is to directly measure the change information according to some features of the image without any prior information. Both methods have their own advantages and have made great progress in the field of SAR image change detection research.

[0004] Threshold method is the most typical unsupervised change detection method. It analyzes the difference image by statistical modeling, sets the optimal threshold according to the image distribution characteristics, and divides the image into change and non-change regions. However, SAR images contain a large amount of speckle noise, which greatly interferes with statistical modeling. The clustering method does not need to perform statistical modeling on the difference image, but relies on the membership function to gather similar pixels to minimize intra-class differences and maximize inter-class differences. It has more flexibility than the threshold method and has attracted more attention from scholars. In addition to the above, common unsupervised change detection methods include: level set-based change detection method, field theory-based change detection method, kernel theory-based change detection method, etc.

[0005] At present, the supervised change detection method mainly takes the deep learning network model as the core, takes the labeled data as the driving, excavates the deep internal features of the SAR image through the network training, and then realizes the full characterization and description of the ground object information, and assists the execution of the ground object change detection task. For example, Gong et al. in the document (Gong M., Zhao J., Liu J., Miao Q., Jiao L., Change detection in synthetic aperture radar images based on deep neural networks[J]. IEEE Transactions on Neural Networks and Learning Systems, 2017, 27(1): 125-138.) adopted the Boltzmann machine model to study the change detection of SAR images, which avoided the construction requirement of difference map by means of the special structure of end-to-end of the Boltzmann machine, and directly obtained the change feature information in the SAR images of two time points to execute the change detection task. The Chinese patent with the authorized announcement number CN 108776968B and the title of “SAR image change detection method based on deep forest” discloses a SAR image change detection method combining deep forest and image gradient features, which combines the detection results of the deep forest model with the gradient feature map of the image, strengthens the description ability of the change region boundary, and improves the precision of change detection.

[0006] Compared with the unsupervised change detection method, the change detection method based on deep learning can exhibit better detection results under the condition of complete training. However, the difficulty in obtaining high-quality training samples restricts the practical application of such methods. In addition, the deep learning model often has a certain tendency, which causes the detection result to still have optimization space, and the post-processing mode of most such methods is relatively simple, which needs further research. SUMMARY

[0007] The present application provides an unsupervised synthetic aperture radar image change detection method based on deep feature map to solve the problem of insufficient detection result precision in the prior art change detection method for SAR images.

[0008] In order to achieve the above purpose, the technical scheme adopted by the present application is:

[0009] The unsupervised synthetic aperture radar image change detection method based on deep feature map comprises the following steps:

[0010] Step 1, obtaining the original SAR data of synthetic aperture radar, obtaining two SAR images X1 and X2 of different time points to be changed from the original SAR data;

[0011] Step 2, mean ratio operation is performed on the to-be-change-detection SAR images X1 and X2 obtained in step 1 to generate a mean ratio difference image;

[0012] Step 3, the mean ratio difference image obtained in step 2 is processed by three-classification using a classical fuzzy C-means clustering algorithm to obtain an initial result image containing a change region, a non-change region and an uncertain region; pseudo labels are selected from the pixel points in the change region and the non-change region in the initial result image, and based on the position information of the pseudo labels, image blocks are divided from the to-be-change-detection SAR images X1 and X2 respectively, and a training sample set is constructed by the divided image blocks and the corresponding pseudo labels;

[0013] Step 4, the training sample set obtained in step 3 is used to train a convolutional neural network, and then the to-be-change-detection SAR images X1 and X2 are input into the trained convolutional neural network to obtain a prediction result image containing change and non-change category probabilities;

[0014] Step 5, the pixel points on the prediction result image obtained in step 4 are set as vertices of a deep feature graph model, and the vertices are connected to each other using K-nearest neighbor rule to obtain the deep feature graph model;

[0015] Step 6, on the basis of the deep feature graph model, a deep feature graph energy function is constructed according to the change and non-change category probability information in the prediction result image, and the deep feature graph energy function is optimized and solved using a min-cut algorithm to obtain the category distribution of the vertices of the deep feature graph when the energy is minimized, thereby generating a final change detection result.

[0016] Further, step 1 further comprises: performing radiation correction and geometric registration preprocessing on the original SAR image, and then obtaining two to-be-change-detection SAR images X1 and X2 at different times from the preprocessed image.

[0017] Further in step 2, the local block mean of the to-be-change-detection SAR images X1 and X2 obtained in step 1 is calculated, and a difference image is generated by combining a ratio operator to obtain the mean ratio difference image.

[0018] Further in step 3, 28-32% of the pixel points in the change region and the non-change region in the initial result image are selected as pseudo labels.

[0019] Further, the selection mode is random selection.

[0020] Further, the convolutional neural network in step 4 is a LeNet-5 convolutional neural network with maximum pooling.

[0021] Further, the network parameters of the convolutional neural network are trained according to the principle of back propagation in step 4.

[0022] Further, the deep feature map energy function in step 6 is composed of a probability constraint term and a spatial context constraint term.

[0023] Compared with the prior art, the present application has the following beneficial effects:

[0024] 1) The present application uses the fuzzy C-means clustering algorithm to perform three-class processing on the mean ratio difference map, selects reliable labels of the change region and the non-change region, and pairs the original image block to jointly form a high-quality training sample data set; on this basis, the training and detection of the LeNet-5 convolutional neural network model are completed, and a high-quality prediction result map is generated.

[0025] 2) The present application constructs a deep feature map model on the prediction result map, describes the spatial context relationship of the prediction probability using the edges of the deep feature map model, constructs a deep feature map energy function driven by probability information, and uses the minimum cut algorithm to optimize the energy to obtain the class distribution of the deep feature map vertex when the energy is minimized, thereby completing the division of the change region and the non-change region, so that the change detection result is more smooth and accurate. BRIEF DESCRIPTION OF DRAWINGS

[0026] Figure 1 is a method flowchart in the embodiment one of the present application.

[0027] Figure 2 is the change detection result on the SAR image in the Huaihe River region in the experiment of the embodiment two of the present application, wherein: Figure 2 (a) is the SAR image of the flood season of the Huaihe River, Figure 2 (b) is the SAR image of the dry season of the Huaihe River, Figure 2 (c) is the artificial interpretation reference map, Figure 2 (d)-(h) respectively correspond to the change detection results of the graph cut, the principal component analysis, the convolutional neural network, the fuzzy C-means clustering and the method of the present application.

[0028] Figure 3 is the change detection result on the SAR image in the Huaihe River region in the experiment of the embodiment two of the present application, wherein: Figure 3 (a) is the SAR image of the flood season of the Huaihe River, Figure 3 (b) is the SAR image of the dry season of the Huaihe River, Figure 3 (c) is the artificial interpretation reference map, Figure 3 (d)-(h) respectively correspond to the change detection results of the graph cut, the principal component analysis, the convolutional neural network, the fuzzy C-means clustering and the method of the present application. DETAILED DESCRIPTION

[0029] The application will be further described below in conjunction with the drawings and examples.

[0030] Example 1

[0031] As shown in the drawings, the embodiment discloses a deep feature map-based unsupervised synthetic aperture radar image change detection method, comprising the following steps: Figure 1 Step 1, obtaining the original SAR image of the synthetic aperture radar, sequentially performing radiometric correction and geometric registration preprocessing on the original SAR image, and then obtaining two SAR images X1 and X2 to be changed from the preprocessed image China at different times.

[0032] Step 2, obtaining the local block mean of the SAR images X1 and X2 to be changed in step 1, and generating a difference map combined with the ratio operator. Let the mean ratio difference map be DI(x, y), then the pixel point in the mean ratio difference map is (x, y), and the change degree represented by the pixel point (x, y) in the mean ratio difference map DI(x, y) is calculated by the following formula:

[0033]

[0034]

[0035] Wherein, f1(x, y) is the 3x3 block mean centered at pixel point (x, y) in the SAR image X1 to be changed; f2(x, y) is the 3x3 block mean centered at pixel point (x, y) in the SAR image X2 to be changed.

[0036] Step 3, using the classical fuzzy C-means clustering algorithm to perform three-class processing on the mean ratio difference map obtained in step 2, to obtain an initial result map containing change regions, non-change regions and uncertain regions.

[0037] From the pixel points in the change region and the non-change region in the initial result map, a plurality of pixel points in the range of 28-32% are randomly selected as pseudo-labels. In this embodiment, the optimal selection ratio is 30%.

[0038] Then, taking any one pseudo-label (x', y') as the center, dividing ωxω size image blocks P1 and P2 on the SAR images X1 and X2 to be changed respectively, splicing the divided image blocks P1 and P2, and pairing with the corresponding position pseudo-labels to jointly constitute a training sample set.

[0039] Step 4, inputting the training sample set with pseudo-labels into the LeNet-5 convolutional neural network applying maximum pooling, training the network parameters of the LeNet-5 convolutional neural network according to the back propagation principle, and obtaining the trained convolutional neural network.

[0040] ​Then the SAR images X1 and X2 to be changed are input into the trained convolutional neural network model for change detection analysis, and the prediction result graph containing the change and non-change category probabilities is obtained by using the Softmax function. The prediction result graph has a size of m×n×z, where m×n is the row and column number of the SAR images X1 and X2 to be changed, z is the number of change detection categories, and z is divided into change probability z c , non-change probability z u , and two kinds.

[0041] Step 5, the pixel points on the prediction result graph are set as the vertices of the deep feature graph model, and the K-nearest neighbor rule is used to connect the vertices to each other to obtain the deep feature graph model DFG={V, E, W}, and the internal elements are specifically represented as follows:

[0042] V={v1, v2,..., u R}, |V|=m·n, v i ={z c，i , z u，i}

[0043] E={(v i , c j ); v j ∈KNN(v i )}

[0044]

[0045] Where: V is the vertex set of the deep feature graph model; v i ={z c，i , z u，i} is any vertex on the deep feature graph, which contains the predicted change probability z c，i and the non-change probability z u，i ; E is the edge connected according to the K-nearest neighbor rule, KNN(v i ) is the K-nearest neighbor vertex set of vertex v i , and v j is any vertex in the K-nearest neighbor vertex set KNN(v i ); W is the weight on the edge, which is used to represent the similarity of the category probability distribution between the vertices.

[0046] Step 6, according to the change and non-change category probability information in the prediction result graph, a deep feature graph energy function is constructed, and a minimum cut algorithm is used to optimize and solve the deep feature graph energy function to obtain the category distribution of the deep feature graph vertices when the energy is minimized, thereby generating the final change detection result.

[0047] The deep feature graph energy function is represented as follows:

[0048]

[0049] In the formula, D(l) i ) represents a probability constraint term, referring to the label l i (Change l) c，i or non-change l u，i ) Assigned to vertex v i The required overhead, D(l) i It can be obtained by the following formula:

[0050] D(l i )=-lnP r(v i |l i )

[0051]

[0052] Wherein, Pr(v i |l i Let z be a conditional probability function. c，i With z u，i This represents the probability of the pixel category being changed and the probability of the pixel category being unchanged within the predicted result image in step 4. U(l) i , l j ) is a spatial context constraint term used to penalize discontinuities in vertex labels (i.e., l) i ≠lj ) ,U(l i , l j The calculation formula is as follows:

[0053] U(l i , l j )=δ(l i , l j )·W(v i c j )

[0054]

[0055] The minimum cut algorithm is used to optimize the energy function φ(l) of the depth feature map, and the class distribution of the vertices of the depth feature map when the energy is minimized is obtained, thereby generating the final change detection result.

[0056] Example 2

[0057] This embodiment is an experimental example. Two sets of real SAR image data were selected for comparative experiments to further illustrate the effectiveness of the change detection method described in Embodiment 1.

[0058] 1. Experimental setup:

[0059] Experimental environment: MATLAB 2022, Intel(R) Core(TM) i9-12700K, Windows 10.

[0060] Experimental data: Figure 2 The SAR images of the Huaihe River in the flood season and the dry season collected by the Sentinel-1 SAR satellite have a spatial resolution of 5 m and an image size of 439*379 pixels; Figure 3 The SAR images of Huoqiu County before and after the flood disaster collected by the Sentinel-1 SAR satellite have a spatial resolution of 10 m and an image size of 429*580 pixels.

[0061] Experimental parameters: image block size ω = 21, number of near neighbors K = 8.

[0062] Experimental comparison method: graph cut, principal component analysis, convolutional neural network, fuzzy C-means clustering are used respectively.

[0063] Experimental evaluation index: the number of missed detections, the number of false detections, the correct rate and the Kappa coefficient are used to quantitatively evaluate the change detection results.

[0064] 2. Experimental results and analysis:

[0065] In experiment 1, the graph cut, principal component analysis, convolutional neural network, fuzzy C-means clustering and the method of the present application are applied to the change detection of the SAR images of the Huaihe River in the flood season and the dry season respectively. The results are shown in Figure 2 Figure 2 (a) is the SAR image of the Huaihe River in the flood season, Figure 2 (b) is the SAR image of the Huaihe River in the dry season, Figure 2 (c) is the artificial interpretation reference map, Figure 2 (d)-(h) correspond to the change detection results of the graph cut, principal component analysis, convolutional neural network, fuzzy C-means clustering and the method of the present application respectively.

[0066] In experiment 2, four existing change detection methods and the method of the present application are used to detect the SAR images near Huoqiu County. The results are shown in Figure 3 Figure 3 (a) is the SAR image of Huoqiu County in the flood season, Figure 3 (b) is the SAR image of Huoqiu County in the dry season, Figure 3 (c) is the artificial interpretation reference map, Figure 3 (d)-(h) correspond to the change detection results of the graph cut, principal component analysis, convolutional neural network, fuzzy C-means clustering and the method of the present application respectively.

[0067] From Figure 2 , Figure 3 ​​As can be seen from Table 1, the overall detection accuracy of the change detection method in the embodiment one is better than that of other change detection methods. The reason is that the change detection method in the embodiment one applies a new reliable sample generation method, which improves the training and prediction effect of the convolutional neural network model; on this basis, a deep feature map model is constructed, the two-class probability information of prediction is summarized by a graph vertex, the spatial context information of different vertex probabilities is constructed by a graph edge, and the prediction result of the convolutional neural network is post-optimized by means of a graph cut algorithm, so that more accurate change detection results are obtained. Table 1 is as follows:

[0068] Table 1: Change detection results of SAR images in Huaihe River and Lu'an City

[0069]

[0070] The above experimental results show that, compared with the existing SAR image change detection method, the present application has clear advantages in noise resistance and detection accuracy.

[0071] The embodiments of the present application are only used to describe the preferred embodiments of the present application, and do not limit the concept and scope of the present application. Without departing from the design idea of the present application, various modifications and improvements of the technical solutions of the present application made by those skilled in the art shall fall within the protection scope of the present application. The technical content of the present application has been fully recorded in the claims.

Claims

1. An unsupervised synthetic aperture radar image change detection method based on depth feature maps, characterized in that, Includes the following steps: Step 1: Acquire the raw SAR data from the synthetic aperture radar, and obtain two SAR images of the changes to be detected at different times from the raw SAR data. and ; Step 2: Analyze the SAR image to be changed obtained in Step 1. and Perform the mean ratio calculation and generate a mean ratio difference chart; Step 3: Use the classic fuzzy C-means clustering algorithm to perform three-class classification on the mean ratio difference map obtained in Step 2 to obtain an initial result map containing the changed region, the unchanged region, and the uncertain region; Pseudo-labels are selected from pixels in the changed and unchanged regions of the initial result image, and based on the location information of the pseudo-labels, the changes are detected in the SAR image to be changed. and The image is divided into patches, and the training sample set is constructed by dividing the image patches and their corresponding pseudo-labels; Step 4: Train the convolutional neural network using the training sample set obtained in Step 3, and then apply the changes to the SAR image to be detected. and The input is fed into a trained convolutional neural network to obtain a prediction result image containing the probabilities of changed and unchanged categories; Step 5: Set the pixels on the predicted result map as vertices of the deep feature map model, and utilize... The nearest neighbor rule connects the vertices to obtain a deep feature map model. Its internal elements are specifically represented as follows: in: For the vertex set of the deep feature map model; For any vertex in the depth feature map, this vertex contains the predicted probability of change. Non-change probability ; Based on Edges connected by the nearest neighbor rule, As vertices of Nearest neighbor vertex set, Nearest neighbor vertex set Any vertex in the middle; The edge weights are used to characterize the similarity of the class probability distributions between vertices. Step 6: Based on the probability information of changed and non-changed categories in the prediction result map, construct the energy function of the deep feature map, and use the minimum cut algorithm to optimize the energy function of the deep feature map to obtain the category distribution of the deep feature map vertices when the energy is minimized, thereby generating the final change detection result; The energy function of this depth feature map is expressed as follows: In the formula, For probability constraints, it refers to the label ,change or no change Assigned to vertices Required expenses, It can be obtained from the following formula: in, It is a conditional probability function. and The predicted class probabilities of pixels within the image from step 4 are the probabilities of changed and unchanged categories. This is a spatial context constraint term used to penalize discontinuities in vertex labels, i.e. , The calculation formula is as follows: Using the minimum cut algorithm to analyze the energy function of the deep feature map The optimization solution is performed to obtain the class distribution of vertices in the depth feature map when the energy is minimized, thereby generating the final change detection result.

2. The unsupervised synthetic aperture radar image change detection method based on depth feature maps according to claim 1, characterized in that, Step 1 also includes: performing radiometric correction and geometric registration preprocessing on the original SAR image, and then obtaining two SAR images of the changes to be detected at different times from the preprocessed image. and .

3. The unsupervised synthetic aperture radar image change detection method based on depth feature maps according to claim 1, characterized in that, In step 2, the SAR image to be detected for change obtained in step 1 is retrieved. and The local block mean is used to generate the mean-to-difference map by combining it with the ratio operator.

4. The unsupervised synthetic aperture radar image change detection method based on depth feature maps according to claim 1, characterized in that, In step 3, 28-32% of the pixels in the changed and unchanged areas of the initial result image are selected as pseudo-labels.

5. The unsupervised synthetic aperture radar image change detection method based on depth feature maps according to claim 4, characterized in that, The selection method is random selection.

6. The unsupervised synthetic aperture radar image change detection method based on depth feature maps according to claim 1, characterized in that, The convolutional neural network mentioned in step 4 is a max-pooled LeNet-5 convolutional neural network.

7. The unsupervised synthetic aperture radar image change detection method based on depth feature maps according to claim 6, characterized in that, In step 4, the network parameters of the convolutional neural network are trained according to the backpropagation principle.

8. The unsupervised synthetic aperture radar image change detection method based on depth feature maps according to claim 1, characterized in that, In step 6, the deep feature map energy function consists of a probability constraint term and a spatial context constraint term.

Images