A remote sensing image change detection method based on adaptive image regression

Abstract

The invention discloses a remote sensing image change detection method based on adaptive image regression, comprising the following steps: step 1): generating a regression image by processing the pre- and post-images based on a multi-output adaptive regression model; step 2): calculating the difference image between the regression image obtained in step 1) and the pre- and post-images; step 3): analyzing the difference image obtained in step 2) based on a fuzzy local information C-means algorithm to obtain a certain number of significant sample pairs, i.e., change sample pairs and unchanged sample pairs; step 4): strengthening the features of the pre- and post-images based on an association-driven fusion method (AF), and then fusing the original image features and the enhanced features; step 5): constructing a training set with the significant sample pairs in step 3), and then training a classifier model in combination with the fused features in step 4) to predict and obtain a change detection result. The invention has high detection accuracy, low time cost, effectively avoids the interference of noise information, and can adapt to the change detection task of complex ground object scenes.

Description

Technical Field

[0001] The invention relates to the technical field of remote sensing image processing and pattern recognition, and in particular to a remote sensing image change detection method based on adaptive image regression. Background Art

[0002] Since heterogeneous remote sensing images have different imaging characteristics due to the inconsistency of the feature space of the pre- and post-images, direct difference comparison often generates a lot of noise. Therefore, it is difficult for homogeneous remote sensing image change detection methods to obtain high detection accuracy in heterogeneous scenes. However, heterogeneous remote sensing image change detection methods effectively improve this limitation. Existing heterogeneous remote sensing image change detection methods are mainly divided into classification-based, similarity-based and deep learning-based methods. Classification-based methods rely on the performance of the classifier and are prone to cumulative errors. In particular, they are easily affected by the coherent speckle noise of synthetic aperture radar (SAR) images and are difficult to classify correctly. Similarity-based methods rely only on invariant pixel pairs and are difficult to characterize the heterogeneity between data. The detection accuracy in complex ground object scenes is low. Deep learning-based methods require a large amount of labeled training data for supervision, and the training process is complex and time-consuming. In summary, the main disadvantages of existing heterogeneous remote sensing image change detection methods are as follows: 1) They are easily disturbed by noise information and difficult to classify correctly; 2) The detection accuracy is not high and cannot adapt to complex ground object scenes; 3) They rely on labeled training data, and the training process is complex and time-consuming. Summary of the invention

[0003] In order to overcome the shortcomings of the above-mentioned prior art, the purpose of the present invention is to provide a remote sensing image change detection method based on adaptive image regression, which has high detection accuracy and low time cost, effectively avoids the interference of noise information, and can adapt to change detection tasks in complex land scene.

[0004] In order to achieve the above object, the technical solution adopted by the present invention is:

[0005] A remote sensing image change detection method based on adaptive image regression comprises the following steps:

[0006] Step 1): Processing the pre- and post-images based on a multi-output adaptive regression model to generate a regression image;

[0007] Step 2): Calculate the difference image between the regression image obtained in step 1) and the before and after images;

[0008] Step 3): Analyze the difference image obtained in step 2) based on the fuzzy local information C-means algorithm to obtain a certain number of significant sample pairs, namely, change sample pairs and unchanged sample pairs;

[0009] Step 4): Based on the association-driven fusion method (AF), the features of the pre- and post-images are enhanced, and then the original image features and the enhanced features are fused;

[0010] Step 5): construct a training set with the significant sample pairs in step 3), and then train a classifier model with the features fused in step 4 to predict the change detection result.

[0011] In the step 1), based on the multi-output adaptive regression model, the regression image is generated by taking the one containing more information in the before-and-after images as the feature of the training set and the test set, and the other image as the label of the training set. The regression model is trained according to the adaptive regression direction to obtain the regression image for subsequent analysis. This strategy can ensure that the model extracts potential features from the remote sensing image containing more information to characterize the remote sensing image containing less information.

[0012] The quality of the difference image in step 2) directly affects the final change detection result. The larger the grayscale value of the difference image, the greater the possibility that the corresponding position has changed. Since the regression image and the original remote sensing image have more consistent feature representations, a higher quality difference image can be obtained. Therefore, the difference image between the regression image and the original pre- and post-remote sensing images is calculated to more accurately describe the changed area.

[0013] The step 3) uses the FLICM clustering method to segment the difference image, and the number of clusters is set to 3 (i.e., the unchanged area with a small difference value, the uncertain area with a difference value in the middle range, and the changed area with a large difference value). It can identify the uncertain area in the middle that is difficult to divide, and then filter out isolated noise to further ensure the reliability of significant samples, and finally obtain reliable significant sample pairs, i.e., the changed sample pairs and the unchanged sample pairs.

[0014] The step 4) uses the correlation between the high-order information of the data and the features to encode the data into a new feature space; first, the pre- and post-remote sensing images are stacked to construct an initial data set, the high-order information of the initial data set is modeled and the nonlinear expression ability of the original data is improved to obtain enhanced features; further, since the obtained high-order data set can assist in training the classification model to obtain better classification results, the original remote sensing data and the enhanced features are fused to obtain a feature data set for decision-making.

[0015] The step 5) constructs a training data set with the reliable and significant sample pairs obtained in step 3), and then uses the features fused in step 4) as auxiliary training to obtain a multi-layer perceptron classifier model, inputs the pre- and post-remote sensing images into the model, and finally generates a high-precision binary change detection result map.

[0016] Beneficial effects of the present invention:

[0017] The present invention proposes a remote sensing image change detection method based on adaptive image regression, which uses the expression ability of nonlinear depth features of multi-layer perceptron to obtain a regression image, and does not consume a lot of computing time while utilizing the depth features; then, based on the fuzzy local information C-means algorithm, difference analysis is performed, which can identify uncertain areas that are difficult to divide in the middle, filter out isolated noise, and further ensure the reliability of significant sample pairs; in addition, the correlation between different features is considered and the high-order information of remote sensing image data is fully utilized, thereby enhancing the expression ability of the features.

[0018] The method proposed in the present invention has high detection accuracy, can effectively suppress the influence of noise, and can still provide high-precision change detection results in complex terrain scenes.

[0019] This method first uses the difference in information volume between multimodal data to adaptively adjust the regression direction based on information entropy theory, and adopts a multi-output multilayer perceptron image regression algorithm to obtain a regression image that is similar to the feature space of the pre-image or post-image; secondly, the regression image and the original remote sensing image data are calculated to obtain a difference image, and a difference analysis method based on the fuzzy local information C-means (FLICM) algorithm is proposed, the purpose of which is to find a certain number of significant sample pairs for subsequent detection; then, in order to consider the correlation between different features of remote sensing data and make full use of the potential high-order information in the data, a fusion algorithm based on correlation features is used to enhance the stacking features of the original image; finally, the fused features are used to train the classification model to generate a high-precision change detection result map. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a flow chart of the present invention.

[0021] Figure 2 Schematic diagram of the multi-output multilayer perceptron adaptive image regression algorithm in the present invention.

[0022] Figure 3 It is a curve diagram showing the influence of the hyperparameter feature enhancement rate L on the change detection results of the method of the present invention.

[0023] Figure 4 This is a binary change detection result diagram of a remote sensing image change detection method based on adaptive image regression in the present invention. DETAILED DESCRIPTION

[0024] The present invention will be further described in detail below in conjunction with the accompanying drawings.

[0025] Reference Figure 1 , Figure 2 The specific steps of the present invention are described in detail.

[0026] Step 1: Process the pre- and post-images based on a multi-output adaptive regression model to generate a regression image. The step 1 determines the regression direction using an adaptive optimal regression direction learning strategy, which is designed to adaptively obtain the optimal regression image and ensure that the regression image has a more consistent visual representation with the original image. The core idea of ​​generating a regression image based on a multi-output adaptive regression model is to use the pre- and post-images that contain more information as the features of the training set and the test set, and the other image as the label of the training set. The regression model is trained according to the adaptive regression direction to obtain a regression image for subsequent analysis. This method can ensure that the model extracts potential features from remote sensing images that contain more information to characterize remote sensing images that contain less information.

[0027] Due to the different imaging principles of different sensor images, the imaging effects of multimodal data are often quite different, and the amount of information contained in remote sensing images is also different. When training a regression model, due to the reconstruction error in the mapping relationship between the pre-image and the post-image, information loss is inevitable in the regression process. In fact, the bidirectional regression of the pre- and post-remote sensing images does not necessarily produce a good regression effect. In order to adaptively obtain the optimal regression image and ensure that the regression image has a more consistent visual representation with the original image, the present invention designs a learning strategy for adaptive optimal regression direction.

[0028] Assume that the before and after images are represented as Where H and W represent the height and width of the image respectively, B X and B Y are the number of channels of X and Y respectively. Entropy is a measure of the uncertainty of a random variable, i.e., the degree of disorder. The information entropy of a remote sensing image can better reflect the amount of information contained in the remote sensing image, and it is directly related to the degree of grayscale change of the remote sensing image. Therefore, the information entropy of the remote sensing image is used to measure the amount of information it contains. Taking the prior image X as an example, assuming that there are Q grayscale levels distributed in its H×W pixels, is the probability of the i-th gray level appearing, f i is the grayscale frequency, then the amount of information contained in X is:

[0029]

[0030] According to the difference in the amount of information contained in remote sensing images, the following adaptive regression direction strategy is proposed:

[0031] ①If H(X)>H(Y) and |H(X)-H(Y)|>ε, then the regression training set is The regression test set is T e = {x|x∈X}, train the regression model to predict the regression image R X ;

[0032] ②If H(X)<H(Y) and |H(X)-H(Y)|>ε, then the regression training set is The regression test set is T e = {y|y∈Y}, train the regression model to predict the regression image R Y ;

[0033] ③If |H(X)-H(Y)|<ε, perform strategies ① and ② at the same time, use X and Y as test sets respectively, and predict the regression image R X and R Y , and fuse the two difference images when calculating the difference image in step 2.

[0034] in, and It is the eigenvalue of X and Y corresponding to the position of the unchanged pixel pair in the prior information, and x and y correspond to a single pixel. In practice, it is possible to obtain a small amount of unchanged prior information, and the original image data contains labels of some unchanged samples. In the present invention, the parameter ε defaults to 0, which is used to improve the robustness of future actual change detection scenarios. When there is a small difference in the amount of information between the before and after images, the difference between the results of forward regression and backward regression is small, and bidirectional regression can be performed by adjusting ε, and a more robust difference image can be obtained by fusing the forward difference and the reverse difference, providing an alternative for future change detection scenarios.

[0035] As the number of hidden layers increases, the feature learning ability of the network becomes stronger. In response to the regression requirements of remote sensing data, the present invention builds a multi-layer perceptron regression model, whose structure includes 1 input layer, 8 intermediate hidden layers and 1 output layer. The specific network structure is set to B Y -16-32-64-128-128-64-32-16-B X , Relu is selected as the activation function. The specific training details are as follows: the maximum number of iterations is set to 100, the Adam optimizer is used to optimize the weights, and the L2 penalty coefficient is set to 0.0001. First, the regression direction is determined according to the adaptive regression strategy, and then N = 8 × 100 training samples are selected from the pre-image and post-image according to the regression direction to form the regression training set T r After the multilayer perceptron regression model is trained, the regression test set T e The predicted regression matrix R is finally imaged as the regression image R Y .

[0036] Step 2: Calculate the difference image between the regression image obtained in step 1 and the before and after images.

[0037] In the present invention, multi-temporal differences of remote sensing images are used to extract pseudo labels for a small number of samples. These significant samples are more able to represent the real trend of change than samples with insignificant feature differences. The quality of the difference image directly affects the final change detection result. The larger the grayscale value of the difference image, the greater the possibility of change in the corresponding position. The regression image has a more consistent feature representation with the original remote sensing image, and a higher quality difference image can be obtained, thereby more accurately describing the area where changes have occurred.

[0038] The calculation formulas for the difference images corresponding to different regression strategies are as follows:

[0039]

[0040]

[0041] D ③ (h,w)=(D ① (h,w)+D ② (h,w)) / 2

[0042] where |·| represents the absolute distance, 1≤h≤H, 1≤w≤W, r X and y are the corresponding positions belonging to R X and the pixel value of Y, r Y and x are the corresponding positions belonging to R Y and the pixel value of X.

[0043] Step 3: Analyze the difference image obtained in step 2 based on the fuzzy local information C-means algorithm to obtain a certain number of significant sample pairs, namely, change sample pairs and unchanged sample pairs.

[0044] The FLICM clustering method is used to segment the difference image, and the number of clusters is set to 3 (i.e., the unchanged area with small difference value, the uncertain area with difference value in the middle range, and the changed area with large difference value). It can identify the uncertain area in the middle that is difficult to divide, and then filter out the isolated noise to further ensure the reliability of the significant samples, and finally obtain reliable significant sample pairs.

[0045] Step 4: Based on the fusion method driven by association (AF), the features of the pre- and post-images are enhanced, and then the original image features and the enhanced features are fused. The purpose of step 4 is to design fusion rules in an interpretable way, and encode the data into a new feature space by using the high-order information of the data and the association between the features. First, the pre- and post-remote sensing images are stacked to construct the initial data set, the high-order information of the initial data set is modeled, and the nonlinear expression ability of the original data is improved to obtain the enhanced features; further, since the obtained high-order data set can assist in training the classification model to obtain more accurate classification results, the original remote sensing data and the enhanced features are fused to obtain a feature data set for decision-making. The feature fusion method based on association makes great use of the complementary information between different modalities in heterogeneous remote sensing image data, fuses the internal features of the data, and the expression ability of the fused features is more powerful, thereby effectively improving the accuracy of change detection.

[0046] First, stack X and Y to construct the initial data set Model the high-order information of S features and improve the nonlinear expression ability of the original data, establish a mapping φ: S→E, and transform the data from the original (B X +B Y ) dimension is raised to (B X +B Y )L dimension, the feature enhancement process is expressed as:

[0047]

[0048] in, The hyperparameter L is the feature enhancement rate, which represents the highest power of feature enhancement. The initial dataset S is transformed into Then the sample s i The corresponding dimension is expanded to {e j , 1≤j≤(Bx+B Y )L}.

[0049] In order to measure the correlation between different features, the feature fusion (AF) algorithm based on correlation relationship defines a relation fusion matrix R based on the enhanced data E. Each element r(e i , e j ) represents feature e i With e j The relationship between the features is measured by the Pearson correlation coefficient, which is expressed as:

[0050]

[0051] in, and They are e i and e jThe standard deviation of and They are e i and e j Then use the relationship fusion matrix to fuse the enhanced data E and establish a mapping The enhanced data is fused with different features. The larger L is, the richer the diversity of the association relationship is. According to the Taylor series, the following fusion strategy is adopted:

[0052]

[0053] in R kj = r(e k , e j ) means e k and e j The converted data set Given an original sample s i , and get the corresponding enhanced features

[0054] The high-order dataset C obtained by AF can assist in training the classifier to obtain better classification results. Therefore, the original remote sensing data S is fused with C to obtain a feature dataset for decision making. L is positively correlated with the diversity of feature relationships, given by Figure 3 It can be seen that when L = 2 or 3, the change detection performance is often better than that based on the original features, and the number of features is also small and the execution time is short. Considering the performance and time cost, the present invention recommends that the hyperparameter L be 2 or 3 as the default parameter. In addition, experiments show that the feature fusion method based on association relationship has a strong feature expression ability of the fusion space, which effectively improves the performance of the change detection method in this paper.

[0055] Step 5: Construct a training set with the significant sample pairs in step 3, and then train the classifier model with the features fused in step 4 to predict the change detection results.

[0056] The step five is to train the classifier model to generate change detection results. The purpose of change detection is to highlight the differences between image pairs at different times, which can essentially be converted into a binary classification task. Compared with the method of clustering or thresholding the difference image, the method based on the classification model can weaken the influence of noise in the difference image and obtain a more robust change result map. The present invention adopts a multi-layer perceptron classification model as a self-supervised classification model, constructs a training data set with the reliable and significant sample pairs obtained in step 3), and then uses the fused features in step 4) as auxiliary training to obtain a multi-layer perceptron classifier model, inputs the pre- and post-remote sensing images into the model, and finally generates a high-precision binary change detection result map.

[0057] Example: Change detection of real-world heterogeneous remote sensing images before and after the event

[0058] The effectiveness and accuracy of the proposed method are demonstrated through a set of actual change detection experiments on heterogeneous remote sensing images before and after the event. Figure 4 (a) and Figure 4 (b) indicates that this set of images is from Sardinia, Italy, with a size of 412×300, including the previous remote sensing images Figure 4 (a) is a Landsat-5TM image acquired by the U.S. Landsat Thematic Mapper (TM) in September 1995. Figure 4 (b) is an RGB optical image taken from Google Earth at the same location in July 1996. In addition, Figure 4 (c) is the real change reference image depicted through field survey.

[0059] The present invention is compared with the heterogeneous remote sensing image change detection method based on symmetric convolutional coupled network (SCCN), and the comparison results are as follows: Figure 4 (d) and 4(e), Figure 4 (d) is the change detection result map generated by the SCCN method. Figure 4(e) is a change detection result diagram of the method of the present invention, in which white pixels represent changed pixels and black pixels represent unchanged pixels. Although the SCCN method can detect most of the changed areas, a large number of false alarm pixels also appear (actually unchanged pixels but mistakenly detected as changed pixels). The change detection result diagram obtained by the method of the present invention is very clear, and the number of missed detection and false alarm pixels in the diagram is very small. This is because for heterogeneous remote sensing image change detection, the multi-output multi-layer perceptron adaptive regression method proposed in the present invention utilizes the characteristics of the difference in the amount of information in multimodal data to quantify the amount of information contained in heterogeneous remote sensing images of different phases, and adapts to the optimal regression direction to obtain the optimal regression image for change detection; then the FLICM algorithm that considers spatial neighborhood information is used to identify uncertain areas in the difference map, which effectively avoids the negative impact of fuzzy areas between changes and invariance on the acquisition of significant sample pairs, and can accurately identify significant sample pairs; finally, the high-order information of the features is modeled based on the fusion of correlation features, while improving the nonlinear expression ability of the original data. Figure 4 (e) Change detection results It can be seen from the figure that the method of the present invention can make the detection results have the least false alarms and missed detection areas, indicating that the method proposed by the present invention can effectively suppress the noise in the final result and improve the change detection performance. Table 1 is an evaluation of the detection accuracy of the two change detection methods, where OE is the overall error, R m is the missed detection rate, R f is the false alarm rate, F 1 The score is an indicator to measure the accuracy of the binary classification model, and the Kappa (KC) coefficient can fully reflect the accuracy of change detection. The heterogeneous remote sensing image change detection results obtained by the method of the present invention are compared with the reference image, and the missed detection rate R m and false alarm rate R f Significantly reduced, F 1 The score is significantly improved, and the method proposed in the present invention performs better than the SCCN method in terms of Kappa coefficient, which is improved to 79.61%.

[0060] Table 1. Accuracy evaluation of change detection results of different methods

[0061] method OE <![CDATA[R m ]]> <![CDATA[R f ]]> <![CDATA[F 1 ]]> KC SCCN method 8298 0.4100 0.4319 0.4191 0.5506 Method of the present invention 2867 0.2069 0.1582 0.1757 0.7961

[0062] The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A remote sensing image change detection method based on adaptive image regression, characterized in that: The steps include: Step 1): Processing the pre- and post-images based on a multi-output adaptive regression model to generate a regression image; Step 2): Calculate the difference image between the regression image obtained in step 1) and the before and after images; Step 3): Analyze the difference image obtained in step 2) based on the fuzzy local information C-means algorithm (FLICM) to obtain a certain number of significant sample pairs, namely, change sample pairs and unchanged sample pairs; Step 4): Based on the association-driven fusion method (AF), the features of the pre- and post-images are enhanced, and then the original image features and the enhanced features are fused; Step 5): construct a training set with the significant sample pairs in step 3), and then train a classifier model with the features fused in step 4) to predict the change detection results; The step 1) is specifically as follows: Set Q gray levels to be distributed among H×W pixels, is the probability of the i-th gray level appearing, f i is the grayscale frequency, then the amount of information contained in X is: The before and after images are represented as Where H and W represent the height and width of the image respectively, B X and B Y are the number of channels for X and Y respectively; According to the difference in the amount of information contained in remote sensing images, the following adaptive regression direction strategy is proposed: ①If H(X)>H(Y) and |H(X)-H(Y)|>ε, then the regression training set is The regression test set is T e = {x|x∈X}, train the regression model to predict the regression image R X ; ② If H(X) < H(Y) and |H(X) - H(Y)| > ε, then the regression training set is The regression test set is T e = {y|y ∈ Y}, and the regression model is trained to predict the regression image R Y ; ③If |H(X)-H(Y)|<ε, perform strategies ① and ② at the same time, use X and Y as test sets respectively, and predict the regression image R X and R Y , and fuse the two difference images when calculating the difference image in step 2; in, and is the position of the unchanged pixel pair in the prior information, corresponding to the eigenvalues ​​of X and Y respectively, x and y correspond to a single pixel, and the parameter ε defaults to 0; When there is a small difference in the amount of information between the images before and after, the difference between the results of forward regression and backward regression is small, and bidirectional regression is performed by adjusting ε; The step 2) is specifically as follows: The calculation formulas for the difference images corresponding to different regression strategies are as follows: D ③ (h,w)=(D ① (h,w)+D ② (h,w)) / 2 where |·| represents the absolute distance, 1≤h≤H, 1≤w≤W, r X and y are the corresponding positions belonging to R X and the pixel value of Y, r Y and x are the corresponding positions belonging to R Y and the pixel value of X; The step 3) uses the FLICM clustering method to segment the difference image; the number of clusters is set to 3; The step 4) uses the correlation between the high-order information of the data and the features to encode the data into a new feature space; first, the pre- and post-remote sensing images are stacked to construct an initial data set, the high-order information of the initial data set is modeled and the nonlinear expression ability of the original data is improved to obtain enhanced features; the original remote sensing data and the enhanced features are fused to obtain a feature data set for decision-making.

2. The remote sensing image change detection method based on adaptive image regression according to claim 1, characterized in that: The step 5) constructs a training data set with the reliable and significant sample pairs obtained in step 3), and then uses the features fused in step 4) as auxiliary training to obtain a multi-layer perceptron classifier model, inputs the pre- and post-remote sensing images into the model, and finally generates a high-precision binary change detection result map.

Images