Change detection method based on label constrained superpixel conditional random field

Abstract

The application discloses a change detection method based on a label constraint super-pixel conditional random field; the technical scheme is characterized in that a super-pixel is used as a basic analysis unit, local features and structural information of an image can be more effectively expressed and extracted, and complex spatial information of the image can be more effectively mined; meanwhile, a conditional random field is optimized through a label constraint strategy, and spatial correlation between super-pixels can be more effectively utilized. The change detection method can effectively solve the problem that existing change detection technologies based on a conditional random field mainly use a pixel as a basic analysis unit, spatial information cannot be sufficiently utilized, and more false detection errors are easily caused in change detection, and optimal change detection precision can be achieved.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing technology, specifically relating to a method for detecting changes in label-constrained superpixel conditional random fields. Background Technology

[0002] Change detection is a research hotspot in the field of remote sensing, with many important applications. In low-to-medium resolution remote sensing imagery, traditional detection methods achieve good accuracy. In recent years, with the continuous advancement of sensor technology, the spatial resolution of remote sensing imagery has become increasingly higher, resulting in richer image details and clearer ground feature information, greatly expanding the application areas of change detection. However, this also brings new challenges, such as decreasing inter-class variance and increasing intra-class variance.

[0003] Utilizing spatial information is an effective way to improve the accuracy of change detection in high-resolution remote sensing images. Therefore, Conditional Random Field (CRF) models have been introduced into change detection tasks with good results. However, existing CRF-based change detection techniques mainly use pixels as the basic analysis unit, which cannot fully utilize spatial information and easily leads to many false detection errors. To address this problem, this invention proposes a change detection method based on label-constrained superpixel CRF. A superpixel is an image sub-region composed of a certain number of adjacent pixels with similar features, which is more conducive to the extraction of local image features and the expression of structural information, and more conducive to mining the complex spatial information of the image. Simultaneously, the label-constrained optimized CRF can more effectively utilize the correlation between superpixels. Summary of the Invention

[0004] This invention provides a change detection method based on label-constrained superpixel conditional random fields, which can more effectively utilize the spatial information of high-resolution remote sensing images. To achieve the above technical effects, the technical solution adopted by this invention is as follows:

[0005] The method for detecting changes in label-constrained superpixel conditional random fields includes the following steps:

[0006] S1: Preprocessing is performed on two high-resolution remote sensing images to reduce the influence of factors such as the sensor itself, atmosphere and lighting conditions. Preprocessing includes registration, radiometric correction and geometric correction. Then, a difference image between the two images is generated by using vector variation analysis.

[0007] S2: A simple linear iterative clustering algorithm is used to perform super-pixel segmentation on the difference image to obtain a super-pixel difference image;

[0008] S3: Introduce the concept of superpixel into fuzzy C-means clustering, and use the superpixel fuzzy C-means clustering algorithm to perform fuzzy clustering on the superpixel differential image to obtain the membership degree of each superpixel to the changed and unchanged classes;

[0009] S4: Introduce the concept of superpixels into the conditional random field model, and use the label constraint strategy to optimize the conditional random field to construct a label-constrained superpixel conditional random field model; and calculate the energy function of the constructed label-constrained superpixel conditional random field based on the superpixel difference images and superpixel membership degrees obtained in S2 and S3.

[0010] S5: Optimize the energy function calculated in step 4 using the graph cutting inference algorithm to generate a change detection map.

[0011] Preferably, in step S3 above, the specific method for calculating the membership degree is as follows:

[0012] Use Y SD ={s i |i=1,2,3,...,M} represents the set of superpixels in the superpixel difference image, G={g i |i=1,2,3…,M} represents the set of gray values ​​of superpixels; g i Represents superpixel s i The gray value is the average gray value of all pixels in the superpixel; M represents the number of superpixels; fuzzy C-means clustering optimizes the objective function J through iterative optimization. C To calculate superpixel membership:

[0013]

[0014] Among them, u ij Represents superpixel s i The membership degree of category j, c j The cluster center of category j is represented by q, the weighting index is q, and the number of categories is C; u ij and c j Calculate using the following equations (2) and (3):

[0015]

[0016]

[0017] In equations (2) and (3), k represents a subscript, and the other symbols have the same meaning as in equation (1). This invention includes two categories: variation and no variation, i.e., C = 2.

[0018] Preferably, in step S4 above, the specific method for calculating the energy function of the constructed label-constrained superpixel conditional random field is as follows:

[0019] Using superpixels as the basic analysis unit, the conditional random field model is used to analyze the superpixel difference image Y. SD ={s i |i=1,2,3,...,M} (i.e., the observation field) and its corresponding label set X={x i We perform posterior probability modeling on |i=1,2,...,M} (i.e., the label field), x i Represents superpixel s i Category labels; Conditional Random Fields (CRFs) solve problems by transforming posterior probabilities into energy functions; The energy function E(X|Y) of a label-constrained superpixel CRF. SD It can be described as the sum of unary and dual potentials:

[0020]

[0021] Where ψ i (x i ,Y SD Let ψ be a univariate potential function. ij (x i ,x j ,Y SD ) is a binary potential function; λ is a non-negative constant used to balance the univariate and binary potentials.

[0022] Furthermore, the univariate potential function ψ i (x i ,Y SD ) indicates the superpixel difference image Y in the observation field. SD Lower superpixel s i Assigned to category x i The cost is defined by the following formula (5):

[0023] ψ i (x i ,Y SD )=-ln(P(x i |s i (5)

[0024] In the formula, ln represents the natural logarithm operator, x i ∈{0,1}, where 0 represents the unchanged class label and 1 represents the changed class label, P(x i |s i ) indicates the superpixel difference image Y in the observation field. SD Lower superpixel s i Assigned to category x i The probability of.

[0025] Furthermore, P(x i |s i The result is obtained by the superpixel blur C-mean algorithm, as shown in the following formula (6):

[0026]

[0027] In the formula, u ixi This represents the superpixel s obtained in step 3 through superpixel fuzzy C-means clustering. i Belongs to category x i The degree of membership.

[0028] Furthermore, the dual potential function ψ ij (x i ,x j ,Y SD Utilizing spatial information to optimize change detection results involves using the spatial correlation of images to classify adjacent superpixels into the same category; the construction of the binary potential function requires the use of superpixels s i The local neighborhood N i The neighborhood system of a superpixel image differs from that of a pixel image:

[0029] The neighborhood system of a pixel image is regular, with each pixel typically having the same neighborhood; while the neighborhood system of a superpixel image is irregular, with each superpixel typically having a different neighborhood system.

[0030] Preferably, the present invention utilizes the adjacency matrix in graph theory to describe the irregular neighborhood relationships between superpixels:

[0031] For any superpixel s i The neighborhood of a supercell is defined as all its neighboring supercells, and the adjacency matrix is ​​used to store the neighborhood information of each supercell: the adjacency matrix Neighbor[M][M] is a binary matrix of size M×M. If the supercell s i and super pixels j If they are adjacent, Neighbor[i][j] is set to 1; otherwise, Neighbor[i][j] is set to 0. In this way, the irregular neighborhood system of the superpixel difference image can be described by the adjacency matrix Neighbor[M][M].

[0032] In existing conditional random field change detection, the binary potential function ψ is usually constructed only by constraining the spectral characteristics of the observed field. ij (x i ,x j ,Y SD This ignores the label information in the label field, which affects the accuracy of change detection to some extent.

[0033] Furthermore, when constructing the binary potential function of the superpixel difference image, label constraints are introduced, and the observation field and label field are considered simultaneously to utilize the spatial correlation of the image, thus constructing the following superpixel conditional random field binary potential function:

[0034]

[0035] Among them, sc ij (Y SD ) represents the spectral characteristic constraint term of the observation field, indicating the neighboring superpixels s i and s j The penalty imposed on the observation field when it is assigned to a different category; lc ij (X) represents the label constraint term of the label field, indicating the neighboring superpixel s. i and s j The penalty imposed by the label field when assigned to different categories.

[0036] Furthermore, regarding sc in equation (7) ij (Y SD Define:

[0037] Generally speaking, superpixels i and s j The closer the spatial distance between features, the closer their values, the stronger their correlation, and the greater the likelihood that they belong to the same category; therefore, the penalty imposed when assigning them to different categories must be greater, hence sc ij (Y SD It can be defined by the following formula:

[0038]

[0039] In the formula, dist(s) i ,s j ) represents superpixel s i and s j The spatial distance between center pixels is preferably Euclidean distance; g i and g j They represent superpixels s and s respectively. i and s j The grayscale value is denoted by |·|, which represents the absolute value operation, and σ represents the difference between the maximum and minimum grayscale values ​​of all superpixels.

[0040] Furthermore, regarding lc in equation (8) ij (X) is defined as follows:

[0041] lc ij (X) Constraining superpixels s through label fields i and s j If superpixel s i and s j The closer the membership degree of superpixels belonging to the same category, the greater the probability that they belong to the same category; therefore, when superpixels s i and s jThe penalty imposed should be greater when the child is assigned to a different category; therefore, lc can be defined by the following formula. ij (X):

[0042]

[0043] The beneficial effects of this invention are as follows:

[0044] The technical solution proposed in this invention uses superpixels as the basic analysis unit, which can more effectively express and extract local features and structural information of images, and more effectively mine complex spatial information of images. At the same time, by optimizing the conditional random field through the label constraint strategy, the spatial correlation between superpixels can be utilized more effectively. Through the above two points, this invention can achieve better change detection accuracy. Attached Figure Description

[0045] Figure 1 This is a remote sensing image at time T1 based on experimental data used in an embodiment of the present invention;

[0046] Figure 2 This is a remote sensing image at time T2 based on experimental data used in an embodiment of the present invention;

[0047] Figure 3 This is a reference image showing the changes in two periods of experimental data used in an embodiment of the present invention, obtained through visual interpretation.

[0048] Figure 4 This is a basic flowchart of the technical solution adopted in this invention;

[0049] Figure 5 This is an example diagram of the superpixel neighborhood structure in a superpixel difference image;

[0050] Figure 6 It is a change detection map generated by a pixel-based fuzzy C-means clustering algorithm;

[0051] Figure 7 It is a change detection map generated by a superpixel fuzzy C-means clustering algorithm;

[0052] Figure 8 It is a change detection map generated based on a fully connected conditional random field of pixels;

[0053] Figure 9 It is a change detection map generated based on pixel conditional random fields;

[0054] Figure 10 It is a change detection map generated by a hybrid conditional random field;

[0055] Figure 11 This is a change detection map generated by an embodiment of the present invention. Detailed Implementation

[0056] This embodiment uses high-resolution remote sensing images from Gaofen-2 for the experiment. The spatial resolution of the images is 4 meters, and the size is 1951×1150 pixels. The corresponding area is a certain region in Yichang, China. Figures 1-3 Imagery at times T1 and T2, along with their variation reference diagrams, are provided. The variation reference diagrams are obtained through visual interpretation.

[0057] Example 1:

[0058] like Figure 4 As shown, a change detection method based on label-constrained superpixel conditional random fields includes the following steps:

[0059] S1: Preprocessing is performed on two high-resolution remote sensing images to reduce the influence of factors such as the sensor itself, atmosphere and lighting conditions. Preprocessing includes registration, radiometric correction and geometric correction. Then, a difference image between the two images is generated by using vector variation analysis.

[0060] S2: A simple linear iterative clustering algorithm is used to perform super-pixel segmentation on the difference image to obtain a super-pixel difference image;

[0061] S3: Introduce the concept of superpixel into fuzzy C-means clustering, and use the superpixel fuzzy C-means clustering algorithm to perform fuzzy clustering on the superpixel differential image to obtain the membership degree of each superpixel to the changed and unchanged classes;

[0062] S4: Introduce the concept of superpixels into the conditional random field model, and use the label constraint strategy to optimize the conditional random field to construct a label-constrained superpixel conditional random field model; and calculate the energy function of the constructed label-constrained superpixel conditional random field based on the superpixel difference images and superpixel membership degrees obtained in S2 and S3.

[0063] S5: Optimize the energy function calculated in step 4 using the graph cutting inference algorithm to generate a change detection map.

[0064] Example 2:

[0065] like Figures 4 to 11 As shown, a high-resolution remote sensing image change detection method based on label-constrained superpixel conditional random fields includes the following steps:

[0066] S1. Preprocessing operations such as registration, radiometric correction, and geometric correction are performed on the two high-resolution remote sensing images to reduce the influence of factors such as the sensor itself, atmosphere, and lighting conditions. Then, a difference image between the two preprocessed images is generated using vector transformation analysis technology. The specific methods are as follows:

[0067] In this embodiment, the high-resolution remote sensing images acquired at times T1 and T2 are denoted as Y1 and Y2, respectively. First, preprocessing operations such as registration, radiometric correction, and geometric correction are performed on Y1 and Y2. Then, change vector analysis is applied to them to generate a differential image Y. D The differential image Y D The calculation formula is as follows:

[0068]

[0069] Where Y t,b (t=1,2) represents image Y t The pixel value of the b-th band (1≤b≤B). In this embodiment, B=3.

[0070] S2. A simple linear iterative clustering algorithm is used to process the differential image Y. D Superpixel segmentation is performed to obtain the superpixel difference image Y. SD The specific method is as follows:

[0071] In the simple linear iterative clustering algorithm, the superpixel difference image is processed by the adaptive K-means algorithm to analyze the difference image Y. D Clustering results in superpixels obtained by iteratively searching for neighboring pixels around the center pixel based on the distance between pixels in the feature space and the location space, thereby generating a superpixel differential image. In this process, parameter K needs to be set, which determines the number of superpixels in the superpixel differential image.

[0072] In this embodiment, the value of K is set to 9000. Using Y... SD ={s i |i=1,2,3,...,M} represents the obtained superpixel difference image, s i Let M represent the i-th superpixel, and M represent the total number of superpixels. In this embodiment, M = 9000.

[0073] S3. Introduce the super-pixel concept into fuzzy C-means clustering and use the super-pixel fuzzy C-means clustering algorithm to analyze the super-pixel difference image Y. SD ={s i Fuzzy clustering is performed on |i=1,2,3,...,M} to obtain the membership degree of each superpixel to the changed and unchanged classes. The specific method is as follows:

[0074] In this embodiment, G = {g} i |i=1,2,3…,M} represents the set of gray values ​​of superpixels, g i Represents superpixel s i The gray value is the average gray value of all pixels in the superpixel, and M represents the number of superpixels. Fuzzy C-means clustering iteratively optimizes the objective function J. CTo calculate superpixel membership:

[0075]

[0076] Among them, u ij Represents superpixel s i The degree of membership to category j, c j Let q represent the cluster center of category j, q be the weighting index, C represent the number of categories, and u represent the cluster center of category j. ij and c j Do not calculate using the following formula:

[0077]

[0078]

[0079] In formulas (3) and (4), k represents a subscript, and the other symbols have the same meaning as in formula (2). This invention includes two categories: modified and unchanged, i.e., C = 2.

[0080] S4. Introduce the concept of superpixels into the conditional random field model, and optimize the conditional random field using a label constraint strategy to construct a label-constrained superpixel conditional random field model; and calculate the energy function of the constructed label-constrained superpixel conditional random field based on the superpixel difference images and superpixel membership degrees obtained in steps 2 and 3, as follows:

[0081] In this embodiment, superpixels are used as basic analysis units, and a conditional random field model is used to analyze the superpixel difference image Y. SD ={s i |i=1,2,3,...,M} (i.e., the observation field) and its corresponding label set X={x i We perform posterior probability modeling on |i=1,2,...,M} (i.e., the label field), x i Represents superpixel s i Category labels; generally, conditional random fields solve problems by transforming posterior probabilities into energy functions; the energy function E(X|Y) of a label-constrained superpixel conditional random field. SD It can be described as the sum of unary and dual potentials:

[0082]

[0083] Where ψ i (x i ,Y SD Let ψ be a univariate potential function. ij (x i ,x j ,Y SDψ is a binary potential function. λ is a non-negative constant used to balance the univariate and binary potentials; in this embodiment, λ = 1; the univariate potential function ψ i (x i ,Y SD ) indicates the superpixel difference image Y in the observation field. SD Lower superpixel s i Assigned to category x i The cost is usually defined by the following formula:

[0084] ψ i (x i ,Y SD )=-ln(P(x i |s i (6)

[0085] Where x i ∈{0,1}, 0 represents the unchanged class label, 1 represents the changed class label, ln represents the natural logarithm operator, P(x i |s i ) indicates the superpixel difference image Y in the observation field. SD Lower superpixel s i Assigned to category x i The probability, P(x) i |s i The result is obtained through the superpixel blur C-means algorithm, i.e.:

[0086]

[0087] Among them, u ixi This represents the superpixel s obtained in step 3 through superpixel fuzzy C-means clustering. i Belongs to category x i The degree of membership.

[0088] Binary potential function ψ ij (x i ,x j ,Y SD Utilizing spatial information to optimize change detection results involves using the spatial correlation of images to classify adjacent superpixels into the same category. The construction of the binary potential function requires the use of superpixels s... i The local neighborhood N i .

[0089] The neighborhood system of a superpixel image differs from that of a pixel image: the neighborhood system of a pixel image is regular, with each pixel typically having the same neighborhood; the neighborhood system of a superpixel image is irregular, with each superpixel typically having a different neighborhood system. Therefore, describing the irregular neighborhood system of a superpixel difference image is more challenging.

[0090] Preferably, this embodiment uses the adjacency matrix in graph theory to describe the irregular neighborhood relationships between superpixels:

[0091] For any superpixel s i This invention defines its neighborhood as all its neighboring supercells and uses an adjacency matrix to store the neighborhood information of each supercell: the adjacency matrix Neighbor[M][M] is a binary matrix of size M×M. If the supercell s i and super pixels j If the pixels are adjacent, Neighbor[i][j] is set to 1; otherwise, Neighbor[i][j] is set to 0. This allows the adjacency matrix Neighbor[M][M] to describe the irregular neighborhood system of the superpixel difference image.

[0092] In existing conditional random field change detection, the binary potential function ψ is usually constructed only by constraining the spectral characteristics of the observed field. ij (x i ,x j ,Y SD This ignores the label information in the label field, which affects the change detection accuracy to some extent. This invention introduces label constraints when constructing the binary potential function of the superpixel difference image, and simultaneously considers both the observation field and the label field to utilize image spatial correlation, constructing the following superpixel conditional random field binary potential function:

[0093]

[0094] Among them, sc ij (Y SD ) represents the spectral characteristic constraint term of the observation field, indicating the neighboring superpixels s i and s j The penalty imposed on the observation field when it is assigned to a different category. ij (X) represents the label constraint term of the label field, indicating the neighboring superpixel s. i and s j The penalty imposed by the label field when assigned to different categories.

[0095] Generally speaking, superpixels i and s j The closer the spatial distance between features, the closer their values, the stronger their correlation, and the greater the likelihood that they belong to the same category; therefore, the penalty imposed when assigning them to different categories must be greater, hence sc ij (Y SD It can be defined by the following formula:

[0096]

[0097] dist(si ,s j ) represents superpixel s i and s j The spatial distance of the center pixels is calculated using Euclidean distance in this invention. i and g j They represent superpixels s and s respectively. i and s j The grayscale value is denoted by |·|, which represents the absolute value operation, and σ represents the difference between the maximum and minimum grayscale values ​​of all superpixels.

[0098] lc ij (X) Constraining superpixels s through label fields i and s j If superpixel s i and s j The closer the membership degree of superpixels belonging to the same category, the greater the probability that they belong to the same category; therefore, when superpixels s i and s j The penalty imposed should be greater when a class is assigned to a different category. Therefore, lc can be defined by the following formula. ij (X):

[0099]

[0100] S5. Optimize the energy function calculated in step 4 using the graph cutting inference algorithm to generate a change detection map; the specific method is as follows:

[0101] In this embodiment, in order to generate the final change detection map, the model inference algorithm graph cutting algorithm is used to solve the super-pixel conditional random field, and the change detection map is generated by minimizing the energy function. The energy function is calculated by the above equation (5).

[0102] Example 3:

[0103] To verify the effectiveness of the technical solution of the present invention, it is compared with existing related change detection algorithms.

[0104] like Figures 6 to 11 As shown, pixel fuzzy C-means clustering (A), super-pixel fuzzy C-means clustering (B), pixel fully connected conditional random field (C), pixel conditional random field (D), mixed conditional random field (E), and the change detection map of the present invention are given respectively; Table 1 below gives the quantitative evaluation index of different change detection maps.

[0105]

[0106] Table 1 Statistical comparison of results from different change detection methods

[0107] from Figures 6 to 11As shown in Table 1, the change detection results of the present invention are significantly better than the other five comparative change detection methods: the change detection map of the present invention contains fewer false positives, has the smallest overall error and the highest Kappa coefficient.

[0108] The overall error of the detection results of this invention is 36380, which is 64929, 12799, 19840, 29742 and 25793 pixels less than that of pixel fuzzy C-means clustering, super-pixel fuzzy C-means clustering, pixel fully connected conditional random field, pixel conditional random field and mixed conditional random field, respectively.

[0109] The Kappa coefficient of the detection results of this invention is 0.8258, which is 19.01%, 4.69%, 7.37%, 9.44%, and 10.87% higher than that of pixel fuzzy C-means clustering, super-pixel fuzzy C-means clustering, pixel fully connected conditional random field, pixel conditional random field, and mixed conditional random field, respectively.

[0110] The above embodiments are merely preferred technical solutions of the present invention and should not be considered as limitations on the present invention. The embodiments and features described in this application can be arbitrarily combined with each other without conflict. The scope of protection of the present invention should be limited to the technical solutions described in the claims, including equivalent substitutions of the technical features described in the claims. That is, equivalent substitutions and improvements within this scope are also within the scope of protection of the present invention.

Claims

1. A method for detecting changes in label-constrained superpixel conditional random fields, characterized in that: Includes the following steps: S1: Preprocessing is performed on two high-resolution remote sensing images to reduce the influence of the sensor itself, atmosphere and lighting conditions. Preprocessing includes registration, radiometric correction and geometric correction. Then, a difference image between the two images is generated by using vector variation analysis. S2: A simple linear iterative clustering algorithm is used to perform super-pixel segmentation on the difference image to obtain a super-pixel difference image; S3: Introducing the concept of superpixels into fuzziness C Mean clustering, utilizing superpixel blur C The mean clustering algorithm performs fuzzy clustering on superpixel difference images to obtain the membership degree of each superpixel to the changed and unchanged classes. The specific method is as follows: use This represents the set of superpixels in a superpixel difference image. The set of grayscale values ​​representing superpixels; Superpixel The gray value is the average gray value of all pixels in the superpixel; Indicates the number of superpixels; fuzzy C Mean clustering optimizes the objective function through iteration. To calculate superpixel membership: （1）； in, Superpixel Category membership degree Indicates category Cluster centers As a weighted index, C Indicates the number of categories; and Calculate using the following formulas (2) and (3): （2）； （3）； S4: The concept of superpixels is introduced into the conditional random field model, and the conditional random field is optimized using a label constraint strategy to construct a label-constrained superpixel conditional random field model. Based on the superpixel difference images and superpixel membership degrees obtained in S2 and S3, the energy function of the constructed label-constrained superpixel conditional random field is calculated. The specific method is as follows: Using superpixels as the basic unit of analysis, the conditional random field model is used to analyze superpixel difference images. and its corresponding tag set Perform posterior probability modeling. Superpixel Category labels; Conditional Random Fields (CRFs) solve problems by transforming posterior probabilities into energy functions; Energy functions of label-constrained superpixel CRFs. Described in the form of a sum of unary and dual potentials: （4）； in It is a univariate potential function. It is a binary potential function; It is a non-negative constant used to balance the univariate and bivariate potentials; S5: Optimize the energy function calculated in step 4 using the graph cutting inference algorithm to generate a change detection map.

Images