Different-source image self-supervision change detection method based on difference algorithm
Through phase consistency and pulse-coupled neural networks, the problem of low change detection accuracy caused by differences between heterologous images is solved, and the problem of low change detection accuracy caused by differences between heterologous images is achieved. More efficient change detection and sample extraction enhance the reliability and sensitivity of detection results.
Patent Information
- Application Number
- CN202410930682.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-11
- Publication Date
- 2024-11-08
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The differences in spectral and texture between heterologous remote sensing images lead to poor quality of differential images in change detection and low accuracy of sample extraction.
The fusion heterologous image is constructed using a fusion method combined with phase consistency and simplified pulse-coupled neural network. The standard deviation image and exponential difference image are extracted based on the improved neighborhood ratio difference algorithm, and the unchanged image is obtained through the dual-layer FCM clustering algorithm. Samples and change samples, build a change detection classifier.
Improves the accuracy and robustness of change detection, reduces the impact of noise, enhances the extraction ability of changing and invariant pixels, and provides complementary information to enhance the comparability and sensitivity of detection results.
Smart Images

Figure CN118918490A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of remote sensing imaging, and in particular relates to a heterogeneous image self-supervised change detection method based on a difference algorithm. Background Art
[0002] With the rapid development of remote sensing platforms and sensor technology, heterogeneous remote sensing image change detection has become a research hotspot that has attracted much attention in recent years. Under disaster emergency conditions, it is often difficult to obtain homogeneous images in a timely manner, and homogeneous image change detection is also difficult to cope with complex actual scenes. However, heterogeneous remote sensing image change detection can break through the limitations of traditional change detection factors such as lighting conditions, imaging cycles and modalities, and can meet the technical requirements of emergency response and other ground applications to a certain extent. This technology has been used in land cover changes, target information extraction and anomaly analysis, and plays an important role in landslide and earthquake inventory mapping, environmental quality evaluation, natural resource monitoring, and urban development decision-making. Therefore, researching and developing efficient heterogeneous image change detection methods has potential positive effects in practical applications.
[0003] Different imaging mechanisms, imaging modes, imaging positions and imaging angles of heterogeneous remote sensing images result in differences in resolution, spectrum, texture, etc., as well as more noise. Producing sample data is a time-consuming manual operation and requires professional background knowledge. In order to obtain robust performance, many deep learning networks require a large number of training and testing samples for module training. However, the change detection method based on the differential algorithm can automatically construct a sample set, while the quality of the differential image is easily affected by the differences in heterogeneous images, resulting in a decrease in the accuracy of sample extraction. Summary of the invention
[0004] Aiming at the problem that the spectral and texture differences between heterogeneous remote sensing images cause poor change detection differential image quality and low sample extraction accuracy, the present invention discloses a heterogeneous image self-supervised change detection method based on a differential algorithm.
[0005] The present invention is implemented as follows: a heterogeneous image self-supervised change detection method based on a difference algorithm, the method comprising the following steps:
[0006] S1. Construct fused heterogeneous images using a fusion method combining phase congruence and a simplified pulse coupled neural network;
[0007] S2, based on heterogeneous images and fused heterogeneous images, the neighborhood ratio difference algorithm is improved to extract standard difference images and exponential difference images;
[0008] S3, executing a double-layer FCM clustering algorithm based on the standard difference image and the exponential difference image, so as to obtain unchanged samples of unchanged pixels and changed samples of changed pixels;
[0009] S4. Construct a change detection classifier using the unchanged samples and the changed samples as training samples, and obtain a detection result.
[0010] Preferably, in step S2, the improved neighborhood ratio difference algorithm includes an improved ratio method, an improved neighborhood intensity mean method and an improved neighborhood ratio difference image method; wherein the improved ratio method is a ratio model D constructed based on heterogeneous images and fused heterogeneous images. IR-CFI The improved neighborhood intensity mean method is a neighborhood intensity mean model D constructed based on heterogeneous images and fused heterogeneous images. MR-CFI The improved neighborhood ratio difference image method is a neighborhood ratio model D constructed based on heterogeneous images and fused heterogeneous images. NR-CFI Among them, the difference image is calculated by the improved ratio method and the improved neighborhood intensity mean method, and on this basis, the standard difference image and the exponential difference image are calculated using the improved neighborhood ratio difference algorithm.
[0011] Preferably, the ratio model D IR-CFI for:
[0012]
[0013] In formula (1), I i (m,n) represents the pixel intensity value at position (m,n) in the image, m∈[1,M], n∈[1,N], M, N correspond to image I respectively. i Dimensions;
[0014] The neighborhood intensity mean model D MR-CFI for:
[0015]
[0016] In formula (2), I1(x) and I2(x) represent the images I1 and I2 centered at position (m, n), x is the intensity mean of the neighborhood space, and F3 is the fused image;
[0017] The neighborhood ratio model D NR-CFI for:
[0018]
[0019]
[0020] In formulas (3) and (4), Ω represents the neighborhood space set centered at position (m, n), I i (x) is defined as the pixel intensity value of the x block matrix in the neighborhood space centered at position (m,n), σ(x) represents the grayscale variance of the neighborhood Ω, and μ(x) represents the grayscale mean of the neighborhood Ω.
[0021] Preferably, step S3 includes the following specific steps:
[0022] S3-1. Perform a hierarchical FCM clustering algorithm on the standard difference image and the exponential difference image, extract the change samples and the unchanged samples independently, form a bi-hierarchical clustering algorithm, and divide the pixels into three clusters: and represents the changed pixels of the corresponding standard difference image or the unchanged pixels of the corresponding exponential difference image, represents the middle pixel, represents the unchanged pixels of the corresponding standard difference image or the changed pixels of the corresponding exponential difference image, The number of pixels in T 1 express;
[0023] S3-2, perform the hierarchical FCM clustering algorithm on the standard difference image and the exponential difference image, independently extract the change samples and the unchanged samples, form a double hierarchical FCM clustering algorithm, and divide the pixels into five clusters: The clusters are sorted in descending order of mean value;
[0024] Among them, for the standard difference image, the cluster with a larger mean has a greater probability of change, and for the exponential difference image, the cluster with a larger mean has a greater probability of not changing; the number of pixels in the 5 clusters is defined as Set the parameter t = 1, Will The pixels in are assigned to the class Ω of the corresponding standard difference image c and the class Ω corresponding to the exponential difference image u ;
[0025] S3-3, set t=t+1,
[0026] S3-4, if c <TT, Pixel allocation in Ω i Otherwise, The pixels in Ω are assigned to u (Ω c ), execute step S3-3 until t=5;
[0027] S3-5, output pre-classification results, represented as labeled images {Ω c ,Ω i ,Ω u};
[0028] S3-6, setting the non-zero elements of the pre-classification result to 1 to obtain a binary matrix, where 1 represents the clustered area and 0 represents other areas;
[0029] S3-7, label the binary matrix to obtain a label matrix and the number of labels;
[0030] S3-8, by traversing each label, remove the clustering results with small areas;
[0031] S3-9, by marking the area where the clustering result value is 1 as 1, obtaining an image corresponding to the standard difference image showing changed pixels or a corresponding image showing unchanged pixels;
[0032] S3-10. Merge the changed pixel area with the unchanged pixel area, wherein the unchanged pixel is marked as 0, the changed pixel is marked as 1, and the intermediate pixel is marked as 0.5.
[0033] Preferably, in step S4, the construction of the change detection classifier is specifically to train the SVM using training samples, extract relevant feature information, and increase the feature information by fusing the images.
[0034] Compared with the shortcomings and deficiencies of the prior art, the present invention has the following beneficial effects:
[0035] (1) When extracting the differential image, the present invention uses the method of fusing the heterogeneous images of the early and late stages to form a transition image, making it comparable with the original image and providing complementary information. This method is not only comparable with the original image in terms of radiation spectrum, number of channels, etc., but also has a higher sensitivity to changed pixels.
[0036] (2) Based on the improved neighborhood ratio difference image algorithm of heterogeneous images and fused images, the present invention can not only balance the weights of suppressing noise and retaining detail information through heterogeneity measurement, but also provide complementary information for fused images, which helps to reduce the impact of heterogeneous image differences and noise on differential image extraction.
[0037] (3) The present invention obtains the change sample and the unchanged sample respectively through the standard difference image and the exponential difference image. Among them, the standard difference image helps to reduce the noise caused by the difference between heterogeneous images and improve the accuracy of the change sample extraction; while the exponential difference image exponentially amplifies the feature difference between heterogeneous images, reduces the noise, and ensures that the unchanged sample obtained is accurate.
[0038] (4) The present invention avoids the problem of small number of changing samples and poor accuracy of unchanged samples by improving the hierarchical FCM clustering algorithm, and has no requirements on the intensity of changing and unchanged pixels, so that the extraction of changing samples and unchanged samples is independent of each other, and does not involve the related problems of double threshold determination.
[0039] (5) When constructing a change detection classifier, the present invention adds feature information by fusing images and makes full use of remote sensing technology and images to extract change information. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] Figure 1 It is a flow chart of the heterogeneous image self-supervised change detection technology based on the difference algorithm of the present invention;
[0041] Figure 2 The improved neighborhood ratio difference algorithm based on heterogeneous images and fused images of the present invention;
[0042] Figure 3 Heterogeneous remote sensing image data used in the experiments provided by the present invention;
[0043] Figure 4 This is a graph of change detection results provided by the present invention. DETAILED DESCRIPTION
[0044] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.
[0045] The embodiment of the present invention discloses a heterogeneous image self-supervised change detection method based on a difference algorithm, characterized in that the method comprises the following steps:
[0046] S1. Construct fused heterogeneous images using a fusion method combining phase congruence and a simplified pulse coupled neural network;
[0047] In step S1, a fusion method of coupled phase congruency information and feature detail injection is used in the low-frequency coefficient fusion process (reference: FU Y, YANG S, LI Y, et al. A novel SAR and optical image fusion algorithm based on an improved SPCNN and phase congruency information [J]. International Journal of Remote Sensing, 2023, 44 (4): 1328-47.), and in the high-frequency sub-band fusion process, a simplified pulse coupled neural network and guided filter improved based on phase congruency information are used to evaluate the pixel activity level. The fusion method has high robustness to nonlinear radiation differences and noise, and can inject clear and complete spatial features into the fused image.
[0048] S2, based on heterogeneous images and fused heterogeneous images, the neighborhood ratio difference algorithm is improved to extract standard difference images and exponential difference images;
[0049] In step S2, the improved neighborhood ratio difference algorithm includes an improved ratio method, an improved neighborhood intensity mean method, and an improved neighborhood ratio difference image method; wherein the improved ratio method is a ratio model D constructed based on heterogeneous images and fused heterogeneous images. IR-CFI The improved neighborhood intensity mean method is a neighborhood intensity mean model D constructed based on heterogeneous images and fused heterogeneous images. MR-CFI The improved neighborhood ratio difference image method is a neighborhood ratio model D constructed based on heterogeneous images and fused heterogeneous images. NR-CFI Among them, the difference image is calculated by the improved ratio method and the improved neighborhood intensity mean method, and on this basis, the standard difference image and the exponential difference image are calculated using the improved neighborhood ratio difference algorithm.
[0050] Specifically, the ratio model D IR-CFI for:
[0051]
[0052] Ratio Model D IR-CFI The construction process is as follows: Assume I i (m,n) represents the pixel intensity value at position (m,n) in the image, m∈[1,M], n∈[1,N], where M and N correspond to image I, respectively. i The fusion image F3 is added on the basis of the ratio method to weaken the influence of heterogeneous image differences and noise on change detection.
[0053] The neighborhood intensity mean model D MR-CFI for:
[0054]
[0055] In formula (2), I1(x) and I2(x) represent the intensity mean of the neighborhood space x (size is R×R) centered at position (m,n) of images I1 and I2, respectively. Based on the neighborhood intensity mean method, the fused image F3 is added to weaken the influence of heterogeneous image differences and noise on change detection.
[0056] The neighborhood ratio model D NR-CFI for:
[0057]
[0058]
[0059] In formulas (3) and (4), Ω represents the neighborhood space set centered at position (m, n), I i(x) is defined as the pixel intensity value of the x-block matrix in the neighborhood space centered at position (m,n), σ(x) represents the grayscale variance of the neighborhood Ω, and μ(x) represents the grayscale mean of the neighborhood Ω (reference: PAN J, LI X, CAI Z, et al. A Self-Attentive Hybrid Coding Network for 3D Change Detection in High-Resolution Optical Stereo Images [J]. Remote Sensing, 2022, 14(9): 2046.). Based on the neighborhood ratio difference image method, the addition of the fused image F3 can not only achieve a balanced weight in suppressing noise and retaining detail information through the heterogeneity measure θ, but also the fused image as a transition image retains the spectral, texture, and spatial detail information, which can reduce the impact of heterogeneous image differences and noise on the differential image and improve the accuracy of changing and invariant sample extraction.
[0060] S3, executing a double-layer FCM clustering algorithm based on the standard difference image and the exponential difference image, so as to obtain unchanged samples of unchanged pixels and changed samples of changed pixels;
[0061] Specifically, step S3 includes the following steps:
[0062] S3-1. Perform a hierarchical FCM clustering algorithm on the standard difference image and the exponential difference image (see reference: Gao F, Dong J, Li B, et al. Change detection from synthetic aperture radar images based on neighborhood-based ratio and extreme learning machine [J]. Journal of Applied Remote Sensing, 2016, 10 (4): 046019-.), extract the change samples and the unchanged samples independently, form a double hierarchical clustering algorithm, and divide the pixels into three clusters: and represents the changed pixels of the corresponding standard difference image or the unchanged pixels of the corresponding exponential difference image, represents the middle pixel, represents the unchanged pixels of the corresponding standard difference image or the changed pixels of the corresponding exponential difference image, The number of pixels in T 1 express;
[0063] S3-2, perform the hierarchical FCM clustering algorithm on the standard difference image and the exponential difference image, independently extract the change samples and the unchanged samples, form a double hierarchical FCM clustering algorithm, and divide the pixels into five clusters: The clusters are sorted in descending order of mean value;
[0064] Among them, for the standard difference image, the cluster with a larger mean has a greater probability of change, and for the exponential difference image, the cluster with a larger mean has a greater probability of not changing; the number of pixels in the 5 clusters is defined as Set the parameter t = 1, Will The pixels in are assigned to the class Ω of the corresponding standard difference image c and the class Ω corresponding to the exponential difference image u ;
[0065] S3-3, set t=t+1,
[0066] S3-4, if c <TT, Pixel allocation in Ω i Otherwise, The pixels in Ω are assigned to u (Ω c ), execute step S3-3 until t=5;
[0067] S3-5, output pre-classification results, represented as labeled images {Ω c ,Ω i ,Ω u};
[0068] S3-6, setting the non-zero elements of the pre-classification result to 1 to obtain a binary matrix, where 1 represents the clustered area and 0 represents other areas;
[0069] S3-7, label the binary matrix to obtain a label matrix and the number of labels;
[0070] S3-8, by traversing each label, remove the clustering results with small areas;
[0071] S3-9, by marking the area where the clustering result value is 1 as 1, obtaining an image corresponding to the standard difference image showing changed pixels or a corresponding image showing unchanged pixels;
[0072] S3-10. Merge the changed pixel area with the unchanged pixel area, wherein the unchanged pixel is marked as 0, the changed pixel is marked as 1, and the intermediate pixel is marked as 0.5.
[0073] S4. Construct a change detection classifier using the unchanged samples and the changed samples as training samples, and obtain a detection result.
[0074] In step S4, the construction of the change detection classifier is specifically to train the SVM using the training samples, extract relevant feature information, and increase the feature information by fusing the image. Therefore, the increased feature information can better detect changes, specifically, it can detect correct change information that cannot be detected from heterogeneous images alone, and obtain change detection results by predicting intermediate pixels.
[0075] The change detection process is as follows Figure 1 As shown in Figure 1, it consists of three parts: improved neighborhood ratio difference image extraction, improved hierarchical FCM clustering algorithm sample set automatic extraction, and change detection classifier construction. The process of the improved neighborhood ratio difference algorithm based on heterogeneous images and fused images is shown in Figure 1. Figure 2 As shown, the difference image is calculated by the improved ratio method and the improved neighborhood intensity mean method, and on this basis, the standard difference image and the exponential difference image are calculated using the improved neighborhood ratio difference algorithm.
[0076] The present invention is verified by implementing cases in lake expansion, building expansion, flooding and snow accumulation using multi-scale and multi-scene remote sensing images. All data sets are resampled and registered. The first and third groups are public data sets that provide ground-truth changes. The second and fourth groups are located in China and have been field-verified to confirm the accuracy of the ground truth. Figure 3 As shown in the figure, data 1 comes from Google Earth and Landsat-5 sensors, data 2 comes from GF1 and GF2 sensors, data 3 comes from TerraSAR-X and QuickBird2 sensors, and data 4 comes from GF-2 and GF-7. Table 1 details the time, size, resolution, and source of the images used in the case.
[0077] Table 1
[0078] Group time event Image size / pixel Resolution / m Data Source Group 1 1995.9 / 1996.7 Lake expansion 412×300 ≈30 / 30 Landsat-5 / Google Earth Group 2 2016.7 / 2018.4 Building extension 412×300 1.0 / 1.2 GF-1 / GF-2 Group 3 2006.7 / 2007.7 flood 802×500 0.61 / 1.0 QuickBird2 / TerraSAR-X Group 4 2021.12 / 2022.12 snow 101×101 1.0 / 0.8 GF-2 / GF-7
[0079] The results of lake expansion, building expansion, flooding and snow cover change information extraction are as follows: Figure 4As shown in the figure, red represents missed pixels, orange represents wrongly detected pixels, and blue represents correct change detection pixels. Dataset 1 has the smallest overestimated change area and clear boundaries; Dataset 2 obtains more accurate change detection results and the smallest underestimated change area; Dataset 3 has fewer errors in detecting changed and unchanged pixels; Dataset 4 has the smallest overestimated change area. The error detection rate, missed detection rate, PCC, Kappa and F1 index accuracy results of change detection are shown in Table 2. Dataset 1 has a low false detection rate and a missed detection rate of 0.0135, and obtains good Kappa values, F1 values and PCC values; Dataset 2 has good final Kappa values, F1 values and PCC values; Dataset 3 has a high Kappa coefficient of 0.8063; Dataset 4 has a low missed detection rate, an F1 value of 0.8605, and a Kappa coefficient of 0.7556.
[0080] Table 2
[0081]
[0082]
[0083] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. A self-supervised change detection method for heterogeneous images based on a difference algorithm, characterized in that: The method comprises the following steps: S1. Construct fused heterogeneous images using a fusion method combining phase congruence and a simplified pulse coupled neural network; S2, based on heterogeneous images and fused heterogeneous images, the neighborhood ratio difference algorithm is improved to extract standard difference images and exponential difference images; S3, executing a double-layer FCM clustering algorithm based on the standard difference image and the exponential difference image, so as to obtain unchanged samples of unchanged pixels and changed samples of changed pixels; S4. Construct a change detection classifier using the unchanged samples and the changed samples as training samples, and obtain a detection result.
2. The method according to claim 1, characterized in that In step S2, the improved neighborhood ratio difference algorithm includes an improved ratio method, an improved neighborhood intensity mean method, and an improved neighborhood ratio difference image method; wherein the improved ratio method is a ratio model D constructed based on heterogeneous images and fused heterogeneous images. IR-CFI The improved neighborhood intensity mean method is a neighborhood intensity mean model D constructed based on heterogeneous images and fused heterogeneous images. MR-CFI The improved neighborhood ratio difference image method is a neighborhood ratio model D constructed based on heterogeneous images and fused heterogeneous images. NR-CFI Among them, the difference image is calculated by the improved ratio method and the improved neighborhood intensity mean method, and on this basis, the standard difference image and the exponential difference image are calculated using the improved neighborhood ratio difference algorithm.
3. The method according to claim 2, characterized in that The ratio model D IR-CFI for: In formula (1), I i (m,n) represents the pixel intensity value at position (m,n) in the image, m∈[1,M], n∈[1,N], M, N correspond to image I respectively. i Dimensions; The neighborhood intensity mean model D MR-CFI for: In formula (2), I1(x) and I2(x) represent the images I1 and I2 centered at position (m, n), x is the intensity mean of the neighborhood space, and F3 is the fused image; The neighborhood ratio model D NR-CFI for: In formulas (3) and (4), Ω represents the neighborhood space set centered at position (m, n), I i (x) is defined as the pixel intensity value of the x block matrix in the neighborhood space centered at position (m,n), σ(x) represents the grayscale variance of the neighborhood Ω, and μ(x) represents the grayscale mean of the neighborhood Ω.
4. The method according to claim 1, characterized in that Step S3 includes the following specific steps: S3-1. Perform a hierarchical FCM clustering algorithm on the standard difference image and the exponential difference image, extract the change samples and the unchanged samples independently, form a bi-hierarchical clustering algorithm, and divide the pixels into three clusters: and represents the changed pixels of the corresponding standard difference image or the unchanged pixels of the corresponding exponential difference image, represents the middle pixel, represents the unchanged pixels of the corresponding standard difference image or the changed pixels of the corresponding exponential difference image, The number of pixels in T 1 express; S3-2, perform the hierarchical FCM clustering algorithm on the standard difference image and the exponential difference image, independently extract the change samples and the unchanged samples, form a double hierarchical FCM clustering algorithm, and divide the pixels into five clusters: The clusters are sorted in descending order of mean value; Among them, for the standard difference image, the cluster with a larger mean has a greater probability of change, and for the exponential difference image, the cluster with a larger mean has a greater probability of not changing; the number of pixels in the 5 clusters is defined as T1 2 , Set parameters t=1, c=T1 2 ,Will The pixels in are assigned to the class Ω of the corresponding standard difference image c and the class Ω corresponding to the exponential difference image u ; S3-3. Set t=t+1, c=c+T t 2 ; S3-4, if c <TT, Pixel allocation in Ω i Otherwise, The pixels in Ω are assigned to u (Ω c ), execute step S3-3 until t=5; S3-5, output pre-classification results, represented as labeled images {Ω c ,Ω i ,Ω u }; S3-6, setting the non-zero elements of the pre-classification result to 1 to obtain a binary matrix, where 1 represents the clustered area and 0 represents other areas; S3-7, label the binary matrix to obtain a label matrix and the number of labels; S3-8, by traversing each label, remove the clustering results with small areas; S3-9, by marking the area where the clustering result value is 1 as 1, obtaining an image corresponding to the standard difference image showing changed pixels or a corresponding image showing unchanged pixels; S3-10. Merge the changed pixel area with the unchanged pixel area, wherein the unchanged pixel is marked as 0, the changed pixel is marked as 1, and the intermediate pixel is marked as 0.
5.
5. The method according to claim 1, characterized in that In step S4, the construction of the change detection classifier specifically includes training the SVM using training samples, extracting relevant feature information, and increasing feature information by fusing images.
Citation Information
Patent Citations
SAR image change detection method based on self-attention image fusion and DEC
CN110555841A
Elevation information-fused processing method for remote sensing images
WO2024109419A1
