A method for pseudo-labeling and change detection of soil changes in large-scale remote sensing images

By using the soil index and spatial moment equivalent peak-to-valley method of multispectral remote sensing images, combined with the clustering method based on point density, the soil change pseudo-label of remote sensing images is automatically generated, and the problem of time-consuming and labor-consuming for manual labeling of samples is solved, and the problem of artificial labeling of samples is realized. Efficient soil change detection.

CN114511724BActive Publication Date: 2024-12-03WUHAN UNIV
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202210061539.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-01-19
Publication Date
2024-12-03
Estimated Expiration
2042-01-19

AI Technical Summary

Technical Problem

Existing machine learning and deep learning methods require a large number of manual labeling of samples in remote sensing image change detection, resulting in high cost and inefficiency.

Method used

A method is adopted to generate pseudo-labels by using multispectral remote sensing images acquired at similar imaging times, including image data in visible light, near-infrared and short-wave infrared bands, through soil index calculation and spatial moment equivalent peak-to-valley method, combined with point-based Clustering method of density, automatic generation of large-scale soil change pseudo-labels.

Benefits of technology

It realizes automatic generation of large-scale soil change labels without manual annotation, dynamically determines segmentation thresholds, removes noise patterns, saves manpower, reduces costs, and improves the effectiveness of remote sensing soil change detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114511724B_ABST
    Figure CN114511724B_ABST
Patent Text Reader

Abstract

The present invention provides a large-scale remote sensing image soil change pseudo label production and change detection method, the pseudo label production includes obtaining a first image and a second image of two phases within the target area and performing image preprocessing; constructing a soil index according to spectral characteristics to calculate the soil index of each pixel in the two phases in the first image; using the spatial moment equivalent peak-valley method to segment the soil index change and generate a soil index change map; using a clustering method based on point density to screen the change spots, filter out the smaller part of the change area, and obtain a soil change label map; projecting the soil change label map onto the second image, forming an image before the change, an image after the change, and a label image sample group on the second image. The pseudo label production generates an image set for training, and the trained model can support machine learning, deep learning, etc., to achieve soil change detection, further reduce the cost of change detection, and improve the effectiveness of remote sensing soil change detection.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of remote sensing detection and sample generation, and in particular to a method for making pseudo labels for soil changes in large-scale remote sensing images and detecting changes. Background Art

[0002] Multi-temporal remote sensing images can be used to detect changes in ground objects. Among them, the more effective methods include artificial intelligence methods such as machine learning and deep learning. In order to obtain high-precision detection results, these methods require training on image pairs with real changes labeled.

[0003] Due to the diversity of remote sensing image data and the complexity of the change detection process, the production of training samples of real changes needs to cover different seasons, landform types, time spans, etc. The number of samples is huge, and it requires a lot of manpower and material resources, which is costly. Summary of the invention

[0004] The present invention mainly solves the problem that existing artificial intelligence methods such as machine learning and deep learning require a large number of manually labeled samples when performing remote sensing image change detection. According to the characteristics that different remote sensing images have different spectral information, a method is proposed that uses auxiliary remote sensing images collected at similar imaging times, including near-infrared and short-wave infrared bands, to automatically generate large-scale pseudo-labels of soil changes in ordinary remote sensing images.

[0005] The above technical problems of the present invention are mainly solved by the following technical solutions:

[0006] A method for producing pseudo labels of soil changes in large-scale remote sensing images comprises the following steps:

[0007] Step 1, obtaining a first image and a second image of two phases within the target area and performing image preprocessing;

[0008] Step 2, constructing a soil index based on spectral characteristics and calculating the soil index of each pixel in the two time phases in the first image;

[0009] Step 3, using the spatial moment equivalent peak-valley method to segment the soil index change and generate a soil index change map;

[0010] Step 4: Use a point density-based clustering method to screen the change patches, filter out the smaller change areas, and obtain a soil change label map;

[0011] Step 5, projecting the soil change label map generated in step 4 onto the second image, forming an image before the change, an image after the change, and a label image sample group on the second image.

[0012] Moreover, the first image data refers to remote sensing image data with visible light, near-infrared band and short-wave infrared band, including images at two time points before and after the change; the second image data refers to remote sensing image data with visible light band but without or incompletely with near-infrared band and short-wave infrared band, also including images at two time points before and after the change; according to the change detection requirements, two time points are selected to obtain the first image and the second image of the same detection area, and both include the RPC parameters required for geometric correction.

[0013] Furthermore, the first image and the second image are subjected to image preprocessing. The implementation method is to perform RPC correction on the first image and the second image respectively, and then perform relative geometric correction on the first image and the second image to complete image registration, so as to obtain an image pair with matching geographic coordinates.

[0014] Moreover, the soil index constructed according to the spectral characteristics is,

[0015]

[0016] Among them, MBI is the soil index, SWIR1 is the reflectance of the shortwave infrared band with a central wavelength near 1600nm, SWIR2 is the reflectance of the shortwave infrared band with a central wavelength near 2200nm, NIR is the reflectance of the near infrared band with a central wavelength near 850nm, and μ is a parameter used to adjust the index distribution.

[0017] Moreover, the implementation of the spatial moment equivalent peak-valley segmentation method is as follows,

[0018] (1) Calculate the histogram of soil index of each pixel on the first image and calculate the area of ​​the histogram:

[0019] (2) Calculate the histogram space moment based on the histogram area;

[0020] (3) Using the point corresponding to the moment of the histogram space as the dividing point, divide the histogram into two intervals, so that the areas of the histograms of the left and right intervals are closest to being equal.

[0021] (4) Find the values ​​with the largest frequencies in the two intervals respectively, and these two values ​​are regarded as equivalent double peaks based on spatial moments;

[0022] (5) Find the lowest point in the interval between the equivalent double peaks as the equivalent valley, and take the median of the soil index interval corresponding to this point as the segmentation threshold.

[0023] Moreover, the clustering method for soil change labels is implemented as follows,

[0024] (1) Detect objects in the data set that have not been checked. If they have not been processed and classified into a certain category set or marked as noise, check their neighborhood. If the number of objects contained is not less than the corresponding threshold, establish a new category candidate set and add all the points in it to the candidate set;

[0025] (2) Check the neighborhood of all unprocessed objects in the candidate category set. If the number of objects contained is greater than the threshold, add these objects to the category set; if they are not classified into any category set, mark them as noise;

[0026] (3) Repeat (2) and continue to check the unprocessed objects in the data set until the current candidate set is empty;

[0027] (4) Repeat (1) to (3) until all objects are classified into a cluster or marked as noise.

[0028] Furthermore, the smaller change areas are filtered out to obtain the soil change label map. The screening conditions are as follows:

[0029] Delete the points marked as noise after clustering from the label map;

[0030] The pixels in the class whose number of objects after clustering is less than the corresponding threshold are deleted from the label map.

[0031] Furthermore, the image before the change, the image after the change and the label image sample group are formed on the second image, and the implementation method is as follows:

[0032] The image before the change in the second image is sliced ​​according to a preset size, and the image after the change and the label image are sliced ​​at the same position, and divided into three groups of samples: before the change, after the change, and label, to form a sample group.

[0033] On the other hand, the present invention provides a large-scale remote sensing image soil change detection method, and implements soil change detection based on the above-mentioned large-scale remote sensing image soil change pseudo-label production method.

[0034] Compared with existing methods, the advantages of this method are:

[0035] A method for automatically generating large-scale soil change labels without manual labeling is provided. The method can use auxiliary remote sensing images that have similar coverage and imaging time as the ordinary remote sensing images to be labeled and contain short-wave infrared bands, calculate soil index based on the auxiliary images, and then project them onto the images to be labeled to generate soil change patches as soil change labels for the images to be labeled.

[0036] In addition, the present invention proposes a spatial moment equivalent peak-valley method based on the state method to dynamically determine the segmentation threshold of the soil change index. Compared with the existing method of manually setting a fixed threshold, this method can dynamically set the threshold according to the histogram of the image.

[0037] The present invention also proposes to use a point density-based clustering method to screen the change spots, which can filter out smaller non-clustered noise spots. Compared with the traditional method, its advantage is that it removes the salt and pepper noise while retaining some pixels close to the change spots.

[0038] The present invention can realize the automatic generation of ultra-large sample sets of remote sensing images without the need for manual labeling of samples, thus saving a lot of manpower, further reducing the cost of change detection, and improving the effectiveness of remote sensing soil change detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 is an overall flow chart of an embodiment of the present invention;

[0040] Figure 2 Schematic diagram of density clustering effect of an embodiment of the present invention, the left side is a schematic diagram before clustering, and the right side is a schematic diagram after clustering and denoising. DETAILED DESCRIPTION

[0041] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiment of the present invention will be further described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is only a part of the embodiment of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0042] Embodiment 1:

[0043] The present invention proposes a method for producing pseudo labels for soil changes in large-scale remote sensing images. The basic idea is to first calculate the soil index from a multispectral image pair with a short-wave infrared band (called the first image), and then use the spatial moment equivalent peak-valley method to segment the soil change index to obtain change spots, which are then projected onto another multispectral image pair of close phase (called the second image) without a short-wave infrared band to generate a large-scale soil change sample set of the second image.

[0044] The technical solution provided by the embodiment of the present invention is a neural network image soil change detection method based on spectral feature automatic sample annotation, such as Figure 1 As shown, the following steps are included:

[0045] Step 1: Obtain the first image and the second image of two phases within the target area and perform image preprocessing:

[0046] The first image data refers to remote sensing image data with visible light, near-infrared bands and short-wave infrared bands, including images at two time points before and after the change. The second image data refers to remote sensing image data with visible light bands but without or incompletely with near-infrared bands and short-wave infrared bands, also including images at two time points before and after the change. According to the change detection requirements, two time points are selected to obtain the first image and the second image of the same detection area, and both include the RPC parameters required for geometric correction.

[0047] Therefore, the preferred scheme proposed by the present invention is: obtain a first image (multispectral image, with visible light band, near infrared band and short-wave infrared band) and a second image (ordinary visible light image) in two phases within the target area, and according to the change detection requirements, select two time points to obtain the first image and the second image of the same detection area, and both include the RPC parameters required for geometric correction, wherein the time difference between the first image and the second image at each time point does not exceed 2 months, and the time difference between the time points before and after the change is more than 1 year.

[0048] Then, the first image and the second image are subjected to image preprocessing, and the implementation method is: RPC correction is performed on the first image and the second image respectively, and then relative geometric correction is performed on the first image and the second image to complete image registration, so as to obtain an image pair with matching geographic coordinates.

[0049] Acquire Sentinel-2 image data of two phases within the target area (Sentinel-2 image data has visible light, near infrared band and short-wave infrared band, called the first image) and GF-1 image data (GF-1 image data has visible light band, called the second image). According to the change detection requirements, Sentinel-2 images of two phases are used as the first image, and two GF-1 image data located at the same location as the first image are used as the second image. The specific time is shown in Table 1.

[0050] Table 1

[0051] First image (Sentinel-2 image) Second image (GF-1 image) Image before change May 2, 2017 May 8, 2017 Image after change April 22, 2019 June 14, 2019

[0052] Step 2: construct a soil index based on the spectral characteristics to calculate the soil index of each pixel in the two phases of the first image.

[0053] The soil index constructed according to the spectral characteristics is:

[0054]

[0055] Among them, MBI is the soil index, SWIR1 is the reflectivity of the shortwave infrared band with a central wavelength near 1600nm, SWIR2 is the reflectivity of the shortwave infrared band with a central wavelength near 2200nm, NIR is the reflectivity of the near infrared band with a central wavelength near 850nm, and μ is a parameter used to adjust the index distribution, which is preferably set to 1.2 in this embodiment.

[0056] Since the emissivity of soil is higher near a wavelength of 1600nm and lower near wavelengths of 850nm and 2200nm, and other land objects such as vegetation, water bodies and impermeable surfaces have different reflectivity near these wavelengths, especially near 1600nm, which is lower than the reflectivity of soil, the embodiment preferably designs soil indexes at these wavelengths and adds a parameter μ to increase the weight of the 1600nm wavelength.

[0057] Step 3: Use the spatial moment equivalent peak-valley method to segment the soil index changes and generate a soil index change map.

[0058] The spatial moment equivalent peak-valley segmentation method proposed by the present invention according to the characteristics of the soil index map is:

[0059] (1) Calculate the histogram of soil index of each pixel on the first image (divided into intervals of length 0.02) and calculate the area of ​​the histogram:

[0060] (2) Calculate the histogram space moment based on the histogram area;

[0061] (3) The spatial moment of the histogram obtained in step (2) can be understood as the centroid of the histogram. Taking this point as the dividing point, the histogram is divided into two intervals, so that the areas of the histograms of the left and right intervals are closest to being equal.

[0062] (4) Find the value with the largest frequency in each of the two intervals. These two values ​​are regarded as equivalent double peaks (based on spatial moments).

[0063] (5) Find the lowest point in the interval between the equivalent double peaks as the equivalent valley, and take the median of the soil index interval corresponding to this point as the segmentation threshold.

[0064] The soil index is constructed based on the spectral characteristics to calculate the soil index of each pixel in the two phases in the first image. The pixels whose difference between the soil index after the change and the soil index before the change is greater than the segmentation threshold calculated in step 3 are marked as soil change pixels to generate a soil change map.

[0065] Step 4: Use a point density-based clustering method to screen the change patches, filter out the smaller change areas, and obtain a soil change label map;

[0066] Filter out the smaller change areas to get the soil change label map. The screening conditions are as follows:

[0067] Delete the points marked as noise after clustering from the label map;

[0068] The pixels in the class whose number of objects after clustering is less than the threshold are deleted from the label map.

[0069] The clustering method for soil change labels proposed by the embodiment of the present invention according to the characteristics of the soil change label map is:

[0070] (1) Detect the soil pixels that have not been checked in the soil label change map. If they have not been processed (classified as soil or marked as noise), check their neighborhood. If the number of soil pixels contained is not less than T1 = 10, establish a soil cluster patch label and add all the pixels in it to the soil change patch;

[0071] (2) Check the neighborhood of all pixels in the soil patch set that have not been processed. If the number of soil pixels contained is greater than T2 = 5, then add these objects to the soil change patch; if they are not included in any soil change patch, mark them as noise;

[0072] (3) Repeat step (2) and continue to check the unprocessed pixels in the data set until the current candidate set is empty;

[0073] (4) Repeat steps (1) to (3) until all objects are classified into a soil change patch or marked as noise.

[0074] The clustering effect is as follows Figure 2 As shown in the figure, after clustering, the smaller part of the change area is filtered out to obtain the soil change label map, and the screening conditions are as follows: the points marked as noise after clustering are deleted from the label map; the pixels in the class with less than T3=10 objects after clustering are deleted from the label map.

[0075] When clustering, if the thresholds T1 and T2 of the number of elements in steps (1) and (2) are too small, too much noise will be classified as soil change, and if they are too large, too much noise will be missed. Combined with the resolution of the image, after experimental testing, T1 = 10 and T2 = 5 are preferred. The parameter threshold T3 in step (4) is set according to the parameters of step (1).

[0076] Step 5, projecting the soil change label map generated in step 4 onto the second image, that is, calculating the soil index from the first image to generate the label, and forming the image before the change, the image after the change, and the label image sample group on the second image, the method is: the image before the change in the second image is sliced ​​according to a preset size (for example, 128×128), and the image after the change and the label image are sliced ​​at the same position, and divided into three groups of samples: before the change, after the change, and label, to form a sample group.

[0077] The core idea of ​​the present invention is to propose using the soil change labels of the Sentinel satellites as the GF1 image change labels.

[0078] Embodiment 2:

[0079] According to the embodiment, a large-scale remote sensing image soil change pseudo-label production method can be used for automatic generation of ultra-large sample sets of remote sensing images for soil change detection using artificial intelligence methods such as machine learning and deep learning, without the need for manual labeling of samples, thus saving a lot of manpower.

[0080] In specific implementation, a large-scale remote sensing image soil change detection method can be realized, including generating an image set for training according to the above large-scale remote sensing image soil change pseudo-label production method, and realizing soil change detection with the trained model. The implementation of machine learning, deep learning, etc. can refer to the existing technology, and the present invention will not elaborate on it.

[0081] In specific implementation, the method proposed in the technical solution of the present invention can be implemented by technical personnel in this field using computer software technology to realize automatic operation process. System devices for implementing the method, such as computer-readable storage media storing the corresponding computer program of the technical solution of the present invention and computer equipment running the corresponding computer program, should also be within the scope of protection of the present invention.

[0082] In some possible embodiments, a large-scale remote sensing image soil change pseudo-label production and change detection system is provided, including a processor and a memory, the memory is used to store program instructions, and the processor is used to call the stored instructions in the memory to execute the large-scale remote sensing image soil change pseudo-label production and change detection method as described above.

[0083] In some possible embodiments, a large-scale remote sensing image soil change pseudo-label production and change detection system is provided, including a readable storage medium, on which a computer program is stored. When the computer program is executed, the large-scale remote sensing image soil change pseudo-label production and change detection method as described above is implemented.

[0084] The specific embodiments described herein are merely examples of the spirit of the present invention. Those skilled in the art may make various modifications or additions to the specific embodiments described or replace them in similar ways, but they will not deviate from the spirit of the present invention or exceed the scope defined by the appended claims.

Claims

1. A method for producing pseudo labels for soil changes in large-scale remote sensing images, characterized in that: The following steps are involved: Step 1, obtaining a first image and a second image of two phases within the target area and performing image preprocessing; Step 2, constructing a soil index based on spectral characteristics and calculating the soil index of each pixel in the two time phases in the first image; The soil index constructed according to the spectral characteristics is: Among them, MBI is the soil index, SWIR1 is the short-wave infrared band reflectance with a central wavelength of about 1600nm, SWIR2 is the short-wave infrared band reflectance with a central wavelength of about 2200nm, NIR is the near-infrared band reflectance with a central wavelength of about 850nm, and μ is a parameter used to adjust the index distribution; Step 3, using the spatial moment equivalent peak-valley method to segment the soil index change and generate a soil index change map; The implementation method of the spatial moment equivalent peak-valley segmentation method is as follows: (1) Calculate the histogram of soil index of each pixel on the first image and calculate the area of ​​the histogram: (2) Calculate the histogram space moment based on the histogram area; (3) Using the point corresponding to the moment of the histogram space as the dividing point, divide the histogram into two intervals, so that the areas of the histograms of the left and right intervals are closest to being equal. (4) Find the values ​​with the largest frequencies in the two intervals respectively, and these two values ​​are regarded as equivalent double peaks based on spatial moments; (5) Find the lowest point in the interval between the equivalent double peaks as the equivalent valley, and take the median of the soil index interval corresponding to this point as the segmentation threshold; Step 4: Use a point density-based clustering method to screen the change patches, filter out the smaller change areas, and obtain a soil change label map; The clustering method for soil change labels is implemented as follows: (1) Detect objects in the data set that have not been checked. If they have not been processed and classified into a certain category set or marked as noise, check their neighborhood. If the number of objects contained is not less than the corresponding threshold, establish a new category candidate set and add all the points in it to the candidate set; (2) Check the neighborhood of all unprocessed objects in the candidate category set. If the number of objects contained is greater than the threshold, add these objects to the category set; if they are not classified into any category set, mark them as noise; (3) Repeat (2) and continue to check the unprocessed objects in the data set until the current candidate set is empty; (4) Repeat (1) to (3) until all objects are classified into a cluster or marked as noise; Step 5, projecting the soil change label map generated in step 4 onto the second image, forming an image before the change, an image after the change, and a label image sample group on the second image.

2. The method for producing pseudo labels for soil changes in large-scale remote sensing images as claimed in claim 1, characterized in that: The first image data refers to remote sensing image data with visible light, near-infrared bands and short-wave infrared bands, including images at two time points before and after the change. The second image data refers to remote sensing image data with visible light bands but without or incompletely with near-infrared bands and short-wave infrared bands, also including images at two time points before and after the change. According to the change detection requirements, two time points are selected to obtain the first image and the second image of the same detection area, and both include the RPC parameters required for geometric correction.

3. The method for producing pseudo labels for soil changes in large-scale remote sensing images as claimed in claim 1, characterized in that: The first image and the second image are subjected to image preprocessing. The implementation method is to perform RPC correction on the first image and the second image respectively, and then perform relative geometric correction on the first image and the second image to complete image registration, so as to obtain an image pair with matching geographic coordinates.

4. The method for producing pseudo labels for soil changes in large-scale remote sensing images as claimed in claim 1, characterized in that: Filter out the smaller change areas to get the soil change label map. The screening conditions are as follows: Delete the points marked as noise after clustering from the label map; The pixels in the class whose number of objects after clustering is less than the corresponding threshold are deleted from the label map.

5. A method for producing pseudo labels for soil changes in large-scale remote sensing images as claimed in claim 1, 2, 3 or 4, characterized in that: The image before the change, the image after the change and the label image sample group are formed on the second image, and the implementation method is as follows: The image before the change in the second image is sliced ​​according to a preset size, and the image after the change and the label image are sliced ​​at the same position, and divided into three groups of samples: before the change, after the change, and label, to form a sample group.

6. A large-scale remote sensing image soil change detection method, characterized by: According to the method for producing pseudo-labels of soil changes in large-scale remote sensing images as described in claims 1 to 5, soil change detection is achieved.

Citation Information

Patent Citations

  • land utilization change detection method for a high-resolution multi-source remote sensing image

    CN109657610A

  • Machine learning training data generation

    JP2019046269A

  • Change detection and change monitoring of natural and man-made features in multispectral and hyperspectral satellite imagery

    US20160307073A1

  • Method and system for crop type identification using satellite observation and weather data

    US20180189564A1