Small-scale-oriented deep learning change detection method and device, equipment and medium
By using deep learning change detection method in small-scale areas, combined with multi-scale transformation and semantic segmentation technology, differential probability and geographic feature map images are generated, and the problem of insufficient samples for small-scale area change detection is solved, and the change area is achieved The efficient detection and classification of the system has improved the ecological environment supervision capabilities.
Patent Information
- Application Number
- CN202411104643.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-13
- Publication Date
- 2024-11-29
- Estimated Expiration
- 2044-08-13
AI Technical Summary
The existing deep learning change detection methods are difficult to effectively apply in small-scale areas, mainly because they cannot obtain sufficient change samples, and due to differences in geographical characteristics, provincial or national models are difficult to directly apply to municipal and county-level regions.
Using a small-scale deep learning change detection method, the two-phase remote sensing images are obtained for multi-scale transformation and semantic segmentation, and differential probability images and geographic feature map images are generated, and the change region and change category are determined in combination with the spatial index algorithm. Realize the change detection of small-scale monitoring areas.
It realizes the change detection of small-scale areas, which is suitable for many categories of land such as forest land, water bodies, buildings, etc., without the need for change detection samples, has strong applicability and fast computing speed, and has improved the ecological environment supervision and governance of small-scale monitoring areas. ability.
Smart Images

Figure CN119048902A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of change detection technology, and in particular to a small-scale deep learning change detection method, device, equipment and medium. Background Art
[0002] Remote sensing change detection is an important technology in the field of earth observation. Remote sensing change detection refers to the comparative analysis of two images and the determination of their change range through relevant methods. The purpose is to filter out non-change information and extract change information. Remote sensing change monitoring has a wide range of applications, such as disaster monitoring, urban supervision, forest protection, military operations, and topographic changes.
[0003] As a branch of machine learning, deep learning has developed rapidly with its multi-level feature learning capabilities. Deep learning change detection algorithms have become the mainstream algorithm for remote sensing change detection. Most deep learning change detection algorithms use deep neural networks such as convolutional neural networks (CNN), generative adversarial networks (GAN), and transformer attention mechanisms to build a change detection network framework to achieve change detection. These methods basically require a large number of labeled samples and are suitable for provincial and national monitoring areas where there are many changes. However, due to the small number of changes in small-scale areas at the city and county level in a short period of time, it is difficult to obtain samples for deep learning. At the same time, deep learning models trained with provincial and national regional samples are difficult to be directly applied to the city and county scale due to differences in geographical features such as phenology and topography.
[0004] In summary, for change detection in small-scale areas, the deep learning method cannot obtain enough change samples in the monitoring area, which seriously restricts the application of change detection in small-scale areas. There is an urgent need for a deep learning change detection algorithm for small-scale areas. Summary of the invention
[0005] In view of this, the purpose of the present invention is to provide a small-scale deep learning change detection method, device, equipment and medium, which realizes change detection in small-scale monitoring areas and provides support for improving the ecological environment supervision and governance capabilities of small-scale monitoring areas.
[0006] In a first aspect, an embodiment of the present invention provides a small-scale deep learning change detection method, comprising:
[0007] Obtain two phases of remote sensing images corresponding to the small-scale monitoring area;
[0008] The two phases of remote sensing images are processed by multi-scale transformation to generate difference probability images, and the two phases of remote sensing images are processed by semantic segmentation through a deep learning model to generate two phases of ground feature image patches; the difference probability image is used to characterize the confidence level corresponding to the sub-regions that have changed in the small-scale monitoring area, and the ground feature image patch is used to characterize the distribution of different ground features in the small-scale monitoring area;
[0009] Based on the difference probability image and the two-phase ground feature patch images, the change patch results corresponding to the small-scale monitoring area are generated; among which, the change patch results are used to characterize the changes of different ground features in the small-scale monitoring area.
[0010] In one implementation, performing multi-scale transformation processing on two phases of remote sensing images to generate a difference probability image includes:
[0011] Perform band-by-band difference processing on the two phases of remote sensing images to obtain a difference image;
[0012] A multi-scale transformation model is constructed based on the difference image to perform multi-scale transformation processing on the difference image through the multi-scale transformation model;
[0013] The difference image after multi-scale transformation is stretched to generate a difference probability image.
[0014] In one embodiment, semantic segmentation of two-phase remote sensing images is performed using a deep learning model to generate two-phase ground feature image spots, including:
[0015] Annotate the semantic information of any one or two remote sensing images to construct semantic samples corresponding to different ground features;
[0016] The deep learning model is trained using the semantic samples corresponding to each ground feature to obtain the semantic segmentation model corresponding to each ground feature;
[0017] The semantic segmentation model corresponding to each land feature is used to perform semantic segmentation processing on the two-phase remote sensing images to generate two-phase land feature patch images corresponding to each land feature.
[0018] In one embodiment, the method further comprises:
[0019] In the process of training the deep learning model, target points are randomly sampled from the image output by the deep learning model, and the prediction results of the ground features to which the target points belong are determined. The loss value is calculated based on the semantic information annotated by the target points, and the model parameters of the deep learning model are adjusted using the loss value.
[0020] In one embodiment, based on the difference probability image and the two-phase ground feature image, a change image result corresponding to the small-scale monitoring area is generated, including:
[0021] Based on the confidence of the difference probability image representation and the preset confidence threshold, candidate sub-regions are extracted from the small-scale monitoring area;
[0022] Using the spatial index algorithm, the distribution of different ground features in the candidate sub-area is determined based on the two-phase ground feature image patch images;
[0023] Based on the distribution of different ground features in the candidate sub-areas, determine the target sub-areas that have changed in the small-scale monitoring area and their corresponding change categories;
[0024] Based on the target sub-area and its corresponding change category, the change patch results corresponding to the small-scale monitoring area are generated.
[0025] In one implementation, a spatial index algorithm is used to determine the distribution of different ground features in the candidate sub-region based on two-phase ground feature image spots, including:
[0026] For each land feature, a spatial index algorithm is used to determine the number of first pixels corresponding to the land feature in the candidate sub-region based on a land feature spot image of one period, and the number of second pixels corresponding to the land feature in each candidate sub-region is determined based on another land feature spot image;
[0027] The distribution of the land feature in the candidate sub-region is described by using a first pixel ratio between the first pixel number and the total number of pixels in the candidate sub-region and a second pixel ratio between the second pixel number and the total number of pixels in the candidate sub-region.
[0028] In one embodiment, based on the distribution of different ground features in the candidate sub-areas, determining the target sub-areas that have changed in the small-scale monitoring area and their corresponding change categories includes:
[0029] Determine whether the first pixel ratio and the second pixel ratio are less than a first ratio threshold;
[0030] If so, it is determined that the candidate subregion has not changed;
[0031] If no, if the first pixel ratio is greater than or equal to the first ratio threshold, and the second pixel ratio is less than the second ratio threshold, then it is determined that the candidate sub-region belongs to the changed target sub-region and the change category is a reduction in ground objects; if the first pixel ratio is less than the second ratio threshold, and the second pixel ratio is greater than or equal to the first ratio threshold, then it is determined that the candidate sub-region belongs to the changed target sub-region and the change category is an increase in ground objects.
[0032] In a second aspect, an embodiment of the present invention further provides a small-scale deep learning change detection device, comprising:
[0033] Image acquisition module, used to obtain two phases of remote sensing images corresponding to the small-scale monitoring area;
[0034] The image processing module is used to perform multi-scale transformation processing on the two-phase remote sensing images to generate difference probability images, and to perform semantic segmentation processing on the two-phase remote sensing images through a deep learning model to generate two-phase ground feature image spots; wherein, the difference probability image is used to characterize the confidence corresponding to the sub-regions that have changed in the small-scale monitoring area, and the ground feature image spot is used to characterize the distribution of different ground features in the small-scale monitoring area;
[0035] The change detection module is used to generate change patch results corresponding to the small-scale monitoring area based on the difference probability image and the two-phase ground feature patch images; among which, the change patch results are used to characterize the changes of different ground features in the small-scale monitoring area.
[0036] In a third aspect, an embodiment of the present invention further provides an electronic device, comprising a processor and a memory, wherein the memory stores computer-executable instructions that can be executed by the processor, and the processor executes the computer-executable instructions to implement any one of the methods provided in the first aspect.
[0037] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement any one of the methods provided in the first aspect.
[0038] The embodiments of the present invention provide a small-scale deep learning change detection method, device, equipment and medium. First, two-phase remote sensing images corresponding to the small-scale monitoring area are obtained; then, the two-phase remote sensing images are subjected to multi-scale transformation processing to generate difference probability images, and at the same time, the two-phase remote sensing images are subjected to semantic segmentation processing through a deep learning model to generate two-phase ground object element spot images. The difference probability image is used to characterize the confidence corresponding to the sub-area that has changed in the small-scale monitoring area, and the ground object element spot image is used to characterize the distribution of different ground object elements in the small-scale monitoring area; finally, based on the difference probability image and the two-phase ground object element spot image, a change spot result corresponding to the small-scale monitoring area is generated, and the change spot result is used to characterize the change of different ground object elements in the small-scale monitoring area. The embodiment of the present invention constructs a deep learning change detection algorithm suitable for small-scale monitoring areas, combines multi-scale transformation and semantic segmentation algorithms, uses multi-scale transformation processing to extract high-confidence change areas, and then uses the semantic segmentation algorithm to determine the change areas and change categories. The embodiment of the present invention can be applied to multiple categories of land features such as woodlands, water bodies, and buildings, and does not require change detection samples. It has the advantages of strong applicability and fast computing speed, and provides support for the improvement of ecological environmental supervision and governance capabilities in small-scale monitoring areas.
[0039] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.
[0040] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0042] Figure 1 A schematic diagram of a process flow of a small-scale deep learning change detection method provided by an embodiment of the present invention;
[0043] Figure 2 A schematic diagram of a remote sensing image preprocessing process provided by an embodiment of the present invention;
[0044] Figure 3A schematic diagram of a multi-scale transformation process provided by an embodiment of the present invention;
[0045] Figure 4 A schematic diagram of a semantic segmentation process provided by an embodiment of the present invention;
[0046] Figure 5 A schematic diagram of the Mask2Former structure provided by an embodiment of the present invention;
[0047] Figure 6 A schematic diagram of a change detection process provided by an embodiment of the present invention;
[0048] Figure 7 A schematic diagram of the overall process of a small-scale deep learning change detection method provided by an embodiment of the present invention;
[0049] Figure 8 A schematic diagram of a two-phase remote sensing image provided by an embodiment of the present invention;
[0050] Fig. 9 A schematic diagram of a difference probability image provided by an embodiment of the present invention;
[0051] Fig.10 A schematic diagram of building distribution in two phases provided by an embodiment of the present invention;
[0052] Fig.11 A schematic diagram of the structure of a small-scale deep learning change detection device provided by an embodiment of the present invention;
[0053] Fig.12 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0054] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution of the present invention will be clearly and completely described in combination with the embodiments below. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0055] At present, for change detection in small-scale areas, the deep learning method cannot obtain enough change samples in the monitoring area, which seriously restricts the application of change detection in small-scale areas. There is an urgent need for a deep learning change detection algorithm for small-scale areas. Based on this, the present invention implements a deep learning change detection method, device, equipment and medium for small-scale, which realizes change detection in small-scale monitoring areas and provides support for the improvement of ecological environment supervision and governance capabilities in small-scale monitoring areas.
[0056] To facilitate understanding of this embodiment, a small-scale deep learning change detection method disclosed in an embodiment of the present invention is first introduced in detail. Figure 1 The flowchart of a small-scale deep learning change detection method shown in FIG. 1 mainly includes the following steps S102 to S106:
[0057] Step S102, obtaining two phases of remote sensing images corresponding to the small-scale monitoring area.
[0058] Among them, the small-scale monitoring area refers to the smaller monitoring area at the city and county levels, and the two remote sensing images can be two adjacent remote sensing images, which are called the front remote sensing image and the back remote sensing image.
[0059] Step S104, performing multi-scale transformation processing on the two-phase remote sensing images to generate a difference probability image, and performing semantic segmentation processing on the two-phase remote sensing images through a deep learning model to generate two-phase ground feature patch images.
[0060] Among them, the difference probability image is used to characterize the confidence corresponding to the sub-region that has changed in the small-scale monitoring area. The higher the confidence, the greater the possibility of change in the sub-region. The ground feature spot image is used to characterize the distribution of different ground features in the small-scale monitoring area. In one example, the two-period remote sensing images can be processed by band-by-band difference, multi-scale transformation, and image stretching to obtain the difference probability image. In one example, the semantic samples corresponding to different ground features can be constructed using any one or two periods of remote sensing images, and the semantic samples corresponding to different ground features can be used to train the deep learning model to obtain the semantic segmentation model corresponding to different ground features, and then the semantic segmentation model is used to output the ground feature spot image corresponding to each of the two remote sensing images.
[0061] Step S106, based on the difference probability image and the two-phase ground feature image, a change image result corresponding to the small-scale monitoring area is generated.
[0062] Among them, the change map results are used to characterize the changes of different ground objects in the small-scale monitoring area. In one example, the confidence represented by the difference probability image can be used to screen out candidate sub-areas with a high probability of change, and then combined with the distribution of different ground objects represented by the ground object element spot images of the two periods in the two remote sensing images, the target sub-areas that have changed and the change categories of each sub-area, such as ground object reduction or ground object increase, can be further screened out, and finally the change map results are generated.
[0063] The embodiment of the present invention constructs a deep learning change detection algorithm suitable for small-scale monitoring areas, combines multi-scale transformation and semantic segmentation algorithms, uses multi-scale transformation processing to extract high-confidence change areas, and then uses the semantic segmentation algorithm to determine the change areas and change categories. The embodiment of the present invention can be applied to multiple categories of land features such as woodlands, water bodies, and buildings, and does not require change detection samples. It has the advantages of strong applicability and fast computing speed, and provides support for the improvement of ecological environmental supervision and governance capabilities in small-scale monitoring areas.
[0064] For ease of understanding, an embodiment of the present invention provides a specific implementation of a small-scale deep learning change detection method.
[0065] For the aforementioned step S102, after obtaining two phases of remote sensing images corresponding to the small-scale monitoring area, it is necessary to perform geometric correction and cropping processing on the two phases of images, such as Figure 2 A schematic diagram of a remote sensing image preprocessing process is shown. Specifically, two images of the small-scale monitoring area are obtained, and the two images are geometrically corrected to minimize the geometric offset of the image for change detection, and then the small-scale monitoring vector is used to crop the geometrically corrected image to obtain the two preprocessed images in the small-scale monitoring area, which are recorded as A image and B image. A image and B image have the same number of bands n and scale (xsize, ysize). Among them, the small-scale monitoring vector is used to characterize the scope of the small-scale monitoring area, which can be the administrative boundary of a city, county, or township, or a customized small-range boundary of interest to the customer.
[0066] For the aforementioned step S104, the embodiment of the present invention provides an implementation method for performing multi-scale transformation processing on two remote sensing images to generate a difference probability image. In practical applications, since remote sensing images have multi-spectral characteristics and the spectral characteristics of each ground feature are inconsistent, the redundancy of remote sensing image information and the inability to measure the spectra of different ground features with a unified standard are caused. Therefore, it is necessary to perform multi-scale transformation on the data of image A and image B to obtain the difference area as much as possible with less data redundancy. Figure 3 The diagram shows a multi-scale transformation process flow. First, the difference between A and B is processed band by band. Then, the difference image is subjected to multi-scale transformation for spectral compression. Finally, the difference probability image is constructed by image stretching, which is recorded as C image.
[0067] For details, please refer to steps 1A to 1C below:
[0068] Step 1A, perform band-by-band difference processing on the two remote sensing images to obtain a difference image. The expression for band-by-band difference processing is as follows:
[0069] C1=[B1-A1,B i -Ai ,…,B n -A n ];
[0070] Among them, A i , B i is the i-th band image between image A and image B, C1 is the difference image, and the size of C1 is (n, xsize, ysize).
[0071] Step 1B, construct a multi-scale transformation model based on the difference image, so as to perform multi-scale transformation processing on the difference image through the multi-scale transformation model. The expression of the multi-scale transformation processing is as follows:
[0072] C2 = MSD (C1);
[0073] Among them, MDS (Multi-dimensional scaling) is a multi-scale transformation, C2 is a difference image processed by the multi-scale transformation, and C2 is (xsize, ysize).
[0074] The MSD algorithm process is as follows:
[0075] Transform the array size of C1 to generate C ′ 1, the size is (n, m), m = xsize * ysize. In order to obtain the MSD model, it speeds up the calculation. ′ In 1, randomly select pixels with a percentage of p to generate a sample S with a size of (n, m1), m1 = INT(p*m).
[0076] First calculate the pairwise distance dist of S:
[0077]
[0078] dist=add(add(-2*(sum_S·sum_S).T),sum_S);
[0079] Where add is the accumulation of matrix elements, sum_S·sum_S is the dot product operation between sum_S, and sum_S is the cumulative sum of squares in the band direction. The size of the dist array is (m1,m1).
[0080] Calculate the inner product matrix Be of dist, assuming one is the initial matrix with the value of (m1,m1) being 1:
[0081] Be = -(T1-T2-T3+dist) / 2;
[0082] T1 = one*sum(dist) / (m1*m1);
[0083] T2 = sum(dist0) / (m1);
[0084] T3 = sum(dist1) / (m1);
[0085] Among them, sum(dist0) and sum(dist1) are the accumulated values of row and column vectors.
[0086] Perform eigendecomposition on Be, obtain the eigensolution and eigenvector, sort the eigenfeatures, select the index value where the maximum value is located, and use the eigensolution, eigenvector and maximum eigenindex as the update parameters of the MSD model. The MSD model is recorded as model.
[0087] Using model to C ′ 1 performs dimensionality reduction processing to generate a 1-dimensional vector C3, and the size (m, 1) is used to convert the array size of C3 to generate C, with a size of (xsize, ysize) and a threshold range of 0-100.
[0088] Step 1C, performing image stretching processing on the difference image after multi-scale transformation processing to generate a difference probability image. The expression of image stretching processing is as follows:
[0089]
[0090] Among them, max(C2) and min(C2) are the maximum and minimum values of C2, and C is the difference probability image.
[0091] For the aforementioned step S104, the embodiment of the present invention also provides an implementation method for generating two-phase ground feature image patches by semantically segmenting two-phase remote sensing images through a deep learning model. In practical applications, although there are fewer changing areas in small-scale areas, there is a large amount of semantic information. For example, there are fewer changing areas in forest land, cultivated land, buildings, etc., but their distribution areas are large. Based on this basic reality, semantic segmentation of ground features is carried out to optimize the boundaries of changing areas and assign attribute information. Figure 4 The figure shows a flow chart of semantic segmentation processing, in which semantic information is annotated based on the distribution information of ground objects, a large number of semantic samples are produced, and the Mask2Former (Masked-attention MaskTransformer) deep learning network framework is used for training to obtain the mask2former model (also known as the semantic segmentation model), and then the A and B images are predicted to output two phases of ground object patch images D and E.
[0092] For details, please refer to steps 2A to 2C below:
[0093] Step 2A, annotate the semantic information of any one or two remote sensing images to construct semantic samples corresponding to different land features; or annotate the semantic information of existing samples outside the monitoring area to construct semantic samples corresponding to different land features.
[0094] Step 2B, using the semantic samples corresponding to each land feature to train the deep learning model to obtain a semantic segmentation model corresponding to each land feature.
[0095] Among them, Mask2Former is built on a simple meta-framework (MaskFormer) and a new Transformer decoder. Its key component is masked-attention, which extracts local features by limiting cross attention to the predicted mask area. The framework consists of three parts, see Figure 5 The figure shows a schematic diagram of the Mask2Former structure, which includes a backbone feature extractor, a pixel decoder, and a Transformer decoder. The backbone feature extractor extracts low-resolution features from the image; the pixel decoder gradually upsamples the low-resolution features from the output of the backbone to generate a high-resolution per-pixel embedding; the Transformer decoder operates on the image features to process the object query and finally obtains the predicted image (i.e., the ground feature patch image).
[0096] Since Mask2Former is designed for visual images, and remote sensing has unique multi-scale and multi-spectral features and requires large amounts of stored data, it needs to be modified to be suitable for remote sensing images. The modifications are as follows: (1) Modify the number and order of bands in the pre-trained weights to be suitable for multi-spectral features of remote sensing; (2) Optimize Transformer; use random sampling methods to improve computational efficiency and save memory to be suitable for remote sensing features with large storage volumes.
[0097] By optimizing the Transformer, the embodiment of the present invention can randomly sample target points from the image output by the deep learning model during the training of the deep learning model, determine the prediction results of the ground features to which the target points belong, calculate the loss value in combination with the semantic information annotated by the target points, and use the loss value to adjust the model parameters of the deep learning model. During the training process, in order to solve the problem that the amount of calculation of remote sensing images is large relative to natural images, the sampling method of calculating the loss of Mask2Former during the training process is modified from the original calculation of the loss for all pixels of the image of img_size*img_size to a random point calculation according to a certain ratio (50%), which can reduce the amount of calculation to 1 / 4 of the original amount while ensuring the accuracy of the model.
[0098] Step 2C, using the semantic segmentation model corresponding to each land feature, the two-phase remote sensing images are processed with semantic segmentation to generate two-phase land feature patch images corresponding to each land feature.
[0099] Based on steps 2A to 2C above, the specific process of generating a patch image of a land feature is as follows: first, use the previous and next images to make a sample of a certain land feature, and feed the sample into the modified Mask2Former deep learning framework, and then set the main hyperparameters of Mask2Former, which are the number of semantic segmentation categories num, slice size img_szie, number of iterations epochs, loss function loss, optimizer optimizer, and learning rate lr. Mask2Former is trained to obtain a deep learning model. Finally, images A and B are sliced and predicted, and finally spliced to generate a patch image of a single land feature, which are recorded as images D and E.
[0100] For the aforementioned step S106, the embodiment of the present invention also provides an implementation method for generating change spot results corresponding to the small-scale monitoring area based on the difference probability image and the two-phase ground feature element spot images. In practical applications, since the land class boundaries of semantic segmentation are relatively accurate and the land class attributes are clear, they can be used as a basis for optimizing the boundaries of the change spots and judging the change categories, determining the change boundaries of the ground features and the types of reduction and increase, and generating change spot results, which are recorded as F images.
[0101] For details, please refer to steps 3A to 3D below:
[0102] Step 3A: Based on the confidence of the difference probability image representation and the preset confidence threshold, candidate sub-regions are extracted from the small-scale monitoring area. Figure 6 The diagram of a change detection process is shown in FIG. 1 . First, the difference probability image C is divided according to the empirical threshold. The candidate sub-regions with probabilities greater than the preset confidence threshold v1 are high-confidence candidate sub-regions, i.e., suspected change spots. The suspected change spots are marked and numbered. If the number of suspected change spots is s, the numbering starts from 1 and ends at s, with an interval of 1.
[0103] Step 3B, using a spatial index algorithm, based on the two-phase ground feature image patch images, determines the distribution of different ground features in the candidate sub-region.
[0104] In one example, for each land feature, a spatial index algorithm is used to determine the first number of pixels corresponding to the land feature in the candidate sub-region based on a land feature spot image of one period, and the second number of pixels corresponding to the land feature in each candidate sub-region is determined based on another land feature spot image; then, the first pixel ratio between the first pixel number and the total number of pixels contained in the candidate sub-region, and the second pixel ratio between the second pixel number and the total number of pixels contained in the candidate sub-region are used to describe the distribution of the land feature in the candidate sub-region.
[0105] Please continue to see Figure 6 , step 3B is the process of superimposing and analyzing the suspected change spots and the two-phase ground feature spot images. In the specific implementation, the suspected change spots are numbered k, and the number of pixels is nk. The distribution of ground feature elements in the D image and the E image in the k area can be obtained according to the spatial indexing technology. Calculate the number of pixels determined to be a certain ground feature element in the k area, where the first pixel number in the D image is nd, and the second pixel number in the E image is ne, and the proportion is calculated as follows:
[0106] p1 = nd / nk;
[0107] p2 = ne / nk;
[0108] Among them, p1 is the first pixel ratio, and p2 is the second pixel ratio.
[0109] Step 3C, based on the distribution of different ground features in the candidate sub-areas, determine the target sub-areas that have changed in the small-scale monitoring area and their corresponding change categories;
[0110] In one example, it is determined whether the first pixel ratio and the second pixel ratio are less than a first ratio threshold; if so, it is determined that the candidate sub-region has not changed; if not, if the first pixel ratio is greater than or equal to the first ratio threshold, and the second pixel ratio is less than the second ratio threshold, it is determined that the candidate sub-region belongs to a changed target sub-region and the change category is a reduction in ground objects; if the first pixel ratio is less than the second ratio threshold, and the second pixel ratio is greater than or equal to the first ratio threshold, it is determined that the candidate sub-region belongs to a changed target sub-region and the change category is an increase in ground objects.
[0111] Please continue to see Figure 6 , using decision trees to hierarchically determine the change areas and change categories, the specific process is as follows:
[0112] (1) When p1 or p2 is greater than or equal to the first ratio threshold v2, this condition is recorded as condition 1. If condition 1 is met, the change occurs; if condition 1 is not met, the change does not occur.
[0113] (2) When the further category judgment of condition 1 is met, when condition 1 is met and p1 is greater than or equal to the first ratio threshold v2 and p2 is less than the second ratio threshold v3, this condition is recorded as condition 2. If this condition is met, the ground object is reduced;
[0114] (3) When condition 1 is met and p2 is greater than or equal to the first ratio threshold v2 and p1 is less than the second ratio threshold v3, this condition is recorded as condition 3, and meeting this condition means that the ground object increases;
[0115] (4) When condition 1 is met but conditions 2 and 3 are not met, it should be a changing noise and is recorded as unchanged.
[0116] Step 3D, based on the target sub-area and its corresponding change category, generates the change spot result corresponding to the small-scale monitoring area. In one example, all suspected change spots are traversed according to step 3C, and the change spot image is generated by screening and judging, and then the raster-to-vector processing is performed to generate the change spot result.
[0117] For ease of understanding, the present invention provides an application example of a small-scale deep learning change detection method. Figure 7 The figure shows an overall process diagram of a small-scale deep learning change detection method, including preprocessing and multi-scale transformation of two-phase images to obtain difference probability images, semantic sample annotation of the two-phase images to obtain semantic samples, using the mask2former network framework suitable for the task to output ground feature patch images, combining the difference probability images and ground feature patch images to determine change information and change categories, and finally outputting change patch results.
[0118] Taking Qibin District of Hebi City as an example of a small-scale monitoring area, the deep learning change detection method for small scale mainly includes the following steps:
[0119] 1. Remote sensing image processing:
[0120] The embodiment of the present invention selects Qibin District of Hebi City, a small-scale area, to monitor common features in northern cities and counties, including water bodies, cultivated land, buildings, and woodlands, which are typical and representative. GF1 images of this area in July 2022 and March 2023 are obtained. After geometric correction and vector clipping, images A and B are generated, such as Figure 8 A schematic diagram of a two-phase remote sensing image is shown, where the number of bands of image A and image B is 4, and the size is (20164, 14066).
[0121] (II) Construction of difference probability image:
[0122] Perform difference processing on image A and image B, perform multi-scale transformation on the difference image for spectral compression, and then construct the difference probability image C by image stretching, such as Fig. 9 A schematic diagram of a difference probability image is shown in FIG. The number of samples selected by MSD spectrum compression is 5%, and the number of sample data is 40993412.
[0123] (III) Semantic Deep Learning Segmentation:
[0124] Select buildings for semantic segmentation, mark 569 patches in the building area, and slice them into 256*256 size to generate 2160 sample sets. Based on the slice set, the Mask2Former network is trained to extract the building distribution (that is, the aforementioned ground feature patch image), such as Fig.10 A schematic diagram of building distribution before and after the two phases is shown, including images D and E. Among them, the number of semantic segmentation categories of Mask2Former is 2, the slice size is 256, the number of iterations is 1000, and the learning rate lr is 0.01.
[0125] (IV) Change monitoring area extraction:
[0126] The difference probability image C is divided according to the empirical threshold, and the probability greater than 50 is a high confidence change area, and the change area is marked and numbered. The total number of change spots is 40268, and the numbering starts from 1 and ends at 40268, with an interval of 1. The change spots are superimposed with the building distribution images D and E, and the v2 and v3 thresholds of the decision tree layer are 0.75 and 0.15 respectively.
[0127] In summary, the embodiment of the present invention constructs a deep learning change detection algorithm suitable for small-scale areas, combines multi-scale transformation with the mask2former semantic segmentation algorithm, uses the multi-scale transformation algorithm to extract high-confidence change areas, and then uses the mask2former semantic information to determine the change area and change category. This method can be applied to multiple categories of objects such as woodlands, water bodies, and buildings, and does not require change detection samples. It has the advantages of strong applicability and fast computing speed, and provides support for the improvement of ecological environmental supervision and governance capabilities in small-scale areas.
[0128] Based on the above embodiments, the present invention provides a small-scale deep learning change detection device. Fig.11 The schematic diagram of the structure of a small-scale deep learning change detection device is shown in FIG. The device mainly includes the following parts:
[0129] The image acquisition module 1102 is used to acquire two phases of remote sensing images corresponding to the small-scale monitoring area;
[0130] The image processing module 1104 is used to perform multi-scale transformation processing on the two-phase remote sensing images to generate a difference probability image, and to perform semantic segmentation processing on the two-phase remote sensing images through a deep learning model to generate two-phase ground feature image spots; wherein the difference probability image is used to characterize the confidence corresponding to the sub-region that has changed in the small-scale monitoring area, and the ground feature image spot is used to characterize the distribution of different ground features in the small-scale monitoring area;
[0131] The change detection module 1106 is used to generate change patch results corresponding to the small-scale monitoring area based on the difference probability image and the two-phase ground feature patch images; wherein the change patch results are used to characterize the changes of different ground features in the small-scale monitoring area.
[0132] The embodiment of the present invention constructs a deep learning change detection device suitable for small-scale monitoring areas, combines multi-scale transformation and semantic segmentation algorithms, uses multi-scale transformation processing to extract high-confidence change areas, and then uses the semantic segmentation algorithm to determine the change areas and change categories. The embodiment of the present invention can be applied to multiple categories of land features such as woodlands, water bodies, and buildings, and does not require change detection samples. It has the advantages of strong applicability and fast computing speed, and provides support for the improvement of ecological environmental supervision and governance capabilities in small-scale monitoring areas.
[0133] In one implementation, the image processing module 1104 is specifically configured to:
[0134] Perform band-by-band difference processing on the two phases of remote sensing images to obtain a difference image;
[0135] A multi-scale transformation model is constructed based on the difference image to perform multi-scale transformation processing on the difference image through the multi-scale transformation model;
[0136] The difference image after multi-scale transformation is stretched to generate a difference probability image.
[0137] In one implementation, the image processing module 1104 is specifically configured to:
[0138] Annotate the semantic information of any one or two remote sensing images to construct semantic samples corresponding to different ground features;
[0139] The deep learning model is trained using the semantic samples corresponding to each ground feature to obtain the semantic segmentation model corresponding to each ground feature;
[0140] The semantic segmentation model corresponding to each land feature is used to perform semantic segmentation processing on the two-phase remote sensing images to generate two-phase land feature patch images corresponding to each land feature.
[0141] In one implementation, the image processing module 1104 is specifically configured to:
[0142] In the process of training the deep learning model, target points are randomly sampled from the image output by the deep learning model, and the prediction results of the ground features to which the target points belong are determined. The loss value is calculated based on the semantic information annotated by the target points, and the model parameters of the deep learning model are adjusted using the loss value.
[0143] In one implementation, the change detection module 1106 is specifically configured to:
[0144] Based on the confidence of the difference probability image representation and the preset confidence threshold, candidate sub-regions are extracted from the small-scale monitoring area;
[0145] Using the spatial index algorithm, the distribution of different ground features in the candidate sub-area is determined based on the ground feature image patches of the two phases.
[0146] Based on the distribution of different ground features in the candidate sub-areas, determine the target sub-areas that have changed in the small-scale monitoring area and their corresponding change categories;
[0147] Based on the target sub-area and its corresponding change category, the change patch results corresponding to the small-scale monitoring area are generated.
[0148] In one implementation, the change detection module 1106 is specifically configured to:
[0149] For each land feature, a spatial index algorithm is used to determine the number of first pixels corresponding to the land feature in the candidate sub-region based on a land feature spot image of one period, and the number of second pixels corresponding to the land feature in each candidate sub-region is determined based on another land feature spot image;
[0150] The distribution of the land feature in the candidate sub-region is described by using a first pixel ratio between the first pixel number and the total number of pixels in the candidate sub-region and a second pixel ratio between the second pixel number and the total number of pixels in the candidate sub-region.
[0151] In one implementation, the change detection module 1106 is specifically configured to:
[0152] Determine whether the first pixel ratio and the second pixel ratio are less than a first ratio threshold;
[0153] If so, it is determined that the candidate subregion has not changed;
[0154] If no, if the first pixel ratio is greater than or equal to the first ratio threshold, and the second pixel ratio is less than the second ratio threshold, then it is determined that the candidate sub-region belongs to the changed target sub-region and the change category is a reduction in ground objects; if the first pixel ratio is less than the second ratio threshold, and the second pixel ratio is greater than or equal to the first ratio threshold, then it is determined that the candidate sub-region belongs to the changed target sub-region and the change category is an increase in ground objects.
[0155] The device provided in the embodiment of the present invention has the same implementation principle and technical effects as those of the aforementioned method embodiment. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment.
[0156] An embodiment of the present invention provides an electronic device. Specifically, the electronic device includes a processor and a storage device. The storage device stores a computer program, and when the computer program is executed by the processor, it executes the method described in any one of the above-mentioned implementation methods.
[0157] Fig.12 A structural diagram of an electronic device provided in an embodiment of the present invention, the electronic device 100 includes: a processor 120, a memory 121, a bus 122 and a communication interface 123, the processor 120, the communication interface 123 and the memory 121 are connected via the bus 122; the processor 120 is used to execute an executable module stored in the memory 121, such as a computer program.
[0158] The memory 121 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 123 (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.
[0159] The bus 122 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.12 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.
[0160] Among them, the memory 121 is used to store programs, and the processor 120 executes the program after receiving the execution instruction. The method executed by the device for flow process definition disclosed in any of the embodiments of the present invention can be applied to the processor 120 or implemented by the processor 120.
[0161] The processor 120 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 120. The above processor 120 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present invention can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiment of the present invention can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module may be located in a storage medium mature in the art, such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 121, and the processor 120 reads the information in the memory 121 and completes the steps of the above method in combination with its hardware.
[0162] The computer program product of the readable storage medium provided in the embodiment of the present invention includes a computer-readable storage medium storing program code, and the instructions included in the program code can be used to execute the methods described in the previous method embodiments. The specific implementation can be referred to the previous method embodiments, which will not be repeated here.
[0163] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.
[0164] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims
1. A small-scale deep learning change detection method, characterized in that: include: Obtain two phases of remote sensing images corresponding to the small-scale monitoring area; Performing multi-scale transformation processing on the two phases of remote sensing images to generate difference probability images, and performing semantic segmentation processing on the two phases of remote sensing images through a deep learning model to generate two phases of ground feature image spots; wherein the difference probability image is used to characterize the confidence corresponding to the sub-regions that have changed in the small-scale monitoring area, and the ground feature image spot is used to characterize the distribution of different ground features in the small-scale monitoring area; Based on the difference probability image and the two phases of the ground object element spot images, the change spot results corresponding to the small-scale monitoring area are generated; wherein the change spot results are used to characterize the changes of different ground objects in the small-scale monitoring area.
2. The small-scale deep learning change detection method according to claim 1, characterized in that: The remote sensing images of the two phases are subjected to multi-scale transformation processing to generate difference probability images, including: Performing band-by-band difference processing on the remote sensing images of the two phases to obtain a difference image; Constructing a multi-scale transformation model based on the difference image, so as to perform multi-scale transformation processing on the difference image through the multi-scale transformation model; The difference image after the multi-scale transformation is subjected to image stretching processing to generate a difference probability image.
3. The small-scale deep learning change detection method according to claim 1, characterized in that: The two phases of remote sensing images are processed by semantic segmentation through a deep learning model to generate two phases of ground feature image spots, including: Annotating the remote sensing images of any one or both periods with semantic information to construct semantic samples corresponding to different ground features; The deep learning model is trained using the semantic samples corresponding to each ground feature to obtain a semantic segmentation model corresponding to each ground feature; The semantic segmentation model corresponding to each of the land feature elements is used to perform semantic segmentation processing on the two phases of remote sensing images to generate two phases of land feature element spot images corresponding to each of the land feature elements.
4. The small-scale deep learning change detection method according to claim 3, characterized in that: The method further comprises: During the training of the deep learning model, target points are randomly sampled from the image output by the deep learning model, and the prediction results of the ground features to which the target points belong are determined. The loss values are calculated in combination with the semantic information annotated with the target points, and the model parameters of the deep learning model are adjusted using the loss values.
5. The small-scale deep learning change detection method according to claim 1, characterized in that: Based on the difference probability image and the two phases of the ground feature image, a change image result corresponding to the small-scale monitoring area is generated, including: Extracting a candidate sub-region from the small-scale monitoring area based on the confidence represented by the difference probability image and a preset confidence threshold; Using a spatial index algorithm, the distribution of different ground features in the candidate sub-area is determined based on the ground feature image spots of the two phases; Based on the distribution of different ground objects in the candidate sub-area, determining the target sub-area that has changed in the small-scale monitoring area and its corresponding change category; Based on the target sub-region and the corresponding change category, a change patch result corresponding to the small-scale monitoring area is generated.
6. The small-scale deep learning change detection method according to claim 5, characterized in that: Using a spatial index algorithm, the distribution of different ground features in the candidate sub-area is determined based on the ground feature image spots of the two phases, including: For each of the land feature elements, using a spatial index algorithm, based on a first phase of the land feature element spot image, determine the first number of pixels corresponding to the land feature element in the candidate sub-area, and based on another phase of the land feature element spot image, determine the second number of pixels corresponding to the land feature element in each of the candidate sub-areas; The distribution of the land feature in the candidate sub-region is described using a first pixel ratio between the first number of pixels and the total number of pixels in the candidate sub-region, and a second pixel ratio between the second number of pixels and the total number of pixels in the candidate sub-region.
7. The small-scale deep learning change detection method according to claim 6, characterized in that: Based on the distribution of different ground objects in the candidate sub-area, determining the target sub-area that has changed in the small-scale monitoring area and its corresponding change category, including: Determine whether the first pixel ratio and the second pixel ratio are less than a first ratio threshold; If yes, determining that the candidate sub-region has not changed; If no, if the first pixel ratio is greater than or equal to the first ratio threshold, and the second pixel ratio is less than the second ratio threshold, then it is determined that the candidate sub-region belongs to the changed target sub-region and the change category is a reduction in ground objects; if the first pixel ratio is less than the second ratio threshold, and the second pixel ratio is greater than or equal to the first ratio threshold, then it is determined that the candidate sub-region belongs to the changed target sub-region and the change category is an increase in ground objects.
8. A small-scale deep learning change detection device, characterized in that: include: Image acquisition module, used to obtain two phases of remote sensing images corresponding to the small-scale monitoring area; An image processing module is used to perform multi-scale transformation processing on the two phases of remote sensing images to generate difference probability images, and to perform semantic segmentation processing on the two phases of remote sensing images through a deep learning model to generate two phases of ground feature image spots; wherein the difference probability image is used to characterize the confidence corresponding to the sub-region that has changed in the small-scale monitoring area, and the ground feature image is used to characterize the distribution of different ground features in the small-scale monitoring area; The change detection module is used to generate a change spot result corresponding to the small-scale monitoring area based on the difference probability image and the two-phase ground feature spot images; wherein the change spot result is used to characterize the change of different ground features in the small-scale monitoring area.
9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores computer executable instructions that can be executed by the processor, and the processor executes the computer executable instructions to implement the method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are called and executed by a processor, the computer-executable instructions prompt the processor to implement the method according to any one of claims 1 to 7.
Citation Information
Patent Citations
Time sequence remote sensing image dimension reduction method
CN111062267A
High-resolution remote sensing image ground object change detection method based on multi-task learning
CN111582043A
Illegal land use detection method and device
CN113160239A
Remote sensing change detection method and device, electronic equipment and storage medium
CN116091497A
MROCNet model construction and multi-source remote sensing image change detection method and system
CN116363526A
Cited By
Remote sensing image change detection method and system based on pattern spot-by-pattern confidence aggregation
CN121685452A
