Method and device for detecting change area of remote sensing image

Abstract

The invention provides a method and device for detecting a change area of a remote sensing image, and belongs to the technical field of image processing, and the method comprises the steps: obtaining a first remote sensing image feature of a first remote sensing image and a second remote sensing image feature of a second remote sensing image in a dual-time-phase remote sensing image based on an encoder; according to the difference degree of the first remote sensing image feature and the second remote sensing image feature at the same position, obtaining a change heat map of the double-time-phase remote sensing image, and according to local peak points in the change heat map, generating a prompt point set; and embedding the prompt point set as a prompt, and generating a change region mask of the dual-temporal remote sensing image based on a decoder so as to carry out change region detection. According to the method, the stability and generalization of a universal feature extraction model in the existing computer vision field are combined, and stable and universal remote sensing image change area detection is realized by designing a flexible and universal change area detection framework and utilizing the existing basic encoder and decoder.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular, to a method and device for detecting changed areas in remote sensing images. Background Art

[0002] Traditional methods for monitoring river and lake shorelines have many limitations. Manual inspections are inefficient, involve a large workload, and it is difficult to inspect some areas thoroughly; optical remote sensing images are greatly affected by weather conditions, and data acquisition is discontinuous; manual visual interpretation is time-consuming and laborious, and is not conducive to the efficient interpretation of large-scale images. Remote sensing image change detection is an important means of environmental monitoring using remote sensing technology. This technology is based on the analysis of multi-source remote sensing images and related geospatial data covering the same surface area at different times, combined with the characteristics of ground objects and the remote sensing imaging mechanism, and uses image and graphic processing theories and mathematical models to determine and analyze the changes of ground objects in the region.

[0003] In recent years, remote sensing image change detection technology has been continuously developing. Methods based on SAR images are not affected by weather and lighting conditions and can provide a stable data source; methods based on deep learning have achieved high-precision recognition by constructing a large-scale sample set; methods based on the identity residual type Unet and remote sensing water indices combine multi-temporal images to obtain more accurate detection results. However, these methods usually require a large number of labeled samples for training, which are difficult to obtain in practical applications.

[0004] Therefore, it is particularly necessary to develop a new technical solution for detecting changed areas in remote sensing images. Summary of the Invention

[0005] The present invention provides a method and device for detecting changed areas in remote sensing images to solve the problem that existing change detection models are difficult to directly migrate to scenarios that have not been trained or have few samples.

[0006] In a first aspect, the present invention provides a method for detecting changed areas in remote sensing images, including: obtaining first remote sensing image features of a first remote sensing image and second remote sensing image features of a second remote sensing image in dual-temporal remote sensing images based on an encoder; obtaining a change heat map of the dual-temporal remote sensing images according to the difference degree of the features at the same position of the first remote sensing image features and the second remote sensing image features, and generating a set of hint points according to the local peak points in the change heat map; using the set of hint points as a hint embedding, and generating a changed area mask of the dual-temporal remote sensing images based on a decoder to perform changed area detection.

[0007] A method for detecting changed regions in remote sensing images provided by the present invention, which obtains the remote sensing image features of any one of the two-temporal remote sensing images, including: converting the remote sensing image into remote sensing images at different scales; using an encoder to respectively perform feature encoding on the remote sensing images at different scales to obtain corresponding image features at different scales; through upsampling, unifying the image features at different scales to the image features at the maximum scale; wherein, the maximum scale is the same as the scale of the original remote sensing image; generating fused remote sensing image features based on the multiple image features after unifying the scales.

[0008] A method for detecting changed regions in remote sensing images provided by the present invention, which obtains a change heat map of the two-temporal remote sensing images according to the difference degree of the features at the same position of the first remote sensing image feature and the second remote sensing image feature, including: obtaining the difference degree of the features of the first remote sensing image feature and the second remote sensing image feature in the same spatial domain; generating an initial change heat map according to the difference degree of the features of the two-temporal remote sensing images in each spatial domain; performing upsampling on the initial change heat map to obtain a change heat map with the same scale as the remote sensing image, which is used as the change heat map of the two-temporal remote sensing images.

[0009] A method for detecting changed regions in remote sensing images provided by the present invention, which obtains the difference degree of the features of the first remote sensing image feature and the second remote sensing image feature in the same spatial domain, including: calculating the cosine similarity of the corresponding feature vectors of the first remote sensing image feature and the second remote sensing image feature in the same spatial domain; calculating the difference degree of the features according to the cosine similarity.

[0010] A method for detecting changed regions in remote sensing images provided by the present invention, which generates a set of hint points according to the local peak points in the change heat map, including: traversing the change heat map through a preset sliding window to determine the peak points of the difference degree in each sliding window during the traversal process; filtering the peak points by using a preset threshold, and taking the peak points with the remaining difference degree greater than the preset threshold as hint points to generate a set of hint points.

[0011] A method for detecting changed regions in remote sensing images provided by the present invention, which uses the set of hint points as hint embeddings to generate a changed region mask of the two-temporal remote sensing images for changed region detection, including: inputting any hint point in the set of hint points and the first remote sensing image into a decoder to obtain a first single-point segmentation mask; and inputting the any hint point and the second remote sensing image into the decoder to obtain a second single-point segmentation mask; taking the intersection of the first single-point segmentation mask and the second single-point segmentation mask to obtain a single-body segmentation mask; determining the changed region mask of the two-temporal remote sensing images based on all the single-body segmentation masks corresponding to all the hint points.

[0012] In a second aspect, the present invention also provides a device for detecting changed regions in remote sensing images, including: A first processing module, configured to obtain first remote sensing image features of a first remote sensing image and second remote sensing image features of a second remote sensing image in a dual-temporal remote sensing image; A second processing module, configured to obtain a change heat map of the dual-temporal remote sensing image according to the difference degree of features at the same position between the first remote sensing image features and the second remote sensing image features, and generate a set of prompt points according to local peak points in the change heat map; A third processing module, configured to generate a change region mask of the dual-temporal remote sensing image by using the set of prompt points as a prompt embedding for change region detection.

[0013] In a third aspect, the present invention provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the steps of the method for detecting a change region of a remote sensing image as described in any one of the above are implemented.

[0014] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for detecting a change region of a remote sensing image as described in any one of the above are implemented.

[0015] In a fifth aspect, the present invention further provides a computer program product. When the computer program is executed by a processor, the steps of the method for detecting a change region of a remote sensing image as described in any one of the above are implemented.

[0016] The method and device for detecting a change region of a remote sensing image provided by the present invention have the following beneficial effects compared with the prior art: (1) The present invention provides a method and device for detecting a change region of a remote sensing image, which fully combines the stability and generalization of a general feature extraction model in the existing computer vision field. By designing a flexible and general change region detection framework and using existing basic encoders (feature extractors) and decoders, stable and general detection of a change region of a remote sensing image is achieved.

[0017] (2) The present invention proposes a feature extraction method of multi-scale fusion, which further strengthens the capture of features of the basic semantics of ground objects.

[0018] (3) The present invention proposes a prompt generation process based on matching search, which efficiently generates a change heat map from basic feature pairs and effectively makes a prompt embedding for subsequent segmentation of a change region mask by a decoder. Description of the Drawings

[0019] To more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0020] Figure 1 It is one of the schematic flowcharts of the method for detecting the changed area of remote sensing images provided by the present invention; Figure 2 It is the second schematic flowchart of the method for detecting the changed area of remote sensing images provided by the present invention; Figure 3 It is the schematic diagram of the basic feature extraction process in the existing method; Figure 4 It is the schematic diagram of the multi-scale fusion feature extraction process provided by the present invention; Figure 5 It is the schematic flowchart of the process for obtaining the change heat map provided by the present invention; Figure 6 It is the schematic flowchart of the extraction process of the hint points provided by the present invention; Figure 7 It is the schematic flowchart of the generation process of the changed area mask provided by the present invention; Figure 8 It is the schematic diagram of the qualitative segmentation result of the present invention on the LEVIR-CD dataset; Figure 9 It is the schematic diagram of partial detection results of the present invention on the CD_samples dataset; Figure 10 It is the schematic diagram of the detection results of other types of ground objects of the present invention on the CD_samples dataset; Figure 11 It is the schematic diagram of the structure of the electronic device provided by the present invention. Detailed implementation manners

[0021] To make the objectives, technical solutions, and advantages of the present invention clearer, the following will clearly and completely describe the technical solutions in the present invention in conjunction with the drawings in the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. Based on the embodiments in the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.

[0022] It should be noted that in the description of the embodiments of the present invention, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method, article or device including the said element. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0023] The terms "first", "second", etc. in this application are used to distinguish similar objects, rather than to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of this application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are usually of the same type, and the number of objects is not limited. For example, the first object can be one or multiple.

[0024] Figure 1 is one of the schematic flowcharts of the method for detecting the changed area of remote sensing images provided by the present invention. Figure 2 is the second of the schematic flowcharts of the method for detecting the changed area of remote sensing images provided by the present invention. The following combines Figure 1 and Figure 2 to illustrate the technical solution of the present invention. The present invention includes but is not limited to the following steps: Step 101: Based on the encoder, obtain the first remote sensing image feature of the first remote sensing image and the second remote sensing image feature of the second remote sensing image in the dual-temporal remote sensing images.

[0025] Optionally, obtaining the remote sensing image feature of any remote sensing image in the dual-temporal remote sensing images includes: converting the remote sensing image into remote sensing images at different scales; using the encoder to perform feature encoding on the remote sensing images at different scales respectively to obtain the corresponding image features at different scales; through the method of upsampling, unify the image features at different scales to the image features at the maximum scale; wherein, the maximum scale is the same as the scale of the original remote sensing image; based on the multiple image features after unifying the scales, generate a fused remote sensing image feature (i.e., a multi-scale fusion feature).

[0026] Specifically, in the process of encoding the basic semantics of the image in the existing method, most directly input the image into the encoder for feature encoding, as Figure 3 shown. Figure 3It is a schematic diagram of the basic feature extraction process in the existing method. Since most of the general basic feature extraction encoders are pre-trained using the VIT architecture, the size of the image patches encoded by each token is fixed. However, due to the species, inter-class differences, and image resolution of different ground objects, there may be scale differences, and it is difficult for the fixed image patch encoding size to capture the ground object features that may vary flexibly with the scale.

[0027] Figure 4 It is a schematic diagram of the multi-scale fusion feature extraction process provided by the present invention. As Figure 4 shown, in order to better capture the complete features within the ground object contour, this method adopts the idea of multi-scale fusion. First, the image is linearly transformed into different scales (1, 1 / 2, 1 / 4, 1 / 8). Since, therefore, when encoding different-scale images, the image patches encoded by each token correspond to different-scale regions in the original image. Taking the encoder with 16-pixel encoding as an example, the image patches with sizes of 16, 32, 64, and 128 pixels in the original image are respectively encoded into corresponding feature tokens. As Figure 4 shown, specifically, in the multi-scale fusion feature extraction process proposed by this method, for the input image (3 represents the number of channels of the input image, h represents the height of the input image, w represents the width of the input image), it is linearly transformed into different-scale sizes , , and the image patches are respectively segmented and encoded, and then input into the basic general encoder for feature extraction. Different token features are obtained for each scale image. The single token in the four types of token feature sets corresponds to the scale sizes , , ( is the scale size of the image patch encoding in the adopted basic general encoder) in sequence. For ground objects of different sizes, their local and overall structures can be effectively encoded into different tokens. For the obtained token feature set, it is transformed into a three-dimensional feature block , , and is linearly interpolated and mapped to the same size as . Finally, the transformed feature and are added together to obtain the final multi-scale fusion feature F .

[0028] The present invention focuses on the problem of change detection. The input is a multi-temporal image pair. Therefore, in the process of basic feature extraction in the first stage, multi-scale fusion feature extraction is respectively performed on the image pair (A and B, that is, the first remote sensing image and the second remote sensing image) to obtain an initial feature pair ( ; that is, the first remote sensing image feature and the second remote sensing image feature) as the basis for subsequent detection.

[0029] Step 102: According to the difference degree of the features of the first remote sensing image feature and the second remote sensing image feature at the same position, obtain the change heat map of the bi-temporal remote sensing image, and generate a set of hint points according to the local peak points in the change heat map.

[0030] The present invention proposes a hint generation process based on the idea of matching search as the hint input for subsequent segmentation. After obtaining the initial features of the input image pair in the first stage , the change value in the image space is obtained by corresponding spatial domain feature matching, and the regional points with the highest change degree are extracted based on the set search strategy as the hint points for decoder mask segmentation. The second stage is divided into two processes, namely the change degree calculation based on feature matching and the hint point generation based on search, which will be introduced in turn below.

[0031] (1) Change degree calculation based on feature matching Optionally, according to the difference degree of the features of the first remote sensing image feature and the second remote sensing image feature at the same position, obtaining the change heat map of the bi-temporal remote sensing image further includes: obtaining the difference degree of the features of the first remote sensing image feature and the second remote sensing image feature in the same spatial domain; generating an initial change heat map according to the difference degree of the features of the bi-temporal remote sensing image in each spatial domain; performing upsampling on the initial change heat map to obtain a change heat map with the same scale as the remote sensing image as the change heat map of the bi-temporal remote sensing image.

[0032] Among them, obtaining the difference degree of the features of the first remote sensing image feature and the second remote sensing image feature in the same spatial domain includes: calculating the cosine similarity of the corresponding feature vectors of the first remote sensing image feature and the second remote sensing image feature in the same spatial domain; calculating the difference degree of the features of the first remote sensing image feature and the second remote sensing image feature in the same spatial domain according to the cosine similarity.

[0033] Figure 5 is the schematic flow chart of obtaining the change heat map (change degree heat map) provided by the present invention. As Figure 5 shown, the purpose of the change degree calculation based on feature matching is to calculate the difference value at the same position in different time phases of two images.

[0034] Specifically, in the initial feature pair , for each corresponding feature vector in the spatial domain of each image block respectively , Calculate the difference degree (difference value) of the eigenvectors , The calculation process is shown in formula (1):

[0035] Finally, it is calculated that feature difference values are combined into corresponding spatial feature differences based on the corresponding spatial structure . Since is a three-dimensional feature matrix, in the actual implementation process, the calculation method is shown in formula (2):

[0036] where (N = ) is formed by dimensional transformation of the feature matrix , is the norm calculation operation in the first dimension, is the operation of taking the diagonal value of the matrix. Thus, the difference degree of the corresponding spatial feature can be directly calculated through the basic feature pair . Finally, through linear interpolation, is transformed to the size of the original remote sensing image to obtain the change heat map . Each pixel point in

[0037] represents the probability value of the ground object change in this spatial domain. The larger the value, the greater the possibility of the ground object change in this area. Therefore, the highlighted area represents the change area. (2) Generation based on the found hint points The change heat map generated in the previous step

[0038] can roughly perceive the change area in the multi-temporal image pair. In order to more accurately capture the mask of the changed ground object, the method extracts a series of change area hint points on the basis of the change heat map as the hint embedding of the large model encoder, and then uses the powerful segmentation function of the existing pre-trained encoder to achieve high-precision change area detection. Specifically, the method sets a sliding window operation based on a specific search strategy to extract the peaks in the change heat map as hint points.

[0039] Figure 6 is the schematic diagram of the extraction process of the hint points provided by the present invention, as shown in Figure 6 ​As shown, for the difference values within each window, first perform a max pooling operation, and then perform an exclusive NOR calculation on the original heat value and the value after max pooling, which retains the peaks inside the window. Therefore, the window search operation can be expressed as formula (3):

[0040] Where is the heat value within the window range, is the size of the sliding window, is the value after window operation, is the exclusive NOR operation, is the negation operation, is 's max pooling value. Based on the above window operation, perform a sliding calculation in the change heat map to obtain the image retaining the change peaks . Due to the existence of some noise peaks, set a certain threshold to filter the peaks, and finally obtain the hint point set by coordinate point extraction. This process is shown in formulas (4)-(5):

[0041]

[0042] Where is the heat value part within the sliding window, is the number of sliding windows, is the set of pixel coordinates in the extracted image where the pixel value is greater than .

[0043] Step 103: Use the hint point set as a hint embedding, and generate a change region mask for the dual-temporal remote sensing image based on the decoder to perform change region detection.

[0044] Based on the hint point set generated by the above process as the hint embedding for the large model decoding, utilize the powerful segmentation ability of the pre-trained general large model decoder to generate an accurate overall change region mask. Optionally, using the hint point set as a hint embedding to generate a change region mask for the dual-temporal remote sensing image to perform change region detection includes: inputting any hint point in the hint point set together with the first remote sensing image into the decoder to obtain a first single-point segmentation mask; and inputting the any hint point together with the second remote sensing image into the decoder to obtain a second single-point segmentation mask; taking the intersection of the first single-point segmentation mask and the second single-point segmentation mask to obtain a single-body segmentation mask; and determining the change region mask of the dual-temporal remote sensing image based on all single-body segmentation masks corresponding to all hint points.

[0045] Figure 7It is a schematic flow diagram of the generation of the change area mask provided by the present invention. As Figure 7 shown, for each hint point in the hint point set , taking as an example ( ), it is respectively input into the encoder together with the basic feature pair generated in the first step to generate the corresponding single-point segmentation mask and (i.e., the first single-point segmentation mask and the second single-point segmentation mask), and then the single-body change mask is obtained through an intersection operation. The process is shown in Formulas (6)-(8):

[0046]

[0047]

[0048] where ( ) is the decoding and segmentation process of the large model. Finally, all the single-body change masks generated by the hint point set , ……, are union-operated to obtain the final change detection mask (i.e., the change area mask) M, and the generation process is shown in Formula (9):

[0049] where is the number of hint points in.

[0050] To evaluate the effectiveness of the proposed method, the present invention has conducted qualitative experiments on different urban and different-resolution image datasets, including LEVIR-CD and CD_samples respectively. In the specific experimental process, the decoder and encoder in the method adopt the SAM model fine-tuned by LORA in the remote sensing ground object segmentation task. Note that this model is only fine-tuned on the target detection and segmentation task and not trained on any change detection task. The experimental parameters used are shown in Table 1: Table 1 Experimental Parameter Table

[0051] LEVIR-CD is a large-scale remote sensing building change detection dataset, aiming to provide a new benchmark for evaluating change detection algorithms. This dataset consists of 637 pairs of very high-resolution (VHR, 0.5 m / pixel) Google Earth image patches, with each pair of images sized 1024×1024 pixels. The time span of these images is from 5 to 14 years, covering significant land use changes, especially building growth. LEVIR-CD includes various types of buildings, such as villa residences, high-rise apartments, small garages, and large warehouses. Figure 8 It is a schematic diagram of the qualitative segmentation result of the present invention on the LEVIR-CD dataset.

[0052] From the qualitative results, the present invention can roughly detect the changed areas in different images of LEVIR-CD. It is worth noting that the present invention has not undergone any training fine-tuning on LEVIR-CD. Since the LEVIR-CD dataset only annotates the building change areas, while the present invention is not limited to building change detection, but can perceive changes in various ground objects. Therefore, in some results, the changed areas detected by the present invention are not all reflected in the true annotations of the dataset. For example Figure 8 For the part boxed by the red rectangle in the second row, the method of the present invention detects a land use change area not annotated in the GT; for the part marked by the red rectangle in the fourth row, the result of the present invention includes a water body change area not covered in the annotation. The qualitative experimental results show the generalization of the method and the potential for perceiving changes in multiple types of ground objects.

[0053] The CD_samples dataset is a new building change detection dataset constructed by the present invention using daily updated images as the image source. The resolution of this dataset is 2 m / pixel, and each pair of images is sized 512×512 pixels. The image range covers the Yangtze River Basin, including various rich topographies such as farmland, hillsides, rivers, and forestlands. The annotation of the building change areas in this dataset is still under optimization. The experiment tests the performance of the present method in detecting changed areas on the CD_samples dataset.

[0054] Figure 9 It is a schematic diagram of some detection results of the present invention on the CD_samples dataset. Generally, the method can capture the changed areas within the images. However, since the annotation of the CD_samples dataset is still under optimization, some true results detected by the present method are not reflected in the annotation, such as Figure 9In the parts marked by the red rectangles in the second and third rows, the method detected the truly changed areas in the image, but the manual annotation results missed some areas. From this aspect, the detection results of the method are helpful for reviewing the manual annotation. In addition, the method also detected the changed areas of features other than buildings, such as the parts marked by the yellow rectangles in the images of the second and fourth rows. The method detected the changes in forest land and water bodies. Figure 10 It is a schematic diagram of the detection results of other types of features of the present invention on the CD_samples dataset. For other unannotated changed features such as roads, farmlands, construction lands, shorelines, etc., the method also demonstrated a certain change detection ability.

[0055] The method for detecting changed areas of remote sensing images provided by the present invention has the following beneficial effects compared with the prior art: (1) The present invention provides a method and device for detecting changed areas of remote sensing images, which fully combines the stability and generalization of the general feature extraction models in the existing computer vision field. By designing a flexible and general changed area detection framework and using the existing basic encoders (feature extractors) and decoders, stable and general detection of changed areas of remote sensing images is achieved.

[0056] (2) The present invention proposes a feature extraction method of multi-scale fusion, which further strengthens the capture of features of the basic semantics of features.

[0057] (3) The present invention proposes a hint generation process based on matching search, which efficiently generates a change heat map from the basic feature pairs and effectively makes hint embedding for subsequent mask segmentation of changed areas by the decoder.

[0058] On the other hand, the present invention also provides a device for detecting changed areas of remote sensing images, and the device includes: A first processing module, configured to obtain the first remote sensing image feature of the first remote sensing image and the second remote sensing image feature of the second remote sensing image in the dual-temporal remote sensing images based on an encoder; A second processing module, configured to obtain a change heat map of the dual-temporal remote sensing images according to the difference degree of the features at the same positions of the first remote sensing image feature and the second remote sensing image feature, and generate a set of hint points according to the local peak points in the change heat map; A third processing module, configured to use the set of hint points as hint embedding and generate a changed area mask of the dual-temporal remote sensing images based on a decoder to perform changed area detection.

[0059] It should be noted that the device for detecting changed areas of remote sensing images provided by the embodiments of the present invention can execute the method for detecting changed areas of remote sensing images described in any of the above embodiments during specific operation, and details are not described in this embodiment.

[0060] Figure 11 is a schematic structural diagram of the electronic device provided by the present invention. As Figure 11 shown, the electronic device may include: a processor 1110, a communications interface 1120, a memory 1130, and a communication bus 1140. Among them, the processor 1110, the communications interface 1120, and the memory 1130 complete mutual communication through the communication bus 1140. The processor 1110 may call logic instructions in the memory 1130 to execute a method for detecting a changed area of a remote sensing image. The method includes: obtaining a first remote sensing image feature of a first remote sensing image and a second remote sensing image feature of a second remote sensing image in a dual-temporal remote sensing image based on an encoder; obtaining a change heat map of the dual-temporal remote sensing image according to the difference degree of the features at the same position of the first remote sensing image feature and the second remote sensing image feature, and generating a set of prompt points according to local peak points in the change heat map; using the set of prompt points as a prompt embedding, and generating a change area mask of the dual-temporal remote sensing image based on a decoder to perform changed area detection.

[0061] In addition, when the logic instructions in the foregoing memory 1130 are implemented in the form of software functional units and sold or used as an independent product, they may be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of the technical solution, may be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The foregoing storage medium includes: various media such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk, or an optical disc that can store program codes.

[0062] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium. The computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the change area detection method of remote sensing images provided in the above embodiments. The method includes: obtaining the first remote sensing image feature of the first remote sensing image and the second remote sensing image feature of the second remote sensing image in the dual-temporal remote sensing images based on an encoder; obtaining a change heat map of the dual-temporal remote sensing images according to the difference degree of the features at the same position of the first remote sensing image feature and the second remote sensing image feature, and generating a set of prompt points according to the local peak points in the change heat map; using the set of prompt points as a prompt embedding, and generating a change area mask of the dual-temporal remote sensing images based on a decoder to perform change area detection.

[0063] In another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it is configured to execute the change area detection method of remote sensing images provided in the above embodiments. The method includes: obtaining the first remote sensing image feature of the first remote sensing image and the second remote sensing image feature of the second remote sensing image in the dual-temporal remote sensing images based on an encoder; obtaining a change heat map of the dual-temporal remote sensing images according to the difference degree of the features at the same position of the first remote sensing image feature and the second remote sensing image feature, and generating a set of prompt points according to the local peak points in the change heat map; using the set of prompt points as a prompt embedding, and generating a change area mask of the dual-temporal remote sensing images based on a decoder to perform change area detection.

[0064] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the essence of the above technical solutions, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disc, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0065] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for detecting changed areas in remote sensing images, characterized in that, Including: Based on an encoder, obtaining first remote sensing image features of a first remote sensing image and second remote sensing image features of a second remote sensing image in dual-temporal remote sensing images; According to the difference degree of features at the same position between the first remote sensing image features and the second remote sensing image features, obtaining a change heat map of the dual-temporal remote sensing images, and generating a set of hint points based on local peak points in the change heat map; Taking the set of hint points as a hint embedding, and generating a change region mask of the dual-temporal remote sensing images based on a decoder to perform change region detection.

2. The method for detecting a changed area of a remote sensing image according to claim 1, wherein Obtaining remote sensing image features of any remote sensing image in dual-temporal remote sensing images, including: Converting the remote sensing image into remote sensing images at different scales; Using an encoder to respectively perform feature encoding on the remote sensing images at different scales to obtain corresponding image features at different scales; Through an upsampling method, unifying the image features at different scales to the image features at the maximum scale; wherein, the maximum scale is the same as the scale of the original remote sensing image; Generating fused remote sensing image features based on the multiple image features after unifying the scales.

3. The method for detecting a changed area of a remote sensing image according to claim 1, wherein According to the difference degree of features at the same position between the first remote sensing image features and the second remote sensing image features, obtaining a change heat map of the dual-temporal remote sensing images, including: Obtaining the difference degree of features between the first remote sensing image features and the second remote sensing image features in the same spatial domain; Generating an initial change heat map according to the difference degree of features in each spatial domain of the dual-temporal remote sensing images; Performing upsampling on the initial change heat map to obtain a change heat map with the same scale as the remote sensing image as the change heat map of the dual-temporal remote sensing images.

4. The method for detecting a changed area of a remote sensing image according to claim 3, wherein Obtaining the difference degree of features between the first remote sensing image features and the second remote sensing image features in the same spatial domain, including: Calculating the cosine similarity of corresponding feature vectors of the first remote sensing image features and the second remote sensing image features in the same spatial domain; Calculating the difference degree of features according to the cosine similarity.

5. The method for detecting a changed area of a remote sensing image according to claim 1, wherein Generating a set of hint points based on local peak points in the change heat map, including: Traversing the change heat map through a preset sliding window to determine peak points of the difference degree in each sliding window during the traversal process; Filtering the peak points using a preset threshold, and taking the peak points with the difference degree greater than the preset threshold that are retained as hint points to generate a set of hint points.

6. The method for detecting a changed area of a remote sensing image according to claim 1, wherein Taking the set of hint points as a hint embedding to generate a change region mask of the dual-temporal remote sensing images to perform change region detection, including: Combining any hint point in the set of hint points with the first remote sensing image and inputting it into the decoder to obtain a first single-point segmentation mask; and, combining the any hint point with the second remote sensing image and inputting it into the decoder to obtain a second single-point segmentation mask; Taking the intersection of the first single-point segmentation mask and the second single-point segmentation mask to obtain a single-body segmentation mask; Determining the change region mask of the dual-temporal remote sensing images based on all single-body segmentation masks corresponding to all hint points.

7. A change region detection device for remote sensing images, characterized in that Including: A first processing module, configured to obtain first remote sensing image features of a first remote sensing image and second remote sensing image features of a second remote sensing image in dual-temporal remote sensing images based on an encoder; The second processing module is configured to obtain a change heat map of the dual-temporal remote sensing images according to the difference degree of the features at the same position between the first remote sensing image feature and the second remote sensing image feature, and generate a set of hint points according to the local peak points in the change heat map; The third processing module is configured to use the set of hint points as a hint embedding, and generate a change region mask of the dual-temporal remote sensing images based on a decoder to perform change region detection.

8. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, When the processor executes the computer program, the steps of the method for detecting a change region of a remote sensing image according to any one of claims 1 to 6 are implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, the steps of the method for detecting a change region of a remote sensing image according to any one of claims 1 to 6 are implemented.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, the steps of the method for detecting a change region of a remote sensing image according to any one of claims 1 to 6 are implemented.

Images