Multi-temporal remote sensing image change detection method and system based on interactive prompt

Abstract

The invention discloses a multi-temporal remote sensing image change detection method and system based on interactive prompt, and belongs to the field of cross combination of remote sensing images and AI, and the method comprises the steps: obtaining paired multi-temporal remote sensing images and corresponding binary change tag maps, and forming a sample set comprising a front temporal image, a rear temporal image and a tag map; simulating to generate a positive sample prompt graph and a negative sample prompt graph by adopting three interactive modes of clicking, frame selection and line drawing; during first iteration, the positive sample prompt graph, the negative sample prompt graph and the sample set which are generated through simulation are input into the interactive prompt remote sensing change detection model; during subsequent iteration, the currently generated positive and negative sample prompt graph and the previous segmentation result graph are jointly input into an interactive prompt remote sensing change detection model; repeating the iteration step for at least three times to optimize parameters of the interactive prompt remote sensing change detection model; and inputting a to-be-detected multi-temporal remote sensing image into the trained interaction prompt remote sensing change detection model, and outputting a change detection result graph in combination with user real-time interaction prompt.

Description

Technical Field

[0001] The present invention belongs to the technical field of cross-integration of remote sensing images and AI, and specifically relates to a multi-temporal remote sensing image change detection method and system based on interactive prompts. Background Art

[0002] Remote sensing imagery is digital image data that records the characteristics and spatial information of the Earth's surface (or other targets) using sensors (remote sensing technology) on platforms such as satellites and aircraft. This data is generated by receiving electromagnetic wave information (visible light, infrared, microwaves, etc.) without physical contact with the target. Remote sensing image change detection provides key technical support for monitoring land cover changes, climate change, and assessing the impact of human activities. In recent years, intelligent change detection methods based on deep learning have significantly improved the level of automation, but existing methods generally rely on massive amounts of labeled samples for model training. Sample collection requires manual comparison of multiple image periods and pixel-by-pixel annotation, which consumes a considerable amount of time and labor, severely limiting the application of this technology in time-sensitive scenarios.

[0003] To reduce sample dependence, existing technologies have proposed interactive remote sensing image change detection models based on clicks and box selections, allowing users to generate detection results through a small number of interactive prompts. Although the above two methods can reduce the workload of manual interaction and improve work efficiency to a certain extent, such methods have obvious limitations: (1) The interaction method is single and only supports one mode of click or box selection. It can only realize change detection of two-phase remote sensing images and cannot flexibly select the best interaction strategy according to the target morphology; (2) There is a lack of iterative optimization mechanism. The results are directly output after a single interaction, and the detection accuracy of complex change areas is insufficient; (3) Historical segmentation information is not utilized, and there is no collaborative optimization relationship between interactive prompts and intermediate results, resulting in error accumulation. In addition, for irregular changing targets such as building demolition and farmland encroachment, the line drawing interaction method proposed by existing technologies can provide more accurate guidance.

[0004] However, existing technologies have not yet achieved the unified integration of the three interaction modes of clicking, selecting boxes, and drawing lines to complete multi-temporal remote sensing change detection tasks; at the same time, there is a lack of a progressive optimization mechanism that integrates the previous segmentation results with the current interactive prompts, making it difficult to achieve a closed-loop detection process of "interaction-feedback-optimization". This is a technical bottleneck that urgently needs to be broken through in the current field. Summary of the Invention

[0005] The present invention aims to solve the above problems, remedy the deficiencies of the prior art, and provide a method and system for detecting changes in multi-temporal remote sensing images based on interactive prompts.

[0006] To achieve the above objectives, the present invention adopts the following technical solutions.

[0007] In a first aspect, the present invention provides a method for detecting changes in multi-temporal remote sensing images based on interactive prompts, comprising the following steps: S1. Constructing a training sample set: Obtain paired multi-temporal remote sensing images and corresponding binary change label maps to form a sample set containing pre-temporal images, post-temporal images, and label maps; S2. Computer simulation of interactive prompts: Using three interactive methods: clicking, box selection, and line drawing, simulate the generation of positive sample prompt images and negative sample prompt images; S3. Progressively iteratively train the interactive prompt remote sensing change detection model: In the first iteration, the positive sample prompt map and negative sample prompt map generated by the simulation in step S2 and the sample set containing the previous phase image, the next phase image, and the label map in step S1 are input into the interactive prompt remote sensing change detection model; in subsequent iterations, the currently generated positive and negative sample prompt maps and the previous segmentation result map are input into the interactive prompt remote sensing change detection model; repeat the iterative steps at least three times to optimize the parameters of the interactive prompt remote sensing change detection model; S4. Execute change detection: Input the multi-temporal remote sensing image to be detected into the trained interactive prompt remote sensing change detection model, and output the change detection result graph in combination with the user's real-time interactive prompt.

[0008] Furthermore, in step S2, the simulation method of click interaction is: generating a circular prompt area with the specified coordinate point as the center; the simulation method of frame selection interaction is: filling the interior of the rectangular box with the foreground value to generate a prompt image; the simulation method of line drawing interaction includes: boundary lines extracted and smoothed by edge detection operators, central axes extracted by central axis transformation, and Bezier curves based on central axis smoothing.

[0009] Furthermore, in step S3, the positive and negative sample prompt maps are generated as follows: the positive sample prompt map is taken from the largest connected area in the true value of the label that is not covered by the previous segmentation; the negative sample prompt map is taken from the largest misdetected area in the previous segmentation result that exceeds the true value of the label, and the maximum area unit is pixel.

[0010] Furthermore, in step S3, the fusion method of the progressive iterative training interactive prompt remote sensing change detection model is as follows: the positive sample prompt map, the negative sample prompt map and the previous segmentation result map are respectively used as independent channels, and the three-channel bands are superimposed to form a multi-band data input.

[0011] Furthermore, in step S3, the interactive prompt remote sensing change detection model includes a segmentation backbone network and an optimization module.

[0012] Furthermore, the segmentation backbone network adopts any one of SegNeXt, ResNet, ResNeXt, SegFormer or SwinTransformer, and the optimization module adopts a large-kernel convolutional layer for feature extraction.

[0013] Furthermore, the sample set types in step S1 include building change, farmland change or forest change detection datasets, and in applications, the change sample set is any other change category.

[0014] Furthermore, the boundary line of the line drawing interaction is generated by extracting the boundary of the change area through the Canny operator and then performing smoothing processing.

[0015] Furthermore, the maximum connected region is the region with the largest area in the binary graph.

[0016] In a second aspect, the present invention further provides a multi-temporal remote sensing image change detection system based on interactive prompts, which is used to implement the multi-temporal remote sensing image change detection method based on interactive prompts, including a sample construction module, an interactive simulation module, an iterative training module, and a detection execution module; The sample construction module is used to obtain paired multi-temporal remote sensing images and corresponding binary change label maps to form a sample set including the previous phase image, the next phase image and the label map; The interactive simulation module is used to simulate the generation of positive sample prompt images and negative sample prompt images using three interactive methods: clicking, box selection, and line drawing; The iterative training module is configured to input the positive sample prompt image and the negative sample image generated by simulation in step S2 and the sample set including the previous phase image, the next phase image and the label image in step S1 into the interactive prompt remote sensing change detection model during the first iteration; in subsequent iterations, input the currently generated positive and negative sample prompt images and the previous segmentation result image into the interactive prompt remote sensing change detection model; and repeat the iterative steps at least three times to optimize the parameters of the interactive prompt remote sensing change detection model; The detection execution module is used to input the multi-temporal remote sensing image to be detected into the trained interactive prompt remote sensing change detection model, and output a change detection result graph in combination with the user's real-time interactive prompt.

[0017] Beneficial effects of the present invention: 1. This invention achieves a significant reduction in manual annotation costs and a breakthrough improvement in detection accuracy for complex scenes by integrating computer-simulated interactive prompts for clicking, selecting boxes, and drawing lines, combined with a progressive iterative training mechanism. During the training phase, interactive prompts are automatically generated to replace manual annotation, requiring only a small amount of real-world interaction to construct a high-quality sample set. After at least three iterative training sessions, historical segmentation results are combined with positive and negative sample prompts in a three-channel band superposition, enabling the model to autonomously identify and correct missed or misdetected areas.

[0018] 2. Furthermore, for irregular targets such as building outlines and farmland boundaries, the interactive line drawing method uses multimodal guidance methods such as boundary lines, central axes, and Bezier curves, combined with a dynamic selection strategy for the largest connected area, to effectively improve the recognition accuracy of complex and changing targets. This invention can adapt to the change analysis of non-fixed types of targets and expand the scope of application of remote sensing change detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0020] Figure 1 The present invention is a flowchart of a method for detecting changes in multi-temporal remote sensing images based on interactive prompts.

[0021] Figure 2 The diagram is a structural diagram of a multi-temporal remote sensing image change detection system based on interactive prompts according to the present invention.

[0022] Markings in the figure: 200 is a sample construction module, 201 is an interactive simulation module, 202 is an iterative training module, 203 is a detection execution module, and 204 is a multi-temporal remote sensing image change detection system based on interactive prompts. DETAILED DESCRIPTION

[0023] In order to make the technical problems, technical solutions and beneficial effects solved by the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0024] like Figure 1 As shown, an embodiment of the present invention provides a method for detecting changes in multi-temporal remote sensing images based on interactive prompts, comprising the following steps: S1. Constructing a training sample set: Obtain paired multi-temporal remote sensing images and corresponding binary change label maps to form a sample set containing pre-temporal images, post-temporal images, and label maps; The sample set type in step S1 includes a building change, cultivated land change or forest change detection dataset. The sample set type in step S1 includes a building change, cultivated land change or forest change detection dataset; Among them, the construction of the sample set mainly uses the Forest Communication desktop GIS software to mark the change targets of interest in the two phases of remote sensing images, and generate sample data sets that meet the change detection tasks, such as: building change detection data sets, cultivated land change detection data sets, forest change detection data sets or other types of change detection data sets, etc. The main format is the before and after two phase images, plus a binary label map. Each sample has a total of three image maps as the input of the interactive prompt remote sensing change detection model.

[0025] S2. Computer-simulated interactive prompts: Using click, box selection, and line drawing to simulate the generation of positive and negative sample prompt images. By using click, box selection, and line drawing to achieve computer-simulated human interaction effects, it facilitates the gradual iterative training of the interactive prompt remote sensing change detection model. In the specific interactive prompt remote sensing change detection model training process, it is necessary to use algorithm simulation to realize the human-computer interaction process. This is because relying entirely on manual training of the interactive prompt remote sensing change detection model requires a huge investment of manpower and material resources. In order to accurately simulate the human-computer interaction mode, the present invention constructs three interaction modes: click, box selection, and line drawing to simulate human-computer interaction. Click, box selection, and line drawing can simulate the human-computer interaction process in most cases. In step S2, the simulation method of click interaction is: generate a circular prompt area with the specified coordinate point as the center, and simulate by buffering the coordinate point by a certain number of pixels, generally 5 pixels; the simulation method of frame selection interaction is: use the foreground value to fill the interior of the box, and fill the interior of the rectangular box with the foreground value to generate a prompt image; the simulation method of line drawing interaction includes: the boundary line extracted and smoothed by the edge detection operator, the center axis extracted by the medial axis transformation, and the Bezier curve based on the smoothing of the center axis. Preferably, the boundary line of the line drawing interaction is generated by extracting the boundary of the change area by the Canny operator and then smoothing it; specifically, the center axis is the center axis of the binary change area extracted by the medial axis transformation method in computer vision; specifically, the Bezier curve first extracts the center axis of the binary change area, and then obtains the smoothed center axis by smoothing and removing burrs.

[0026] S3. Progressively iteratively train the interactive prompt remote sensing change detection model: In the first iteration, the positive sample prompt map and negative sample prompt map generated by the simulation in step S2 and the sample set containing the previous phase image, the next phase image, and the label map in step S1 are input into the interactive prompt remote sensing change detection model; in subsequent iterations, the currently generated positive and negative sample prompt maps and the previous segmentation result map are input into the interactive prompt remote sensing change detection model; repeat the iterative steps at least three times to optimize the parameters of the interactive prompt remote sensing change detection model; Specifically, in step S3, the positive and negative sample prompt maps are generated in the following manner: the positive sample prompt map is taken from the largest connected area in the true value of the label that is not covered by the previous segmentation; the negative sample prompt map is taken from the largest misdetected area in the previous segmentation result that exceeds the true value of the label; the criterion for determining the maximum connected area is the area of ​​the change region whose area ratio exceeds the preset threshold; for any group of samples, in the first interaction process, only positive sample simulation is performed, and the negative sample and the previous segmentation result are replaced by a 0-value map; in the second and subsequent interaction processes, positive and negative sample prompt maps are generated respectively. Among them, the formation process of the positive sample prompt map is mainly: take the automatically extracted result and the missing part of the true value of the label, use any one of the three interactive prompting methods to simulate the area with the largest missing area, and obtain the positive sample prompt map; the formation process of the negative sample prompt map is mainly: take the automatically extracted result and the wrongly extracted part of the true value of the label, use any one of the three prompting methods to simulate the wrongly extracted part, and obtain the negative sample prompt map. Specifically, in step S3, the fusion method of the progressive iterative training interactive prompt remote sensing change detection model is as follows: the positive sample prompt image, the negative sample prompt image and the previous segmentation result image are respectively used as independent channels, and the three-channel bands are superimposed to form fusion input data; Specifically, in step S3, the interactive prompt remote sensing change detection model includes a segmentation backbone network and an optimization module; preferably, the segmentation backbone network adopts any one of SegNeXt, ResNet, ResNext or SegFormer networks, and may also adopt other modern backbone neural networks such as Swin Transformer, and the segmentation backbone network has good versatility; the optimization module adopts a large-core convolutional layer for feature extraction, which can improve the interactive segmentation accuracy; The progressive iterative training method in step S3 can integrate the previous segmentation result image into the interactive prompt, and can gradually optimize the segmentation result.

[0027] S4. Execute change detection: Input the multi-temporal remote sensing image to be detected into the trained interactive prompt remote sensing change detection model, and output the change detection result graph in combination with the user's real-time interactive prompt.

[0028] like Figure 2 As shown, an embodiment of the present invention provides a multi-temporal remote sensing image change detection system based on interactive prompts, which is used to implement the multi-temporal remote sensing image change detection method based on interactive prompts. The multi-temporal remote sensing image change detection system based on interactive prompts 204 includes a sample construction module 200, an interactive simulation module 201, an iterative training module 202, and a detection execution module 203; The sample construction module 200 is used to obtain paired multi-temporal remote sensing images and corresponding binary change label maps to form a sample set including the previous phase image, the next phase image and the label map; The interactive simulation module 201 is used to simulate and generate positive sample prompt images and negative sample prompt images by using three interactive methods: clicking, box selection, and line drawing; The iterative training module 202 is configured to input the positive sample prompt image and the negative sample image generated by simulation in step S2 and the sample set including the previous phase image, the next phase image and the label image in step S1 into the interactive prompt remote sensing change detection model during the first iteration; in subsequent iterations, input the currently generated positive and negative sample prompt images and the previous segmentation result image into the interactive prompt remote sensing change detection model; and repeat the iterative steps at least three times to optimize the parameters of the interactive prompt remote sensing change detection model. The detection execution module 203 is used to input the multi-temporal remote sensing image to be detected into the trained interactive prompt remote sensing change detection model, and output a change detection result graph in combination with the user's real-time interactive prompt.

[0029] In summary, the interactive prompting model of the present invention, which undergoes progressive iterative training, is versatile and can effectively perform interactive segmentation across different tasks. It also features a simple structure and good transferability. The interactive segmentation of the present invention utilizes a segmentation backbone network and a pure convolutional architecture, which supports domestic deep learning frameworks and has the advantage of operating in a secure environment.

[0030] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "illustrative embodiments," "preferred embodiments," "specific implementations," or "preferred implementations" means that the specific features, structures, materials, or characteristics described in conjunction with that embodiment or example are included in at least one embodiment or example of the present invention. In this specification, illustrative uses of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0031] It can be understood that the above specific description of the present invention is only used to illustrate the present invention and is not limited to the technical solutions described in the embodiments of the present invention. Those skilled in the art should understand that the present invention can still be modified or replaced by equivalents to achieve the same technical effects; as long as the use requirements are met, they are within the scope of protection of the present invention.

Claims

1. A multi-temporal remote sensing image change detection method based on interactive prompts, characterized by: The following steps are included: S1. Constructing a training sample set: Obtain paired multi-temporal remote sensing images and corresponding binary change label maps to form a sample set containing pre-temporal images, post-temporal images, and label maps; S2. Computer-simulated interactive prompts: Using click, box selection, and line drawing, the system simulates the generation of positive and negative sample prompts. S3. Progressively iteratively train the interactive prompt remote sensing change detection model: In the first iteration, the positive sample prompt map and negative sample prompt map generated by the simulation in step S2 and the sample set containing the previous phase image, the next phase image, and the label map in step S1 are input into the interactive prompt remote sensing change detection model; in subsequent iterations, the currently generated positive and negative sample prompt maps and the previous segmentation result map are input into the interactive prompt remote sensing change detection model together; Repeat the iterative steps at least three times to optimize the parameters of the interactive prompt remote sensing change detection model; S4. Execute change detection: Input the multi-temporal remote sensing image to be detected into the trained interactive prompt remote sensing change detection model, and output the change detection result graph in combination with the user's real-time interactive prompt.

2. The method for detecting changes in multi-temporal remote sensing images based on interactive prompts according to claim 1, characterized in that: In step S2, the simulation method of click interaction is: generating a circular prompt area with the specified coordinate point as the center; the simulation method of frame selection interaction is: filling the interior of the rectangular box with the foreground value to generate a prompt image; the simulation method of line drawing interaction includes: boundary lines extracted and smoothed by edge detection operators, central axes extracted by central axis transformation, and Bezier curves based on central axis smoothing.

3. The method for detecting changes in multi-temporal remote sensing images based on interactive prompts according to claim 1, characterized in that: In step S3, the positive and negative sample prompt maps are generated as follows: the positive sample prompt map is taken from the largest connected area in the true value of the label that is not covered by the previous segmentation; the negative sample prompt map is taken from the largest misdetected area in the previous segmentation result that exceeds the true value of the label, and the maximum area unit is pixel.

4. The method for detecting changes in multi-temporal remote sensing images based on interactive prompting according to claim 1, characterized in that: In step S3, the fusion method of the progressive iterative training interactive prompt remote sensing change detection model is as follows: the positive sample prompt map, the negative sample prompt map and the previous segmentation result map are respectively used as independent channels, and the three-channel bands are superimposed to form a multi-band data input.

5. The method for detecting changes in multi-temporal remote sensing images based on interactive prompting according to claim 1, characterized in that: In step S3, the interactive prompt remote sensing change detection model includes a segmentation backbone network and an optimization module.

6. The method for detecting changes in multi-temporal remote sensing images based on interactive prompts according to claim 5, characterized in that: The segmentation backbone network adopts any one of SegNeXt, ResNet, ResNeXt, SegFormer or Swin Transformer, and the optimization module adopts a large-core convolutional layer for feature extraction.

7. The method for detecting changes in multi-temporal remote sensing images based on interactive prompting according to claim 1, characterized in that: The sample set types in step S1 include building change, farmland change or forest change detection datasets. In the application, the change sample set is any other change category.

8. The method for detecting changes in multi-temporal remote sensing images based on interactive prompts according to claim 2, characterized in that: The boundary line of the line drawing interaction is generated by extracting the boundary of the change area through the Canny operator and then performing smoothing processing.

9. The method for detecting changes in multi-temporal remote sensing images based on interactive prompts according to claim 3, characterized in that: The maximum connected region is the region with the largest area in the binary graph.

10. A multi-temporal remote sensing image change detection system based on interactive prompts, characterized by: Used to implement the multi-temporal remote sensing image change detection method based on interactive prompting as described in claim 1, comprising a sample construction module, an interactive simulation module, an iterative training module, and a detection execution module; The sample construction module is used to obtain paired multi-temporal remote sensing images and corresponding binary change label maps to form a sample set including the previous phase image, the next phase image and the label map; The interactive simulation module is used to simulate the generation of positive sample prompt images and negative sample prompt images using three interactive methods: clicking, box selection, and line drawing; The iterative training module is used to input the positive sample prompt map and negative sample map generated by simulation in step S2 and the sample set including the previous phase image, the next phase image and the label map in step S1 into the interactive prompt remote sensing change detection model during the first iteration; in subsequent iterations, the currently generated positive and negative sample prompt map and the previous segmentation result map are input into the interactive prompt remote sensing change detection model together; Repeat the iterative steps at least three times to optimize the parameters of the interactive prompt remote sensing change detection model; The detection execution module is used to input the multi-temporal remote sensing image to be detected into the trained interactive prompt remote sensing change detection model, and output a change detection result graph in combination with the user's real-time interactive prompt.

Images