Change Detection Annotation Method, Device, System and Storage Medium for Remote Sensing Images

Abstract

An embodiment of the present invention discloses a method, apparatus, system and storage medium for change detection annotation of remote sensing images. The method includes: obtaining two remote sensing images corresponding to a target scene at different time phases; determining a plurality of sub-regions with changes between the two remote sensing images and the corresponding outer contours of each sub-region; clustering the outer contours of the plurality of sub-regions to obtain a plurality of clustering regions, where one clustering region includes the outer contours of one or more sub-regions, and the sub-regions included in any two clustering regions are different from each other; according to the outer contour position information of each sub-region included in the first clustering region, performing outer contour annotation of each sub-region in the two remote sensing images, and controlling a display device to display the images of the annotated parts of the two remote sensing images. The method for change detection annotation of remote sensing images of the present invention solves the problem that it is inconvenient to perform change detection annotation on remote sensing images in the prior art.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing, and in particular, to a method, device, system and storage medium for change detection annotation of remote sensing images. Background Art

[0002] Remote sensing refers to a non-contact and long-distance detection technology, generally referring to the use of sensors / remote sensors to detect the electromagnetic wave radiation and reflection characteristics of target objects.

[0003] Remote sensing images have important applications in the fields of natural resource detection, geological disaster warning, landform change detection, urban planning, etc. The change detection annotation of remote sensing images plays a significant role in remote sensing change tasks and determines the performance upper limit of the change detection model. However, for remote sensing change detection annotation, it is necessary to manually compare two-phase remote sensing data with the naked eye, compare pixel by pixel and mark the areas where changes exist. Different application scenarios require professional engineers in different fields to analyze remote sensing images. This process consumes a large amount of manpower and has low efficiency.

[0004] Therefore, in the prior art, there is a problem that it is inconvenient to perform change detection annotation on remote sensing images. For the above problems, no effective solution has been proposed yet.

[0005] The above information disclosed in the background art section is only used to enhance the understanding of the background art of the technology described in this article. Therefore, the background art may contain certain information that is not known to those skilled in the art as the existing prior art. Summary of the Invention

[0006] Embodiments of the present invention provide a method, device, system and storage medium for change detection annotation of remote sensing images, so as to at least solve the problem that it is inconvenient to perform change detection annotation on remote sensing images in the prior art.

[0007] According to one aspect of the embodiments of the present invention, a method for change detection annotation of remote sensing images is provided, which includes: obtaining two remote sensing images corresponding to a target scene at different time phases; determining a plurality of sub-regions where changes exist between the two remote sensing images and the outer contour corresponding to each sub-region; clustering the outer contours of the plurality of sub-regions to obtain a plurality of clustering regions, where one clustering region includes the outer contours of one or more sub-regions, and the sub-regions included in any two clustering regions are different from each other; according to the outer contour position information of each sub-region included in the first clustering region, performing outer contour annotation of each sub-region in the two remote sensing images, and controlling a display device to display the images of the marked parts of the two remote sensing images, where the first clustering region includes one, more or all of the plurality of clustering regions.

[0008] Optionally, clustering the outer contours of multiple sub-regions to obtain multiple clustering regions, including: determining the circumscribed horizontal rectangles corresponding to the outer contours of each sub-region; determining a first target circumscribed horizontal rectangle from the circumscribed horizontal rectangles, where the size of the first target circumscribed horizontal rectangle is greater than or equal to a first preset size; determining the region corresponding to one first target circumscribed horizontal rectangle as one clustering region.

[0009] Optionally, clustering the outer contours of multiple sub-regions to obtain multiple clustering regions, further including: determining a second target circumscribed horizontal rectangle from the circumscribed horizontal rectangles, where the size of the second target circumscribed horizontal rectangle is less than the first preset size; determining multiple target horizontal rectangles, with one target horizontal rectangle surrounding multiple second target circumscribed horizontal rectangles; determining the region corresponding to one target horizontal rectangle as one clustering region.

[0010] Optionally, determining multiple target horizontal rectangles, with one target horizontal rectangle surrounding multiple second target circumscribed horizontal rectangles, including: determining multiple reference points, with the multiple reference points corresponding one by one to the multiple second target circumscribed horizontal rectangles; performing K-means clustering processing on the multiple reference points, where the value of K traverses from 1 to the total number of the second target circumscribed horizontal rectangles, so that all the second target circumscribed horizontal rectangles corresponding to the reference points belonging to each class are included within the range of a corresponding second preset size; determining each target horizontal rectangle as the smallest horizontal rectangle that includes all the second target circumscribed horizontal rectangles corresponding to the reference points of the corresponding class.

[0011] Optionally, determining multiple sub-regions where there are changes between two remote sensing images and the outer contour corresponding to each sub-region, including: inputting the two remote sensing images into a pre-trained change detection segmentation model to obtain a change detection binary map representing the difference between the two remote sensing images; using a connected region analysis algorithm to analyze the change detection binary map to obtain the outer contours of each connected region.

[0012] Optionally, after annotating the outer contours of each sub-region in the two remote sensing images according to the outer contour position information included in the first clustering region and controlling the display device to display the images of the annotated parts of the two remote sensing images, the change detection annotation method for remote sensing images further includes: receiving contour modification information for the outer contour annotation, where the contour modification information corresponds to at least one of the following operations: deleting at least one outer contour, adjusting at least one outer contour, adding an outer contour; modifying the change detection binary map according to the contour modification information.

[0013] Optionally, obtaining two remote sensing images corresponding to a target scene at different time phases includes: receiving two overall remote sensing maps at different time phases; obtaining multiple pairs of feature points corresponding on the two overall remote sensing maps; aligning the two overall remote sensing maps according to the multiple pairs of feature points; and performing grid division on the two aligned overall remote sensing maps to obtain multiple remote sensing image pairs, where each remote sensing image pair includes two remote sensing images at different time phases.

[0014] Optionally, according to the outer contour position information of each sub-region included in the first clustering region, performing outer contour annotation of each sub-region in the two remote sensing images, and controlling a display device to display the images of the annotated parts of the two remote sensing images, includes: during the process of controlling the display device to display the images of the annotated parts of the two remote sensing images, controlling to perform mask processing on the target regions of the two remote sensing images, where the target region is the region where the first clustering region overlaps with the sub-regions belonging to the second clustering region, and the first clustering region and the second clustering region are different clustering regions.

[0015] According to another aspect of the embodiments of the present invention, there is also provided a change detection annotation device for remote sensing images, including: an acquisition unit for obtaining two remote sensing images corresponding to a target scene at different time phases; a determination unit for determining multiple sub-regions where changes exist between the two remote sensing images and the outer contour corresponding to each sub-region; a clustering unit for clustering the outer contours of the multiple sub-regions to obtain multiple clustering regions, where one clustering region includes the outer contours of one or more sub-regions, and the sub-regions included in any two clustering regions are different from each other; a control unit for performing outer contour annotation of each sub-region in the two remote sensing images according to the outer contour position information of each sub-region included in the first clustering region, and controlling a display device to display the images of the annotated parts of the two remote sensing images, where the first clustering region includes one, multiple, or all of the multiple clustering regions.

[0016] Optionally, the clustering unit includes: a first determination module for determining the circumscribed horizontal rectangles corresponding to the outer contours of each sub-region; a second determination module for determining a first target circumscribed horizontal rectangle from the circumscribed horizontal rectangles, wherein the size of the first target circumscribed horizontal rectangle is greater than or equal to a first preset size; a third determination module for determining the region corresponding to one first target circumscribed horizontal rectangle as a clustering region; the clustering unit includes: a fourth determination module for determining a second target circumscribed horizontal rectangle from the circumscribed horizontal rectangles, wherein the size of the second target circumscribed horizontal rectangle is less than the first preset size; a fifth determination module for determining a plurality of target horizontal rectangles, with one target horizontal rectangle surrounding a plurality of second target circumscribed horizontal rectangles; a sixth determination module for determining the region corresponding to one target horizontal rectangle as a clustering region; the fifth determination module includes: a first determination sub-module for determining a plurality of reference points, with the plurality of reference points corresponding one-to-one to the plurality of second target circumscribed horizontal rectangles; a clustering sub-module for performing K-means clustering processing on the plurality of reference points, with the value of K traversing from 1 to the total number of the second target circumscribed horizontal rectangles, so that all the second target circumscribed horizontal rectangles corresponding to the reference points belonging to each class are included within the range of a corresponding second preset size; a second determination sub-module for determining each target horizontal rectangle as the smallest horizontal rectangle containing all the second target circumscribed horizontal rectangles corresponding to the reference points of the corresponding class; the determination unit includes: an input module for inputting two remote sensing images into a pre-trained change detection segmentation model to obtain a change detection binary map representing the difference between the two remote sensing images; an analysis module for analyzing the change detection binary map using a connected region analysis algorithm to obtain the outer contours of each connected region; the change detection annotation device for remote sensing images further includes: a receiving unit for, after performing outer contour annotation of each sub-region in the two remote sensing images according to the outer contour position information of each sub-region included in the first clustering region and controlling the display device to display the images of the annotated parts of the two remote sensing images, receiving contour modification information for the outer contour annotation, where the contour modification information corresponds to at least one of the following operations: deleting at least one outer contour, adjusting at least one outer contour, adding an outer contour; a modification unit for modifying the change detection binary map according to the contour modification information; the acquisition unit includes: a receiving module for receiving two overall remote sensing maps at different times; an acquisition module for acquiring a plurality of pairs of corresponding feature points on the two overall remote sensing maps; an alignment module for aligning the two overall remote sensing maps according to the plurality of pairs of feature points; a division module for dividing the two aligned overall remote sensing maps into a grid to obtain a plurality of pairs of remote sensing images, with each pair of remote sensing images including two remote sensing images at different times;The clustering unit includes a control module, which is configured to control masking of the target regions of two remote sensing images during the process of controlling the display device to display the images of the labeled parts of the two remote sensing images. The target region is the region where the first clustering region overlaps with the sub-region belonging to the second clustering region, and the first clustering region and the second clustering region are different clustering regions.;

[0017] An embodiment of the present invention further provides a non-volatile storage medium, which includes a stored program. When the program runs, it controls the device where the non-volatile storage medium is located to execute the above-mentioned change detection and annotation method for remote sensing images.

[0018] An embodiment of the present invention further provides a processor, which is used to run a program. When the program runs, it executes the above-mentioned change detection and annotation method for remote sensing images.

[0019] An embodiment of the present invention further provides a change detection and annotation device for remote sensing images, which includes a display device, a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the above-mentioned change detection and annotation method for remote sensing images.

[0020] The change detection annotation method for remote sensing images in the embodiments of the present invention includes: obtaining two remote sensing images corresponding to a target scene at different time phases; determining multiple sub-regions where changes exist between the two remote sensing images and the outer contour corresponding to each sub-region; clustering the outer contours of the multiple sub-regions to obtain multiple clustering regions, where one clustering region includes the outer contours of one or more sub-regions, and the sub-regions included in any two clustering regions are different from each other; according to the outer contour position information of each sub-region included in the first clustering region, performing outer contour annotation of each sub-region in the two remote sensing images, and controlling a display device to display the images of the annotated parts of the two remote sensing images, where the first clustering region includes one, multiple, or all of the multiple clustering regions. After obtaining two remote sensing images (i.e., an image pair) corresponding to a target scene at different time phases, through comparative analysis, it is possible to determine the sub-regions where changes exist between the two remote sensing images, and the outer contours of these sub-regions where changes exist can be determined. By clustering these outer contours, multiple clustering regions are formed, and each clustering region includes at least one outer contour. Then, according to the outer contour position information of each sub-region included in the first clustering region, outer contour annotation of each sub-region is performed in the two remote sensing images, and a display device is controlled to display the images of the annotated parts of the two remote sensing images. In this way, the change detection and annotation of remote sensing images are realized, and at the same time, the fragmented outer contours are clustered and then centrally displayed, which is convenient for operators to check or modify the annotation content later, helps improve the inspection efficiency of the annotation content and reduces the risk of omission of some fragmented outer contours during the inspection process, effectively improves the annotation processing speed, reduces the labor cost, and solves the problem that it is inconvenient to perform change detection annotation on remote sensing images in the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation to the present invention. In the drawings:

[0022] Figure 1 is a schematic flowchart of an optional embodiment of the change detection annotation method for remote sensing images according to the present invention;

[0023] Figure 2 is a schematic diagram of an optional embodiment of the change detection annotation device for remote sensing images according to the present invention;

[0024] Figure 3 is a schematic diagram of obtaining a change detection binary map based on two remote sensing images at different time phases by the change detection annotation method for remote sensing images according to the present invention;

[0025] Figure 4It is a schematic diagram of a change detection binary map obtained by the change detection annotation method of remote sensing images according to the present invention;

[0026] Figure 5 It is a schematic diagram of determining the circumscribed horizontal rectangles corresponding to each outer contour according to the change detection annotation method of remote sensing images of the present invention;

[0027] Figure 6 It is a schematic diagram of clustering multiple outer contours according to the change detection annotation method of remote sensing images of the present invention to obtain multiple clustering regions;

[0028] Figure 7 It is a schematic diagram of controlling a display device to display the parts corresponding to a clustering region of two remote sensing images and display the corresponding outer contours according to the change detection annotation method of remote sensing images of the present invention. Detailed implementation manners

[0029] In order to enable those skilled in the art to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0030] It should be noted that the terms "first", "second", etc. in the specification, claims and drawings of the present invention are used to distinguish different objects, rather than to limit a specific order.

[0031] Figure 1 It is a change detection annotation method for remote sensing images according to an embodiment of the present invention. As Figure 1 shown, the method includes the following steps:

[0032] Step S102, obtaining two remote sensing images corresponding to a target scene at different time phases;

[0033] Step S104, determining multiple sub-regions where changes exist between the two remote sensing images and the outer contour corresponding to each sub-region;

[0034] Step S106, clustering the outer contours of the multiple sub-regions to obtain multiple clustering regions. One clustering region includes the outer contours of one or more sub-regions, and the sub-regions included in any two clustering regions are different from each other;

[0035] Step S108: Based on the outer contour position information of each sub-region included in the first clustering region, perform outer contour annotation of each sub-region in the two remote sensing images, and control the display device to display the images of the annotated parts of the two remote sensing images, where the first clustering region includes one, multiple, or all of the multiple clustering regions.

[0036] After obtaining two remote sensing images (i.e., an image pair) corresponding to a target scene at different time phases by using the above-described method for change detection and annotation of remote sensing images, through comparative analysis, it is possible to determine the sub-regions where changes exist between the two remote sensing images, and the outer contours of these sub-regions where changes exist can be determined. By clustering these outer contours, multiple clustering regions are formed, and each clustering region includes at least one outer contour. Then, based on the outer contour position information of each sub-region included in the first clustering region, perform outer contour annotation of each sub-region in the two remote sensing images, and control the display device to display the images of the annotated parts of the two remote sensing images. In this way, change detection and annotation of remote sensing images are achieved, and at the same time, the fragmented outer contours are clustered and then centrally displayed, which is convenient for operators to subsequently check or modify the annotated content, helps improve the inspection efficiency of the annotated content and reduces the risk of omission of some fragmented outer contours during the inspection process, effectively improves the annotation processing speed, reduces the labor cost, and solves the problem that it is inconvenient to perform change detection and annotation on remote sensing images in the prior art. The schematic diagram of the final display device displaying the annotated parts of the two remote sensing images is as Figure 7 shown, Figure 7 which shows the first clustering region and the annotation contours of the image pair. These contours represent the regions where changes exist in the identified image pair, which can effectively facilitate subsequent inspection or modification of the annotated content by operators and improve the annotation processing speed. Of course, the display device can display only two remote sensing images at a time, or multiple remote sensing images at a time. During the process of clustering multiple outer contours, the clustering basis can be in various forms. For example, clustering can be performed according to distance, according to the size of the outer contour, according to the type of image elements corresponding to the outer contour, and so on.

[0037] In this embodiment, clustering the outer contours of multiple sub-regions to obtain multiple clustering regions includes: determining the circumscribed horizontal rectangles corresponding to the outer contours of each sub-region, as Figure 5 shown; determining a first target circumscribed horizontal rectangle from the circumscribed horizontal rectangles, where the size of the first target circumscribed horizontal rectangle is greater than or equal to a first preset size; determining the region corresponding to one first target circumscribed horizontal rectangle as one clustering region, for example Figure 6 the 1 region and the 2 region in.

[0038] In the process of clustering the outer contours, by circumscribing a horizontal rectangle for each outer contour, it is more convenient to evaluate the approximate size of each outer contour. If the size of the circumscribed horizontal rectangle of a single outer contour is greater than or equal to the first preset size, it indicates that the size of this outer contour is also relatively large. At this time, the circumscribed horizontal rectangle of this outer contour is determined as the first target circumscribed horizontal rectangle, and the corresponding area is separately used as a clustering area. Furthermore, the corresponding parts of the two remote sensing images to this clustering area are separately marked and displayed, which is convenient for subsequent operators to check the change detection marking results.

[0039] On this basis, clustering the outer contours of multiple sub-regions to obtain multiple clustering areas further includes: determining a second target circumscribed horizontal rectangle from the circumscribed horizontal rectangles, where the size of the second target circumscribed horizontal rectangle is less than the first preset size; determining multiple target horizontal rectangles, with one target horizontal rectangle surrounding multiple second target circumscribed horizontal rectangles; determining the area corresponding to one target horizontal rectangle as a clustering area, such as Figure 6 Region 3 and Region 4 in

[0040] For small rectangles whose circumscribed horizontal rectangle size is less than the first preset size, they will be determined as the second target circumscribed horizontal rectangles. In the process of clustering these second target circumscribed horizontal rectangles, multiple target horizontal rectangles will be determined such that each target horizontal rectangle surrounds multiple small rectangles (the second target circumscribed horizontal rectangles), and the area corresponding to each target horizontal rectangle is separately used as a clustering area. In this way, subsequently, the display device will be controlled to separately display the corresponding parts of the two remote sensing images to this clustering area, so as to cluster and centrally display the outer contours of these smaller regions with changes. In this way, the centralized display of smaller fragmented regions is achieved, which is beneficial for subsequent inspection of the marking results.

[0041] In specific implementation, the measurement criteria for the first preset size can be various. For example, it can be measured by the area size, by the length size, by the width size, and so on. In a specific embodiment, the length of the maximum side of the circumscribed horizontal rectangle is used for measurement. If the length of the maximum side of the circumscribed horizontal rectangle, that is, max_len, is greater than or equal to a preset value (such as 256 pixels), it indicates that the size of this circumscribed horizontal rectangle is greater than or equal to the first preset size, and then it is determined as the first target circumscribed horizontal rectangle. If the length of the maximum side of the circumscribed horizontal rectangle, that is, max_len, is less than this preset value (such as 256 pixels), it indicates that the size of this circumscribed horizontal rectangle is less than the first preset size, and then it is determined as the second target circumscribed horizontal rectangle.

[0042] In this embodiment, determining a plurality of target horizontal rectangles, with one target horizontal rectangle surrounding a plurality of second target circumscribed horizontal rectangles, includes: determining a plurality of reference points, where the plurality of reference points correspond one-to-one to the plurality of second target circumscribed horizontal rectangles; performing K-means clustering on the plurality of reference points, with the value of K traversing from 1 to the total number of second target circumscribed horizontal rectangles, so that all the second target circumscribed horizontal rectangles corresponding to the reference points belonging to each class are included within the corresponding second preset size range; determining each target horizontal rectangle as the smallest horizontal rectangle that includes all the second target circumscribed horizontal rectangles corresponding to the reference points of the corresponding class.

[0043] In this embodiment, the K-means clustering processing method is used to cluster the circumscribed horizontal rectangles with sizes smaller than the first preset size according to the distances between the respective outer contours. Specifically, for each circumscribed horizontal rectangle, first select a reference point, and this reference point can be flexibly selected. For example, the center of each circumscribed horizontal rectangle can be selected as its respective reference point. During the K-means clustering process, let the value of K traverse from 1 to the total number of second target circumscribed horizontal rectangles, and cluster each reference point to the nearest clustering center until all the small rectangles in each class can be completely included within the second preset size range (within the cropped image), at which point the clustering ends. During the clustering process, the value of K takes values in ascending order from small to large, and clustering operations are continuously performed. The final clustering result is that when K takes the smallest value, all the second target circumscribed horizontal rectangles belonging to each class can be completely included within the second preset size range. That is to say, with as few clustering regions as possible, all the outer contours are included, which is conducive to more efficient and centralized display and annotation of the positions with changes in the remote sensing image. Among them, the second preset size can be flexibly determined according to the actual situation. It can be the same as the first preset size or different from the first preset size. For example, the second preset size can be a size of 512 pixels × 512 pixels. The specific selection of the first preset size and the second preset size can be considered from aspects such as the size of the display area of the display device or based on facilitating the operator's observation of the image (saving operations such as zooming in and out of the image by the operator). For the small rectangles (second target circumscribed horizontal rectangles) belonging to each class, determine a horizontal rectangle that can include all the small rectangles (second target circumscribed horizontal rectangles) in this class and ensure that the size of this horizontal rectangle is the smallest. That is to say, the four sides of this horizontal rectangle overlap with the sides of at least one small rectangle (second target circumscribed horizontal rectangle). This horizontal rectangle is the smallest circumscribed horizontal rectangle for the combination of all the small rectangles (second target circumscribed horizontal rectangles) in this class. This can more reasonably achieve the centralized display of the outer contours corresponding to each small rectangle (second target circumscribed horizontal rectangle) while ensuring that the size of the obtained clustering region is appropriate, thus facilitating subsequent annotation inspection.

[0044] There are various ways to determine multiple sub-regions with changes between two remote sensing images and the corresponding outer contours of each sub-region. In this embodiment, this step includes: inputting the two remote sensing images into a pre-trained change detection segmentation model to obtain a change detection binary map representing the difference between the two remote sensing images; using a connected component analysis algorithm to analyze the change detection binary map to obtain the outer contours of each connected component. By inputting the two remote sensing images into a pre-trained change detection segmentation model to obtain a change detection binary map representing the difference between the two remote sensing images, this binary map can clearly reflect the difference between the two remote sensing images. Then, by using a connected component analysis algorithm to analyze the change detection binary map, the outer contours of at least one connected component can be obtained, and this outer contour is the outer contour of the position where the change exists. Figure 3 FIG. is a schematic diagram of obtaining a change detection binary map based on two remote sensing images of different time phases in the change detection annotation method of remote sensing images according to the present invention. The obtained change detection binary map is as Figure 4 shown. In the process of pre-segmenting the two remote sensing images by using a change detection segmentation model to obtain a change detection binary map, by setting the confidence threshold of the change detection segmentation model, the adjustment of the model output result can be realized. In order to ensure a high recall rate, the confidence can be set relatively low.

[0045] In this embodiment, after annotating the outer contours of each sub-region in the two remote sensing images according to the outer contour position information of each sub-region included in the first clustering region and controlling the display device to display the images of the annotated parts of the two remote sensing images, the change detection annotation method of remote sensing images further includes: receiving contour modification information for the outer contour annotation, and the contour modification information corresponds to at least one of the following operations: deleting at least one outer contour, adjusting at least one outer contour, adding a new outer contour; modifying the change detection binary map according to the contour modification information.

[0046] The contour modification information for the outer contour annotation can be information input by an operator, or information generated by an electronic device in response to a user's modification operation, or even information automatically generated by the electronic device based on an algorithm. Taking the contour modification information as input by the operator as an example, after the display device displays the images of the annotated parts of the two remote sensing images, the operator can manually confirm whether the outer contour annotation for the corresponding object is accurate, and input contour modification information to perform operations such as deleting, adding, and adjusting the outer contour when the accuracy does not meet the standard. For these operations, the change detection binary map will be modified according to the corresponding contour modification information. Of course, the change detection binary map needs to be saved after modification.

[0047] Specifically, obtaining two remote sensing images corresponding to a target scene at different time phases includes: receiving two overall remote sensing maps at different time phases; obtaining multiple pairs of feature points corresponding on the two overall remote sensing maps; aligning the two overall remote sensing maps according to the multiple pairs of feature points; and performing grid division on the two aligned overall remote sensing maps to obtain multiple remote sensing image pairs, where each remote sensing image pair includes two remote sensing images at different time phases.

[0048] That is to say, after receiving two overall remote sensing maps at different time phases, in order to facilitate the change detection annotation of the very large overall remote sensing maps, the two overall remote sensing maps will be aligned and grid-divided. Alignment is to ensure the accuracy of subsequent change detection judgment. Grid division means dividing the two overall remote sensing maps into multiple small images (i.e., remote sensing images) respectively. After grid division, the remote sensing images belonging to two time phases correspond to each other pairwise, thus facilitating the subsequent change detection and annotation. During the process of aligning the two overall remote sensing maps, first obtain multiple pairs of feature points corresponding on the two overall remote sensing maps. These feature points can be arbitrarily selected as long as they can serve as a positioning reference. By using these pairs of feature points, the accurate alignment of the two overall remote sensing maps can be achieved, and then grid division can be carried out according to the actual situation.

[0049] Among them, according to the outer contour position information of each sub-region included in the first clustering region, perform the outer contour annotation of each sub-region in the two remote sensing images, and control the display device to display the images of the marked parts of the two remote sensing images, including: during the process of controlling the display device to display the images of the marked parts of the two remote sensing images, control the masking process of the target regions of the two remote sensing images, where the target region is the region where the first clustering region overlaps with the sub-regions belonging to the second clustering region, and the first clustering region and the second clustering region are different clustering regions. By masking the target regions of the two remote sensing images, the hiding of the sub-regions in the overlapping region can be achieved. The overlapping region refers to the part where the sub-regions of the second clustering region overlap with the first clustering region. In this way, the situation of repeated annotation of partially overlapping sub-regions during the annotation of different clustering regions can be avoided, ensuring the simplicity of the annotation process, thus avoiding repeated work and being beneficial to ensuring the efficiency of image change detection annotation.

[0050] Secondly, as Figure 2As shown in the figure, an embodiment of the present invention further provides a change detection annotation device for remote sensing images, which includes: an acquisition unit for acquiring two remote sensing images corresponding to a target scene at different time phases; a determination unit for determining a plurality of sub-regions where changes exist between the two remote sensing images and the outer contour corresponding to each sub-region; a clustering unit for clustering the outer contours of the plurality of sub-regions to obtain a plurality of clustering regions, where one clustering region includes the outer contours of one or more sub-regions, and the sub-regions included in any two clustering regions are different from each other; a control unit for performing outer contour annotation of each sub-region in the two remote sensing images according to the outer contour position information of each sub-region included in the first clustering region, and controlling a display device to display the images of the annotated parts of the two remote sensing images, where the first clustering region includes one, multiple or all of the plurality of clustering regions. For the change detection annotation device of remote sensing images adopting this setting method, after the acquisition unit acquires two remote sensing images (i.e., an image pair) corresponding to a target scene at different time phases, through comparative analysis, the determination unit can determine the sub-regions where changes exist between the two remote sensing images and can determine the outer contours of these sub-regions where changes exist. The clustering unit forms a plurality of clustering regions by clustering these outer contours. Each clustering region includes at least one outer contour. The control unit then performs outer contour annotation of each sub-region in the two remote sensing images according to the outer contour position information of each sub-region included in the first clustering region, and controls the display device to display the images of the annotated parts of the two remote sensing images. In this way, the change detection and annotation of remote sensing images are realized, and at the same time, the fragmented outer contours are clustered and then centrally displayed, which is convenient for the operator to check or modify the annotation content subsequently, is beneficial to improving the efficiency of checking the annotation content and reducing the risk of omission of some fragmented outer contours during the checking process, effectively improves the annotation processing speed, reduces the labor cost, and solves the problem that it is inconvenient to perform change detection annotation on remote sensing images in the prior art. The schematic diagram of the final display device displaying the annotated parts of the two remote sensing images is as Figure 7 shown Figure 7 which shows the first clustering region and the annotation contours of the image pair, and these contours represent the regions where changes exist in the identified image pair, which can effectively facilitate the operator to check or modify the annotation content subsequently and improve the annotation processing speed. Of course, the display device can display only two remote sensing images at a time, or can display multiple images at a time. During the process of clustering the plurality of outer contours, the basis for clustering can be in various forms. For example, clustering can be performed according to distance, according to the size of the outer contour, according to the type of image elements corresponding to the outer contour, and so on.

[0051] In this embodiment, the clustering unit includes: a first determination module for determining the circumscribed horizontal rectangle corresponding to the outer contour of each sub-region, as Figure 5as shown in the figure; a second determination module, configured to determine a first target circumscribed horizontal rectangle from the circumscribed horizontal rectangles, where the size of the first target circumscribed horizontal rectangle is greater than or equal to a first preset size; a third determination module, configured to determine the area corresponding to one first target circumscribed horizontal rectangle as a clustering area, for example Figure 6 Regions 1 and 2 in Figure 6 . During the process of clustering the outer contours, the first determination module can more conveniently evaluate the approximate sizes of the respective outer contours by means of the circumscribed horizontal rectangles of each outer contour. If the size of the circumscribed horizontal rectangle of a single outer contour is greater than or equal to the first preset size, it indicates that the size of this outer contour is also relatively large. At this time, the second determination module determines the circumscribed horizontal rectangle of this outer contour as the first target circumscribed horizontal rectangle. Furthermore, the third determination module takes the corresponding area as a separate clustering area, and then separately marks and displays the corresponding parts of the two remote sensing images in this clustering area, facilitating subsequent operators to check the change detection annotation results.

[0052] Specifically, the clustering unit further includes: a fourth determination module, configured to determine a second target circumscribed horizontal rectangle from the circumscribed horizontal rectangles, where the size of the second target circumscribed horizontal rectangle is less than the first preset size; a fifth determination module, configured to determine a plurality of target horizontal rectangles, with one target horizontal rectangle surrounding a plurality of second target circumscribed horizontal rectangles; a sixth determination module, configured to determine the area corresponding to one target horizontal rectangle as a clustering area, for example Figure 6 Regions 3 and 4 in Figure 6 . For small rectangles whose circumscribed horizontal rectangle sizes are less than the first preset size, the fourth determination module will determine them as second target circumscribed horizontal rectangles. During the process of clustering these second target circumscribed horizontal rectangles, the fifth determination module will determine a plurality of target horizontal rectangles such that each target horizontal rectangle surrounds a plurality of small rectangles (second target circumscribed horizontal rectangles), and the sixth determination module takes the areas corresponding to the respective target horizontal rectangles as separate clustering areas. In this way, subsequently, the display device will be controlled to separately display the corresponding parts of the two remote sensing images in this clustering area, thereby clustering and centrally displaying the outer contours of these smaller changed areas, thus achieving the centralized display of smaller fragmented areas, which is beneficial for subsequent checking of the annotation results.

[0053] In specific implementation, the measurement criteria for the first preset size can be diverse. For example, it can be measured by the size of the area, the size of the length, the size of the width, and so on. In a specific embodiment, the length of the maximum side of the circumscribed horizontal rectangle is used for measurement. If the length of the maximum side of the circumscribed horizontal rectangle, that is, max_len, is greater than or equal to a preset value (for example, 256 pixels), it indicates that the size of the circumscribed horizontal rectangle is greater than or equal to the first preset size, and then it is determined as the first target circumscribed horizontal rectangle. If the length of the maximum side of the circumscribed horizontal rectangle, that is, max_len, is less than the preset value (for example, 256 pixels), it indicates that the size of the circumscribed horizontal rectangle is less than the first preset size, and then it is determined as the second target circumscribed horizontal rectangle.

[0054] In this embodiment, the fifth determination module includes: a first determination sub-module for determining a plurality of reference points, where the plurality of reference points correspond one-to-one to a plurality of second target circumscribed horizontal rectangles; a clustering sub-module for performing K-means clustering on the plurality of reference points, with the value of K traversing from 1 to the total number of the second target circumscribed horizontal rectangles, so that all the second target circumscribed horizontal rectangles corresponding to the reference points belonging to each class are included within the corresponding second preset size range; a second determination sub-module for determining each target horizontal rectangle as the smallest horizontal rectangle that includes all the second target circumscribed horizontal rectangles corresponding to the reference points of the corresponding class. In this embodiment, the K-means clustering processing method is adopted to cluster the circumscribed horizontal rectangles with sizes smaller than the first preset size according to the distances between the respective outer contours. Specifically, for each circumscribed horizontal rectangle, the first determination sub-module first selects a reference point, and this reference point can be flexibly selected. For example, the center of each circumscribed horizontal rectangle can be selected as its respective reference point. During the K-means clustering process of the clustering sub-module, the value of K traverses from 1 to the total number of the second target circumscribed horizontal rectangles, clustering each reference point to the nearest clustering center until all the small rectangles of each class can be completely included within the second preset size range (within the cropped image), at which point the clustering ends. During the clustering process, the value of K takes values in ascending order from small to large, continuously performing clustering operations. The final clustering result is that when K takes the smallest value, all the second target circumscribed horizontal rectangles belonging to each class can be completely included within the second preset size range. That is to say, with as few clustering regions as possible, all the outer contours are included, which is conducive to more efficiently and centrally displaying and annotating the positions with changes in the remote sensing image. Among them, the second preset size can be flexibly determined according to the actual situation. It can be the same as the first preset size or different from the first preset size. For example, the second preset size can be a size of 512 pixels × 512 pixels. The specific selection of the first preset size and the second preset size can be considered from aspects such as the size of the display area of the display device or based on facilitating the operator's observation of the image (saving operations such as zooming in and out of the image by the operator). For the small rectangles (second target circumscribed horizontal rectangles) belonging to each class, a horizontal rectangle is determined. This horizontal rectangle can include all the small rectangles (second target circumscribed horizontal rectangles) under this class and ensure that the size of this horizontal rectangle is the smallest. That is to say, the four sides of this horizontal rectangle overlap with the sides of at least one small rectangle (second target circumscribed horizontal rectangle). This horizontal rectangle is the smallest circumscribed horizontal rectangle for the combination of all the small rectangles (second target circumscribed horizontal rectangles) under this class. This can more reasonably achieve the centralized display of the outer contours corresponding to each small rectangle (second target circumscribed horizontal rectangle) while ensuring that the size of the obtained clustering region is appropriate, thus facilitating subsequent annotation inspection.

[0055] In specific implementation, the determination unit can have various forms. For example, the determination unit includes: an input module for inputting two remote sensing images into a pre-trained change detection segmentation model to obtain a change detection binary map representing the difference between the two remote sensing images; an analysis module for analyzing the change detection binary map using a connected component analysis algorithm to obtain the outer contours of each connected component. By inputting two remote sensing images into a pre-trained change detection segmentation model, the input module obtains a change detection binary map representing the difference between the two remote sensing images. This binary map can clearly reflect the difference between the two remote sensing images. Then, the analysis module analyzes the change detection binary map using a connected component analysis algorithm to obtain the outer contours of at least one connected component, and this outer contour is the outer contour of the position where the change exists. Figure 3 is a schematic diagram of obtaining a change detection binary map based on two remote sensing images with different time phases in the change detection annotation method of remote sensing images according to the present invention. The obtained change detection binary map is as Figure 4 shown. In the process of pre-segmenting two remote sensing images using a change detection segmentation model to obtain a change detection binary map, by setting the confidence threshold of the change detection segmentation model, the adjustment of the model output result can be realized. In order to ensure a relatively high recall rate, the confidence can be set relatively low.

[0056] The change detection annotation device for remote sensing images further includes: a receiving unit for, after performing the outer contour annotation of each sub-region in the two remote sensing images according to the outer contour position information of each sub-region included in the first clustering region and controlling the display device to display the images of the annotated parts of the two remote sensing images, receiving contour modification information for the outer contour annotation. The contour modification information corresponds to at least one of the following operations: deleting at least one outer contour, adjusting at least one outer contour, adding a new outer contour; a modification unit for modifying the change detection binary map according to the contour modification information. The contour modification information for the outer contour annotation can be information input by an operator, or information generated by an electronic device in response to a user's modification operation, or even information automatically generated by an electronic device based on an algorithm. Taking the contour modification information as input by the operator as an example, after the display device displays the images of the annotated parts of the two remote sensing images, the operator can manually confirm whether the outer contour annotation for the corresponding object is accurate, and in the case of insufficient accuracy, input the contour modification information to perform operations such as deleting, adding, and adjusting the outer contour. After the receiving unit receives the contour modification information, for these operations, the modification unit will modify the change detection binary map according to the corresponding contour modification information. Of course, after modification, the change detection binary map needs to be saved.

[0057] In this embodiment, the acquisition unit includes: a receiving module for receiving two overall remote sensing maps at different time phases; an acquisition module for acquiring multiple pairs of corresponding feature points on the two overall remote sensing maps; an alignment module for aligning the two overall remote sensing maps according to the multiple pairs of feature points; and a division module for performing grid division on the two aligned overall remote sensing maps to obtain multiple remote sensing image pairs, where each remote sensing image pair includes two remote sensing images at different time phases. That is to say, after receiving two overall remote sensing maps at different time phases, in order to facilitate the change detection annotation of the overall remote sensing maps with very large sizes, the two overall remote sensing maps are aligned and grid-divided. Alignment is to ensure the accuracy of subsequent change detection judgment. Grid division means dividing the two overall remote sensing maps into multiple small images (i.e., remote sensing images) respectively. After grid division, the remote sensing images belonging to the two time phases correspond to each other pairwise, thus facilitating the subsequent change detection and annotation. During the process of aligning the two overall remote sensing maps, first, multiple pairs of corresponding feature points on the two overall remote sensing maps are acquired. These feature points can be arbitrarily selected as long as they can serve as a positioning reference. By using these pairs of feature points, the accurate alignment of the two overall remote sensing maps can be achieved, and then the grid division can be performed according to the actual situation.

[0058] The clustering unit further includes a control module for controlling the masking of the target areas of the two remote sensing images during the process of controlling the display device to display the images of the marked parts of the two remote sensing images, where the target area is the area where the first clustering area overlaps with the sub-area belonging to the second clustering area, and the first clustering area and the second clustering area are different clustering areas. By masking the target areas of the two remote sensing images, the hiding of the sub-areas of the overlapping area can be realized. The overlapping area refers to the part where the sub-area of the second clustering area overlaps with the first clustering area. This can avoid the situation of repeated annotation of the partially overlapping sub-areas when annotating different clustering areas, ensure the simplicity of the annotation process, thus avoiding repeated work, and is beneficial to ensuring the efficiency of image change detection annotation.

[0059] Again, an embodiment of the present invention further provides a processor for running a program, where when the program runs, it executes the above-mentioned method for change detection annotation of remote sensing images.

[0060] Finally, an embodiment of the present invention further provides a device for change detection annotation of remote sensing images, including a display device, a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the above-mentioned method for change detection annotation of remote sensing images. The above is only a preferred embodiment of the present invention and is not used to limit the protection scope of the present invention.

Claims

1. A method for change detection annotation of remote sensing images, characterized in that Including: Obtain two remote sensing images corresponding to a target scene at different time phases; Determine multiple sub-regions where changes exist between the two remote sensing images and the outer contour corresponding to each sub-region; Cluster the outer contours of the multiple sub-regions to obtain multiple clustering regions, where one clustering region includes the outer contours of one or more of the sub-regions, and the sub-regions included in any two clustering regions are different from each other; According to the outer contour position information of each sub-region included in the first clustering region, perform outer contour annotation of each sub-region in the two remote sensing images, and control a display device to display the images of the annotated parts of the two remote sensing images, where the first clustering region includes one, multiple, or all of the multiple clustering regions; Among them, according to the outer contour position information of each sub-region included in the first clustering region, performing outer contour annotation of each sub-region in the two remote sensing images and controlling the display device to display the images of the annotated parts of the two remote sensing images includes: During the process of controlling the display device to display the images of the annotated parts of the two remote sensing images, control to perform mask processing on the target regions of the two remote sensing images, where the target region is the region where the first clustering region overlaps with the sub-regions belonging to the second clustering region, and the first clustering region and the second clustering region are different clustering regions.

2. The change detection annotation method for remote sensing images according to claim 1, wherein Clustering the outer contours of the multiple sub-regions to obtain multiple clustering regions includes: Determine the circumscribed horizontal rectangles corresponding to the outer contours of each sub-region; Determine a first target circumscribed horizontal rectangle from the circumscribed horizontal rectangles, where the size of the first target circumscribed horizontal rectangle is greater than or equal to a first preset size; Determine the region corresponding to one first target circumscribed horizontal rectangle as one clustering region.

3. The change detection annotation method for remote sensing images according to claim 2, characterized in that Clustering the outer contours of the multiple sub-regions to obtain multiple clustering regions further includes: Determine a second target circumscribed horizontal rectangle from the circumscribed horizontal rectangles, where the size of the second target circumscribed horizontal rectangle is smaller than the first preset size; Determine multiple target horizontal rectangles, where one target horizontal rectangle surrounds multiple second target circumscribed horizontal rectangles; Determine the region corresponding to one target horizontal rectangle as one clustering region.

4. The change detection annotation method for remote sensing images according to claim 3, characterized in that, Determining multiple target horizontal rectangles, where one target horizontal rectangle surrounds multiple second target circumscribed horizontal rectangles includes: Determine multiple reference points, where the multiple reference points correspond to the multiple second target circumscribed horizontal rectangles one by one; Perform K-means clustering processing on the multiple reference points, where the value of K traverses from 1 to the total number of the second target circumscribed horizontal rectangles, so that all the second target circumscribed horizontal rectangles corresponding to the reference points belonging to each class are included within a corresponding second preset size range; Determine each target horizontal rectangle as the smallest horizontal rectangle that includes all the second target circumscribed horizontal rectangles corresponding to the reference points of the corresponding class.

5. The change detection annotation method for remote sensing images according to claim 1, characterized in that Determining multiple sub-regions where changes exist between the two remote sensing images and the outer contour corresponding to each sub-region includes: Input two pieces of the remote sensing images into a pre-trained change detection segmentation model to obtain a change detection binary map representing the differences between the two pieces of the remote sensing images; Adopt a connected component analysis algorithm to analyze the change detection binary map to obtain the outer contours of each connected component.

6. The method for detecting and labeling changes in remote sensing images according to claim 5, characterized in that, After marking the outer contours of each of the sub-regions included in the first clustering region in the two remote sensing images according to the outer contour position information of each of the sub-regions included in the first clustering region, and controlling a display device to display the images of the marked parts of the two remote sensing images, the change detection marking method for the remote sensing images further includes: Receiving contour modification information for the outer contour marking, where the contour modification information corresponds to at least one of the following operations: deleting at least one of the outer contours, adjusting at least one of the outer contours, and adding new outer contours; Modifying the change detection binary map according to the contour modification information.

7. The change detection annotation method for remote sensing images according to any one of claims 1 to 6, characterized in that, Obtaining two remote sensing images corresponding to a target scene at different time phases, including: Receiving two overall remote sensing maps at different time phases; Obtaining multiple pairs of feature points corresponding to the two overall remote sensing maps; Aligning the two overall remote sensing maps according to the multiple pairs of feature points; Performing grid division on the two aligned overall remote sensing maps to obtain multiple pairs of remote sensing images, and each pair of remote sensing images includes two pieces of the remote sensing images at different time phases.

8. A change detection annotation device for remote sensing images, characterized in that, Including: An obtaining unit configured to obtain two remote sensing images corresponding to a target scene at different time phases; A determining unit configured to determine multiple sub-regions where changes exist between the two remote sensing images and the outer contour corresponding to each sub-region; A clustering unit configured to cluster the outer contours of the multiple sub-regions to obtain multiple clustering regions, where one clustering region includes the outer contours of one or more of the sub-regions, and the sub-regions included in any two clustering regions are different from each other; A control unit configured to mark the outer contours of each of the sub-regions in the two remote sensing images according to the outer contour position information of each of the sub-regions included in the first clustering region, and control the display device to display the images of the marked parts of the two remote sensing images, where the first clustering region includes one, multiple, or all of the multiple clustering regions; The control unit is specifically configured to, during the process of controlling the display device to display the images of the marked parts of the two remote sensing images, control to perform mask processing on the target regions of the two remote sensing images, where the target regions are the regions where the first clustering region overlaps with the sub-regions belonging to the second clustering region, and the first clustering region and the second clustering region are different clustering regions.

9. The change detection marking device for remote sensing images according to claim 8, wherein The clustering unit includes: a first determination module, configured to determine an externally circumscribed horizontal rectangle corresponding to the outer contour of each of the sub-regions; a second determination module, configured to determine a first target externally circumscribed horizontal rectangle from the externally circumscribed horizontal rectangles, wherein the size of the first target externally circumscribed horizontal rectangle is greater than or equal to a first preset size; a third determination module, configured to determine a region corresponding to one of the first target externally circumscribed horizontal rectangles as one of the clustering regions; The clustering unit includes: a fourth determination module, configured to determine a second target externally circumscribed horizontal rectangle from the externally circumscribed horizontal rectangles, wherein the size of the second target externally circumscribed horizontal rectangle is less than the first preset size; a fifth determination module, configured to determine a plurality of target horizontal rectangles, with one of the target horizontal rectangles surrounding a plurality of the second target externally circumscribed horizontal rectangles; a sixth determination module, configured to determine a region corresponding to one of the target horizontal rectangles as one of the clustering regions; The fifth determination module includes: a first determination sub-module, configured to determine a plurality of reference points, with the plurality of reference points corresponding to the plurality of second target externally circumscribed horizontal rectangles one by one; a clustering sub-module, configured to perform K-means clustering processing on the plurality of reference points, with the value of K traversing from 1 to the total number of the second target externally circumscribed horizontal rectangles, so that all the second target externally circumscribed horizontal rectangles corresponding to the reference points belonging to each class are included within a corresponding second preset size range; a second determination sub-module, configured to determine each of the target horizontal rectangles as the smallest horizontal rectangle that includes all the second target externally circumscribed horizontal rectangles corresponding to the reference points of the corresponding class; The determination unit includes: an input module, configured to input two of the remote sensing images into a pre-trained change detection segmentation model to obtain a change detection binary map representing the difference between the two remote sensing images; an analysis module, configured to analyze the change detection binary map by using a connected component analysis algorithm to obtain the outer contours of the respective connected components; The remote sensing image change detection annotation device further includes: a receiving unit, configured to, after performing outer contour annotation of each of the sub-regions in the two remote sensing images according to the outer contour position information included in the first clustering region and controlling a display device to display the images of the annotated parts of the two remote sensing images, receive contour modification information for the outer contour annotation, where the contour modification information corresponds to at least one of the following operations: deleting at least one of the outer contours, adjusting at least one of the outer contours, adding an outer contour; a modification unit, configured to modify the change detection binary map according to the contour modification information; The acquisition unit includes: a receiving module, configured to receive two overall remote sensing maps at different times; an acquisition module, configured to acquire a plurality of pairs of corresponding feature points on the two overall remote sensing maps; an alignment module, configured to align the two overall remote sensing maps according to the plurality of pairs of feature points; a partitioning module, configured to perform grid partitioning on the two overall remote sensing maps after alignment to obtain a plurality of pairs of remote sensing images, with each pair of remote sensing images including two of the remote sensing images at different times.

10. A non-volatile storage medium, characterized in that, The non-volatile storage medium includes a stored program, wherein, when the program runs, it controls the device where the non-volatile storage medium is located to execute the change detection annotation method for remote sensing images according to any one of claims 1 to 7.

11. A processor, characterized in that, The processor is used to run a program, wherein, when the program runs, it executes the change detection annotation method for remote sensing images according to any one of claims 1 to 7.

12. A change detection annotation device for remote sensing images, comprising a display device, a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the change detection annotation method for remote sensing images according to any one of claims 1 to 7.

Images