System, method and device for labeling remote sensing image

Abstract

The invention discloses a remote sensing image labeling system, method and device. The system comprises a processing platform and an optimization platform, wherein the processing platform comprises an image labeling unit and a labeling data set generation unit, and the optimization platform is in communication connection with the processing platform. The image labeling unit is used for performing region segmentation on the remote sensing image to be labeled to obtain an initial segmentation data set; and performing data adjustment on the initial segmentation data set to obtain a target segmentation data set. And the optimization platform is used for correcting the target segmentation data set according to the obtained manual correction instruction to obtain a target correction data set. And the annotation data set generation unit is used for generating an annotation data set according to the correction data set. Therefore, rapid and efficient image segmentation and annotation are realized through the image annotation unit. On the basis, manual correction is carried out on the target segmentation data set through the optimization platform, and the labeling accuracy is improved while the labeling efficiency is considered. In addition, by automatically generating the annotation data set, the annotation efficiency of the remote sensing image is further improved.

Description

Technical Field

[0001] The present application relates to the field of remote sensing technology, and in particular to a remote sensing image annotation system, method and device. Background Art

[0002] With the continuous development of satellite technology, satellite remote sensing images have become one of the important means of obtaining information about the earth's surface. The contours, positions, areas and other information of geographic targets in remote sensing images are extracted and annotated. The obtained annotated data sets can be used for image tasks such as scene recognition, speech segmentation, and target detection, and can thus be widely used in scenarios such as environmental monitoring, urban planning, natural disaster response and scientific research.

[0003] At present, when annotating remote sensing images, people can do it manually. However, although manual annotation can obtain highly accurate annotation data sets, the annotation efficiency is very low and cannot meet the needs of large-scale remote sensing image annotation. Although automatic annotation using algorithms such as deep learning has significant advantages in improving efficiency, it often cannot guarantee annotation accuracy when dealing with complex scenes.

[0004] It can be seen that how to balance the efficiency and accuracy of remote sensing image annotation is an urgent problem to be solved by technical personnel in this field. Summary of the invention

[0005] In view of this, one aspect of the present application provides a remote sensing image annotation system, the system comprising: a processing platform, and an optimization platform communicatively connected to the processing platform; wherein the processing platform comprises an image annotation unit and an annotation data set generation unit;

[0006] The image annotation unit is used to perform regional segmentation on the remote sensing image to be annotated to obtain an initial segmentation data set after regional annotation; perform data adjustment on the initial segmentation data set to obtain a target segmentation data set; and transmit the target segmentation data set to the optimization platform;

[0007] The optimization platform is used to correct the region annotation result of the target segmentation data set according to the acquired manual correction instruction to obtain a target correction data set; and transmit the target correction data set to the annotation data set generation unit;

[0008] The annotated data set generating unit is used to generate an annotated data set according to the target corrected data set.

[0009] Optionally, the optimization platform includes: a labeling correction unit, and a quality inspection unit communicatively connected to the labeling correction unit;

[0010] The annotation correction unit is used to perform initial correction on the region annotation result of the target segmentation data set according to the acquired first correction instruction to obtain an initial corrected data set; and transmit the initial corrected data set to the quality inspection unit;

[0011] The quality inspection unit is used to obtain a second correction instruction; and perform a second correction on the region labeling result of the initial correction data set according to the second correction instruction to obtain the target correction data set.

[0012] Optionally, there are multiple labeling correction units and multiple quality inspection units, and one quality inspection unit is communicatively connected to multiple labeling correction units.

[0013] Optionally, the image annotation unit is used to:

[0014] Get image segmentation instructions;

[0015] According to the image segmentation instruction, a target image segmentation model is retrieved from a model database;

[0016] The target image segmentation model is used to perform regional segmentation on the remote sensing image to be labeled to obtain the initial segmentation data set.

[0017] Optionally, the processing platform further includes: a model optimization unit;

[0018] The model optimization unit is used to:

[0019] Acquire a specified data set; wherein the specified data set includes at least one of the labeled data set and a specified training data set, and the specified training data set is a data set obtained by manually annotating regions of the original remote sensing image;

[0020] The model in the model database is fine-tuned using the specified data set.

[0021] Optionally, the processing platform further includes: a data preprocessing unit, and the data preprocessing unit is communicatively connected with the image annotation unit;

[0022] The data preprocessing unit is used for:

[0023] Acquire an original remote sensing image; wherein the original remote sensing image includes at least one of an optical remote sensing image and a radar image;

[0024] Preprocessing the original remote sensing image to obtain a preprocessed image;

[0025] The pre-processed image is sliced ​​to obtain the remote sensing image to be labeled; and the remote sensing image to be labeled is transmitted to the image labeling unit.

[0026] Optionally, the image annotation unit is used to:

[0027] Performing a first optimization process on the initial segmented data set to obtain a first data set; wherein the first optimization process includes region boundary detection, image denoising, image erosion, image expansion and hole filling;

[0028] After converting the region boundary in the first data set into vector data, a second data set is obtained;

[0029] Based on the vector data, the second data set is subjected to a second optimization process to obtain the target segmentation data set; wherein the second optimization process includes simplifying region boundaries, eliminating jagged boundaries, and removing patches whose areas are smaller than an area threshold.

[0030] Optionally, the annotated dataset includes at least a target detection dataset, a semantic segmentation dataset, a graphic description dataset, a ground feature segmentation dataset and a change detection dataset.

[0031] Another aspect of the present application provides a remote sensing image annotation method, which is applied to the remote sensing image annotation system, and the method includes:

[0032] Perform regional segmentation on the remote sensing image to be labeled to obtain an initial segmentation data set after regional labeling;

[0033] Performing data adjustment on the initial segmentation data set to obtain a target segmentation data set;

[0034] The target segmentation data set is transmitted to the optimization platform, so that the optimization platform corrects the region labeling result of the target segmentation data set according to the acquired manual correction instruction to obtain the target correction data set;

[0035] The dataset is modified according to the target to generate a labeled dataset.

[0036] Another aspect of the present application provides a remote sensing image annotation device, the device comprising:

[0037] The segmentation module is used to perform regional segmentation on the remote sensing image to be labeled and obtain the initial segmentation data set after regional labeling;

[0038] An optimization module, used for optimizing the initial segmentation data set to obtain a target segmentation data set;

[0039] A transmission module, used for transmitting the target segmentation data set to the optimization platform, so that the optimization platform corrects the region labeling result of the target segmentation data set according to the acquired manual correction instruction to obtain a target correction data set;

[0040] A generation module is used to modify the data set according to the target and generate a labeled data set.

[0041] The present application provides a remote sensing image annotation system, method and device, which have the following beneficial effects: fast and efficient image region segmentation and annotation are achieved through an image annotation unit. On this basis, the target segmentation data set is manually corrected through an optimization platform, which takes into account the annotation efficiency and improves the annotation accuracy. In addition, the annotation data set is automatically generated through an annotation data set generation unit, further improving the remote sensing image annotation efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A schematic diagram of the structure of a remote sensing image annotation system provided in an embodiment of the present application;

[0043] Figure 2 A schematic diagram of a correction interface of an optimization platform provided in an embodiment of the present application;

[0044] Figure 3 A schematic diagram of the structure of a remote sensing image annotation system provided by another embodiment of the present application;

[0045] Figure 4 A schematic diagram of a labeling process of a remote sensing image labeling system provided in an embodiment of the present application;

[0046] Figure 5 A schematic diagram of a flow chart of a remote sensing image annotation method provided in an embodiment of the present application;

[0047] Figure 6 A schematic diagram of the structure of a remote sensing image annotation device provided in an embodiment of the present application.

[0048] The accompanying drawings are marked as follows: 1 is a processing platform, 2 is an optimization platform, 10 is an image annotation unit, 11 is an annotation data set generation unit, 12 is a model database, 13 is a model optimization unit, 14 is a designated training data set, 15 is a data preprocessing unit, 20 is a annotation correction unit, 21 is a quality inspection unit, 60 is a segmentation module, 61 is an adjustment module, 62 is a transmission module, and 63 is a generation module. DETAILED DESCRIPTION

[0049] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0050] It should be understood that although the terms first, second, third, etc. may be used in the present application to describe various information, these information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of the present application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0051] Figure 1 A schematic diagram of a remote sensing image annotation system provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the system includes: a processing platform 1, and an optimization platform 2 communicatively connected to the processing platform 1, wherein the processing platform 1 includes an image annotation unit 10 and an annotation data set generation unit 11.

[0052] In a specific embodiment, the image annotation unit 10 is used to perform region segmentation on the remote sensing image to be annotated to obtain an initial segmentation data set after region annotation. Specifically, region segmentation of the remote sensing image refers to dividing the remote sensing image into multiple non-overlapping regions, and the pixels in each region have similar attributes. For example, the color, texture, and brightness in each region are similar. Through region segmentation, forests, lakes, residential areas, cultivated land and other areas can be divided.

[0053] In an optional embodiment, when performing regional segmentation on remote sensing images, an image segmentation model can be called for automatic segmentation, thereby ensuring the efficiency of image segmentation in large-scale remote sensing image processing scenarios. It should be noted that the image segmentation model called can be selected according to actual business needs, and this application does not make specific limitations on this.

[0054] Figure 2 This is a schematic diagram of a correction interface of an optimization platform provided in an embodiment of the present application. For ease of understanding, the following will be combined with Figure 2 For explanation. Figure 2 As shown in an optional embodiment, Figure A is a remote sensing image to be annotated. After the image A is segmented and optimized by the image annotation unit 10, a segmented image as shown in Figure B on the right is obtained. Figure 2 As shown, after segmentation, Figure A can obtain multiple non-overlapping areas and classify different areas.

[0055] Furthermore, the image annotation unit 10 is also used to adjust the initial segmentation data set to obtain the target segmentation data set; and transmit the target segmentation data set to the optimization platform 2. In order to facilitate the subsequent conversion of the region-annotated image into a directly usable annotation data set, it is necessary to adjust the initial segmentation data set. Specifically, it is necessary to convert the segmented region boundary into vector data, where vector data refers to a data structure used to represent geographic spatial information, as opposed to raster data. Vector data is mainly composed of geometric objects such as points, lines, and surfaces. It can be understood that each segmented region has a clear coordinate position, and the shape, size, and position of the region can be accurately described by vector data.

[0056] like Figure 1 As shown, the remote sensing image annotation system provided in the embodiment of the present application includes an optimization platform 2. In an optional embodiment, the optimization platform 2 is used to receive a target segmentation data set transmitted by an image annotation unit 10, and to correct the regional annotation results of the target segmentation data set according to the acquired manual correction instructions to obtain a target correction data set.

[0057] It can be understood that the remote sensing image annotation system provided in the embodiment of the present application realizes fast and efficient regional segmentation of large-scale remote sensing images through the image annotation unit 10. On this basis, in order to improve the annotation accuracy of remote sensing images, it is necessary to manually correct the regional segmentation results of the image annotation unit 10.

[0058] Specifically, the optimization platform 2 obtains the manual correction instruction, and according to the manual correction instruction, the region annotation result in the target segmentation data set transmitted by the image annotation unit 10 is corrected. It is worth noting that, in an optional embodiment, the manual correction instruction includes a graphic correction instruction and an attribute correction instruction, the graphic correction instruction refers to drawing, translating, node editing, etc. of the region graphic, and the attribute correction instruction refers to the annotation of the graphic quality, modification of the category, etc.

[0059] Further, the optimization platform 2 transmits the corrected target correction data set to the annotated data set generation unit 11 in the processing unit, so that the annotated data set generation unit 11 generates an annotated data set according to the target correction data set. The annotated data set refers to a data set that can be directly applied to fields such as urban planning, environmental monitoring, and scientific research, and may include but is not limited to target detection data sets, semantic segmentation data sets, graphic description data sets, object segmentation data sets, and change detection data sets.

[0060] It should be noted that if Figure 1As shown, in a specific embodiment, the present application does not limit the number of optimization platforms 2. In an optional embodiment, in order to improve the labeling efficiency of remote sensing images, the optimization platform 2 is set to be multiple, and in a specific embodiment, the processing unit distributes the target segmentation data set to different optimization platforms 2 for parallel correction processing, thereby further improving the labeling efficiency while ensuring the labeling accuracy.

[0061] Therefore, the remote sensing image annotation system provided in the embodiment of the present application realizes fast and efficient image region segmentation and annotation through the image annotation unit 10. On this basis, the target segmentation data set is manually corrected through the optimization platform 2, while taking into account the annotation efficiency and improving the annotation accuracy. In addition, the annotation data set is automatically generated through the annotation data set generation unit 11, further improving the remote sensing image annotation efficiency.

[0062] In an optional embodiment, if Figure 1 As shown, the optimization platform 2 includes: a label correction unit 20, and a quality inspection unit 21 that is in communication with the label correction unit 20. The label correction unit 20 is used to perform an initial correction on the region labeling result of the target segmentation data set according to the acquired first correction instruction to obtain an initial corrected data set; and transmit the initial corrected data set to the quality inspection unit 21.

[0063] In a specific embodiment, the annotation correction unit 20 refers to a platform on which a corrector can directly modify the target segmentation data set. The first correction instruction includes a graphic correction instruction and an attribute correction instruction. The graphic correction instruction includes but is not limited to graphic drawing, graphic translation, graphic display and hiding, and graphic node editing. The attribute correction instruction includes but is not limited to the annotation of graphic quality and modification of category.

[0064] It should be noted that, in order to facilitate the correction personnel to better correct the regional segmentation, in an optional embodiment, the annotation correction unit 20 superimposes and displays the target segmentation data set with the original remote sensing image data set, specifically, superimposes and displays the vector data with the original image data. In a specific embodiment, the vector data is converted into GeoJson format, and then converted into Json format, so as to achieve seamless superposition with the original image, and distinguish different types of objects through different colors and labels. This makes it convenient for the correction personnel to correct the regional annotation results of the target segmentation data set according to the original image.

[0065] It is worth noting that, in an optional embodiment, the annotation correction unit 20 includes a display screen, which is used for correction interaction with the correction operator, and the display screen displays Figure 2The interface shown allows the reviser to revise the regional segmentation and labeling results of each image in the target segmentation data set through command input devices such as keyboard, mouse, and microphone, and this application does not specifically limit the first correction instruction.

[0066] In an optional embodiment, if Figure 2 As shown, the graphic drawing includes but is not limited to adding drawing of building categories, adding drawing of transportation categories, adding drawing of sports venues categories, and adding drawing of natural resources categories. Figure 2 In the example shown, the annotation of the image quality can be performed by selecting high quality or low quality to calibrate the segmentation and annotation quality of the currently displayed image area.

[0067] If the quality of the current sample is poor, that is, the current segmentation and annotation results are not high, or the image resolution is low, the editor can click Figure 2 It should be noted that the sample quality may include high quality, medium quality, low quality, or primary quality, secondary quality, and tertiary quality, and the quality heights of primary quality, secondary quality, and tertiary quality decrease in sequence, and this application does not limit the classification levels.

[0068] In an optional embodiment, in the target segmentation data set, the canvas area of ​​the image supports multi-level scaling and supports enlarging to a resolution exceeding the original image. After the reviser selects a type of graphic drawing with the mouse, the cross-shaped cursor of the mouse enters the canvas area, and a polygon can be placed on the canvas by clicking the left button of the mouse, whereby the polygon frames a certain area as the currently added area. When the polygon drawing is completed, that is, after the area addition is completed, the drawing is completed by double-clicking the left mouse button, at which time the polygon is automatically closed, and the polygon no longer supports revision. In addition, it also supports selecting an area for deletion by right-clicking the mouse, that is, the reviser can highlight the drawn polygon and delete it by right-clicking the mouse.

[0069] Furthermore, when the editor double-clicks the left side of the mouse to complete the drawing, Figure 2 A new record is added in the annotation results shown, and the default name is empty. The editor can double-click the record to edit it, so as to mark the new area, for example, to add a new residential area. It should be noted that after adding a new area, the new records displayed in the annotation results can be sorted by creation time or by the name of the new area, which is not limited in this application.

[0070] After the editor clicks on the new record in the annotation result to edit, a label will pop up in the corresponding figure B, and the editor can modify, delete, etc. the label. For example, a new record of living in a district is added to the annotation result, and the corresponding label with the serial number 9 of the district will pop up in figure B.

[0071] In another optional embodiment, the graphic node editing allows the reviser to drag any vertex of the current polygon, thereby adjusting the polygon outline, that is, adjusting the boundary of the region. The graphic translation allows the reviser to select the drawn graphic for translation.

[0072] It is understood that in an optional embodiment, when any area is selected in Figure B, the corresponding annotation result of the area is also selected in the corresponding annotation result, and the serial number label in the area can also be highlighted, indicating that the label can be modified. When any target in the annotation result is highlighted, the target can be switched to display or hide, that is, the graphic display and hiding operation is performed on the graphic.

[0073] In another optional embodiment, after determining that all regions corresponding to the annotation results have been marked with serial number labels, the initial corrected image (initial corrected data set) can be submitted and transmitted to the quality inspection unit 21. If the annotation information of any region is incomplete, a prompt message will pop up so that the corrector can complete the annotation of all regions in time, wherein the annotation information includes but is not limited to the region boundary, vector data, annotation results, and serial number labels.

[0074] In another optional embodiment, an interactive interface of the display screen is designed to overlay point of interest (POI, electronic map and other annotation prompt information, so that the editor can easily and quickly correct the segmentation graphics and view the segmentation results in real time, thereby realizing large-scale auxiliary online data production.

[0075] like Figure 1 As shown, the optimization platform 2 includes a quality inspection unit 21, and in an optional embodiment, the quality inspection unit 21 is used to obtain a second correction instruction; and perform a second correction on the area labeling result of the initial correction data set according to the second correction instruction to obtain a target correction data set.

[0076] Specifically, in an optional embodiment, the second correction instruction can be of the same quality as the first correction, see the description above, that is, in the quality inspection unit 21, the quality inspector can perform a quality review of the modifications made by the reviser. If it is found that the reviser has omitted any corrections or made unqualified corrections, a second correction can be performed to obtain the target correction data set.

[0077] Of course, in another optional embodiment, if the quality inspector determines that the correction quality of the reviser is unqualified, the corresponding initial correction image can be returned to the corresponding reviser, who will make modifications and submit and inspect the image again. In this embodiment, the second correction instruction refers to a return instruction, that is, an instruction to return the initial correction data set to the annotation correction unit 20.

[0078] Furthermore, after the quality inspection is passed, the quality inspection unit 21 converts the vector data in the Json format in the segmentation result into the Shapefile format, and finally converts it into a Mask image.

[0079] Therefore, the remote sensing image annotation system provided in the embodiment of the present application sets up two-level quality control units in the optimization platform 2, namely, the annotation correction unit 20 and the quality inspection unit 21. Based on the correction made by the annotation correction unit 20, the quality inspection unit 21 further performs a secondary correction on the initial correction result, thereby further improving the annotation accuracy of the remote sensing image.

[0080] Based on the above embodiment, as an optional embodiment, Figure 1 As shown, there are multiple marking correction units 20 and multiple quality inspection units 21, and one quality inspection unit 21 is communicatively connected with multiple marking correction units 20.

[0081] It is understandable that although the remote sensing image can be efficiently and automatically segmented through the image annotation unit 10, the efficiency of manually correcting the regional annotation results in the optimization platform 2 is relatively low. In order to ensure the correction efficiency, in an optional embodiment, the annotation correction unit 20 is set to multiple, and multiple annotation correction units 20 perform annotation in parallel, thereby greatly improving the annotation efficiency.

[0082] Furthermore, in order to ensure the efficiency of quality inspection, multiple quality inspection units 21 are also provided. It can be understood that the quality inspection unit 21 performs secondary correction on the regional annotation result based on the annotation correction unit 20, so the inspection is already performed on the basis of a higher annotation quality. In order to control labor costs, one quality inspection unit 21 can be connected to multiple annotation correction units 20.

[0083] In an optional embodiment, the quality inspection unit 21 is connected in communication with the annotation correction unit 20, and the quality inspection unit 21 is connected in communication with the annotation data set generation unit 11, so as to transmit the obtained target correction data set to the annotation data set generation unit 11 for generating the annotation data set. However, it should be noted that the target segmentation data set sent by the image annotation unit 10 can be received by the annotation correction unit 20 or the quality inspection unit 21, and this application does not limit this.

[0084] like Figure 1 As shown, when the quality inspection unit 21 is communicatively connected with the image annotation unit 10, as an optional embodiment, the image annotation unit 10 may send the target segmentation data set to each quality inspection unit 21 on average, and then the quality inspection unit 21 distributes the target segmentation data set according to the field of expertise of each annotation correction unit 20, the speed of correction, etc.

[0085] Figure 3A schematic diagram of a remote sensing image annotation system provided in another embodiment of the present application is shown in FIG. Figure 3 As shown, in another optional embodiment, the annotation correction unit 20 is communicatively connected to the image annotation unit 10. In this embodiment, the image annotation unit 10 evenly distributes the target segmentation data set to different annotation correction units 20. After the annotation correction unit 20 corrects the data set, it transmits the initial corrected data set to the corresponding quality inspection units 21.

[0086] Therefore, the remote sensing image annotation system provided in the embodiment of the present application, by setting up multiple annotation correction units 20 and quality inspection units 21, and one quality inspection unit 21 is communicatively connected with multiple annotation correction units 20, on the basis of multi-level correction, through the parallel correction of multiple annotation correction units 20 and multiple quality inspection units 21, the annotation efficiency of remote sensing images is further improved.

[0087] As an optional embodiment, the image annotation unit 10 is used to:

[0088] Get image segmentation instructions;

[0089] According to the image segmentation instruction, the target image segmentation model is retrieved from the model database 12;

[0090] The target image segmentation model is used to perform regional segmentation on the remote sensing image to be labeled, and an initial segmentation data set is obtained.

[0091] In an optional embodiment, if Figure 3 As shown, in order to facilitate the image annotation unit 10 to quickly perform region segmentation on the remote sensing image to be annotated, the image segmentation model is pre-stored in a model database 12. After the image segmentation instruction is obtained, the target image segmentation model is retrieved from the model database 12 according to the image segmentation instruction, so as to perform region segmentation on the remote sensing image to be annotated by the target image segmentation model to obtain an initial segmentation data set.

[0092] In a specific embodiment, the image segmentation models stored in the model database 12 include but are not limited to the SAM (Segment Anything) model, the DeepLab model and the U-Net model, wherein the SAM model is suitable for target boundary detection and fine-grained classification in images, the DeepLab model is suitable for large-scale target recognition and classification in images, and the U-Net model is suitable for object boundary extraction and region segmentation.

[0093] Therefore, in an optional embodiment, a suitable model can be selected for image segmentation according to specific application requirements, that is, according to the image segmentation quality. Of course, in another optional embodiment, the remote sensing image to be labeled is analyzed to determine the data characteristics, and a suitable model is selected according to the analysis results, which is not limited in this application.

[0094] It is understandable that training the image segmentation model with a high-quality training sample set can effectively improve the accuracy and robustness of the image segmentation model. The SAM model has a strong zero-sample generalization. If the basic model of the SAM model is continuously fine-tuned, the SAM model finally obtained can be applied to process multimodal remote sensing images, that is, the remote sensing images that can be applied are relatively wide. However, the DeepLab model and the U-Net model need to organize a large number of high-quality training sample data sets, and the newly generated annotation data sets need to be continuously added to the fine-tuning iterative training of the DeepLab model and the U-Net model to obtain a better DeepLab model and U-Net model. Therefore, in a preferred embodiment, the target image segmentation model can be a SAM model.

[0095] Therefore, in the remote sensing image annotation system provided in the embodiment of the present application, the image annotation unit 10 can select an image segmentation model suitable for the current application scenario and actual needs to perform region segmentation according to the image segmentation instruction, thereby further ensuring the accuracy of the remote sensing image annotation.

[0096] Based on the above embodiment, in an optional embodiment, as Figure 3 As shown, the processing platform 1 further includes: a model optimization unit 13;

[0097] The model optimization unit 13 is used for:

[0098] Acquire a specified data set; wherein the specified data set includes at least one of a labeled data set and a specified training data set 14, and the specified training data set 14 is a data set obtained by manually annotating regions of the original remote sensing image;

[0099] By specifying a data set, the model in the model database 12 is fine-tuned.

[0100] In a specific embodiment, in order to further improve the accuracy of remote sensing image annotation, a model optimization unit 13 is provided in the processing platform 1, and the image segmentation model in the model database 12 is continuously optimized by the model optimization unit 13.

[0101] It can be understood that a high-quality annotated data set can be obtained after processing by the image processing platform 1 and the optimization platform 2 of the embodiment of the present application. Figure 3As shown, the designated data set may be a labeled data set, and the image segmentation model in the model database 12 is fine-tuned by the labeled data set, so that the segmentation quality of the model is continuously improved, thereby continuously improving the labeling accuracy of the remote sensing image labeling system provided by the present application.

[0102] In another optional embodiment, if Figure 3 As shown, the designated data set may also be a data set obtained by manually pre-marking the original remote sensing image. By continuously fine-tuning each model in the database through the designated training data set 14, a model with higher accuracy can also be obtained.

[0103] Therefore, the remote sensing image annotation system provided in the embodiment of the present application further improves the annotation accuracy of the remote sensing image by setting a model optimization unit 13, which continuously optimizes the model in the model database 12. In other words, processing multimodal remote sensing image data can make full use of the advantages of different modal data and improve the overall segmentation effect of the image segmentation model. In addition, in the process of fine-tuning the image segmentation model, active learning and data enhancement technology are used to generate better quality training samples.

[0104] In an optional embodiment, if Figure 3 As shown, the processing platform 1 further includes: a data preprocessing unit 15, and the data preprocessing unit 15 is communicatively connected with the image annotation unit 10;

[0105] The data preprocessing unit 15 is used for:

[0106] Acquire original remote sensing images; wherein the original remote sensing images include at least one of optical remote sensing images and radar images;

[0107] Preprocessing the original remote sensing image to obtain a preprocessed image;

[0108] The pre-processed image is sliced ​​to obtain a remote sensing image to be labeled; and the remote sensing image to be labeled is transmitted to the image labeling unit 10 .

[0109] On the basis of the above embodiment, in order to further improve the accuracy of remote sensing image annotation, as an optional embodiment, Figure 3 As shown, a data preprocessing unit 15 is added to the remote sensing image annotation system.

[0110] In a specific embodiment, the data preprocessing unit 15 obtains the original remote sensing image. Specifically, the data preprocessing unit 15 supports obtaining remote sensing image data collected by different sensors, for example, including but not limited to high-resolution optical remote sensing images and radar images.

[0111] In addition, the data preprocessing unit 15 also supports the acquisition of remote sensing image data of different resolutions, and the present application does not limit the processing level and acquisition method of the original remote sensing image. Generally speaking, the higher the resolution of the image, the better the effect of automatic region segmentation and annotation.

[0112] After obtaining the original remote sensing image, the data preprocessing unit 15 preprocesses the original remote sensing image. Specifically, the optical remote sensing image is processed by image correction, full color fusion, etc., wherein the image correction includes but is not less than radiation correction, geometric correction, and orthorectification. Further, the three channel data of RGB of the processed optical remote sensing image are extracted to obtain the preprocessed image corresponding to the optical remote sensing image.

[0113] When the original remote sensing image is a radar image, the SAR (Synthetic Aperture Radar) amplitude image to be coordinated is obtained through calibration, multi-view processing and geocoding, that is, a single-band grayscale image is obtained, and then the preprocessed image corresponding to the radar image is obtained.

[0114] It is understandable that the obtained pre-processed image is an image with a large resolution, which is not conducive to the image annotation unit 10 to perform region segmentation and annotation. Therefore, the data pre-processing unit 15 slices the pre-processed image to obtain the remote sensing image to be annotated. Specifically, the pre-processed image is sliced ​​by sliding window. In an optional embodiment, the length and width can be set to the same size when slicing. For example, the pre-processed image can be switched to a slice data set (i.e., the remote sensing image to be annotated) of different pixel sizes such as 256×256, 512×512, 1024×1024.

[0115] It should be noted that in some scenes, some areas are relatively large, and if the slice size is small, a completed area (for example, a completed forest area) will be divided into multiple slices. Therefore, in an optional embodiment, when performing sliding window slicing, sliding window slicing of multiple sizes is performed simultaneously.

[0116] Of course, in another optional embodiment, the user can set the slice size according to actual business needs, thereby avoiding the need to label images of multiple sizes and improving labeling efficiency.

[0117] Therefore, the remote sensing image annotation system provided in the embodiment of the present application, by adding a data preprocessing unit 15, preprocesses the original remote sensing image in advance, thereby extracting some images of poor quality and ensuring that the image annotation unit 10 obtains higher quality images for segmentation, thereby further improving the accuracy of remote sensing image annotation.

[0118] Based on the above embodiment, as an optional embodiment, the image annotation unit 10 is used to:

[0119] Performing a first optimization process on the initial segmented data set to obtain a first data set; wherein the first optimization process includes region boundary detection, image denoising, image erosion, image expansion and hole filling;

[0120] After converting the region boundary in the first data set into vector data, a second data set is obtained;

[0121] Based on the vector data, a second optimization process is performed on the second data set to obtain a target segmentation data set; wherein the second optimization process includes simplifying region boundaries, eliminating jagged boundaries, and removing patches whose areas are smaller than an area threshold.

[0122] In an optional embodiment, to ensure the accuracy of image region segmentation and annotation by the image annotation unit 10, the image annotation unit 10 performs a first optimization process on the initial segmentation data set after performing region segmentation on the remote sensing image to be annotated using a target image segmentation model to obtain an initial segmentation data set.

[0123] Specifically, it is understood that the slice data of the remote sensing image to be annotated retains the spatial information of the original image, and the first data set can be obtained by performing a first optimization process on the region boundary through the OpenCV morphological algorithm. The first optimization process includes but is not limited to region boundary detection, image denoising, image erosion, image expansion, and hole filling.

[0124] Further, the region boundaries in the first data set are converted into vector data to obtain a second data set. In an optional embodiment, the data format of the vector data may be a Shapefile format, and the conversion into vector data facilitates subsequent rapid identification of the region. In order to further improve the accuracy of annotation, the second data set is subjected to a second optimization process based on the vector data to obtain a target segmentation data set. The second optimization process may include, but is not limited to, simplifying region boundaries, eliminating jagged boundaries, and removing patches whose areas are less than an area threshold.

[0125] Specifically, the boundary lines corresponding to the vector data are simplified, and the jagged edges of the boundary lines are eliminated. At the same time, some patches with too small areas are removed, that is, patches with areas smaller than an area threshold are removed.

[0126] Therefore, the remote sensing image annotation system provided in the embodiment of the present application optimizes the initial segmented data set after segmentation through the image annotation unit 10, which facilitates the subsequent optimization platform 2 to make corrections, thereby improving the accuracy of remote sensing image annotation.

[0127] In an optional embodiment, the annotated dataset includes at least a target detection dataset, a semantic segmentation dataset, a graphic description dataset, a ground feature segmentation dataset, and a change detection dataset.

[0128] In a specific embodiment, after the annotated data set generation unit 11 obtains the target correction data set transmitted by the quality inspection unit 21, it can call the corresponding visual model to recognize the image in the target correction data set, thereby automatically generating a corresponding annotated data set.

[0129] Specifically, when it is necessary to annotate a designated target in an image, in an optional embodiment, a target detection model may be selected to perform multi-category segmentation and annotation on the image in the target correction data set, thereby extracting the designated target. For example, important land features such as the segmentation boundaries of an overpass and a railway station are extracted, and the minimum bounding rectangle of the area is determined, and the minimum bounding rectangle is converted into an HBB (Horizontal Bounding Box) annotation text.

[0130] In an optional embodiment, the HBB annotation text can be expressed as {x1, y1, x2, y2, x3, y3, x4, y4, category, difficult}, where (x1, y1), (x2, y2), (x3, y3) and (x4, y4) are the vertex coordinates of the minimum outer rectangle, category is the type of the specified target, difficult is the recognition difficulty of the specified target, the recognition difficulty can be divided into highly difficult, moderately difficult and easy, and can also be divided into first level difficulty, second level difficulty and third level difficulty, and the difficulty levels of first level difficulty, second level difficulty and third level difficulty decrease successively, and can also be divided into difficult recognition and not difficult recognition. This application does not limit the division of recognition difficulty.

[0131] If it is necessary to obtain a ground feature segmentation dataset, the ground feature segmentation model is used to sort the patches in the segmentation results of each image in the target correction data from large to small in area, and a preset number of ground feature categories are selected before sorting, and the corresponding multi-classification labels are obtained after deduplication, thereby obtaining a ground feature segmentation dataset.

[0132] In an optional embodiment, the feature segmentation dataset may include but is not limited to four categories: building, transportation, sports venues, and natural resources. Among them, the building category includes but is not limited to residential areas and industrial areas, the transportation category includes but is not limited to bridges and ports, the sports venue category includes but is not limited to courts and stadiums, and the natural resource category includes but is not limited to cultivated land, grassland, and woodland.

[0133] It should be noted that, in an optional embodiment, different types of land features can be distinguished by using different colors, or by using different numbers, text, etc., which is not limited in this application.

[0134] If it is necessary to obtain a picture-text interpretation dataset, the segmentation results of each image in the target correction data are analyzed in terms of position, color, and shape, and the picture-text is parsed through the picture-text description model to generate a long text picture-text interpretation result, thereby obtaining a picture-text interpretation dataset.

[0135] In an optional embodiment, each slice image in the target correction data set is spliced ​​and merged to obtain a ground object segmentation vector map of a larger area, and the merged target area coverage map is output and stored.

[0136] It should be noted that the labeled dataset may also include, but is not limited to, a semantic segmentation dataset and a change detection dataset. The semantic segmentation dataset can be applied to autonomous driving, for example, road segmentation, obstacle detection, etc., and can also be applied to urban planning and other fields. The change detection dataset can be used for natural disaster detection.

[0137] Therefore, the remote sensing image annotation system provided by the embodiment of the present application can quickly perform target detection, object classification and image-text interpretation, realizing multiple uses of one label. The annotated data set can be used in practical applications such as target discovery, tracking, natural disaster detection, and urban planning.

[0138] Figure 4 This is a schematic diagram of a remote sensing image annotation system provided in an embodiment of the present application. In order to facilitate those skilled in the art to better understand the technical solution of the present application, the following will be combined with Figure 3 and Figure 4 Provide explanation.

[0139] like Figure 4 As shown, in a specific embodiment, the processing platform 1 obtains the original remote sensing image, specifically, by Figure 3 The data preprocessing unit 15 shown is used to acquire and preprocess the original remote sensing image to obtain a preprocessed image. Further, the preprocessed image is sliced ​​by the data preprocessing unit 15 to obtain a remote sensing image to be labeled.

[0140] Furthermore, the image annotation unit 10 in the processing platform 1 retrieves the target image segmentation model from the model database 12 according to the image segmentation instruction, and performs regional segmentation on the remote sensing image to be annotated through the target image segmentation model to obtain an initial segmentation data set.

[0141] Then, the image annotation unit 10 performs a first optimization process on the initial segmented data set to obtain a first data set; wherein the first optimization process includes region boundary detection, image denoising, image erosion, image expansion and hole filling. Further, after converting the region boundary in the first data set into vector data, a second data set is obtained, and based on the vector data, a second optimization process is performed on the second data set to obtain a target segmented data set. wherein the second optimization process includes region boundary simplification, anti-aliasing of jagged boundaries and removal of patches whose area is less than an area threshold.

[0142] like Figure 4 As shown, after obtaining the target segmentation data set, the processing platform 1 transmits the target segmentation data set to the optimization platform 2. At this time, the annotation correction unit 20 in the optimization platform 2 performs an initial correction on the regional annotation result of the target segmentation data set according to the obtained first correction instruction to obtain an initial corrected data set.

[0143] Furthermore, the quality inspection unit 21 obtains a second correction instruction, and performs a second correction on the region labeling result of the initial correction data set according to the second correction instruction to obtain a target correction data set.

[0144] The optimization platform 2 returns the corrected target correction data set to the processing platform 1, so that the annotation data set generation unit 11 in the processing platform 1 automatically generates an annotation data set according to the target correction data set, thereby completing the entire annotation process of the remote sensing image. The specific implementation method of each step refers to the description of the above embodiment.

[0145] In the above-mentioned embodiments, the remote sensing image annotation system is described in detail. The present application also provides an embodiment corresponding to a remote sensing image annotation method.

[0146] Figure 5 A flowchart of a remote sensing image annotation method provided in an embodiment of the present application is shown in FIG. Figure 5 As shown, the method is applied to a remote sensing image annotation system of any of the above embodiments, including:

[0147] S50: performing regional segmentation on the labeled remote sensing image to obtain an initial segmentation data set after regional labeling;

[0148] S51: performing data adjustment on the initial segmentation data set to obtain a target segmentation data set;

[0149] S52: transmitting the target segmentation data set to the optimization platform, so that the optimization platform corrects the region labeling result of the target segmentation data set according to the acquired manual correction instruction to obtain the target correction data set;

[0150] S53: Correct the data set according to the target and generate a labeled data set.

[0151] Since the remote sensing image annotation method provided in the embodiment of the present application corresponds to the remote sensing image annotation system provided in the above embodiment, the specific implementation method of the embodiment of the present application refers to the description of the above embodiment.

[0152] Compared with traditional manual labeling, the remote sensing image labeling method provided in the embodiment of the present application can quickly and efficiently complete the image area segmentation through preliminary automatic labeling, realize batch processing of remote sensing images, and reduce the errors and inconsistencies that may occur in the manual labeling process. On this basis, manual correction is performed to ensure the labeling efficiency while improving the labeling accuracy.

[0153] In the above-mentioned embodiments, the remote sensing image annotation method is described in detail. The present application also provides a corresponding embodiment of a remote sensing image annotation device.

[0154] Figure 6 A schematic diagram of the structure of a remote sensing image annotation device provided in an embodiment of the present application is shown in FIG. Figure 6 As shown, the device comprises:

[0155] A segmentation module 60 is used to perform regional segmentation on the remote sensing image to be labeled, and obtain an initial segmentation data set after regional labeling;

[0156] An adjustment module 61 is used to adjust the initial segmentation data set to obtain a target segmentation data set;

[0157] A transmission module 62 is used to transmit the target segmentation data set to the optimization platform, so that the optimization platform corrects the region labeling result of the target segmentation data set according to the acquired manual correction instruction to obtain a target correction data set;

[0158] The generating module 63 is used to correct the data set according to the target and generate the labeled data set.

[0159] Since the remote sensing image annotation device provided in the embodiment of the present application corresponds to the remote sensing image annotation method provided in the above embodiment, and the remote sensing image annotation method corresponds to the remote sensing image annotation system provided in the above embodiment, the specific implementation method of the embodiment of the present application refers to the description of the remote sensing image annotation system in the above embodiment.

[0160] It should be noted that, although the operations are depicted in a specific order in the accompanying drawings, this should not be understood as requiring these operations to be performed in the specific order shown or to be performed sequentially, or requiring all illustrated operations to be performed to achieve the desired results. In some cases, multitasking and parallel processing may be advantageous. In addition, the separation of various system modules and components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product, or packaged into multiple software products.

Claims

1. A remote sensing image annotation system, characterized in that: The system comprises: a processing platform, and an optimization platform communicatively connected to the processing platform; wherein the processing platform comprises an image annotation unit and an annotation data set generation unit; The image annotation unit is used to perform regional segmentation on the remote sensing image to be annotated to obtain an initial segmentation data set after regional annotation; perform data adjustment on the initial segmentation data set to obtain a target segmentation data set; and transmit the target segmentation data set to the optimization platform; The optimization platform is used to correct the region annotation result of the target segmentation data set according to the acquired manual correction instruction to obtain a target correction data set; and transmit the target correction data set to the annotation data set generation unit; The annotated data set generating unit is used to generate an annotated data set according to the target corrected data set.

2. The remote sensing image annotation system according to claim 1, characterized in that: The optimization platform includes: a marking correction unit, and a quality inspection unit connected to the marking correction unit in communication; The annotation correction unit is used to perform initial correction on the region annotation result of the target segmentation data set according to the acquired first correction instruction to obtain an initial corrected data set; and transmit the initial corrected data set to the quality inspection unit; The quality inspection unit is used to obtain a second correction instruction; and perform a second correction on the region labeling result of the initial correction data set according to the second correction instruction to obtain the target correction data set.

3. The remote sensing image annotation system according to claim 2, characterized in that: There are multiple labeling correction units and multiple quality inspection units, and one quality inspection unit is communicatively connected with multiple labeling correction units.

4. The remote sensing image annotation system according to claim 1, characterized in that: The image annotation unit is used to: Get image segmentation instructions; According to the image segmentation instruction, a target image segmentation model is retrieved from a model database; The target image segmentation model is used to perform regional segmentation on the remote sensing image to be labeled to obtain the initial segmentation data set.

5. The remote sensing image annotation system according to claim 4, characterized in that: The processing platform also includes: a model optimization unit; The model optimization unit is used to: Acquire a specified data set; wherein the specified data set includes at least one of the labeled data set and a specified training data set, and the specified training data set is a data set obtained by manually annotating regions of the original remote sensing image; The model in the model database is fine-tuned using the specified data set.

6. The remote sensing image annotation system according to claim 1, characterized in that: The processing platform further includes: a data preprocessing unit, and the data preprocessing unit is communicatively connected with the image annotation unit; The data preprocessing unit is used for: Acquire an original remote sensing image; wherein the original remote sensing image includes at least one of an optical remote sensing image and a radar image; Preprocessing the original remote sensing image to obtain a preprocessed image; The pre-processed image is sliced ​​to obtain the remote sensing image to be labeled; and the remote sensing image to be labeled is transmitted to the image labeling unit.

7. The remote sensing image annotation system according to claim 1, characterized in that: The image annotation unit is used to: Performing a first optimization process on the initial segmented data set to obtain a first data set; wherein the first optimization process includes region boundary detection, image denoising, image erosion, image expansion and hole filling; After converting the region boundary in the first data set into vector data, a second data set is obtained; Based on the vector data, the second data set is subjected to a second optimization process to obtain the target segmentation data set; wherein the second optimization process includes simplifying region boundaries, eliminating jagged boundaries, and removing patches whose areas are smaller than an area threshold.

8. The remote sensing image annotation system according to claim 1, characterized in that: The annotated dataset includes at least a target detection dataset, a semantic segmentation dataset, a graphic description dataset, a ground feature segmentation dataset and a change detection dataset.

9. A remote sensing image annotation method, characterized in that: The remote sensing image annotation system according to any one of claims 1 to 8, wherein the method comprises: Perform regional segmentation on the remote sensing image to be labeled to obtain an initial segmentation data set after regional labeling; Performing data adjustment on the initial segmentation data set to obtain a target segmentation data set; The target segmentation data set is transmitted to the optimization platform, so that the optimization platform corrects the region labeling result of the target segmentation data set according to the acquired manual correction instruction to obtain the target correction data set; The dataset is modified according to the target to generate a labeled dataset.

10. A remote sensing image annotation device, characterized in that: The device comprises: The segmentation module is used to perform regional segmentation on the remote sensing image to be labeled and obtain the initial segmentation data set after regional labeling; An optimization module, used for optimizing the initial segmentation data set to obtain a target segmentation data set; A transmission module, used for transmitting the target segmentation data set to the optimization platform, so that the optimization platform corrects the region labeling result of the target segmentation data set according to the acquired manual correction instruction to obtain a target correction data set; A generation module is used to modify the data set according to the target and generate a labeled data set.

Images