A method and system for constructing and labeling a remote sensing ground object sample library

Abstract

The application provides a remote sensing ground object sample library construction and labeling method and system, and belongs to the technical field of remote sensing image information extraction; solves the technical problems of remote sensing ground object sample library classification level, space-time difference expression, sample labeling precision, and poor sample quality control; the technical scheme adopted is S1: collecting remote sensing image data and preprocessing to obtain remote sensing orthographic image data; S2: establishing a remote sensing ground object hierarchical classification system; S3: performing space-time sensitivity discrimination to distinguish space-time sensitive ground objects and space-time insensitive ground objects; S4: respectively establishing a fine-grained sub-library and a unified sample library; S5: cropping the remote sensing orthographic image data to obtain sample images; S6: screening the sample images; S7: performing polygon labeling on the ground object instances in the sample images; S8: organizing the sample images and labeling information to obtain sample labeling results; S9: quality inspection, storing the sample images and labeling information that pass the quality inspection to obtain a remote sensing ground object sample library; and application in land survey.

Description

Technical Field

[0001] This application relates to the field of remote sensing image information extraction technology, and in particular to a method and system for constructing and labeling a remote sensing ground feature sample database. Background Technology

[0002] With the development of high-resolution remote sensing image acquisition technology, UAV aerial surveying technology, and artificial intelligence image recognition technology, the automatic identification and intelligent interpretation of rural features using remote sensing imagery has been widely applied in scenarios such as rural planning, land surveys, farmland protection, rural infrastructure management, ecological environment monitoring, and disaster assessment. Rural areas have complex feature types, including common features such as rural buildings, roads, vegetation, and water systems, as well as smaller or uniquely shaped features such as above-ground pipelines, poles, and graves. Different features exhibit significant differences in color, texture, boundary, spatial distribution, and scale characteristics in remote sensing images, placing high demands on the classification system and annotation quality of the sample database.

[0003] Existing methods for constructing remote sensing sample databases typically employ a unified classification and annotation approach, collecting samples according to fixed land cover categories and organizing them in the same way across all land cover types. While this approach facilitates management, it fails to adequately represent the differences in rural land cover images across different regions and time periods. For example, vegetation exhibits significant variations in color, coverage, and texture characteristics in spring, summer, autumn, and winter; rural buildings may show substantial differences across regions due to variations in roof materials, building density, spatial layout, and surrounding environment; and water systems may experience changes in water surface area across seasons. If the sample database fails to reflect these spatiotemporal differences, the land cover recognition model subsequently trained is prone to problems such as insufficient cross-regional generalization ability and unstable cross-temporal recognition performance.

[0004] Meanwhile, rural remote sensing imagery contains a large number of small, elongated, and complex-boundary targets. For example, above-ground pipelines appear as thin, linear structures in the images; independent features such as poles and graves are small in scale; and rural building roofs, waterway edges, and vegetation canopies often have irregular boundaries. Traditional rectangular bounding box annotation methods are insufficient to accurately represent the true contours of these targets, easily leading to problems such as large boundary errors, category confusion, and unstable sample quality.

[0005] In addition, the existing sample library production process often lacks a systematic sample screening and quality inspection mechanism. Some blurry, severely occluded, excessively projected, or truncated samples will reduce the quality of the sample library after entering the training set, affecting the training effect of subsequent object detection, semantic segmentation, and instance segmentation models. Summary of the Invention

[0006] To address the aforementioned technical issues, this application proposes a method and system for constructing and labeling a remote sensing ground feature sample database.

[0007] The technical solution adopted in this application is: a method for constructing and labeling a remote sensing ground feature sample database, comprising the following steps: Step 1: Collect remote sensing image data, preprocess the remote sensing image data to obtain remote sensing orthophoto data; Step 2: Establish a hierarchical classification system for remotely sensed ground features; Step 3: Based on the changes in remote sensing image data of land cover categories in different regions and time phases, determine the spatiotemporal sensitivity of land cover categories in the remote sensing land cover hierarchical classification system. Land cover categories with obvious temporal or spatial differences in remote sensing image data are determined to be spatiotemporally sensitive land cover categories; land cover categories without obvious temporal or spatial differences in remote sensing image data are determined to be spatiotemporally insensitive land cover categories. Step 4: Establish fine-grained sub-databases for spatiotemporally sensitive features, categorized by region and time phase; establish a unified sample database for spatiotemporally insensitive features; Step 5: Perform sliding window cropping on the remote sensing orthophoto data to obtain sample images; Step 6: Screen the sample images and remove blurry, severely occluded, or excessively large projection differences. Step 7: Perform polygon annotation on the ground feature instances in the sample image to obtain the outlines of the ground feature instances that fit the true boundaries of the ground feature instances; Step 8: Organize the sample images and annotation information to obtain the sample annotation results; Step 9: Perform quality checks on the sample annotation results and verify the fine-grained sub-libraries separately. Input the sample images and annotation information that pass the quality check and separate verification into the database to obtain the remote sensing ground feature sample database. Spatiotemporal sensitivity discrimination includes temporal sensitivity discrimination and spatial sensitivity discrimination. Temporal sensitivity discrimination is used to determine whether the color, texture, coverage, boundary range or morphological characteristics of the same land cover category change at different times. Spatial sensitivity discrimination is used to determine whether there are differences in the material, color, scale, spatial distribution, background environment or image performance of the same land cover category in different regions.

[0008] Furthermore, the annotation information includes the sample image number, file name, image width, image height, category number, category name, segmented polygon coordinates, target area, circumscribed rectangle, attribute information, region label, time label, image source, and quality label.

[0009] Furthermore, the land cover categories in the remote sensing land cover hierarchical classification system include buildings, roads, vegetation, water systems, pipelines, and independent land cover.

[0010] Furthermore, buildings include tiled houses, bungalows, prefabricated steel houses, photovoltaic roof houses, and greenhouses; roads include asphalt roads, cement roads, dirt roads, and railways; vegetation includes trees, shrubs, solitary trees, and cultivated land with seedlings; water systems include lakes, rivers, ponds, canals, and reservoirs; pipelines include poles, power lines, and above-ground pipelines; and independent features include grave mounds.

[0011] Furthermore, the fine-grained sub-library includes regional labels and temporal labels. The regional labels include at least one of North China, South China, Northwest China, and Southwest China, and the temporal labels include at least one of Spring, Summer, Autumn, and Winter.

[0012] Furthermore, in step 1, preprocessing includes coordinate unification, orthorectification, image stitching, color equalization, abnormal area removal, and image quality inspection.

[0013] Furthermore, a system for implementing the construction and annotation method of the remote sensing ground object sample library as described above includes an image data acquisition and preprocessing module whose output is electrically connected to the input of a hierarchical classification system construction module; an image data acquisition and preprocessing module whose output is electrically connected to the input of a spatiotemporal sensitivity discrimination module; an image data acquisition and preprocessing module whose output is electrically connected to the input of a spatiotemporal adaptation sub-library construction module; a spatiotemporal adaptation sub-library construction module whose output is electrically connected to the input of a sliding window cropping module; and a sliding window cropping module whose output is electrically connected to the input of a sample screening and annotation module.

[0014] The advantages of this application over the prior art are as follows: 1. It establishes a hierarchical classification system for rural remote sensing ground features, which improves the standardization and scalability of the sample database category organization by combining primary categories and subcategories; 2. Introduce a spatiotemporal adaptation mechanism. Based on the sensitivity of different land cover categories to regional differences and temporal changes, adopt a differentiated sample creation strategy to avoid mechanically dividing all land cover into regions and time phases to build databases. 3. Establishing fine-grained sub-databases for land features that are sensitive to time and space or have significant regional differences, such as vegetation, rural buildings, and water systems, is beneficial to improving the generalization ability of subsequent land feature identification models under different regional and seasonal conditions. 4. An overlapping sliding window clipping method is adopted to preserve the contextual information of the ground features while expanding the number of samples, thereby reducing the probability of the target being truncated by the clipping boundary. 5. The polygonal fine annotation method can more accurately express the true shape of targets such as rural building roofs, road boundaries, vegetation canopy, water system edges and pipelines compared with rectangular annotation. 6. Sample annotation information is organized using COCO JSON format, which facilitates subsequent training of target detection, semantic segmentation, instance segmentation, and remote sensing intelligent interpretation models; 7. By combining self-inspection by annotators, cross-inspection by peers, and random inspection by experts, the accuracy, consistency, and traceability of the annotation results in the sample library have been improved. Attached Figure Description

[0015] The following description, in conjunction with the accompanying drawings, further illustrates this application: Figure 1 Here is a flowchart of the construction and annotation methods in this application; Figure 2 This is a schematic diagram of the building category in the remote sensing land cover hierarchical classification system of this application; Figure 3 This is a schematic diagram of the road category in the remote sensing land cover hierarchical classification system of this application; Figure 4 This is a schematic diagram of the water system category in the remote sensing land cover hierarchical classification system of this application; Figure 5 This is a schematic diagram of vegetation categories in the remote sensing land cover hierarchical classification system of this application; Figure 6 This is a schematic diagram of the pipeline category in the remote sensing land cover hierarchical classification system of this application; Figure 7 This is a schematic diagram of an independent land cover category in the remote sensing land cover hierarchical classification system of this application. Detailed Implementation

[0016] like Figures 1 to 7 As shown, this application provides a method for constructing and labeling a remote sensing ground feature sample database, including the following steps: Step 1: Acquire remote sensing image data and preprocess the remote sensing image data to obtain remote sensing orthophoto data; the preprocessing includes coordinate unification, orthorectification, image stitching, color equalization, abnormal area removal and image quality check.

[0017] Step 2: Establish a hierarchical classification system for remote sensing ground features. The ground feature categories in the hierarchical classification system include buildings, roads, vegetation, water systems, pipelines, and independent ground features. The above six ground feature categories are further subdivided according to the ground feature morphology, material differences, usage attributes, spatial distribution patterns, texture features, and image representation differences: buildings include tiled houses, bungalows, prefabricated steel houses, photovoltaic roof houses, and greenhouses; roads include asphalt roads, cement roads, dirt roads, and railways; vegetation includes trees, shrubs, isolated trees, and cultivated land with seedlings; water systems include lakes, rivers, ponds, canals, and reservoirs; pipelines include poles, power lines, and above-ground pipelines; and independent ground features include graves.

[0018] Step 3: Based on the changes in remote sensing image data of land cover categories in different regions and time phases, perform spatiotemporal sensitivity discrimination on land cover categories in the remote sensing land cover hierarchical classification system. Spatiotemporal sensitivity discrimination includes temporal sensitivity discrimination and spatial sensitivity discrimination. Temporal sensitivity discrimination is used to determine whether the color, texture, coverage, boundary range, or morphological characteristics of the same land cover category change at different times. For example, the color, coverage, and texture characteristics of vegetation differ significantly in different seasons (spring, summer, autumn, and winter); the water surface range of water systems changes at different times; and some surface coverage areas will exhibit different image characteristics due to seasonal changes. Spatial sensitivity discrimination is used to determine whether there are differences in the material, color, scale, spatial distribution, background environment, or image representation of the same land cover category in different regions. For example, buildings in North China, South China, Northwest China, and Southwest China exhibit differences due to differences in roof materials, building density, settlement patterns, and surrounding environments; roads in different regions present different forms such as paved roads, dirt roads, and field roads; and the texture and coverage of vegetation also differ in different climatic regions. Land cover categories with significant temporal or spatial differences within remote sensing image data are classified as spatiotemporally sensitive land cover categories; land cover categories without significant temporal or spatial differences within remote sensing image data are classified as spatiotemporally insensitive land cover categories.

[0019] Step 4: Establish fine-grained sub-databases for spatiotemporally sensitive features, categorized by region and time phase; establish a unified sample database for spatiotemporally insensitive features; The fine-grained sub-libraries include regional and temporal labels. Regional labels include at least one of North China, South China, Northwest China, and Southwest China, while temporal labels include at least one of spring, summer, autumn, and winter. For example, for vegetation features, separate vegetation sample sub-libraries are established for different regions and times; for building features, regional sub-libraries are established based on regional differences, and temporal labels are added according to the season of image acquisition; for water system features, corresponding temporal sample sub-libraries are established in conjunction with seasonal water surface changes.

[0020] Step 5: Perform sliding window cropping on the remote sensing orthophoto data according to the preset window size and preset step size to obtain the sample image; Step 6: Screen the sample images, removing blurry, severely occluded, or excessively projected images. For multi-scale samples of small targets, manual visual verification is required to determine whether the target has been truncated by clipping boundaries or has undergone severe deformation. Samples that do not meet the annotation requirements will not proceed to subsequent steps.

[0021] Step 7: Perform polygon annotation on the ground feature instances in the sample image to obtain the outlines of the ground feature instances that fit the true boundaries of the ground feature instances; specifically, this includes the following steps: Step 7.1: For buildings within the remote sensing feature hierarchical classification system, mark the outer contour of the corresponding feature instance; Step 7.2: For roads within the remote sensing land cover classification system, mark the road surface area (excluding the shoulder green area). Step 7.3: For vegetation within the remote sensing land cover stratification system, label the continuous canopy area; Step 7.4: For water systems within the remote sensing land cover stratification system, mark the outer contour of the perennial or seasonal water surface; Step 7.5: For pipelines within the remote sensing feature hierarchical classification system, label the visible line segments of the corresponding feature instances; Step 7.6: For independent features within the remote sensing feature hierarchical classification system, mark the outer contour of the corresponding feature instance.

[0022] Step 8: Organize the sample images and annotation information according to the Context Common Objects Dataset (COCO JSON) format to obtain the sample annotation results; the annotation information includes the sample image number, file name, image width, image height, category number, category name, segmentation polygon coordinates, target area, bounding rectangle, attribute information, region label, time label, image source and quality label.

[0023] Step 9: Perform quality checks on the sample annotation results and verify the fine-grained sub-libraries separately. Input the sample images and annotation information that pass the quality check and separate verification into the database to obtain the remote sensing ground feature sample database.

[0024] In the embodiments of this application, the remote sensing image data is a high-resolution remote sensing image, preferably a UAV remote sensing DOM orthophoto with a spatial resolution of 0.1 meters; in step 5, the preset window size is 1024×1024 pixels and the preset step size is 512 pixels, so that there is a 50% overlap between adjacent sample images. The format of the sample image is TIFF or JPEG, the channel is 3-channel RGB, the bit depth is 8-bit integer, and the pixel size is 1024×1024; in step 7, the error between the polygon annotation boundary of the ground feature instance and the actual ground feature boundary is no more than 2 pixels. Ground feature instances with an occlusion area greater than 70% are not annotated or are marked as ignored areas; the minimum annotation area threshold for buildings is set to 10 pixels, and the minimum line width threshold for pipelines is set to 5 pixels.

[0025] In step 9, quality checks include self-checking by annotators, cross-checking by peers, and expert sampling. Self-checking involves annotators reviewing their own annotation results, focusing on issues such as missing annotations, incorrect annotations, boundary offsets, category confusion, missing attributes, and format errors. Cross-checking involves other annotators reviewing the annotation results, focusing on whether annotation boundaries conform to the feature outlines, whether category classifications are consistent, and whether the COCO JSON format is standardized. Expert sampling involves the quality control team or experts randomly selecting 30% of the total annotation results. Expert sampling focuses on the quality of complex features, small targets, features with ambiguous boundaries, and samples in regional and temporal sub-databases. The overall annotation pass rate must not be lower than 98%. Unqualified samples are returned to the annotation stage for modification and then re-enter step 9. Individual verification of fine-grained sub-databases includes verifying whether the number of samples under each category combination meets the standard, whether category labels, regional labels, and temporal labels are accurate, whether small target samples are truncated or severely deformed, and whether the sample images correspond one-to-one with the annotation files.

[0026] A system for implementing the method for constructing and labeling a remote sensing ground object sample library as described above includes an image data acquisition and preprocessing module whose output is electrically connected to the input of a hierarchical classification system construction module; an image data acquisition and preprocessing module whose output is electrically connected to the input of a spatiotemporal sensitivity discrimination module; an image data acquisition and preprocessing module whose output is electrically connected to the input of a spatiotemporal adaptation sub-library construction module; a spatiotemporal adaptation sub-library construction module whose output is electrically connected to the input of a sliding window cropping module; and a sliding window cropping module whose output is electrically connected to the input of a sample screening and labeling module.

[0027] In practical work, the image data acquisition and preprocessing module is used to acquire remote sensing image data and preprocess it; the hierarchical classification system construction module is used to establish a hierarchical classification system for remote sensing ground features, including buildings, roads, vegetation, water systems, pipelines, and independent ground features; the spatiotemporal sensitivity discrimination module is used to discriminate the temporal and spatial sensitivity of different ground feature categories and determine the corresponding sample production strategy; the spatiotemporal adaptation sub-library construction module is used to establish fine-grained sub-libraries for spatiotemporally sensitive ground features by region and time, and to establish a unified sample library for spatiotemporally insensitive ground features; the sliding window cropping module is used to perform sliding window cropping on remote sensing orthophotos according to preset window size and preset step size; the sample screening and annotation module is used to screen valid sample images and perform polygon annotation on ground feature instances in the sample images; the annotation information organization module is used to organize sample images and their annotation information according to COCO JSON format; and the quality inspection and storage module is used to perform quality inspection on the sample annotation results and store qualified samples in the storage. The quality inspection and warehousing module includes a self-inspection unit, a cross-inspection unit, an expert sampling inspection unit, and a sub-library verification unit. The self-inspection unit is used by annotators to check the sample annotation results they have completed. The cross-inspection unit is used by peer annotators to review the sample annotation results. The expert sampling inspection unit is used by the quality inspection team or experts to conduct random checks on the sample annotation results. The sub-library verification unit is used to verify the sample quantity, category label, region label, and phase label in the fine-grained sub-library separately.

[0028] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A method for constructing and labeling a remote sensing ground feature sample database, characterized in that, Includes the following steps: Step 1: Collect remote sensing image data, preprocess the remote sensing image data to obtain remote sensing orthophoto data; Step 2: Establish a hierarchical classification system for remotely sensed ground features; Step 3: Based on the changes in remote sensing image data of land cover categories in different regions and time phases, determine the spatiotemporal sensitivity of land cover categories in the remote sensing land cover hierarchical classification system. Land cover categories with obvious temporal or spatial differences in remote sensing image data are determined to be spatiotemporally sensitive land cover categories; land cover categories without obvious temporal or spatial differences in remote sensing image data are determined to be spatiotemporally insensitive land cover categories. Step 4: Establish fine-grained sub-databases for spatiotemporally sensitive features, categorized by region and time phase; establish a unified sample database for spatiotemporally insensitive features; Step 5: Perform sliding window cropping on the remote sensing orthophoto data to obtain sample images; Step 6: Screen the sample images and remove blurry, severely occluded, or excessively large projection differences. Step 7: Perform polygon annotation on the ground feature instances in the sample image to obtain the outlines of the ground feature instances that fit the true boundaries of the ground feature instances; Step 8: Organize the sample images and annotation information to obtain the sample annotation results; Step 9: Perform quality checks on the sample annotation results and verify the fine-grained sub-libraries separately. Input the sample images and annotation information that pass the quality check and separate verification into the database to obtain the remote sensing ground feature sample database. Spatiotemporal sensitivity discrimination includes temporal sensitivity discrimination and spatial sensitivity discrimination. Temporal sensitivity discrimination is used to determine whether the color, texture, coverage, boundary range or morphological characteristics of the same land cover category change at different times. Spatial sensitivity discrimination is used to determine whether there are differences in the material, color, scale, spatial distribution, background environment or image performance of the same land cover category in different regions.

2. The method for constructing and labeling a remote sensing ground feature sample database according to claim 1, characterized in that: The annotation information includes the sample image number, file name, image width, image height, category number, category name, segmented polygon coordinates, target area, circumscribed rectangle, attribute information, region label, time label, image source, and quality label.

3. The method for constructing and labeling a remote sensing ground feature sample database according to claim 1, characterized in that: The land cover categories in the remote sensing land cover hierarchical classification system include buildings, roads, vegetation, water systems, pipelines, and independent land cover.

4. The method for constructing and labeling a remote sensing ground feature sample database according to claim 3, characterized in that: Buildings include tiled houses, bungalows, prefabricated steel houses, photovoltaic roof houses, and greenhouses; roads include asphalt roads, cement roads, dirt roads, and railways; vegetation includes trees, shrubs, solitary trees, and cultivated land with seedlings; water systems include lakes, rivers, ponds, canals, and reservoirs; pipelines include poles, power lines, and above-ground pipelines; and independent features include grave mounds.

5. The method for constructing and labeling a remote sensing ground feature sample database according to claim 1, characterized in that: The fine-grained sub-library includes regional labels and temporal labels. Regional labels include at least one of North China, South China, Northwest China, and Southwest China, while temporal labels include at least one of Spring, Summer, Autumn, and Winter.

6. The method for constructing and labeling a remote sensing ground feature sample database according to claim 1, characterized in that: In step 1, preprocessing includes coordinate unification, orthorectification, image stitching, color equalization, abnormal area removal, and image quality inspection.

7. A system for implementing the method for constructing and labeling a remote sensing ground feature sample database as described in any one of claims 1-6, characterized in that, The output of the image data acquisition and preprocessing module is electrically connected to the input of the hierarchical classification system construction module. The output of the hierarchical classification system construction module is electrically connected to the input of the spatiotemporal sensitivity discrimination module. The output of the spatiotemporal sensitivity discrimination module is electrically connected to the input of the spatiotemporal adaptation sub-library construction module. The output of the spatiotemporal adaptation sub-library construction module is electrically connected to the input of the sliding window cropping module. The output of the sliding window cropping module is electrically connected to the input of the sample screening and labeling module.

Images