A remote sensing image sample generation method and device based on spatial perception division
Patent Information
- Application Number
- CN202611496855.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-24
- Publication Date
- 2026-10-30
AI Technical Summary
然而，现有的遥感影像样本生成方案主要面临两个方面的问题，一方面，训练集与验证集的划分多采用随机打乱或按类别平衡的方式，未能考虑遥感影像的地理空间连续特性
[0017]本说明书实施例提供的方案，通过按照设定的训练集重叠率生成候选边界框，按最小空间间隔约束下确定的间隔生成候选验证中心点并以最近邻方式选取验证边界框，再剔除与验证边界框存在空间重叠的边界框作为训练边界框的方式，实现了训练集边界框与验证集边界框在地理空间上的完全隔离，使训练集与验证集之间不存在任何空间交叉，从而消除因传统随机划分导致同一地理区域样本同时出现在训练集与验证集中所造成的空间数据泄漏问题，使模型评估结果能够真实反映泛化能力。此外，处理过程中根据各统计网格内候选边界框密度计算密度加权采样权重，实现对稀疏标注区域的优先采样，保证了训练集与验证集各自在空间维度上的分布均匀性，避免因标注数据空间分布不均导致的模型局部盲区与泛化能力衰退。
Smart Images
Figure CN123016124A_ABST
Abstract
Description
Technical Field
[0001] The embodiments in this specification pertain to the field of remote sensing image sample generation, and particularly relate to a method and apparatus for generating remote sensing image samples based on spatial perception division. Background Technology
[0002] In the field of intelligent interpretation of remote sensing images, the training of deep learning models relies on a large number of high-quality labeled samples. Remote sensing images typically cover a vast area (a single image can reach tens to hundreds of square kilometers), requiring the generation of fixed-size training samples through sliding window tiling. However, existing remote sensing image sample generation schemes mainly face two problems. First, the division of training and validation sets often uses random shuffling or class balancing, failing to consider the geospatial continuity of remote sensing images. Since there is usually a large overlap between adjacent samples during sliding window tiling, random division can easily lead to slices of the same geographic area appearing in both the training and validation sets, causing spatial data leakage and inflating model evaluation results that fail to accurately reflect generalization ability. Although some schemes have circumvented this problem in individual projects by manually defining training / prediction areas, a systematic automatic spatial isolation and partitioning scheme is still lacking.
[0003] On the other hand, existing remote sensing image sample export systems are typically designed for a single task type, with labeled data stored directly in a task-specific output format. When generating samples for different task types for the same area, labeling or format conversion must be performed for each task type, making labeling work unusable. Furthermore, labeled data stored in raster format loses its original geometric information, making lossless conversion between different task types impossible. In addition, raster-formatted labeled data is tied to a specific resolution; when outputting training samples at different resolutions, rasterization must be performed again, making it difficult to flexibly meet the diverse input resolution requirements of different model architectures. Summary of the Invention
[0004] The embodiments of this disclosure provide a method and apparatus for generating remote sensing image samples based on spatially perceived segmentation, which aims to solve one or more of the above-mentioned problems and other potential problems.
[0005] According to a first aspect of this disclosure, a method for generating remote sensing image samples based on spatially perceived partitioning is provided. The method includes determining request parameters and a minimum spatial interval constraint for the remote sensing image based on an export task request for the remote sensing image. The request parameters include a training set ratio and a training set overlap rate. The minimum spatial interval constraint is set based on a preset scaling factor of the sample geographic size.
[0006] Based on the overlap rate of the training set, multiple candidate bounding boxes are generated within the remote sensing image, and the remote sensing image is divided into multiple statistical grids to determine the number of candidate bounding boxes in each statistical grid. The density-weighted sampling weight of each statistical grid is calculated based on the number of candidate bounding boxes, and the density-weighted sampling weight is higher for statistical grids with sparser density.
[0007] Multiple candidate validation center points are generated in the remote sensing image. Weighted random sampling is performed based on the density-weighted sampling weight of the statistical grid to which each candidate validation center point belongs until the number of validation center points of the target is obtained. The interval between any two candidate validation center points satisfies the minimum spatial interval constraint. The number of targets is determined based on the proportion of the training set.
[0008] Using each validation center point as a reference, the nearest unoccupied bounding box in each candidate bounding box is used as the validation set bounding box, and the candidate bounding boxes that do not have spatial overlap with the validation set bounding boxes are used as the training set bounding boxes.
[0009] Based on the geographical extent of the bounding boxes of the training set and the bounding boxes of the validation set, image slices are cropped from remote sensing images to obtain the training sample set and the validation sample set.
[0010] According to a second aspect of this disclosure, a remote sensing image sample generation device based on spatial perception partitioning is provided. The device includes a task parsing module configured to determine request parameters and minimum spatial interval constraints of the remote sensing image based on an export task request for the remote sensing image. The request parameters include a training set ratio and a training set overlap rate, and the minimum spatial interval constraint is set based on a preset magnification of the sample geographic size.
[0011] The bounding box generation module is configured to generate multiple candidate bounding boxes within the remote sensing image based on the overlap rate of the training set, and divide the remote sensing image into multiple statistical grids to determine the number of candidate bounding boxes in each statistical grid. The density-weighted sampling weight of each statistical grid is calculated based on the number of candidate bounding boxes, and the density-weighted sampling weight is higher for statistical grids with sparser density.
[0012] The random sampling module is configured to generate multiple candidate validation center points in the remote sensing image, and perform weighted random sampling based on the density-weighted sampling weight of the statistical grid to which each candidate validation center point belongs, until the target number of validation center points is obtained; the interval between any two candidate validation center points satisfies the minimum spatial interval constraint, and the target number is determined based on the proportion of the training set;
[0013] The bounding box classification module is configured to use each validation center point as a reference to find the nearest unoccupied bounding box in each candidate bounding box as the validation set bounding box, and to use the candidate bounding boxes that do not have spatial overlap with the validation set bounding boxes as the training set bounding boxes.
[0014] The sample set construction module is configured to crop image slices from remote sensing images based on the geographic extent of the training set bounding boxes and validation set bounding boxes to obtain the training sample set and validation sample set.
[0015] According to a third aspect of this disclosure, an electronic device is provided, including one or more processors and a memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform a method provided according to a first scheme.
[0016] According to a fourth aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the method provided according to the first aspect.
[0017] The scheme provided in this specification generates candidate bounding boxes according to a set training set overlap rate, generates candidate validation center points at intervals determined under minimum spatial interval constraints, selects validation bounding boxes using a nearest neighbor approach, and then discards bounding boxes that spatially overlap with validation bounding boxes as training bounding boxes. This achieves complete geographical isolation between training and validation bounding boxes, ensuring no spatial overlap between them. This eliminates the spatial data leakage problem caused by traditional random partitioning, where samples from the same geographical area appear simultaneously in both the training and validation sets, allowing the model evaluation results to accurately reflect generalization ability. Furthermore, the processing calculates density-weighted sampling weights based on the density of candidate bounding boxes within each statistical grid, prioritizing sampling of sparsely labeled regions. This ensures the uniform spatial distribution of both the training and validation sets, avoiding local blind spots and generalization ability degradation caused by uneven spatial distribution of labeled data. Attached Figure Description
[0018] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein:
[0019] Figure 1 A flowchart illustrating a remote sensing image sample generation method based on spatially perceptual segmentation, according to some embodiments of this disclosure, is shown.
[0020] Figure 2 A schematic diagram of the structure of a remote sensing image sample generation apparatus based on spatial perception partitioning according to some embodiments of the present disclosure is shown.
[0021] Figure 3 A schematic block diagram of an electronic device according to some embodiments of the present disclosure is shown. Detailed Implementation
[0022] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.
[0023] The terms “comprising” and “having”, and any variations thereof, in this specification, claims, and the foregoing drawings are intended to cover a non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus. Depending on the context, the word “if” as it applies herein may be interpreted as “when”, “in response to determination”, or “in response to detection”.
[0024] Figure 1 The diagram illustrates a flowchart of a remote sensing image sample generation method based on spatially sensed partitioning, representing some embodiments of this disclosure. Method 100 can be executed by a terminal, which may include, but is not limited to, mobile phones, tablets, desktop computers, servers, etc. Figure 1 As shown, in method 100, step 102 can determine the request parameters and minimum spatial interval constraints of the remote sensing image based on the export task request for the remote sensing image. The request parameters include the training set ratio and the training set overlap rate, and the minimum spatial interval constraint is based on a preset scaling factor of the sample geographic size.
[0025] In this embodiment, the user sends an export task request to the terminal. The user inputs request parameters and a minimum spatial interval constraint set based on the sample geographic size in the export task request. This minimum spatial interval constraint is used to constrain the minimum spatial interval between subsequently generated validation samples. The export task request indicates that the user wants to divide the selected remote sensing imagery according to the input data to generate a training sample set and a validation sample set for model training. The request parameters include the training set proportion (i.e., the percentage of the total number of samples) and the training set overlap rate (i.e., the overlap rate between training sets). The sample geographic size can be the product of the sample pixel size and the image resolution; the sample pixel size and image resolution can also be input by the user. By parsing the export task request, the corresponding request parameters and minimum spatial interval constraint can be determined, facilitating subsequent processing.
[0026] As an example, the minimum spatial interval constraint can be set to a spacing that is not less than a preset multiple of the sample geographic size, for example:
[0027]
[0028] in, To verify the minimum spatial interval between samples, This represents the geographic size of the sample.
[0029] In method 100, step 104 can generate multiple candidate bounding boxes within the remote sensing image based on the overlap rate of the training set, and divide the remote sensing image into multiple statistical grids to determine the number of candidate bounding boxes in each statistical grid. The density-weighted sampling weight of each statistical grid is calculated based on the number of candidate bounding boxes, and the density-weighted sampling weight is higher for statistical grids with sparser density.
[0030] In this embodiment, the sliding frame step size can be calculated based on the training set overlap rate set by the user, as follows:
[0031]
[0032] in, The slider step size, For sample geographic dimensions, This represents the overlap rate of the training set.
[0033] The sliding frame step size determines the geographic distance between the center points of two adjacent candidate slices, allowing sliding frame operations to be performed on remote sensing images to generate multiple candidate bounding boxes. Simultaneously, since remote sensing images are generally large, for ease of processing, they are divided into multiple statistical grids according to a pre-defined statistical grid dimension. For example, assuming a statistical grid dimension M of 4, 16 statistical grids will be generated. For each statistical grid, the number of candidate bounding boxes within that grid (i.e., the number of candidate bounding box center points falling within the grid) is determined. The number of candidate bounding boxes represents the density of candidate bounding boxes within that grid, and the density-weighted sampling weight of the statistical grid is then calculated. Sparsely labeled regions receive higher sampling weights and are prioritized for selection in subsequent sampling processes, thus ensuring spatial distribution balance. As an example, the formula for calculating the density-weighted sampling weight is:
[0034]
[0035] in, For density-weighted sampling weights, For the first Line 1 Statistical grid of columns The number of candidate bounding boxes within, This is the smoothing constant.
[0036] In method 100, step 106 can generate multiple candidate verification center points in the remote sensing image, and perform weighted random sampling based on the density-weighted sampling weight of the statistical grid to which each candidate verification center point belongs, until the target number of verification center points is obtained; the interval between any two candidate verification center points satisfies the minimum spatial interval constraint, and the target number is determined based on the proportion of the training set.
[0037] In this embodiment, the target number of validation sets can first be calculated based on the proportion of the training set as follows:
[0038]
[0039] in, The target number of validation sets. The total number of candidate bounding boxes. This represents the proportion of the training set.
[0040] Under the minimum spatial interval constraint, multiple candidate verification center points are generated in the remote sensing image. Besides directly generating candidate verification center points based on the minimum spatial interval constraint, to make the distribution of candidate verification center points more uniform, several initial candidate points with a number greater than the target number can be generated first, based on a preset step size (if the number of center points satisfying the minimum spatial interval constraint is less than the target number, then the preset step size does not need to satisfy the minimum spatial interval constraint; otherwise, it should). Then, a greedy selection algorithm is used; for each selected initial candidate point, other initial candidate points within the interval range that do not satisfy the minimum spatial interval constraint are disabled, thus ultimately selecting the candidate verification center points. For each candidate verification center point, its corresponding statistical grid is determined to obtain the density-weighted sampling weight of that statistical grid. Next, based on the density-weighted sampling weight, a weighted random sampling method using roulette wheel selection is employed to finally select the target number of verification center points.
[0041] In method 100, step 108 can use each validation center point as a reference to find the nearest unoccupied bounding box in each candidate bounding box as the validation set bounding box, and use the candidate bounding boxes that do not have spatial overlap with the validation set bounding boxes as the training set bounding boxes.
[0042] In this embodiment, using each validation center point as a reference, the Euclidean distance from each validation center point to the center point of each unoccupied bounding box can be calculated. The nearest unoccupied bounding box (i.e., the one with the smallest Euclidean distance) is then found as the validation set bounding box corresponding to that validation center point. Thus, since the geographical distance between each validation center point is greater than the geographical size of each bounding box under the minimum spatial distance constraint, there is no geographical overlap between any two validation bounding boxes obtained. Furthermore, after removing all candidate bounding boxes that spatially overlap with the determined validation set bounding boxes, the remaining candidate bounding boxes will be used as the training set bounding boxes.
[0043] In addition, to speed up the search for the nearest unoccupied bounding box, a spatial index can be set for each candidate bounding box in advance, and the ID of each candidate bounding box can be recorded in the associated list of the corresponding spatial index. In this way, when calculating the Euclidean distance, only the index cell of the spatial index and its 8 adjacent index cells, i.e., the 3x3 neighborhood, can be calculated based on the spatial index coordinates of the verification center point, so as to reduce the computational complexity.
[0044] In method 100, step 110 can crop image slices from remote sensing images based on the geographical extent of the training set bounding boxes and the validation set bounding boxes to obtain the training sample set and the validation sample set.
[0045] In this embodiment, based on the geographical extent of the determined training set and validation set bounding boxes, pixel regions within each bounding box range can be cropped from the original remote sensing image, i.e., image slices. These slices are then stored in the training sample set directory or the validation sample set directory, depending on the type of the bounding box, to ultimately obtain the training sample set and the validation sample set. Through this method, the complete geographical isolation between the training set and the validation set eliminates spatial data leakage problems at the source of the sample set, providing a high-quality, standardized training data foundation for the intelligent interpretation model of remote sensing images.
[0046] In one possible implementation, the method further includes:
[0047] In response to the existence of a user-specified cropping region in the export task request, the intersection of the cropping region and the remote sensing image is determined as the valid export region for the export task.
[0048] In this embodiment, under certain circumstances, users may not want to use the entire remote sensing image for dividing the training and validation sets, but instead want to train the model on a specific portion of the remote sensing image. In this case, when generating the export task request, the user can specify a cropping region in the remote sensing image by selecting a coordinate range or setting the vector file. The terminal will calculate the intersection of the specified cropping region and the remote sensing image, and determine the intersection region as the valid export region for this export task. That is, subsequent processing operations on the training and validation sample sets will only be performed within the identified valid export region, without considering other regions.
[0049] In one possible implementation, multiple candidate bounding boxes are generated within the remote sensing image based on the overlap rate of the training set, including:
[0050] The sliding frame step size is calculated based on the overlap rate of the training set and the geographic size of the samples. The sliding frame step size is used to uniformly slide the remote sensing image in the horizontal and / or vertical directions to generate multiple candidate bounding boxes. The union of all candidate bounding boxes covers the entire remote sensing image. The geographic size of the samples is determined based on the product of the preset sample pixel size and the image resolution.
[0051] In this embodiment, after calculating the sliding box step size based on the training set overlap rate and sample geographic size, the sliding box is uniformly slid across the remote sensing image both horizontally and vertically with this step size until the entire remote sensing image is covered, thus obtaining each candidate bounding box. In this way, since the training set overlap rate is a user-controllable input parameter, it allows users to flexibly adjust the density of candidate samples based on sample scarcity, providing a sufficient candidate pool for subsequent spatial isolation partitioning and avoiding the inability to select a sufficient number of training and validation samples due to an initially insufficient number of candidate bounding boxes.
[0052] In one possible implementation, before obtaining the training sample set and validation sample set by cropping image slices from remote sensing imagery based on the geographical extent of the training set bounding boxes and validation set bounding boxes, the method further includes:
[0053] Calculate the spatial coverage and coefficient of variation of the training set bounding box set and the validation set bounding box set in the statistical grid, respectively. The spatial coverage is the ratio of the number of statistical grids covered by the bounding box set to the total number of statistical grids. The coefficient of variation of the spatial distribution is the ratio of the standard deviation to the mean of the number of target bounding boxes in each statistical grid. The number of target bounding boxes is the number of bounding boxes in a single statistical grid.
[0054] In response to the spatial coverage of either the training set bounding box set or the validation set bounding box set being lower than a preset coverage threshold or the spatial distribution coefficient of variation exceeding a preset coefficient of variation threshold, candidate validation center points are regenerated, and the training set bounding boxes and validation set bounding boxes are redefined.
[0055] In this embodiment, after determining the bounding boxes of each training set and each validation set, the spatial coverage and spatial distribution variation coefficient of the training set bounding box set and the validation set bounding box set will be calculated to verify the spatial distribution balance of each set. The spatial coverage is the ratio of the number of statistical grids covered by the bounding box set to the total number of statistical grid cells. The spatial distribution variation coefficient is calculated by counting the number of bounding boxes belonging to the bounding box set within each statistical grid, and then using the number of bounding boxes within each statistical grid as a sample, calculating the ratio of the standard deviation to the mean of each sample. The spatial distribution variation coefficient characterizes the ratio of the dispersion of the number of bounding boxes within each statistical grid to the average level. The closer this value is to 0, the closer the sample size of each grid is to the average, and the more uniform the spatial distribution; the larger the value, the more concentrated the samples are in some grids and the more sparse they are in other grids, and the more uneven the spatial distribution. If the spatial coverage of any set is lower than a preset coverage threshold or the coefficient of variation of the spatial distribution exceeds a preset coefficient of variation threshold, then the selected training set bounding box set and validation set bounding box set are considered to not meet the spatial distribution balance requirement. Candidate validation center points will be regenerated to re-select and determine the training set bounding boxes and validation set bounding boxes. In this way, by jointly validating the results using two quantitative indicators, coverage and coefficient of variation, the spatial distribution uniformity of the partitioning results can be objectively and quantitatively evaluated. Automatic re-partitioning is triggered when the threshold is not met, thus ensuring that the final output training set and validation set each have sufficient coverage and distribution uniformity in the spatial dimension, avoiding model blind spots caused by certain geographical areas being completely uncovered by any set.
[0056] In one possible implementation, image slices are cropped from remote sensing imagery based on the geographical extent of the training set bounding boxes and the validation set bounding boxes to obtain the training sample set and the validation sample set, including:
[0057] For any training set bounding box or validation set bounding box, the geographic extent of the bounding box is used as the query window to retrieve matching vector polygons from the vector polygon annotation database, and image slices are cropped from the remote sensing image based on the geographic extent of the bounding box.
[0058] The vector polygon is rendered into a label format that matches the preset sample type identifier to obtain the labeled data;
[0059] By associating image slices with labeled data, training or validation samples corresponding to a single bounding box can be obtained.
[0060] Integrate all training and validation samples to obtain the training sample set and validation sample set.
[0061] In this embodiment, in addition to cropping image slices based on the geographical extent of the bounding boxes, vector polygons are retrieved from the vector polygon annotation database based on the geographical extent of each bounding box. A vector polygon is a closed shape formed by a series of connected geographical coordinate points; all vector polygons within the same geographical area constitute the complete annotation layer for that area. The vector polygon annotation database can be obtained through manual annotation interpretation or generated by automatic or semi-automatic remote sensing image interpretation algorithms. When initiating an export task request, the user can also set a sample type identifier. The retrieved vector polygons will be rendered in an annotation format matching that sample type identifier, resulting in annotated data. Different sample type identifiers have different rendering methods. For example, in semantic segmentation, the rendering method can be to rasterize the vector polygons into a palette mask image. Finally, the annotation data and image slices are associated to form the training or validation samples corresponding to the bounding box. By using real-time querying and on-demand rendering of the vector polygon annotation database, the same vector polygon data can be reused repeatedly in different export tasks. Each time, it is rendered into the corresponding annotation format according to the user-specified task type, while the original vector data itself is never modified or converted, thus achieving lossless reuse across tasks.
[0062] In one possible implementation, the method further includes:
[0063] In response to a single export task request that sets multiple target resolution values, the annotation data and image tiles of each bounding box are calculated separately according to different target resolution values, so that the same bounding box corresponds to multiple training samples and validation samples under different target resolutions.
[0064] In this embodiment, in practical applications, different deep learning model architectures have different requirements for the resolution of input images. Therefore, it can also support specifying multiple target resolution values in a single export task, and output training and validation sample sets with multiple resolutions based on the same vector polygon annotation data and image tiles. Specifically, if multiple target resolution values are set in the export task request, the remote sensing image within the corresponding bounding box will be resampled with each target resolution value to crop image tiles of different pixel sizes. At the same time, when generating annotation data, the vector polygons will also be rasterized and rendered using each target resolution value. The higher the resolution, the more pixels the canvas has, and the more finely the vector polygon boundaries are depicted; conversely, the fewer the canvas pixels, the coarser the vector polygon boundaries. However, the coordinate values of the vector polygons themselves (such as the latitude and longitude of the corner points of building outlines) remain unchanged at any resolution. In this way, the same spatial partitioning result and the same vector annotation data can meet the diverse needs of different model architectures for input resolution, without having to re-perform spatial partitioning and annotation queries for each resolution, thus improving the flexibility and efficiency of sample generation.
[0065] In one possible implementation, the sample type identifier includes at least one of semantic segmentation, object detection, rotated bounding box detection, change detection, and instance segmentation; the annotation format corresponding to semantic segmentation is a palette mask image, the annotation format corresponding to object detection is normalized bounding box coordinate text, the annotation format corresponding to rotated bounding box detection is rotated rectangle four-point coordinate text, the annotation format corresponding to change detection is a binary change mask image, and the annotation format corresponding to instance segmentation is run-length encoded JSON data.
[0066] In this embodiment, the sample type is identified by a preset integer code, and different values represent different deep learning task types that the user expects to export.
[0067] The goal of semantic segmentation is to classify each pixel in an image, with the corresponding annotation format being a palette mask image. When rendering semantic segmentation annotations, a zero-based matrix of the same size as the sample can be created. For each vector polygon retrieved from the vector database, its category label is determined, and the pixel positions covered by that vector polygon are assigned that index value. Finally, the matrix is saved as a PNG format palette mask image. If the same pixel position is covered by multiple vector polygons (this usually doesn't happen because two polygons of different categories won't exist simultaneously in the same area of the annotation data), the final category is determined according to a preset priority rule (e.g., the largest area takes precedence).
[0068] The purpose of object detection is to identify target objects in an image and provide their bounding boxes, with the corresponding annotation format being normalized bounding box coordinate text (TXT format). When rendering object detection annotations, for each retrieved vector polygon, its minimum bounding axis aligned rectangle (i.e., the bounding rectangle) is calculated, obtaining the coordinates of the top-left corner (x_min, y_min) and the bottom-right corner (x_max, y_max) of the rectangle. These coordinates are then normalized to YOLO format and saved as a TXT file.
[0069] Rotated bounding box detection is an extension of object detection, used to detect targets that have undergone angular rotation (such as vehicles and ships in remote sensing imagery). Its corresponding annotation format is the four-point coordinate text of the rotated rectangle (TXT format). When rendering the rotated bounding box annotations, for each retrieved vector polygon, its minimum bounding rectangle (i.e., the rectangle with the smallest area that can contain the polygon at any angle) is calculated. The coordinates of the four vertices of the rotated rectangle are obtained, normalized, and then written to a TXT file.
[0070] The purpose of change detection is to detect changes in the same geographic area between two different time phases, with the corresponding annotation format being a binary change mask image (PNG format). When rendering change detection annotations, for dual-temporal images, image slices within the current bounding box are cropped as input for both phases. The change mask is derived from vector polygon annotation databases marked as "change areas" (usually change patches annotated after manual comparison of dual-temporal images).
[0071] The goal of instance segmentation is to segment each target instance in an image at the pixel level and distinguish different instances of the same category. The corresponding annotation format is COCO-formatted run-length encoded JSON data. When rendering instance segmentation annotations, each retrieved vector polygon is encoded using COCO-formatted run-length encoding (RLE). RLE is a lossless compression encoding method: it unfolds the mask image row by row into a one-dimensional sequence, recording the run length of consecutive pixels with the same value. Each vector polygon is treated as an independent instance, assigned a unique instance ID, and output as a JSON file.
[0072] Figure 2 Schematic diagrams of a remote sensing image sample generation apparatus based on spatially perceptual segmentation, according to some embodiments of this disclosure, are shown. The various embodiments in this specification are described in a progressive manner; similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the apparatus embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments. Figure 2 As shown, the device 200 includes a task parsing module 201, which is configured to determine the request parameters and minimum spatial interval constraints of the remote sensing image based on the export task request for the remote sensing image. The request parameters include the training set ratio and the training set overlap rate, and the minimum spatial interval constraint is set based on a preset magnification of the sample geographic size.
[0073] The bounding box generation module 202 is configured to generate multiple candidate bounding boxes in the remote sensing image based on the overlap rate of the training set, and divide the remote sensing image into multiple statistical grids to determine the number of candidate bounding boxes in each statistical grid. The density-weighted sampling weight of each statistical grid is calculated based on the number of candidate bounding boxes, and the density-weighted sampling weight is higher for statistical grids with sparser density.
[0074] The random sampling module 203 is configured to generate multiple candidate verification center points in the remote sensing image, and to perform weighted random sampling based on the density-weighted sampling weight of the statistical grid to which each candidate verification center point belongs, until the target number of verification center points is obtained; the interval between any two candidate verification center points satisfies the minimum spatial interval constraint, and the target number is determined based on the proportion of the training set;
[0075] The bounding box classification module 204 is configured to use each validation center point as a reference to find the nearest unoccupied bounding box in each candidate bounding box as the validation set bounding box, and to use the candidate bounding boxes that do not have spatial overlap with the validation set bounding boxes as the training set bounding boxes.
[0076] The sample set construction module 205 is configured to crop image slices from remote sensing images based on the geographic extent of the training set bounding boxes and validation set bounding boxes to obtain the training sample set and validation sample set.
[0077] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this specification is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in or transmitted through a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., Digital Versatile Discs (DVDs)), or semiconductor media (e.g., Solid State Disks (SSDs)).
[0078] Figure 3 A block diagram of an electronic device 300 that can implement various embodiments of the present disclosure is shown. For example... Figure 3As shown, the electronic device 300 includes a processor 310, a disk drive 320, an input / output interface 330, a network interface 340, and a memory 350. The processor 310, disk drive 320, input / output interface 330, network interface 340, and memory 350 can communicate with each other via a communication bus.
[0079] The processor 310 can be implemented using a general-purpose CPU, microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits to execute relevant programs and implement the technical solution provided in this application.
[0080] The memory 350 can be implemented in the form of ROM (Read Only Memory), RAM (Read Access Memory), static memory, dynamic storage devices, etc. The memory 350 can store the operating system 351 used to control the operation of the electronic device 300, and the basic input / output system (BIOS) 352 used to control the low-level operations of the electronic device 300. Additionally, it can store a web browser 353, a data storage management system 354, etc. In summary, when the technical solution provided in this application is implemented through software or firmware, the relevant program code is stored in the memory 350 and is called and executed by the processor 310.
[0081] Input / output interface 330 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touch screens, microphones, various sensors, etc., and output devices may include displays, vibrators, indicator lights, etc.
[0082] Network interface 340 is used to connect a communication module (not shown in the figure) to enable communication and interaction between the device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).
[0083] Bus 360 includes a pathway for transmitting information between various components of the device, such as processor 310, disk drive 320, input / output interface 330, network interface 340, and memory 350.
[0084] It should be noted that although the above-described device only shows the processor 310, disk drive 320, input / output interface 330, network interface 340, memory 350, bus 360, etc., in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the method of this application, and does not necessarily include all the components shown in the figures.
[0085] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.
[0086] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing. Furthermore, although operations are depicted in a specific order, this should be understood as requiring that such operations be performed in the specific order shown or in sequential order, or requiring that all illustrated operations be performed to achieve the desired result. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the foregoing discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single implementation. Conversely, various features described in the context of a single implementation may also be implemented individually or in any suitable sub-combination in multiple implementations.
[0087] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.
Claims
1. A method for generating remote sensing image samples based on spatial perception segmentation, characterized in that, The method includes: Based on the export task request for remote sensing imagery, the request parameters and minimum spatial interval constraints of the remote sensing imagery are determined. The request parameters include the training set ratio and the training set overlap rate. The minimum spatial interval constraint is based on a preset multiplier setting of the sample geographic size. Based on the overlap rate of the training set, multiple candidate bounding boxes are generated in the remote sensing image, and the remote sensing image is divided into multiple statistical grids to determine the number of candidate bounding boxes in each statistical grid. The density-weighted sampling weight of each statistical grid is calculated based on the number of candidate bounding boxes, and the density-weighted sampling weight is higher for statistical grids with sparser density. Multiple candidate verification center points are generated in the remote sensing image. Weighted random sampling is performed based on the density-weighted sampling weight of the statistical grid to which each candidate verification center point belongs until the target number of verification center points is obtained. The interval between any two candidate verification center points satisfies the minimum spatial interval constraint. The target number is determined based on the training set ratio. Based on each verification center point, the nearest unoccupied bounding box in each candidate bounding box is used as the verification set bounding box, and the candidate bounding boxes that do not have spatial overlap with the verification set bounding boxes are used as the training set bounding boxes. Based on the geographical extent of the bounding boxes of the training set and the bounding boxes of the validation set, image slices are cropped from the remote sensing image to obtain the training sample set and the validation sample set.
2. The method for generating remote sensing image samples based on spatial perception partitioning according to claim 1, characterized in that, The method further includes: In response to the existence of a user-specified cropping region in the export task request, the intersection of the cropping region and the remote sensing image is determined as the valid export region for the export task.
3. The method for generating remote sensing image samples based on spatial perception partitioning according to claim 1, characterized in that, The step of generating multiple candidate bounding boxes within the remote sensing image based on the overlap rate of the training set includes: The sliding frame step size is calculated based on the overlap rate of the training set and the geographic size of the samples. The sliding frame step size is used to uniformly slide the remote sensing image in the horizontal and / or vertical directions to generate multiple candidate bounding boxes. The union of all candidate bounding boxes covers the entire remote sensing image. The geographic size of the samples is determined based on the product of the preset sample pixel size and the image resolution.
4. The method for generating remote sensing image samples based on spatial perception partitioning according to claim 1, characterized in that, Before obtaining the training sample set and validation sample set by cropping image slices from the remote sensing image based on the geographical extent of the training set bounding boxes and validation set bounding boxes, the method further includes: Calculate the spatial coverage and spatial distribution variation coefficient of the training set bounding box set and the validation set bounding box set in the statistical grid, respectively; the spatial coverage is the ratio of the number of statistical grids covered by the bounding box set to the total number of statistical grids, the spatial distribution variation coefficient is the ratio of the standard deviation to the mean of the number of target bounding boxes in each statistical grid, and the number of target bounding boxes is the number of bounding boxes in a single statistical grid; In response to the spatial coverage of either the training set bounding box set or the validation set bounding box set being lower than a preset coverage threshold or the spatial distribution coefficient of variation exceeding a preset coefficient of variation threshold, candidate validation center points are regenerated, and the training set bounding boxes and validation set bounding boxes are redefined.
5. The method for generating remote sensing image samples based on spatial perception partitioning according to claim 1, characterized in that, The geographical extent of the training set bounding boxes and validation set bounding boxes is obtained by cropping image slices from the remote sensing image to obtain the training sample set and validation sample set, including: For any of the training set bounding boxes or validation set bounding boxes, the geographic extent of the bounding box is used as the query window to retrieve matching vector polygons from the vector polygon annotation database, and image slices are cropped from the remote sensing image based on the geographic extent of the bounding box. The vector polygon is rendered into a labeling format that matches the preset sample type identifier to obtain labeled data; The image slices are associated with the labeled data to obtain training or validation samples corresponding to a single bounding box; Integrate all the training and validation samples to obtain the training sample set and the validation sample set.
6. The method for generating remote sensing image samples based on spatial perception partitioning according to claim 5, characterized in that, The method further includes: In response to a single export task request that sets multiple target resolution values, the annotation data and image slices of each bounding box are calculated according to different target resolution values, so that the same bounding box corresponds to multiple training samples and validation samples under different target resolutions.
7. The method for generating remote sensing image samples based on spatial perception partitioning according to claim 5, characterized in that, The sample type identifier includes at least one of semantic segmentation, object detection, rotated bounding box detection, change detection, and instance segmentation; the annotation format corresponding to semantic segmentation is a palette mask image, the annotation format corresponding to object detection is normalized bounding box coordinate text, the annotation format corresponding to rotated bounding box detection is rotated rectangle four-point coordinate text, the annotation format corresponding to change detection is a binary change mask image, and the annotation format corresponding to instance segmentation is run-length encoded JSON data.
8. A remote sensing image sample generation apparatus based on spatial perception partitioning, used to perform the steps of the remote sensing image sample generation method based on spatial perception partitioning according to any one of claims 1-7, characterized in that, The device includes: The task parsing module is configured to determine the request parameters and minimum spatial interval constraints of the remote sensing image based on the export task request for the remote sensing image. The request parameters include the training set ratio and the training set overlap rate, and the minimum spatial interval constraint is set based on a preset magnification of the sample geographic size. The bounding box generation module is configured to generate multiple candidate bounding boxes within the remote sensing image based on the overlap rate of the training set, and to divide the remote sensing image into multiple statistical grids to determine the number of candidate bounding boxes in each statistical grid. The density-weighted sampling weight of each statistical grid is calculated based on the number of candidate bounding boxes, and the density-weighted sampling weight is higher for statistical grids with sparser density. A random sampling module is configured to generate multiple candidate verification center points in the remote sensing image, and to perform weighted random sampling based on the density-weighted sampling weight of the statistical grid to which each candidate verification center point belongs, until a target number of verification center points are obtained; the interval between any two candidate verification center points satisfies the minimum spatial interval constraint, and the target number is determined based on the training set ratio; The bounding box classification module is configured to use each of the verification center points as a reference to find the nearest unoccupied bounding box in each of the candidate bounding boxes as the verification set bounding box, and to use the candidate bounding boxes that do not have spatial overlap with the verification set bounding boxes as the training set bounding boxes. The sample set construction module is configured to crop image slices from the remote sensing image based on the geographical extent of the training set bounding box and the validation set bounding box to obtain the training sample set and the validation sample set.
9. An electronic device, characterized in that, include: One or more processors, and a memory associated with the one or more processors, the memory being used to store program instructions that, when read and executed by the one or more processors, perform the steps of the remote sensing image sample generation method based on spatially sensed partitioning as described in any one of claims 1-7.
10. A computer program product, characterized in that, The method includes a computer program that, when executed by a processor, implements a remote sensing image sample generation method based on spatially perceptual partitioning according to any one of claims 1-7.