Remote sensing image sample cutting and optimizing method and device and electronic equipment

Abstract

The invention discloses a remote sensing image sample cutting and optimizing method and device and electronic equipment, and the method comprises the steps: obtaining a time-phase approaching remote sensing image and corresponding vector labeling data, and carrying out the preprocessing to generate a sample vector; traversing the vector elements, and calculating MAR and a dynamic cutting window of the vector elements; cutting the vector and the remote sensing image according to a cutting window, rasterizing the cut vector into a label image, and performing rotation correction on the cut remote sensing image and the label image; performing block cutting on the rotated image according to a fixed window, filtering a target area smaller than a certain threshold value according to a target area proportion in a block label image, otherwise, cutting and outputting an initial sample set; and performing feedback iterative optimization of model training and prediction on the sample set. Compared with the prior art, the method has the advantages that slender or irregular ground features in any direction are flexibly and efficiently processed, the sample pair integrity is remarkably improved, the edge effect and data redundancy are reduced, label noise is eliminated in combination with model feedback, and finally high-quality training samples and models are obtained.

Description

Technical Field

[0001] The present invention relates to the technical field of computer vision and remote sensing image processing, and in particular to a remote sensing image sample cutting and optimization method, device and electronic equipment. Background Art

[0002] In recent years, deep learning-based remote sensing image interpretation technology has become a research hotspot, demonstrating great potential in areas such as land cover classification, object detection, and change detection. Semantic segmentation, a pixel-level classification task, can provide refined ground object information for remote sensing images and is of great significance for applications such as water resource management, precision agriculture, urban planning, and environmental monitoring. In recent years, deep learning technologies, represented by deep convolutional neural networks (DCNNs), have significantly advanced the accuracy and practical application of remote sensing semantic segmentation. However, the performance of these data-driven deep learning models is highly dependent on large-scale, high-quality, and diverse annotated training samples. Remote sensing imagery typically features wide swaths, high spatial resolution, and complex and diverse ground object morphology and scales. Directly processing large-scale imagery would exceed the memory capacity of GPU hardware devices and be inefficient. Ensuring ground object integrity and proper segmentation makes the preparation of semantic segmentation samples a highly challenging task.

[0003] In the field of remote sensing, manually obtaining pixel-level annotated samples is often costly and time-consuming. A common approach is to automatically crop the sample patches required for training using existing vector results and corresponding remote sensing images. This type of cropping method mainly includes:

[0004] Cropping based on vector bounding rectangle: This method is simple, but the generated samples vary in size and may contain a large amount of irrelevant background. It is not suitable for CNN models that require a fixed input size. Direct resampling will cause deformation of the ground objects and reduce the resolution, affecting the model accuracy.

[0005] Fixed-size cropping based on the vector center point: This generates fixed-size samples but may truncate large objects or lose off-center objects.

[0006] Fixed-size window cropping based on vector coverage: A fixed starting point (upper left corner of the vector range) and step size are often used to perform window cropping by rows and columns to generate fixed-size block samples. Before cropping, the entire vector is rasterized, which is time-consuming and occupies a large amount of storage space. At the same time, a large number of redundant, invalid (no target), low-quality (such as a small target area or only containing target edges) and category-unbalanced samples are generated.

[0007] At present, the production of remote sensing semantic segmentation samples mainly faces the following problems:

[0008] 1. Poor cropping adaptability to directional features: In remote sensing images, many important land features, such as rivers, roads, railways, coastlines, strips of farmland with specific orientations, and tilted buildings in cities, often exhibit distinct linear features or are distributed at specific angles. Traditional sample cropping methods, such as sliding windows based on a fixed grid, often struggle to efficiently and completely capture these tilted or curved objects because their windows are aligned with the image coordinate axes. Objects are easily repeatedly cut by the window edges, resulting in the loss or fragmentation of critical morphological structural information.

[0009] 2. Low effective information density of samples and severe background redundancy: To include a portion of the tilted object, axis-aligned cropping windows often have to introduce a large amount of background regions unrelated to the target. This not only reduces the effective information density of individual samples (i.e., the proportion of target pixels), increases the computational burden during model training, but can also affect the model's learning of the target class due to oversampling of the background class.

[0010] 3. High labor costs and low efficiency: Although pixel-level annotation of raw remote sensing images is labor-intensive, further cropping the labeled large images into small sample blocks suitable for model training relies on manual selection and adjustment of cropping boxes to accommodate targets of different orientations and shapes. This is extremely inefficient and difficult to meet the sample size requirements of large-scale model training.

[0011] 4. Sample quality is difficult to ensure and an effective closed-loop optimization loop is lacking: The automatically generated initial sample set may still contain "dirty data" or "difficult samples" due to incorrect labeling, inaccurate boundaries, or improper cropping, making it difficult for the model to learn. Most existing methods lack an effective a posteriori quality verification and feedback optimization mechanism. These low-quality samples can significantly impact the model's training and final performance.

[0012] Therefore, there is an urgent need for an automated remote sensing semantic segmentation sample cropping and optimization technology that can intelligently adapt to objects of different directions and shapes, improve sample information density, and have feedback optimization capabilities, in order to obtain higher-quality training data and thus improve the performance of downstream segmentation models. It is particularly suitable for sample production and extraction applications of water conservancy features such as rivers, lakes, and ponds. Summary of the Invention

[0013] Purpose of the invention: In order to overcome the problems of poor sample cropping adaptability, low sample effective information content, and lack of an effective sample quality iterative optimization closed loop in the existing technology when processing directional remote sensing ground objects, a remote sensing image sample cropping and optimization method, device and electronic equipment are proposed to improve the production efficiency and final quality of remote sensing semantic segmentation samples.

[0014] Technical solution: The present invention discloses a remote sensing image sample cropping and optimization method, comprising the following steps:

[0015] S100, obtaining remote sensing images and corresponding vector label results, generating sample vectors after pre-processing, and obtaining image paths corresponding to features for subsequent cropping;

[0016] S200, traversing the sample vectors to obtain the minimum area rectangle MAR of each element as well as the center point coordinates, length, width and rotation angle. The minimum area rectangle MAR can be rotated at any angle to enclose the target object in a rectangle with the minimum area;

[0017] S300 dynamically calculates the cropping window based on the relationship between the length and width of the minimum area rectangle MAR and the standard cropping size d. When both the length and width of the MAR are less than d, a rotated square window is constructed with the MAR center point as the center, with the rotation angle consistent with the MAR. When the length or width of the MAR is greater than d, the number of standard blocks required in the x and y directions is calculated to construct a rotated rectangle that can completely enclose the object, and the coordinates of the four corner points of the cropping window are calculated.

[0018] S400, the vector elements in the vector layer that fall within the window range are cropped and stored in the memory, and are rasterized into a label image. At the same time, the remote sensing image within the corresponding window outer rectangle is cropped according to the image path, and rotation correction is performed in the memory according to the rotation angle of the MAR.

[0019] S500, performing center cropping or block cropping on the rotated remote sensing image and the labeled image block according to the relationship with the standard cropping size, and calculating the target area ratio and length ratio of the cropped sample. If the ratio is less than the filtering threshold, the filter is not output; otherwise, it is output as a sample pair;

[0020] S600 uses a pre-cut sample set to pre-train the segmentation model, and uses the segmentation model to predict the labels of the original samples. It then screens out "difficult samples" whose intersection-over-union ratio of the predicted label and the label value is lower than a preset threshold for manual inspection and editing. The samples are then returned to the training set for iterative training to continuously optimize sample quality and model performance.

[0021] Furthermore, in step S100, the vector label result is preprocessed:

[0022] S101, performing category filtering and code mapping according to the category of the application design. Category filtering filters out non-interested feature categories and easily confused features through attributes. Code mapping maps feature categories and merges categories through a code correspondence table.

[0023] S102, traversing the elements to automatically match the file path of the image in the specified directory through the spatial range, and saving it to the vector file attribute field.

[0024] Furthermore, in step S200, the label vector elements are traversed, and the minimum area rectangle MAR as well as the length, width and rotation angle are calculated using the rotating caliper algorithm, specifically:

[0025] (a) Calculate the convex hull of vector elements;

[0026] (b) initializing a pair of mutually perpendicular calipers, the calipers comprising: a first pair of parallel calipers parallel to one side of the convex hull; and a second pair of parallel calipers orthogonal to the first pair of calipers;

[0027] (c) rotating the caliper to traverse all rotation positions of the convex hull;

[0028] (d) calculating the area of ​​the bounding rectangle formed by the current caliper at each rotation position;

[0029] (e) Record the bounding rectangle with the smallest area as the minimum area rectangle MAR;

[0030] (f) Output the center point coordinates, length, width and rotation angle of the minimum area rectangle.

[0031] Furthermore, in step S300, the fixed sample size c is set to 512x512 or 1024x1024 pixels, and its standard cropping size d=c*r is calculated according to the resolution r of the original image. The cropping window is dynamically calculated according to the relationship between the length and width of the minimum area rectangle MAR and d, as follows:

[0032] S301, when the length and width of the minimum area rectangle MAR are both less than d, a rotated square window is calculated with the center point of the minimum area rectangle MAR as the center, which is consistent with the MAR rotation angle, i.e., the center cropping window, to ensure that the target object is centered as much as possible;

[0033] S302: When the length or width of the minimum area rectangle MAR is greater than d, the number of standard cropping blocks required in the x and y directions to cover the entire MAR is calculated with the center point of the minimum area rectangle MAR as the center. The relative coordinates of the four corner points with the maximum offset are calculated based on the number of blocks, and the actual coordinates of the four corner points are calculated based on the rotation angle of the minimum area rectangle MAR to generate a large rotated rectangular cropping window that can completely cover all small standard cropping blocks.

[0034] Furthermore, in step S400, the vector image and the remote sensing image are cropped using a cropping window, specifically:

[0035] S401, using a clipping window to clip target features within the vector layer, save them in memory, and rasterize them into a label image, where the range of the generated label image is the minimum bounding rectangle of the clipping window;

[0036] S402, cropping the corresponding remote sensing image using the minimum bounding rectangle of the cropping window according to the image path to obtain a range consistent with the label image;

[0037] S403, performing central rotation on the cropped label image and the remote sensing image using the rotation angle of the minimum area rectangle MAR, so that the minimum area rectangle MAR is parallel to the image coordinate axis in the rotated image;

[0038] Furthermore, the step S500 specifically includes:

[0039] S501: Calculate a non-rotated cropping window based on the center point according to the cropping method used in S300. If a block cropping method is used, calculate a sliding window with a certain degree of overlap:

[0040] Use a certain stride to perform overlapping window cropping:

[0041] stride=w*(1-overlap_ratio)

[0042] Where w is the standard cropping window size, overlap ratio overlap_ratio = (w-stride) / d*100%;

[0043] S502: Calculate the target area ratio in the cropped label image. If the area is less than a certain threshold, discard it and do not output it. Keep a portion of slender object samples based on the length ratio of the target object. If the area is greater than the threshold, output it as a sample image block. For the area without target objects in the cropping window, generate a certain proportion of negative samples according to the spatial distribution.

[0044] Furthermore, in step S600, the sample optimization steps based on model feedback are as follows:

[0045] S601, using the initial sample set generated in step S500, pre-training a deep learning semantic segmentation model;

[0046] S602, using a pre-trained model to predict sample image blocks in the initial sample set to obtain predicted label blocks;

[0047] S603, calculating the intersection-over-union (IOU) ratio between the original sample label block and the predicted label block of each sample;

[0048] S604: Filter out difficult samples with IoU values ​​below a preset threshold and submit them for manual inspection and label editing;

[0049] S605: Replace or add the manually edited samples back to the sample set to form an optimized sample set for subsequent iterative training or final model training.

[0050] Furthermore, the feedback optimization process of steps S601 to S605 is iterated multiple times, and each iteration uses the sample set optimized in the previous round to retrain or fine-tune the semantic segmentation model. After the iteration, the trained semantic segmentation model is used for prediction.

[0051] The present invention also discloses a remote sensing image sample cutting and optimization device, comprising the following modules:

[0052] Data acquisition module, used to analyze and acquire existing vector data and multi-source remote sensing images with similar temporal phases;

[0053] The label vector preprocessing module is used to check and convert the coordinate system of vector and image data, map the feature type code, and obtain the image path attributes, so that the coordinate system and category code of vector data and image are unified, and the elements and images are consistent;

[0054] The remote sensing sample cropping frame generation module is used to traverse all vector elements, calculate the minimum area rectangle MAR and center point coordinates, length, width and rotation angle of each element, use the minimum area rectangle MAR to crop the vector layer and image and save it in memory, rasterize the cropped vector into a label image, rotate the cropped remote sensing and label pairs to be parallel to the image coordinate axis, and calculate the cropping window based on the relationship between the length and width and the set block size. If the length and width of the outer rectangle are both smaller than the block size, the center cropping frame is calculated, otherwise the cropping frame of each windowed block is calculated;

[0055] The cropping box filtering module is used to calculate the target area ratio and length ratio within the cropping box, set the filtering threshold, and filter out the cropping boxes whose area ratio and length ratio are both less than the filtering threshold.

[0056] The sample set acquisition module is used to crop and output the remote sensing image and the label image according to the filtered cropping frame to obtain a remote sensing semantic segmentation sample set of multiple groups of samples;

[0057] The sample optimization module is used for model training and prediction of the sample set. It forms a pre-trained model by pre-training the cropped samples. The trained model is used to predict all samples. The IOU is calculated by the label value and the predicted value. The "difficult samples" with an error below the preset threshold are screened out for subsequent manual inspection. The manually modified samples are then put back into iterative training.

[0058] The manual inspection and editing module is used to manually inspect "difficult samples", edit and crop samples that have been missed or have incorrect boundaries into positive samples, and re-crop falsely detected samples into negative samples.

[0059] The present invention also discloses an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the above-mentioned remote sensing image sample cropping and optimization method is implemented.

[0060] Compared with the prior art, the present invention has the following beneficial effects:

[0061] (1) Most existing sample cropping methods adopt the method of first processing the whole image and then cropping, that is, the remote sensing image needs to be mosaicked and the vector data needs to be rasterized as a whole. There will inevitably be many areas with invalid samples. The core of this application is to process and crop the vector elements one by one. The original remote sensing image of a single scene does not need to be mosaicked first. Only the small area with vector elements is processed. The intermediate processing process can be completed in the memory, which greatly reduces the processing time and storage space, and can process multi-temporal samples (including change detection samples) more flexibly.

[0062] (2) The existing sample cropping method uses the minimum enclosing rectangle to obtain the range of the target object. This application accurately extracts the minimum area rectangle (MAR) of the target as the core geometric prior, and uses this as the basis for an adaptive cropping strategy (center cropping or block cropping). It can retain the original geometric shape and internal structure information of targets with any direction, such as rivers, roads, farmlands with a specific direction, inclined buildings, etc., to the greatest extent, effectively avoiding the problems of repeated cropping, information loss and morphological distortion caused by traditional axis-aligned cropping methods for such targets, while significantly reducing the negative impact of background redundancy and category imbalance.

[0063] (3) The existing sample cutting only considers the area ratio of the feature target to filter the samples. This application calculates the area ratio and length ratio (the ratio of the length of the minimum area rectangle to the block size). In this way, narrow and long ground feature samples with small areas (such as rivers, roads, etc.) can be retained without being filtered, thereby improving the extraction accuracy of such ground features.

[0064] (4) Existing sample optimization methods often use clustering to remove outlier samples, but often some of these small samples are true and correct, and after deletion, there will inevitably be omissions. This application uses model feedback to extract a small number of questionable samples and create different positive and negative samples. These difficult examples are given more weight during training, which better solves the problem of uneven sample distribution.

[0065] It can be seen that the embodiments of the present invention can make full use of existing vectors to flexibly and quickly construct high-quality remote sensing samples, overcome the problem of large workload of manual construction and the problem that samples of existing cropping methods have invalid low-quality samples and samples with unbalanced target proportions, and adopt a method combining model feedback optimization method and a small amount of manual inspection to optimize samples, which greatly improves the accuracy of remote sensing image classification and change detection and has good practicality. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 This is a schematic diagram of a process provided by an embodiment of the present invention;

[0067] Figure 2 is a schematic diagram comparing two cutting methods provided by an embodiment of the present invention in a specific image;

[0068] Figure 3 This is a specific cutting comparison of the two cutting methods in the center of the present invention;

[0069] Figure 4 is a specific schematic diagram of the sample filtering method provided by an embodiment of the present invention;

[0070] Figure 5 is a schematic diagram of a sample optimization process provided by an embodiment of the present invention;

[0071] Figure 6 This is an example diagram of a remote sensing semantic segmentation sample vector result provided by an embodiment of the present invention;

[0072] Figure 7 This is an example diagram of a sample vector result of remote sensing change detection provided by an embodiment of the present invention;

[0073] Figure 8 This is an example diagram of a sample window cutting with a fixed starting point provided by an embodiment of the present invention;

[0074] Figure 9 This is a comparison example diagram of fixed-starting point block cropping and dynamic window cropping provided by an embodiment of the present invention;

[0075] Figure 10 This is an example diagram of semantic segmentation and change detection samples provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0076] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0077] The specific implementation manner and working principle of the present invention will be further described in detail below with reference to the accompanying drawings.

[0078] In view of the large scale variations of water conservancy targets such as ditches, rivers, and lakes, and the problems existing in existing methods, the present invention proposes a remote sensing image sample cropping and optimization method, device, and electronic equipment. The process is as follows: Figure 1 The specific steps are as follows:

[0079] S100: Acquire remote sensing images and corresponding vector label results, and pre-process the label vectors.

[0080] Collect existing vector data (such as surface coverage, basic surveying and mapping, thematic data, etc.) and remote sensing images close to the current time. In addition, the vector data needs to be preprocessed as follows:

[0081] S101: Design an appropriate sample classification system based on the needs of the actual application. Ensure the consistency of sample categories through category filtering and code mapping. Category filtering uses attributes to filter out some non-interested and easily confused feature categories. Code mapping uses a code correspondence table to map feature categories and merge semantically similar categories to ensure label category consistency. For example, for water feature extraction applications, first filter out irrelevant features through category, then merge and code-map the target features (such as rivers, ditches, and ponds) to water code 1, and background to 0. If the vector coordinate system is inconsistent with the image coordinate system, the vector coordinates must be converted to the same coordinate system.

[0082] S102: The traversal element automatically matches the file path of the image closest in time in the specified directory using the spatial range, and saves it to the vector file attribute field. This method can flexibly process single-view and mosaicked images, avoiding the additional processing time and storage space required to mosaic the images before cutting the sample.

[0083] In order to achieve sample diversity and generalization, it is often necessary to obtain remote sensing image samples of different sensors, resolutions and multiple phases. As long as the vector elements are consistent with the corresponding regional images, the present invention batch reads the single-scene image paths with a specified directory and a vector acquisition time close to the vector element range. Unlike other methods, there is no need to first mosaic the single-scene images as a whole, which saves a lot of processing time and storage space, and is more flexible and efficient.

[0084] like Figure 6 As shown in the figure, in the sample vector result, the CC field represents the feature category, and the Image_url field records the corresponding image path of the vector element. The image paths of multiple phases can be recorded to cut multi-phase samples, such as Figure 7 As shown, the image paths of the preceding and following phases are recorded to realize the cutting of remote sensing change samples (i.e., the preceding and following phase images, and the change label images). In addition, the sensor, acquisition time, and location information can be automatically obtained for subsequent sample management.

[0085] S200, read the label vector result, traverse all the elements therein, and obtain the minimum area rectangle MAR and the center point coordinates, length L, width W and rotation angle θ of each element.

[0086] The minimum area rectangle (MAR) algorithm is based on the rotating caliper theorem in computational geometry. The algorithm rotates a pair of orthogonal calipers to traverse all possible directions of the convex hull to find the smallest enclosing rectangle. The algorithm includes the following steps:

[0087] (a) Calculate the convex hull of the vector elements.

[0088] (b) Initializing a pair of mutually perpendicular calipers, the calipers comprising:

[0089] The first pair of parallel calipers: parallel to one side of the convex hull

[0090] Second pair of parallel calipers: orthogonal to the first pair of calipers.

[0091] (c) Rotate the caliper system to traverse all rotational positions of the convex hull.

[0092] (d) Calculate the area of ​​the bounding rectangle formed by the current caliper at each rotation position.

[0093] (e) Record the bounding rectangle with the smallest area as the minimum area rectangle (MAR).

[0094] (f) Output the center point coordinates, length, width and rotation angle of the minimum area rectangle.

[0095] Center point: calculated by the coordinates of the rectangle vertices

[0096]

[0097] Rotation angle θ: defined as the angle between the long side of the rectangle and the north direction (radians).

[0098] Length and width determination: the larger value is recorded as the length L, and the smaller value is recorded as the width W.

[0099] The standard size d is determined by the fixed sample size c and the original remote sensing image resolution r, that is, d = c * r. Typically, the fixed sample size c is set to 512*512 or 1024*1024 pixels. For example, when the image resolution is 0.8 m and the fixed sample size c is set to 1024 pixels, the standard size d = 0.8 x 1024 = 819.2 m.

[0100] S300 is the core cropping logic of the present invention. It dynamically selects the optimal cropping method based on the relationship between the actual size (L, W) of each MAR and the user-preset standard square cropping size d. In other words, the cropping window is dynamically calculated based on the relationship between L, W, the center point coordinates (x_c, y_c) and the standard cropping size d:

[0101] (a) Define the relative offset coordinates of the four corner points half_size = d / 2

[0102] [dx, dy] = [(-half_size, -half_size), # upper left

[0103] (half_size, -half_size), # upper right

[0104] (half_size, half_size), # lower right

[0105] (-half_size, half_size), # lower left

[0106] (-half_size, -half_size)] # closed

[0107] (b) Apply the rotation matrix to each corner point

[0108] rx = dx * cosθ - dy * sinθ

[0109] ry = dx * sinθ + dy * cosθ

[0110] (c) Calculate the coordinates of the rotated corner points [[ID=²9]]

[0111] px = x0 + rx <¢

[0112] py = y0 + ry

[0113] S301 (Small target cropping), if L < d and W < d, calculate a rotated square window with side length d centered at the MAR, with the direction consistent with θ. The generated cropping window is as Figure 3 shown in (a);

[0114] S302 (Large target cropping), if L ≥ d or W ≥ d, adopt a directional block cropping strategy, as Figure 3 shown in (b):

[0115] (a) Calculate the number of blocks required to cover the MAR:

[0116] (b) Calculate the maximum offset relative to the midpoint based on the number of blocks to determine the relative coordinates of the four corner points.

[0117] (c) Calculate the actual coordinates of the corner points according to the θ rotation angle to generate a large rotated rectangular cropping window.

[0118] As Figure 2 It should be noted that there seems to be a small error in the original text where "¢0000242" is likely a misrepresentation. It is presented as it is in the translation for the sake of following the rules.The figure shows the difference between the minimum area rectangle and the minimum outer bounding rectangle block cutting of the present invention: the blue solid line is the minimum area rectangle adopted by the present invention, and the yellow solid line area is the minimum outer bounding rectangle; for the same river area (red), the minimum area rectangle only needs 4 blocks (blue dotted line blocks in the figure) to cover it, while the minimum outer bounding rectangle needs 8 blocks (yellow dotted line blocks in the figure), there are more invalid areas in the upper left and lower right, and the complete characteristics of the river cannot be well preserved.

[0119] S400, use the cropping window to crop vector and remote sensing images respectively:

[0120] S401, using a clipping window to clip target features within a vector layer, saving the clipped features in memory, and rasterizing them into a label image, wherein the range of the generated label image is the minimum bounding rectangle of the clipping window;

[0121] S402, cropping the corresponding remote sensing image using the minimum bounding rectangle of the cropping window to obtain a range consistent with the label image;

[0122] S403, performing central rotation on the cropped label image and the remote sensing image using the rotation angle of the MAR, so that the MAR is parallel to the image coordinate axis in the rotated image;

[0123] In steps S401-S403, since only the images and vectors within the cropping window are read each time, GDAL and OpenCV are used to process them in memory, including rasterizing the vectors within the cropping window and cropping the remote sensing image. At the same time, the labels and remote sensing images are rotated in memory. This can efficiently implement the above steps without generating intermediate result files, saving storage space.

[0124] In step S401, a program is written using GDAL to perform spatial filtering SetSpatialFilterRect on all vectors that fall within the cropping frame, then perform Insection cropping and save the cropping vectors to memory. This can greatly improve the processing efficiency for cropping large-scale vector files. For the cropped vectors, the area ratio and length ratio of the target element to the cropping frame can be counted in the memory. If the ratio is less than a certain threshold, it will be skipped directly to reduce the time of invalid processing.

[0125] S500: Sample cutting is performed based on the relationship between the length and width of the aforementioned MAR:

[0126] S501, calculate the non-rotating cropping window based on the center point according to the cropping method adopted in S300, and calculate the sliding window with a certain degree of overlap if the block method is adopted; while cropping the sample, record its geographic location, sensor type, phase, resolution and other information according to the vector attributes into the sample database, so as to facilitate the subsequent rapid screening of samples that meet the conditions for model training according to different application purposes.

[0127] like Figure 2 The blue dotted box in the middle is the windowed block of the non-overlapping area in S501. When there are fewer samples, a certain stride can also be used to perform overlapping windowing and cropping:

[0128] stride=w*(1-overlap_ratio)

[0129] Where w is the standard cropping window size, overlap ratio overlap_ratio = (w-stride) / d*100%;

[0130] For example, when the window size is 512 and the step size is 256, the overlap rate is 50%.

[0131] S502, calculate the target area ratio in the cropped label image, when the area is less than a certain threshold (such as Figure 4 As shown in the figure, when the threshold is set to 1 / 3, the three yellow areas that are smaller than the threshold are discarded and not output. It is also possible to retain a portion of slender object samples with a smaller area according to the length ratio of the target object, and output them as sample image blocks if they are larger than the threshold.

[0132] Furthermore, in step S502, attribute queries can be flexibly used to filter out unwanted feature types (such as dried-up ponds) in the cropped features, flexibly addressing different sample requirements. Other methods rasterize vector files into labeled images before cropping, which increases processing time and storage space and does not allow for flexible filtering of unwanted feature types and samples.

[0133] The output that meets the threshold in step S502 is a positive sample. For block samples with an area ratio of 0 (completely targetless samples), a small number of negative samples can be randomly output according to the spatial distribution.

[0134] S600, the sample optimization steps based on model feedback are as follows Figure 5 As shown:

[0135] S601, initial sample generation: Generate an initial training set D_initial = {(x_i, y_i)} that may contain noise or errors using step S500, where x_i is the image block and y_i is the corresponding (vector-based rasterized) mask label;

[0136] S602, model pre-training: Train a semantic segmentation model M (such as Mask2Former) on D_initial. Set the number of training epochs (such as 5000) or early stopping training strategy to reach a certain degree of convergence to avoid overfitting.

[0137] Semantic segmentation models can include Unet, HRNet, and Mask2Former. Due to the large scale variations in water bodies, this example uses the Mask2Former model for pre-training. This model uses a Transformer and multi-scale feature fusion architecture to layer-by-layer integrate low-resolution semantic information and the high-resolution enhancement model's ability to capture targets of varying scales. A masked attention mechanism limits attention to the predicted foreground area, reducing complex background interference and improving computational efficiency. Furthermore, because the dynamic sample cropping method ensures that the sample foreground is as centered and complete as possible, sample augmentation (such as random cropping, rotation, and mirroring) during training can still achieve a high foreground target ratio, thereby ensuring sample training accuracy. The loss function combines cross-entropy loss with Dice loss, using the Adam optimizer.

[0138] S603, Sample Prediction and Evaluation: Use model M to predict each sample (x_i, y_i) in D_initial, obtaining a predicted mask p_i = M(x_i). Then, a series of metrics (Intersection-over-Union (IOU) and maximum probability / confidence) are calculated to evaluate the consistency of the predicted result p_i with the original label y_i or the credibility of the prediction itself.

[0139] S604: Based on the indicators calculated in step 3 and the preset threshold, those samples that are considered to be "difficult samples" are automatically screened out to form a purified high-quality data set D_filtered.

[0140] For single-class extraction, the overall pixel IoU is used. Different IoU thresholds are set based on the number of fine-tuning iterations. For example, the initial setting is 0.7, and subsequently increased to 0.8 and 0.9. This gradually reduces the number of difficult examples detected automatically. Examples that are missed by manual inspection are edited and added to the positive sample library, while false positives are added to the negative sample library.

[0141] S605, filter out difficult samples below the preset threshold, provide a human-computer interaction interface for manual inspection and label editing, and produce positive and negative samples: errors (including labeling errors, imprecise boundaries, category confusion, missed detection, etc.) are defined as positive samples as 3, and false detections are defined as negative samples as 4. In training, category mapping and different weights are set, such as 0 (background) and 1 (target) with a weight of 1, while the weights of difficult samples 3 (missed detection) and 4 (false detection) are set to 20 and 10 respectively. During actual training, the category mapping is {3:1, 4:0}. The number of these difficult examples is less than that of the filtered sample set D_filtered. During training, these difficult examples are given more weights to enable the model to be updated in a targeted manner, thereby improving the accuracy and generalization of the model.

[0142] Manually edited samples are replaced or added back to the sample set to form an optimized sample set for subsequent iterative training or final model training. Generally, two to three iterations are sufficient to optimize and purify the sample set. While other automated methods, such as clustering, can also automatically filter out outliers, they can also filter out some atypical target samples, resulting in a high rate of missed detections. Therefore, this method is highly practical.

[0143] This example describes in detail the cutting and optimization of single-class semantic segmentation samples. The generated segmentation samples are as follows: Figure 10 (a). The same is true for the cropping of multi-class ground feature samples. For multi-class features, spatial clustering is first performed to form the cropping range, and then the minimum area rectangle or minimum outer rectangle cropping window is calculated to divide and crop the multi-class samples according to the subsequent steps. In particular, the same is true for change detection samples. It is only necessary to add the previous and next image paths in the vector attributes (such as Figure 6 As shown), the change vector code: 1 represents change, 0 represents no change. This method can be used to directly read the front and back images and the change vector to complete the cropping of the change sample, generating Figure 10 (b) Change detection sample. It only needs to cut two phases of remote sensing images and one label when traversing the features, such as Figure 10 Therefore, the cropping method and implementation of the present invention have good versatility and flexibility, and are suitable for cropping and optimizing a variety of remote sensing deep learning application samples.

[0144] The present invention also provides a remote sensing image sample cutting and optimization device, comprising the following modules:

[0145] The data acquisition module is used to analyze and acquire existing vector data and multi-source remote sensing images with a phase as close as possible.

[0146] The label vector preprocessing module is used to check and convert the coordinate system of vector and image data, map the feature type code, and obtain attributes such as image path, so that the coordinate system and category code of vector data and image are unified, and the features and images are consistent.

[0147] The remote sensing sample cropping box generation module is used to traverse all vector features, calculate the minimum area rectangle (MAR) and center point coordinates, length, width and rotation angle of each feature, use MAR to crop the vector layer and image and save them in memory, rasterize the cropped vector into a label image, rotate the cropped remote sensing and label pairs to be parallel to the image coordinate axis, and calculate the cropping window based on the relationship between the length and width and the set block size. If the length and width of the outer rectangle are both smaller than the block size, the center cropping box is calculated; otherwise, the cropping box of each windowed block is calculated.

[0148] The cropping box filtering module is used to calculate the target area ratio and length ratio within the cropping box, set a certain filtering threshold, and filter out the cropping boxes whose area ratio and length ratio are both less than the threshold and do not crop them. This ensures the balance of pixel ratios and avoids the processing time and storage space of invalid samples.

[0149] The sample set acquisition module is used to crop and output the remote sensing image and the label image according to the filtered cropping frame to obtain a remote sensing semantic segmentation sample set of multiple groups of samples.

[0150] The sample optimization module is used for model training and prediction of the sample set. It forms a pre-trained model by pre-training the cropped samples, uses the trained model to predict all samples, calculates the IOU by the label value and the predicted value, screens out a small number of questionable samples below a certain threshold for subsequent manual inspection, and performs iterative training on the manually modified samples.

[0151] The manual inspection and editing module is used to manually inspect questionable samples, edit and crop samples that are missed or have incorrect boundaries into positive samples, and re-crop falsely detected samples into negative samples.

[0152] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned remote sensing image sample cropping and optimization method when executing the computer program.

[0153] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the above-mentioned remote sensing image sample cropping and optimization method.

[0154] The present invention also provides a computer program product, comprising a computer program, which implements the above-mentioned remote sensing image sample cropping and optimization method when executed by a processor.

[0155] The above are merely embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A remote sensing image sample cropping and optimization method, characterized in that: The following steps are involved: S100, obtaining remote sensing images and corresponding vector label results, generating sample vectors after pre-processing, and obtaining image paths corresponding to features for subsequent cropping; S200, traversing the sample vectors to obtain the minimum area rectangle MAR of each element as well as the center point coordinates, length, width and rotation angle. The minimum area rectangle MAR can be rotated at any angle to enclose the target object in a rectangle with the minimum area; S300 dynamically calculates the cropping window based on the relationship between the length and width of the minimum area rectangle MAR and the standard cropping size d. When both the length and width of the MAR are less than d, a rotated square window is constructed with the MAR center point as the center, with the rotation angle consistent with the MAR. When the length or width of the MAR is greater than d, the number of standard blocks required in the x and y directions is calculated to construct a rotated rectangle that can completely enclose the object, and the coordinates of the four corner points of the cropping window are calculated. S400, the vector elements in the vector layer that fall within the window range are cropped and stored in the memory, and are rasterized into a label image. At the same time, the remote sensing image within the corresponding window outer rectangle is cropped according to the image path, and rotation correction is performed in the memory according to the rotation angle of the MAR. S500, performing center cropping or block cropping on the rotated remote sensing image and the labeled image block according to the relationship with the standard cropping size, and calculating the target area ratio and length ratio of the cropped sample. If the ratio is less than the filtering threshold, the filter is not output; otherwise, it is output as a sample pair; S600 uses a pre-cut sample set to pre-train the segmentation model, and uses the segmentation model to predict the labels of the original samples. It then screens out "difficult samples" whose intersection-over-union ratio of the predicted label to the label value is below a preset threshold for manual inspection and editing. The samples are then returned to the training set for iterative training to continuously optimize sample quality and model performance.

2. The remote sensing image sample cutting and optimization method according to claim 1, characterized in that: In step S100, the vector label results are preprocessed: S101, performing category filtering and code mapping according to the category of the application design. Category filtering filters out non-interested feature categories and easily confused features through attributes. Code mapping maps feature categories and merges categories through a code correspondence table. S102, traversing the elements to automatically match the file path of the image in the specified directory through the spatial range, and saving it to the vector file attribute field.

3. The remote sensing image sample cutting and optimization method according to claim 1, characterized in that: In step S200, the label vector elements are traversed and the minimum area rectangle MAR as well as the length, width and rotation angle are calculated using the rotating caliper algorithm, specifically: (a) Calculate the convex hull of vector features; (b) initializing a pair of mutually perpendicular calipers, the calipers comprising: a first pair of parallel calipers parallel to one side of the convex hull; and a second pair of parallel calipers orthogonal to the first pair of calipers; (c) rotating the caliper to traverse all rotation positions of the convex hull; (d) Calculate the area of ​​the bounding rectangle formed by the current caliper at each rotation position; (e) Record the bounding rectangle with the smallest area as the minimum area rectangle MAR; (f) Output the center point coordinates, length, width and rotation angle of the minimum area rectangle.

4. The remote sensing image sample cutting and optimization method according to claim 1, characterized in that: In step S300, the fixed sample size c is set to 512x512 or 1024x1024 pixels, and its standard cropping size d = c * r is calculated according to the resolution r of the original image. The cropping window is dynamically calculated according to the relationship between the length and width of the minimum area rectangle MAR and d, as follows: S301, when the length and width of the minimum area rectangle MAR are both less than d, a rotated square window is calculated with the center point of the minimum area rectangle MAR as the center, which is consistent with the MAR rotation angle, i.e., the center cropping window, to ensure that the target object is centered as much as possible; S302: When the length or width of the minimum area rectangle MAR is greater than d, the number of standard cropping blocks required in the x and y directions to cover the entire MAR is calculated with the center point of the minimum area rectangle MAR as the center. The relative coordinates of the four corner points with the maximum offset are calculated based on the number of blocks, and the actual coordinates of the four corner points are calculated based on the rotation angle of the minimum area rectangle MAR to generate a large rotated rectangular cropping window that can completely cover all small standard cropping blocks.

5. The remote sensing image sample cutting and optimization method according to claim 1, characterized in that: In step S400, the vector image and the remote sensing image are cropped using the cropping window, specifically: S401, using a clipping window to clip target features within the vector layer, save them in memory, and rasterize them into a label image, where the range of the generated label image is the minimum bounding rectangle of the clipping window; S402, cropping the corresponding remote sensing image using the minimum bounding rectangle of the cropping window according to the image path to obtain a range consistent with the label image; S403 , performing central rotation on the cropped label image and the remote sensing image using the rotation angle of the minimum area rectangle MAR, so that the minimum area rectangle MAR is parallel to the image coordinate axis in the rotated image.

6. The remote sensing image sample cutting and optimization method according to claim 1, characterized in that: The step S500 specifically includes: S501: Calculate a non-rotated cropping window based on the center point according to the cropping method used in S300. If a block cropping method is used, calculate a sliding window with a certain degree of overlap: Use a certain stride to perform overlapping window cropping: stride=w*(1-overlap_ratio)) Among them, w is the standard cropping window size, and the overlap ratio overlap_ratio = (w-stride) / d*100%; S502: Calculate the target area ratio in the cropped label image. If the area is less than a certain threshold, discard it and do not output it. Keep a portion of slender object samples based on the length ratio of the target object. If the area is greater than the threshold, output it as a sample image block. For the area without target objects in the cropping window, generate a certain proportion of negative samples according to the spatial distribution.

7. The remote sensing image sample cutting and optimization method according to claim 1, characterized in that: In step S600, the sample optimization steps based on model feedback are as follows: S601, using the initial sample set generated in step S500, pre-training a deep learning semantic segmentation model; S602, using a pre-trained model to predict sample image blocks in the initial sample set to obtain predicted label blocks; S603, calculating the intersection-over-union (IOU) ratio between the original sample label block and the predicted label block of each sample; S604: Filter out difficult samples with IoU values ​​below a preset threshold and submit them for manual inspection and label editing; S605: Replace or add the manually edited samples back to the sample set to form an optimized sample set for subsequent iterative training or final model training.

8. The remote sensing image sample cutting and optimization method according to claim 7, characterized in that: The feedback optimization process from step S601 to S605 is iterated multiple times, and each iteration uses the sample set optimized in the previous round to retrain or fine-tune the semantic segmentation model. After the iteration, the trained semantic segmentation model is used for prediction.

9. A remote sensing image sample cutting and optimization device, characterized in that: Includes the following modules: Data acquisition module, used to analyze and acquire existing vector data and multi-source remote sensing images with similar temporal phases; The label vector preprocessing module is used to check and convert the coordinate system of vector and image data, map the feature type code, and obtain the image path attributes, so that the coordinate system and category code of vector data and image are unified, and the elements and images are consistent; The remote sensing sample cropping frame generation module is used to traverse all vector elements, calculate the minimum area rectangle MAR and center point coordinates, length, width and rotation angle of each element, use the minimum area rectangle MAR to crop the vector layer and image and save it in memory, rasterize the cropped vector into a label image, rotate the cropped remote sensing and label pairs to be parallel to the image coordinate axis, and calculate the cropping window based on the relationship between the length and width and the set block size. If the length and width of the outer rectangle are both smaller than the block size, the center cropping frame is calculated, otherwise the cropping frame of each windowed block is calculated; The cropping box filtering module is used to calculate the target area ratio and length ratio within the cropping box, set the filtering threshold, and filter out the cropping boxes whose area ratio and length ratio are both less than the filtering threshold. The sample set acquisition module is used to crop and output the remote sensing image and the label image according to the filtered cropping frame to obtain a remote sensing semantic segmentation sample set of multiple groups of samples; The sample optimization module is used for model training and prediction of the sample set. It forms a pre-trained model by pre-training the cropped samples. The trained model is used to predict all samples. The IOU is calculated by the label value and the predicted value. "Difficult samples" with a value less than the preset threshold are screened out for subsequent manual inspection. The manually modified samples are then put back into iterative training. The manual inspection and editing module is used to manually inspect "difficult samples", edit and crop samples that have been missed or have incorrect boundaries into positive samples, and re-crop falsely detected samples into negative samples.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the remote sensing image sample cropping and optimization method according to any one of claims 1 to 8 is implemented.

Images