Satellite remote sensing image dataset enhancement method and system

Abstract

The embodiment of the application provides a satellite remote sensing image dataset enhancement method and system, and relates to the technical field of satellite remote sensing image target detection. The method performs semantic segmentation on the satellite remote sensing image after obtaining the satellite remote sensing image dataset, obtains a semantic segmentation result, and divides the satellite remote sensing image into a target region, a background region and an interference region. Based on the semantic segmentation result, combined with the anchor box size distribution characteristics of the target detection model, differential enhancement rules are formulated for the image regions. Then, the differential enhancement rules are used to perform semantic hierarchical enhancement on the satellite remote sensing image dataset to generate an enhanced dataset. The method can formulate differential enhancement rules for the target, background and interference regions, adapt to the characteristics of the target detection model, and dynamically adjust the size and proportion of the anchor box candidate set, and simultaneously optimize the enhancement rules to solve the problem that the satellite remote sensing image dataset enhancement process affects the accuracy of model training.

Description

Technical Field

[0001] This application relates to the field of satellite remote sensing image target detection technology, and in particular to a method and system for satellite remote sensing image dataset enhancement. Background Technology

[0002] Satellite remote sensing images are obtained by satellites, space shuttles, and other spacecraft performing image detection on specific areas in the space environment. Due to their wide coverage and high acquisition efficiency, satellite remote sensing images serve as an important data source for target detection. When performing target detection based on satellite remote sensing images, target detection models can be used, such as the You Only Look Once (YOLO) model. The YOLO model has the advantages of fast detection speed and strong real-time performance, and can be widely used in target detection tasks based on remote sensing images.

[0003] Object detection models like YOLO treat object detection as an end-to-end regression problem. This involves dividing the input image into a grid, predicting a fixed amount of information for each grid cell, and then combining the predictions from all grid cells to output a final prediction tensor. This allows for the direct prediction of the bounding box locations and class probabilities of all objects from the image pixels. When faced with challenges such as insufficient remote sensing image data, monotonous scenes, and complex target features, dataset augmentation is necessary.

[0004] Dataset augmentation can expand the dataset size by performing basic operations such as image rotation, flipping, brightness adjustment, and noise addition, as well as by generating new data based on generative adversarial networks (GANs), thereby preventing overfitting of object detection models and improving their generalization ability. However, the above-mentioned dataset augmentation methods use uniform augmentation rules, which leads to the loss of features of small targets and the stretching and distortion of medium and large targets, failing to meet the needs of model training. Furthermore, the dataset augmentation process is prone to background and target confusion, and excessive interference regions can negatively impact detection results, making target location annotations prone to misalignment and affecting the accuracy of model training. Summary of the Invention

[0005] In view of this, embodiments of this application provide a satellite remote sensing image dataset enhancement method and system to solve the problem that the satellite remote sensing image dataset enhancement process affects the accuracy of model training.

[0006] According to a first aspect of this application, a method for enhancing a satellite remote sensing image dataset is provided, the method comprising:

[0007] Acquire a satellite remote sensing image dataset, which includes multiple satellite remote sensing images;

[0008] Semantic segmentation is performed on the satellite remote sensing image to obtain a semantic segmentation result, the semantic segmentation result including image regions divided from the satellite remote sensing image, the image regions including target regions;

[0009] Based on the semantic segmentation results and combined with the anchor box size distribution characteristics of the target detection model, differential enhancement rules are formulated for the image regions respectively; the differential enhancement rules include the enhancement rules for the target regions; the enhancement rules for the target regions are used to calculate the adaptation coefficient through the adaptation degree analysis between the target scale and the reference anchor box, and to perform differential scaling on the reference anchor box based on the adaptation coefficient;

[0010] The satellite remote sensing image dataset is subjected to hierarchical enhancement using the differential enhancement rules to generate an enhanced dataset; the enhanced dataset includes multiple enhanced images that are synchronously updated through target detection model-linked sample selection and annotation; the enhanced dataset balances the class distribution by weighted sampling of multiple enhanced images.

[0011] In some embodiments, acquiring a satellite remote sensing image dataset includes:

[0012] Acquire satellite information and raw images captured by the satellite; the satellite information includes the radiation response characteristics of the satellite sensors, satellite orbital parameters, attitude angle data, and ground control point information;

[0013] Perform radiometric correction on the original image based on the radiometric response characteristics information;

[0014] Based on the satellite orbit parameters, the attitude angle data, and the ground control point information, a polynomial correction model is used to perform geometric distortion correction on the original image;

[0015] The original image is subjected to noise removal and size adjustment to generate the satellite remote sensing image.

[0016] In some embodiments, semantic segmentation is performed on the satellite remote sensing image to obtain semantic segmentation results, including:

[0017] The satellite remote sensing image is input into a semantic segmentation model, which optimizes the feature extraction process by introducing an attention mechanism.

[0018] The semantic segmentation model is used to perform semantic segmentation on the satellite remote sensing image to divide the image region from the satellite remote sensing image; the image region includes the target region, the background region and the interference region; the target region is an image region containing the target to be detected; the background region is an image region without the target to be detected; the interference region is an image region containing interference content, the interference content including at least one of clouds, shadows and fog spots;

[0019] The semantic segmentation result is generated by aggregating the target region, the background region, and the interference region.

[0020] Record target information based on semantic segmentation results, including target scale, target location, and target category;

[0021] A semantic hierarchical tag library is constructed based on the target information.

[0022] In some embodiments, based on the semantic segmentation results and combined with the anchor box size distribution characteristics of the object detection model, differential enhancement rules are formulated for the image regions, including:

[0023] Set a target scale determination threshold, which includes a single target pixel ratio threshold and a target bounding rectangle side length threshold;

[0024] Based on the target scale determination threshold, the targets to be detected are divided into small targets and medium-to-large targets; the small targets are the targets to be detected whose single target pixel ratio is lower than the single target pixel ratio threshold and whose target bounding rectangle side length is smaller than the target bounding rectangle side length threshold; the medium-to-large targets are the targets to be detected other than the small targets;

[0025] Read the default anchor box parameters of the object detection model;

[0026] Based on the semantic hierarchical label library, scale features of all targets to be detected in the current batch of satellite remote sensing images are extracted; the scale features include the average bounding rectangle size and the scale standard deviation.

[0027] The adaptation coefficient is calculated based on the default anchor frame parameters and the scale features. The adaptation coefficient includes the adaptation coefficient for the small target and the adaptation coefficient for the medium and large targets.

[0028] The baseline anchor frame is differentially scaled based on the adaptation coefficient to generate a dynamic anchor frame candidate set.

[0029] In some embodiments, the baseline anchor frame is differentially scaled based on the adaptation coefficient to generate a dynamic anchor frame candidate set, including:

[0030] An initial candidate set is generated based on the fit coefficient and the scale standard deviation, the initial candidate set including the adjustment function of the small target and the adjustment function of the medium and large target;

[0031] Set the adjustment step size, which is used to make the anchor frame size continuously adapt to the target scale distribution;

[0032] Acquire target characteristic information of the target to be detected, wherein the target characteristic information includes the shape characteristics of the target to be detected;

[0033] Constraint factors are set according to the target characteristic information, and the constraint factors include proportional constraint factors set based on the shape characteristics;

[0034] Cluster optimization is performed on the initial candidate set based on the adjustment step size and the constraint factor to generate the dynamic anchor box candidate set.

[0035] In some embodiments, the differential enhancement rules are used to perform hierarchical enhancement on the satellite remote sensing image dataset to generate an enhanced dataset:

[0036] Calculate the average intersection-union ratio (IUU) between the dynamic anchor frames in the candidate set and the small target;

[0037] The number of iterations for the enhanced super-resolution generative adversarial network is set based on the average intersection-over-union ratio (IoU), and the number of iterations is positively correlated with the average IoU.

[0038] Calculate the scaling factor based on the adaptation coefficient of the small target, and adjust the scaling range of the dynamic anchor frame according to the scaling factor;

[0039] Based on the number of iterations and the scaling range, the enhanced super-resolution generative adversarial network is used to perform image enhancement on the small target to obtain an enhanced image containing the small target.

[0040] In some embodiments, the differential enhancement rules are used to perform hierarchical enhancement on the satellite remote sensing image dataset to generate an enhanced dataset, including:

[0041] The coverage of the dynamic anchor frame with the medium-to-large target at different rotation angles is statistically analyzed. The coverage is the ratio of the overlapping area of ​​the dynamic anchor frame and the medium-to-large target to the total area of ​​the medium-to-large target.

[0042] Based on the coverage, a candidate enhancement angle is selected, wherein the candidate enhancement angle is a rotation angle where the coverage is greater than or equal to a coverage threshold;

[0043] The optimal enhancement angle is randomly selected from the candidate enhancement angles according to the satellite imaging azimuth angle weight;

[0044] A shearing ratio constraint is set based on the outer rectangle size of the dynamic anchor frame, and the shearing ratio constraint includes an upper limit for the shearing ratio; the upper limit for the shearing ratio is set according to the overlap area between the dynamic anchor frame and the medium-to-large target and the maximum side length of the medium-to-large target.

[0045] The target shape distortion parameters are adjusted according to the aspect ratio of the dynamic anchor frame. The target shape distortion parameters include the occlusion area ratio and the attitude offset angle. The occlusion area ratio and the attitude offset angle are calculated based on the average intersection-to-union ratio of the dynamic anchor frame and the medium-to-large target.

[0046] Image enhancement is performed on the satellite remote sensing image according to the optimal enhancement angle, the shearing ratio constraint, and the target morphology distortion parameters to obtain an enhanced image containing the medium-to-large target.

[0047] In some embodiments, the differential enhancement rules are used to perform hierarchical enhancement on the satellite remote sensing image dataset to generate an enhanced dataset, including:

[0048] Calculate the matching index between the dynamic anchor frame and the target to be detected. The matching index includes at least one of the following: average intersection-over-union ratio, average anchor frame coverage, and similarity between the target to be detected and the dynamic anchor frame.

[0049] Set a qualified threshold, which includes at least one of the following: intersection-over-union threshold, coverage threshold, and similarity threshold;

[0050] Based on the matching index and the qualified threshold, a parameter feedback adjustment strategy is set; wherein, when the average intersection-over-union ratio (IoU) is less than the IoU threshold, the feedback adjustment strategy includes recalculating the adaptation coefficient and adjusting the step size to regenerate the dynamic anchor boxes; when the average anchor box coverage is less than the coverage threshold, the feedback adjustment strategy includes reducing the upper limit of the shearing ratio of the medium and large targets and expanding the scaling range of the small targets; when the similarity between the target to be detected and the dynamic anchor boxes is less than the similarity threshold, the feedback adjustment strategy includes increasing the number of residual blocks in the generative adversarial network and / or adjusting the number of feature extraction channels of the generator in the generative adversarial network;

[0051] Based on the matching accuracy statistics within the current batch, the baseline parameters for dynamic adjustment of the anchor frame are updated. These baseline parameters include the cluster number range and the initial value of the adaptation coefficient.

[0052] In some embodiments, the differential enhancement rules are used to perform hierarchical enhancement on the satellite remote sensing image dataset to generate an enhanced dataset, including:

[0053] Obtain environmental feature information of the remote sensing area, including the topographic features and climate features of the remote sensing area;

[0054] Environmental adjustment parameters are generated based on the environmental feature information. The environmental adjustment parameters include illumination parameters applicable to the terrain features and climate features of the remote sensing area. The illumination parameters are generated by randomly adjusting the brightness, contrast, and hue of the satellite remote sensing image.

[0055] By using random texture migration, similar terrain textures in the background area are migrated across regions to generate an enhanced image that includes the background area.

[0056] In some embodiments, the differential enhancement rules are used to perform hierarchical enhancement on the satellite remote sensing image dataset to generate an enhanced dataset, including:

[0057] An atmospheric scattering model is used to generate interference sample images under different levels of interference content, and the atmospheric scattering model is used to overlay different levels of interference content onto the enhanced image.

[0058] Adaptive interference cancellation is performed based on semantic masks;

[0059] By using threshold segmentation and region restoration, the target region in the interference sample image is preserved, and the background region and interference region in the interference sample image are restored simultaneously.

[0060] In some embodiments, the differential enhancement rules are used to perform hierarchical enhancement on the satellite remote sensing image dataset to generate an enhanced dataset, including:

[0061] Based on the aforementioned differential enhancement rules, hierarchical enhancement is performed on the satellite remote sensing image dataset to obtain hierarchically enhanced data;

[0062] The hierarchical augmented data is input into a pre-trained target detection model to calculate target detection evaluation parameters, which include target detection accuracy and target detection recall.

[0063] Based on the target detection evaluation parameters, high-quality sample data is selected from the hierarchical augmented data. The high-quality sample data includes augmented images with target detection accuracy not lower than the original sample detection accuracy and recall rate not lower than the recall rate threshold.

[0064] A label synchronization mapping function is established based on the affine transformation matrix, and the target position label coordinates are updated using the label synchronization mapping function;

[0065] A sample weighted fusion strategy is adopted to balance the category distribution of the hierarchical augmented data through weighted sampling to generate the augmented dataset; the sample weighted fusion strategy is used to increase the weight of augmented samples for scarce category targets and small targets.

[0066] In some embodiments, the method further includes:

[0067] Build a lightweight enhancement framework;

[0068] Configure iteration parameters, which include parameter update period, enhancement scaling adjustment step size, and disturbance simulation intensity adjustment step size;

[0069] Based on the lightweight enhancement framework and the iteration parameters, the differentiated enhancement strategy is dynamically and iteratively adjusted using the gradient descent algorithm.

[0070] The satellite remote sensing image dataset is subjected to hierarchical enhancement using the adjusted differential enhancement strategy;

[0071] When the iterative adjustment process reaches the set iteration termination condition, the optimal augmented dataset and augmentation parameter configuration information are output.

[0072] According to a second aspect of this application, a satellite remote sensing image dataset enhancement system is provided, the system comprising:

[0073] The data acquisition module is used to acquire a satellite remote sensing image dataset, which includes multiple satellite remote sensing images.

[0074] A semantic segmentation module is used to perform semantic segmentation on the satellite remote sensing image to obtain a semantic segmentation result, the semantic segmentation result including image regions divided from the satellite remote sensing image, the image regions including target regions;

[0075] The rule generation module is used to formulate differentiated enhancement rules for the image regions based on the semantic segmentation results and the anchor box size distribution characteristics of the target detection model. The differentiated enhancement rules include enhancement rules for the target regions. The enhancement rules for the target regions are used to calculate the adaptation coefficient through the adaptation degree analysis between the target scale and the reference anchor box, and to perform differentiated scaling on the reference anchor box based on the adaptation coefficient.

[0076] An image enhancement module is used to perform hierarchical enhancement on the satellite remote sensing image dataset using the differential enhancement rules to generate an enhanced dataset; the enhanced dataset includes multiple enhanced images that are synchronously updated through target detection model-linked sample selection and annotation; the enhanced dataset balances the class distribution by weighted sampling of multiple enhanced images.

[0077] By employing the above technical solutions, embodiments of this application provide a method and system for enhancing satellite remote sensing image datasets. After acquiring a satellite remote sensing image dataset, the method performs semantic segmentation on the satellite remote sensing images to obtain semantic segmentation results, dividing the satellite remote sensing images into target regions, background regions, and interference regions. Based on the semantic segmentation results and combined with the anchor box size distribution characteristics of the target detection model, differentiated enhancement rules are formulated for each image region. Then, semantic hierarchical enhancement is performed on the satellite remote sensing image dataset using these differentiated enhancement rules to generate an enhanced dataset. This method can formulate differentiated enhancement rules for target, background, and interference regions, adapting to the characteristics of the target detection model. Furthermore, it can dynamically adjust the size and proportion of the anchor box candidate set, simultaneously optimizing the enhancement rules, thereby addressing the problem of the satellite remote sensing image dataset enhancement process affecting the accuracy of model training.

[0078] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of this application more obvious and understandable, the following are specific embodiments of this application. Attached Figure Description

[0079] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0080] Figure 1 This is a schematic diagram of the satellite remote sensing image dataset enhancement method provided in the embodiments of this application;

[0081] Figure 2 This is a schematic diagram illustrating the entire process of dataset augmentation provided in the embodiments of this application;

[0082] Figure 3 This is a schematic diagram of the data preprocessing process provided in the embodiments of this application;

[0083] Figure 4 This is a schematic diagram of the adaptive data augmentation process provided in an embodiment of this application;

[0084] Figure 5 This is a schematic diagram of the lightweight iterative optimization mechanism provided in the embodiments of this application;

[0085] Figure 6 A schematic diagram of the structure of the satellite remote sensing image dataset enhancement system provided in this application embodiment. Detailed Implementation

[0086] The present application will be described in detail below with reference to the accompanying drawings and embodiments. It should be noted that, unless otherwise specified, the embodiments and features described in the embodiments of the present application can be combined with each other.

[0087] In this embodiment, a remote sensing satellite is a spacecraft operating in a space environment capable of remotely monitoring an area. During operation, the remote sensing satellite can respond to remote sensing control programs or remote sensing monitoring commands generated by a ground control console to acquire images of a designated remote sensing monitoring area, thereby obtaining satellite remote sensing images. Depending on the principle by which the remote sensing satellite performs remote sensing monitoring, it can include optical satellites, radar satellites, hyperspectral satellites, and infrared satellites, etc.

[0088] Taking optical satellites as an example, they can passively receive solar radiation reflected from the Earth's surface to acquire remote sensing image data in the visible light band for remote sensing monitoring areas. The satellite remote sensing images acquired by optical satellites have high resolution, rich color information, and the data is intuitive and easy to interpret. However, because optical satellites rely on sunlight, they cannot operate at night and are easily obstructed by clouds, shadows, fog, and other factors.

[0089] Because of their wide coverage and high acquisition efficiency, satellite remote sensing images serve as an important data source for target detection. Satellite remote sensing images can be used to identify targets within remotely sensed monitoring areas, such as buildings, terrain features, vegetation, and moving objects like vehicles.

[0090] When performing object detection based on satellite remote sensing imagery, object detection models such as the You Only Look Once (YOLO) model can be used. Object detection models like YOLO treat object detection as an end-to-end regression problem. This involves dividing the input image into a grid, predicting a fixed amount of information for each grid cell, and then combining the predictions from all grid cells to output a final prediction tensor. This allows for the direct prediction of the bounding box locations and class probabilities of all objects from the image pixels.

[0091] Object detection models such as YOLO have advantages in fast detection speed and strong real-time performance, and can be widely used in remote sensing image object detection tasks. When faced with problems such as insufficient remote sensing image data, simple scene, and complex target features, object detection models need to be augmented with datasets.

[0092] Dataset augmentation can expand the dataset size by performing basic operations such as image rotation, flipping, brightness adjustment, and noise addition, as well as by generating new image data based on generative adversarial networks, thereby preventing overfitting of object detection models and improving the model's generalization ability.

[0093] However, because the dataset augmentation uses a uniform augmentation rule, small target features are lost, and medium and large targets are stretched and distorted, failing to meet the needs of model training. Furthermore, the dataset augmentation process is prone to background and target confusion, and excessive interference regions can negatively impact detection performance, leading to misalignment and bias in target location labeling, thus affecting the accuracy of model training.

[0094] To address the issue of satellite remote sensing image dataset augmentation affecting model training accuracy, this application provides a satellite remote sensing image dataset augmentation method in some embodiments. This method constructs a complete data augmentation process covering data acquisition, data preprocessing, semantic layering, adaptive augmentation, intelligent filtering and fusion linked to target detection models, and a lightweight iterative optimization mechanism. Through mechanisms such as semantic layering, adaptive data augmentation, and intelligent filtering and fusion linked to target detection models, differentiated augmentation rules are formulated for targets, backgrounds, and interference regions to adapt to the characteristics of target detection models. Furthermore, the size and proportion of the anchor box candidate set can be dynamically adjusted to simultaneously optimize the augmentation rules and improve the accuracy of model training.

[0095] The method can be applied to electronic devices with data processing capabilities and communication connections with remote sensing satellites. These electronic devices include, but are not limited to, computers, servers, mobile terminals, smart wearable devices, and industrial control units. For ease of description, this embodiment uses an electronic device as the execution subject of the method. It should be understood that the method can also be applied to other types of execution subjects, which are not illustrated in this embodiment. Figure 1 , Figure 2 As shown, the method includes:

[0096] S101. Obtain satellite remote sensing image dataset.

[0097] To perform augmentation on a satellite remote sensing image dataset, the dataset must first be acquired. This dataset comprises multiple satellite remote sensing images. Satellite remote sensing images are obtained by remote sensing satellites through image acquisition of remotely sensed monitoring areas.

[0098] In some embodiments, satellite remote sensing image datasets can be acquired through real-time image acquisition. For example, when acquiring a satellite remote sensing image dataset, an electronic device can send an image acquisition request to a remote sensing satellite. Upon receiving the request, the remote sensing satellite can respond by acquiring images, obtaining multiple satellite remote sensing images. The acquired satellite remote sensing images are then packaged into a dataset and sent to the electronic device, enabling the electronic device to acquire the satellite remote sensing image dataset.

[0099] In some embodiments, a data acquisition period can be set, and a database can be established to retrieve satellite remote sensing image datasets for a specific acquisition period from the database. For example, if the data acquisition period is set to 24 hours, the remote sensing satellite can perform image acquisition during operation within each acquisition period and send the acquired images to a data storage device for storage. This allows the data storage device to store multiple satellite remote sensing image datasets, each including all satellite remote sensing images acquired by the satellite within the 24 hours. Correspondingly, when acquiring a satellite remote sensing image dataset, the electronic device can send a data acquisition request to the data storage device, enabling the data storage device to respond with a request for a satellite remote sensing image dataset for a specified period based on the data acquisition request.

[0100] After acquiring the satellite remote sensing image dataset, data preprocessing can be performed on the dataset, such as... Figure 3 As shown, in some embodiments, when acquiring a satellite remote sensing image dataset, satellite information and raw images captured by the satellite can be acquired first. The satellite information includes the radiometric response characteristics of the satellite sensors, satellite orbital parameters, attitude angle data, and ground control point information.

[0101] Radiometric correction is then performed on the original image based on radiometric response characteristics. Geometric distortion correction is performed on the original image using a polynomial correction model based on satellite orbital parameters, attitude angle data, and ground control point information. Finally, noise removal and size adjustment are performed on the original image to generate a satellite remote sensing image.

[0102] For example, during data preprocessing, radiometric correction can be performed first. This involves performing radiometric correction operations based on the radiometric response characteristics of the satellite sensor, controlling the correction error within 5%, eliminating the influence of sensor bias and atmospheric scattering on image quality, and ensuring that the image grayscale value matches the true radiance of the ground object.

[0103] After radiometric correction, geometric correction is also required. Based on satellite orbital parameters, attitude angle data, and ground control point information, a polynomial correction model is used to correct geometric distortion, ensuring that the image is aligned with the ground position. This method is adaptable to remote sensing images with different resolutions from 1 meter to 10 meters, ensuring the accuracy of the target position and providing a precise spatial reference for subsequent semantic layering and annotation.

[0104] Then, noise removal is performed. For Gaussian noise in the image, an adaptive median filtering algorithm can be used to control the noise variance between 0.01 and 0.05. For strip noise in the image, an algorithm combining wavelet transform and mean filtering can be used, with a removal rate of no less than 90%, effectively improving image clarity and target feature recognition.

[0105] The processed image resolution is then uniformly adjusted to 320×320 pixels, 640×640 pixels, or 1280×1280 pixels to adapt to the input size requirements of different versions of YOLO models. It also supports user-defined adjustments based on custom model input requirements, with the channel format uniformly set to RGB or 3- to 8-channel multispectral formats to ensure the consistency and stability of model input data through channel normalization.

[0106] S102. Perform semantic segmentation on the satellite remote sensing image to obtain the semantic segmentation result.

[0107] After acquiring a satellite remote sensing image dataset, semantic segmentation can be performed on the satellite remote sensing images in the dataset to obtain semantic segmentation results. These semantic segmentation results include image regions divided from the satellite remote sensing images. Furthermore, the image regions include target regions, which characterize areas in the image containing the target to be detected. The image regions may also include background regions and interference regions, which characterize areas in the image without the target to be detected and areas containing interference content, respectively.

[0108] In some embodiments, when performing semantic segmentation on satellite remote sensing images to obtain semantic segmentation results, the satellite remote sensing images can be first input into a semantic segmentation model, wherein the semantic segmentation model optimizes the feature extraction process by introducing an attention mechanism.

[0109] Semantic segmentation is then performed on the satellite remote sensing image using a semantic segmentation model to divide the image into regions. These regions include target regions, background regions, and interference regions. Specifically, the target region is the image region containing the target to be detected; the background region is the image region without the target to be detected; and the interference region is the image region containing interfering content, which may include at least one of clouds, shadows, and fog.

[0110] Then, semantic segmentation results are generated by aggregating the target region, background region, and interference region. Target information, including target scale, target location, and target category, is then recorded based on the semantic segmentation results, and a semantic hierarchical label library is constructed according to this target information.

[0111] For example, when performing semantic hierarchical processing, semantic segmentation can be performed first. This involves using semantic segmentation models such as U-Net++ to perform semantic segmentation on satellite remote sensing images, and optimizing the feature extraction process by introducing an attention mechanism to ensure a segmentation accuracy (mIoU) of no less than 88%. Semantic segmentation can divide satellite remote sensing images into three layers: target region, background region, and interference region. The target region includes buildings, vehicles, vegetation, and other targets; the background region consists of terrain and water bodies without targets; and the interference region includes clouds, shadows, and fog spots, enabling accurate segmentation of different semantic regions.

[0112] After semantic segmentation is completed, a label library can be built based on the semantic segmentation results. This involves automatically recording target information based on the semantic segmentation results. Specifically, it records the pixel percentage, bounding rectangle side length, and other target scales of each target to be detected; pixel coordinates, geographic coordinates, and other target locations; and the category information of the target to be detected. This information is used to construct a semantic hierarchical label library. The label format in the semantic hierarchical label library is compatible with the Visual Object Classes (VOC) and Common Objects in Context (COCO) standards, and can be directly used by the YOLO model, providing accurate data support and localization basis for subsequent hierarchical enhancement.

[0113] S103. Based on the semantic segmentation results and combined with the anchor box size distribution characteristics of the target detection model, differentiated enhancement rules are formulated for each image region.

[0114] After performing semantic segmentation on satellite remote sensing images and obtaining the semantic segmentation results, adaptive data augmentation can be performed. During adaptive data augmentation, based on the semantic segmentation results and the anchor frame size distribution characteristics of the target detection model, differentiated enhancement rules can be formulated for each image region. These differentiated enhancement rules include enhancement rules for the target region. The enhancement rules for the target region are used to calculate the fit coefficient through a fit analysis between the target scale and the reference anchor frame, and then differentiate the scaling of the reference anchor frame based on the fit coefficient.

[0115] In order to formulate differentiated enhancement rules, in some embodiments, when formulating differentiated enhancement rules for image regions based on semantic segmentation results and combined with the anchor box size distribution characteristics of the target detection model, a target scale determination threshold can be set first. The target scale determination threshold includes a single target pixel ratio threshold and a target bounding rectangle side length threshold.

[0116] The targets to be detected are then divided into small targets and medium-to-large targets based on the target scale determination threshold. Small targets are those whose single-target pixel ratio is lower than a single-target pixel ratio threshold, and whose bounding rectangle side length is smaller than a target bounding rectangle side length threshold. Medium-to-large targets are those other than small targets.

[0117] Then, the default anchor box parameters of the target detection model are read, and based on the semantic hierarchical label library, the scale features of all targets to be detected in the current batch of satellite remote sensing images are extracted. The fitting coefficients are then calculated based on the default anchor box parameters and the scale features. The scale features include the average bounding rectangle size and the scale standard deviation; the fitting coefficients include those for small targets and those for medium to large targets.

[0118] Then, the baseline anchor frame is differentially scaled based on the adaptation coefficient to generate a dynamic anchor frame candidate set. To generate the dynamic anchor frame candidate set, an initial candidate set can also be generated based on the adaptation coefficient and scale standard deviation, the initial candidate set including adjustment functions for small targets and adjustment functions for medium and large targets.

[0119] By setting an adjustment step size to continuously adapt the anchor frame size to the target scale distribution and acquiring the target characteristic information of the target to be detected, constraint factors are set according to the target characteristic information. The target characteristic information includes the shape characteristics of the target to be detected; the constraint factors include a proportional constraint factor set based on the shape characteristics. Then, clustering optimization is performed on the initial candidate set based on the adjustment step size and constraint factors to generate a dynamic anchor frame candidate set.

[0120] For example, to enhance an image of a target region, target size can be determined first. This involves using a threshold of 5% pixel percentage for a single target and a target bounding rectangle side length of 32 pixels to classify targets into small and medium-to-large targets. Targets with a single pixel percentage below 5% and a bounding rectangle side length less than 32 pixels are classified as small targets, while the rest are classified as medium-to-large targets, thus achieving accurate target classification and enhancement.

[0121] Next, perform initial anchor box parameter calibration. During the initial anchor box parameter calibration process, you can first obtain the baseline anchor boxes by reading the default anchor box parameters of the currently adapted YOLO model, which can then be used as the baseline dataset for dynamic adjustment of the anchor boxes.

[0122] Then, target scale statistics are performed, that is, based on a semantic hierarchical label library, the scale features of all targets in the current batch of remote sensing images are extracted. The average bounding rectangle size (W) of small targets is also considered. s H s ), the average outer rectangle size of medium and large targets (W) l H l ), and simultaneously calculate the scale standard deviation (σ) of the two types of targets. s, σ l This ensures that the data distribution covers more than 95% of the targets within the batch.

[0123] By analyzing the fit between the target scale and the baseline anchor frame, a dynamic adjustment coefficient is calculated, and the fit coefficient is then calculated. For small targets, the fit coefficient can be calculated using the following formula:

[0124] α s =(W s / W bass +H s / H bass ) / 2;

[0125] Where, α s For small targets, the fit coefficient; W s H represents the average width of the bounding rectangle of a small target. s W represents the average height of the bounding rectangle of a small target. bass H represents the average width of the small target corresponding to the reference anchor frame. bass This represents the average height of the small target corresponding to the reference anchor frame.

[0126] Similarly, the fit coefficient for medium to large-sized targets can be calculated using the following formula:

[0127] α l =(W l / W basl +H l / H basl ) / 2;

[0128] Where, α l For medium to large-sized targets, the adaptation coefficient is W. l H represents the average width of the bounding rectangle of medium to large-sized targets. l W represents the average height of the bounding rectangle of a medium to large-sized target. basl H represents the average width of the medium to large-sized target within the reference anchor frame. basl This represents the average height of the medium to large-sized target within the baseline anchor frame.

[0129] The adaptation coefficient is limited to a range of [0.6, 1.4], and can be adjusted according to the actual application scenario and system requirements. Furthermore, when the calculated adaptation coefficient exceeds the range, it can be truncated to boundary values ​​to avoid over-adjustment of the anchor frame.

[0130] After calculating the adaptation coefficient, a dynamic anchor frame candidate set can be generated based on the adaptation coefficient. During the generation of the dynamic anchor frame candidate set, the anchor frame size can be adjusted first, that is, the baseline anchor frame is differentially scaled based on the adaptation coefficient to generate the initial candidate set. Specifically, when adjusting the anchor frame size, for anchor frames of small targets, the size can be adjusted according to the following adjustment function:

[0131] W sanchor =W bass ×α s ×(1±σ s / 2);

[0132] H sanchor =H bass ×α s ×(1±σ s / 2);

[0133] Among them, W sanchor W is the width of the anchor frame for small targets. bass α is the average width of the small target corresponding to the reference anchor frame; s For small targets, the fit coefficient is σ. s The standard deviation of the scale for small targets; H sanchor The height of the anchor frame for small targets; W bass This represents the average height of the small target corresponding to the reference anchor frame.

[0134] Similarly, for anchor frames of medium to large-sized targets, the size can be adjusted using the following adjustment function:

[0135] W lanchor =W basl ×α l ×(1±σ l / 3);

[0136] H lanchor =H basl ×α l ×(1±σ l / 3);

[0137] Among them, W lanchor W represents the width of the anchor frame for medium to large-sized targets. basl α is the average width of the medium to large-sized target corresponding to the reference anchor frame; l For medium to large-sized targets, the adaptation coefficient is σ. l H represents the scale standard deviation of medium to large-sized targets. lanchor H represents the height of the anchor frame for medium to large-sized targets. basl This represents the average height of the medium to large-sized target within the baseline anchor frame.

[0138] The step size is adjusted to 0.1 to ensure that the anchor frame size continuously adapts to the target scale distribution. Then, through anchor frame ratio optimization, considering the target characteristics in remote sensing images such as buildings being mostly rectangular and vehicles having relatively fixed aspect ratios, a ratio constraint factor is introduced. That is, the aspect ratio range of small target anchor frames is limited to [0.7, 1.3], and the aspect ratio range of medium and large target anchor frames is limited to [0.5, 2.0]. Anchor frames with abnormal ratios are eliminated to avoid invalid matching.

[0139] Then, clustering and purification are performed. Specifically, the K-means++ algorithm is used to optimize the clustering of the initial candidate set. The number of clusters for small target anchor frames is set to 3-5, and the number of clusters for medium and large target anchor frames is set to 4-6. The number of iterations is ≥100 rounds. The clustering stops when the change in the center distance of the anchor frames is ≤1%. Finally, a dynamic anchor frame candidate set that is highly adapted to the target scale of the current batch is generated.

[0140] Corresponding to the background and interference regions, the differential enhancement rules can also include enhancement rules for the background region and enhancement rules for the interference region. The background region enhancement rules are used to simulate different time periods and seasonal changes in illumination by randomly adjusting the brightness, contrast, and hue of satellite remote sensing images based on the terrain and climate characteristics of the remote sensing area, and to enhance the regional image by introducing random texture transfer technology. The interference region enhancement rules are used to simulate interference based on interference samples of different concentrations generated by an atmospheric scattering model, and to perform adaptive interference cancellation based on semantic masks to achieve interference elimination.

[0141] It is evident that, based on the semantic layering results, and considering the anchor box size distribution characteristics of target detection models such as YOLO and the shortcomings of small target detection, differentiated enhancement rules can be formulated for the target region, background region, and interference region respectively, ensuring that the enhancement operation fits the model training requirements and avoiding ineffective enhancement.

[0142] S104. Perform hierarchical enhancement on the satellite remote sensing image dataset using differential enhancement rules to generate an enhanced dataset.

[0143] After formulating differentiated enhancement rules, these rules can be used to perform hierarchical enhancement on satellite remote sensing image datasets to generate enhanced datasets. These enhanced datasets include multiple enhanced images whose sample selection and annotation are synchronously updated through a target detection model. Furthermore, the enhanced datasets balance the class distribution by weighted sampling of multiple enhanced images.

[0144] like Figure 4 As shown, for small targets, the dataset can be augmented using the augmentation rules corresponding to the target area where the small target is located. In some embodiments, when performing hierarchical augmentation on the satellite remote sensing image dataset using differentiated augmentation rules to generate an augmented dataset, the average intersection-union ratio (IUR) between the dynamic anchor boxes in the dynamic anchor box candidate set and the small target can be calculated first. Then, the number of iterations of the augmented super-resolution generative adversarial network can be set based on the average IUR, wherein the number of iterations is positively correlated with the average IUR.

[0145] Then, a scaling factor is calculated based on the adaptation coefficient of the small target, and the scaling range of the dynamic anchor box is adjusted according to the scaling factor. Based on the number of iterations and the scaling range, an enhanced super-resolution generative adversarial network is used to perform image enhancement on the small target to obtain an enhanced image containing the small target.

[0146] For example, the enhancement of small targets can be dynamically adjusted based on the super-resolution reconstruction parameters. For instance, to make the number of iterations of the Enhanced Super-Resolution Generative Adversarial Networks (ESRGAN) model positively correlated with the matching accuracy of the small target anchor boxes, the mapping relationship can be set as iteration number N = 50 + 50 × IoU. anchor Among them, IoU anchor The average intersection-union ratio (IoU) between the dynamic anchor frame and small targets is given by the IoU value. anchor When N is ≥0.8, N takes the maximum value of 100 rounds; when IoU anchor When N is less than 0.6, N takes the minimum value of 50 rounds.

[0147] Then, micro-scale scaling and adaptation are performed, that is, the scaling range is adjusted according to the dynamic anchor frame size, and the scaling factor β=1+(α) s -1)×0.5, the scaling range is adjusted from the original [0.8, 1.2] to [0.8×β, 1.2×β], to ensure that the fit between the small target size and the anchor frame size is improved by ≥30% after scaling.

[0148] For medium to large targets, augmentation rules corresponding to the target area can be used to augment the dataset. In some embodiments, when performing layered augmentation on satellite remote sensing image datasets using differentiated augmentation rules to generate augmented datasets, the coverage of the dynamic anchor frame with the medium to large target at different rotation angles can be calculated first. The coverage is the ratio of the overlap area between the dynamic anchor frame and the medium to large target to the total area of ​​the medium to large target.

[0149] Next, candidate enhancement angles are selected based on coverage, where the candidate enhancement angle is a rotation angle with coverage greater than or equal to a coverage threshold. The optimal enhancement angle is then randomly selected from the candidate enhancement angles according to the satellite imaging azimuth weight.

[0150] For example, in the image enhancement process for medium and large targets, the rotation angle can be adaptively selected first. That is, by statistically analyzing the coverage of the dynamic anchor frame with the medium and large target at different rotation angles, such as 0°, 90°, 180°, 270°, ±45°, etc., the coverage can be obtained by calculating the ratio of the overlapping area between the anchor frame and the target to the total area of ​​the target. That is: Coverage = Overlapping area between the dynamic anchor frame and the medium and large target / Total area of ​​the target.

[0151] Based on the set coverage threshold of 0.7, rotation angles with coverage ≥ 0.7 are preferentially selected as candidate enhancement angles. If multiple candidate enhancement angles meet the conditions, the optimal enhancement angle can be obtained by randomly selecting one according to the satellite imaging azimuth weight.

[0152] Then, a shearing ratio constraint is set based on the outer rectangle size of the dynamic anchor frame. This constraint includes an upper limit for the shearing ratio. The upper limit is set based on the overlap area between the dynamic anchor frame and the medium-to-large target, and the maximum side length of the medium-to-large target.

[0153] Then, the target morphology distortion parameters are adjusted according to the aspect ratio of the dynamic anchor frame. This allows for image enhancement of the satellite remote sensing image based on the optimal enhancement angle, shearing ratio constraints, and target morphology distortion parameters, resulting in an enhanced image containing medium to large-sized targets. The target morphology distortion parameters include the occlusion area ratio and attitude offset angle; these are calculated based on the average intersection-over-union ratio (IoU) between the dynamic anchor frame and the medium to large-sized targets.

[0154] For example, after adaptively selecting the rotation angle, a shearing ratio constraint can be set. That is, based on the outer rectangle size of the dynamic anchor frame, an upper limit for the shearing ratio is set: shearing ratio k. q ≤1-(M / N). Where M is the overlap area between the anchor frame and the target, and N is the maximum side length of the target. By setting an upper limit for the shearing ratio, it can be ensured that the target can still be effectively covered by the anchor frame after shearing, with a coverage of not less than 0.65.

[0155] Then, through shape distortion adaptation, the target shape distortion parameters are adjusted according to the aspect ratio of the anchor frame, and the occlusion area ratio k is determined. z =0.3×(1-z), attitude offset angle θ=30°×(1-z), where z is the anchor frame fit degree, that is, the average intersection-to-union ratio between the dynamic anchor frame and medium-to-large targets. Through morphological distortion fit, the degree of distortion and the anchor frame fit degree can be linked in reverse.

[0156] For the target area, real-time matching verification and feedback adjustments can be performed during data augmentation. Specifically, in some embodiments, when performing hierarchical augmentation on a satellite remote sensing image dataset using differentiated augmentation rules to generate an augmented dataset, the matching index between the dynamic anchor boxes and the target to be detected can be calculated first, and a passing threshold can be set. The matching index includes at least one of the following: average intersection-over-union ratio (IoU), average anchor box coverage, and similarity between the target to be detected and the dynamic anchor boxes. Corresponding to the matching index, the passing threshold includes at least one of the following: IoU threshold, coverage threshold, and similarity threshold.

[0157] Then, a parameter feedback adjustment strategy is set based on the matching index and the qualified threshold. Specifically, when the average cross-union ratio (CUNR) is less than the CUNR threshold, the feedback adjustment strategy includes recalculating the adaptation coefficient and adjusting the step size to regenerate dynamic anchor boxes; when the average anchor box coverage is less than the coverage threshold, the feedback adjustment strategy includes reducing the upper limit of the shearing ratio for medium and large targets and expanding the scaling range for small targets; when the similarity between the target to be detected and the dynamic anchor box is less than the similarity threshold, the feedback adjustment strategy includes increasing the number of residual blocks in the generative adversarial network (GAN) and / or adjusting the number of feature extraction channels of the generator in the GAN.

[0158] Based on the matching accuracy statistics within the current batch, the baseline parameters for dynamic adjustment of the anchor frame are updated. The baseline parameters include the cluster number range and the initial value of the adaptation coefficient, enabling real-time matching verification and feedback adjustment of the data augmentation process.

[0159] For example, when performing real-time matching verification and feedback adjustments, matching accuracy can be calculated first. That is, after each image enhancement is completed, the matching metrics between the dynamic anchor boxes and target features are calculated, including the mean intersection-union ratio (mIoU). anchor ), Anchor frame coverage mean (Avg) Coverage The similarity between target features and anchor boxes is calculated. Specifically, the similarity between target features and anchor boxes can be obtained by extracting feature vectors using Convolutional Neural Networks (CNNs) and calculating cosine similarity.

[0160] Then, based on the set acceptable threshold, parameter feedback adjustments are made. The acceptable threshold is used to represent the average crossover ratio mIoU. anchor ≥0.75, Avg Coverage ≥0.7, feature similarity ≥0.8. If the matching index does not reach the qualified threshold, immediate adjustment can be triggered.

[0161] That is, in mIoU anchor When <0.75, the fit coefficient α for small targets is recalculated. s Or the fit coefficient α for medium to large-sized targets lFurthermore, by adjusting the step size by 0.05, the dynamic anchor frame was regenerated. And in Avg... Coverage When the similarity is less than 0.7, the upper limit of the shearing ratio for medium and large targets can be reduced, while the scaling range for small targets can be expanded. When the feature similarity is less than 0.8, the number of residual blocks in the ESRGAN model can be increased, up to a maximum of 20, or the number of feature extraction channels in the generator can be adjusted.

[0162] Then, through batch adaptive updates, each batch consists of 100 images. Based on the matching accuracy statistics within the batch, the baseline parameters such as the cluster number range of the anchor box adjustment and the initial value of the adaptation coefficient are updated to achieve adaptive optimization across image batches, ensuring continuous adaptation between the anchor box and the enhanced target features.

[0163] For background areas, data augmentation can be performed using background area enhancement rules. In some embodiments, when performing hierarchical enhancement on a satellite remote sensing image dataset using differentiated enhancement rules to generate an enhanced dataset, environmental feature information of the remote sensing area can be obtained first. This environmental feature information includes the terrain features and climate features of the remote sensing area.

[0164] Then, environmental adjustment parameters are generated based on environmental feature information, and similar terrain textures in the background area are transferred across regions through random texture migration to generate an enhanced image that includes the background area. The environmental adjustment parameters include illumination parameters applicable to the terrain and climate features of the remote sensing area. These illumination parameters are generated by randomly adjusting the brightness, contrast, and hue of the satellite remote sensing image.

[0165] For example, when enhancing the background area, the illumination and seasonal changes at different times can be simulated based on the terrain and climate characteristics of the remotely sensed area by randomly adjusting at least one of the following: brightness, contrast, and hue. Specifically, the adjustment range for brightness is ±20%; the adjustment range for contrast is ±30%; and the adjustment range for hue is ±15°.

[0166] Furthermore, by introducing random texture transfer technology, similar terrain textures such as desert and grassland, mountains and plains are transferred across regions to match the multi-region imaging characteristics of satellites. The background pixel perturbation amplitude is strictly limited so that the gray value change is ≤10, so as to avoid confusion between the background and the target area and ensure the accuracy of target detection.

[0167] For interference areas, image data enhancement can be performed using enhancement rules for interference areas. In some embodiments, when performing layered enhancement on satellite remote sensing image datasets using differentiated enhancement rules to generate enhanced datasets, atmospheric scattering models can be used to generate interference sample images under different levels of interference content. The atmospheric scattering model is used to overlay different levels of interference content onto the enhanced images.

[0168] Then, adaptive interference cancellation is performed based on semantic masking. Then, through threshold segmentation and region restoration, the target region in the interference sample image is preserved, and the background region and interference region in the interference sample image are restored simultaneously.

[0169] For example, to enhance interference in certain areas, interference simulation can be performed first. This involves generating interference samples of different concentrations based on an atmospheric scattering model, with cloud cover categorized into low (10%-20%), medium (20%-40%), and high (40%-60%) concentrations. Shadows are simulated at different illumination angles within a range of 0° to 60° with 15° increments, closely reflecting the patterns of solar illumination. Fog concentration parameters are set to 0.2 to 0.8, comprehensively covering all types of interference in real-world scenarios.

[0170] Then, interference cancellation is performed, which involves adaptive interference cancellation based on semantic masks. The target region is accurately preserved through threshold segmentation and region inpainting algorithms, while background interference regions are simultaneously repaired. After repair, the peak signal-to-noise ratio (PSNR) of the image is no less than 30 dB, and the structural similarity (SSIM) is no less than 0.85. Interference-free and interference-free sample pairs are generated to adapt to the feature discrimination learning requirements of the YOLO model and improve the model's adaptability to complex interference scenarios.

[0171] During data augmentation, intelligent filtering and fusion of target detection models can also be performed. Specifically, in some embodiments, when performing hierarchical augmentation on a satellite remote sensing image dataset using differentiated augmentation rules to generate an augmented dataset, hierarchical augmentation can first be performed on the satellite remote sensing image dataset based on the differentiated augmentation rules to obtain hierarchically augmented data. This hierarchically augmented data is then input into a pre-trained target detection model to calculate target detection evaluation parameters. These evaluation parameters include target detection accuracy and target detection recall.

[0172] Then, high-quality sample data is selected from the hierarchical augmented data based on the target detection evaluation parameters. The high-quality sample data includes augmented images with target detection accuracy not lower than the original sample detection accuracy and recall rate not lower than the recall rate threshold.

[0173] Then, a label synchronization mapping function is established based on the affine transformation matrix, and the target location label coordinates are updated using this function. A sample weighted fusion strategy is then employed to balance the class distribution of the hierarchical augmented data through weighted sampling, thereby generating an augmented dataset. Specifically, the sample weighted fusion strategy is used to increase the weight of augmented samples for scarce class targets and small targets.

[0174] For example, when performing intelligent screening and fusion of YOLO models, a pre-trained model can be configured first. By building a YOLO pre-trained model and setting the number of iterations to no less than 100, the learning rate to a range of 0.001 to 0.01, using the SGD optimizer, setting the momentum parameter to 0.9, and the weight decay to 0.0005, the model can be ensured to have stable basic detection capabilities.

[0175] Next, the enhanced samples are initially screened. The stratified enhanced samples are input into the pre-trained YOLO model to calculate the target detection precision and recall. High-quality sample datasets with mean average precision (mAP) no less than 1.1 times that of the original sample detection precision and recall no less than 85% are selected to eliminate invalid enhanced samples and ensure the quality of the dataset.

[0176] Then, the annotation is updated synchronously. An annotation synchronous mapping mechanism is established based on the affine transformation matrix. When the image is rotated, flipped, scaled and other enhancement operations are performed, the target position annotation coordinates are updated in real time through the coordinate transformation formula to ensure that the coordinate error does not exceed 1 pixel. The accuracy of category annotation is maintained at a high level, which solves the problem of annotation misalignment in enhancement.

[0177] Then, through sample weighted fusion, the weight of ordinary samples is set to 1.0, while the weight of rare categories such as special types of buildings and niche vehicles and small targets is increased to 1.2 to 1.5 times. By balancing the class distribution of the dataset through weighted sampling, the class imbalance problem in model training is alleviated.

[0178] The augmented dataset generated by intelligent filtering and fusion through the linkage of object detection models can be adapted to model training. Specifically, based on the optimization requirements of the CIoU loss function of the YOLO model, a strategy of prioritizing hard samples is adopted to adjust the input order. High-loss samples with CIoU loss values ​​≥ 0.5 are given priority to input into the model for training, thereby reducing training bias caused by class imbalance and improving the model training effect.

[0179] In some embodiments, as a refinement and extension of the specific implementation of the above embodiments, and to fully illustrate the specific implementation process of this embodiment, some embodiments of this application also provide a satellite remote sensing image dataset enhancement method. The difference between this method and the above embodiments lies in that it can use a lightweight iterative optimization mechanism to iterate the enhancement rules and enhancement parameters, such as... Figure 5 As shown, the method includes:

[0180] S201. Construct a lightweight enhancement framework;

[0181] S202, Configure iteration parameters;

[0182] S203. Based on a lightweight enhancement framework and iteration parameters, the differentiated enhancement strategy is dynamically and iteratively adjusted through the gradient descent algorithm.

[0183] S204. Perform hierarchical enhancement on the satellite remote sensing image dataset using the adjusted differential enhancement strategy;

[0184] S205. When the set iteration termination condition is reached during the iterative adjustment process, output the optimal augmentation dataset and augmentation parameter configuration information.

[0185] A lightweight iterative optimization mechanism is employed. First, a lightweight enhancement framework is constructed and iterative parameters are configured. These parameters include the parameter update cycle, the enhancement scaling adjustment step size, and the disturbance simulation intensity adjustment step size. Then, based on the lightweight enhancement framework and iterative parameters, a differential enhancement strategy is dynamically and iteratively adjusted using a gradient descent algorithm. The adjusted differential enhancement strategy is then used to perform hierarchical enhancement on the satellite remote sensing image dataset. When the iterative adjustment process reaches the set termination condition, the optimal enhanced dataset and enhancement parameter configuration information are output.

[0186] For example, based on the lightweight iterative optimization mechanism, a lightweight framework can be built first. The lightweight enhanced framework can integrate a multi-threaded parallel computing module, dynamically allocate the number of threads based on the number of CPU cores, such as supporting 8 to 64 threads to run, and improve parallel efficiency through task splitting and synchronization mechanisms to adapt to different hardware environments.

[0187] Based on the enhanced efficiency control strategy, for remote sensing images with a resolution of 640×640 pixels, the tree algorithm can be optimized and the process simplified through feature reuse, parallel computing, etc., to ensure that the enhancement time of a single image does not exceed 0.5 seconds, which is 3 to 5 times more efficient than the traditional serial enhancement method.

[0188] When configuring iterative parameters, the parameter update cycle can be set. That is, after every 20 rounds of YOLO model training iterations, the enhancement parameters are updated synchronously. The step size for adjusting the scaling ratio of small target enhancement is set to 0.05, and the step size for adjusting the intensity of disturbance simulation is set to 0.1. The optimization accuracy is ensured through fine-tuning of parameters.

[0189] Then, closed-loop optimization is performed based on the constructed lightweight enhancement framework and configured iterative parameters. By constructing a closed loop of enhancement, training, feedback, and optimization, the enhancement strategy is dynamically adjusted using gradient descent algorithm based on the metrics such as mAP, recall, and loss value output by the YOLO model training. This adjusts the number of super-resolution iterations for small targets and the range of simulated interference concentration, allowing for YOLO target detection training to be adapted to different remote sensing scenarios without manual intervention.

[0190] During the iteration process, an iteration termination condition can be set to determine the termination of the iteration. When the model's mAP improvement does not exceed 0.05% after three consecutive iterations, the parameter iteration is stopped, and the optimal augmentation sample set and augmentation parameter configuration are output to avoid wasting computing power due to excessive iteration.

[0191] By applying the technical solutions of the above embodiments, the satellite remote sensing image dataset enhancement method described in the above embodiments can achieve semantic hierarchical differential enhancement, adapt to the characteristics of target detection models such as YOLO, achieve high-precision semantic hierarchical layering through the U-Net++ model, formulate exclusive enhancement rules for targets, backgrounds, and interference areas, and accurately solve the shortcomings of the YOLO model in small target detection and the problem of feature distortion of medium and large targets.

[0192] The method can also achieve synchronous and unbiased annotation, that is, establish an annotation synchronization mapping mechanism based on the affine transformation matrix, control the coordinate error within 1 pixel, maintain the accuracy of category annotation at a high level, and solve the problem of enhanced annotation misalignment.

[0193] The method can also enhance anchor boxes dynamically. Based on the target scale obtained from semantic hierarchies, the size and proportion of the anchor box candidate set are dynamically adjusted, and the enhancement rules are optimized simultaneously. This breaks through the data augmentation logic of enhancing first and then adapting anchor boxes, and realizes a three-in-one dynamic matching from enhancement parameters to anchor box generation and finally localization of target features, further improving the model's detection recall rate.

[0194] The method can also improve data processing throughput through lightweight and efficient operation. By integrating a multi-threaded parallel computing module, the time for enhancing a single 640×640 pixel image is ≤0.5 seconds, which is 3 to 5 times more efficient than traditional methods, making it suitable for large-scale dataset processing needs.

[0195] The method also enables closed-loop adaptive optimization. By constructing a closed-loop mechanism of enhancement, training, feedback, and optimization, enhancement parameters can be dynamically adjusted based on the YOLO model training results without manual intervention, ensuring a continuous improvement in the model's average accuracy. Furthermore, it boasts strong compatibility and wide adaptability, supporting remote sensing image processing at resolutions from 1 to 10 meters, compatible with RGB and 3- to 8-channel multispectral formats, and its label format adapts to VOC and COCO standards, allowing direct integration with mainstream object detection models.

[0196] In some embodiments, as a specific implementation of the satellite remote sensing image dataset enhancement method described in the above embodiments, some embodiments of this application also provide a satellite remote sensing image dataset enhancement system, such as... Figure 6 As shown, the system includes:

[0197] The data acquisition module is used to acquire a satellite remote sensing image dataset, which includes multiple satellite remote sensing images.

[0198] A semantic segmentation module is used to perform semantic segmentation on the satellite remote sensing image to obtain a semantic segmentation result, the semantic segmentation result including image regions divided from the satellite remote sensing image, the image regions including target regions;

[0199] The rule generation module is used to formulate differentiated enhancement rules for the image regions based on the semantic segmentation results and the anchor box size distribution characteristics of the target detection model. The differentiated enhancement rules include enhancement rules for the target regions. The enhancement rules for the target regions are used to calculate the adaptation coefficient through the adaptation degree analysis between the target scale and the reference anchor box, and to perform differentiated scaling on the reference anchor box based on the adaptation coefficient.

[0200] An image enhancement module is used to perform hierarchical enhancement on the satellite remote sensing image dataset using the differential enhancement rules to generate an enhanced dataset; the enhanced dataset includes multiple enhanced images that are synchronously updated through target detection model-linked sample selection and annotation; the enhanced dataset balances the class distribution by weighted sampling of multiple enhanced images.

[0201] By applying the technical solutions of the above embodiments, the satellite remote sensing image dataset enhancement system described in the above embodiments can perform semantic segmentation on the satellite remote sensing images after acquiring the satellite remote sensing image dataset, obtaining semantic segmentation results to divide the satellite remote sensing images into target regions, background regions, and interference regions. Based on the semantic segmentation results and combined with the anchor box size distribution characteristics of the target detection model, differentiated enhancement rules are formulated for each image region. Then, semantic hierarchical enhancement is performed on the satellite remote sensing image dataset using the differentiated enhancement rules to generate an enhanced dataset. The system can formulate differentiated enhancement rules for target, background, and interference regions, adapt to the characteristics of the target detection model, and dynamically adjust the size and proportion of the anchor box candidate set, simultaneously optimizing the enhancement rules to solve the problem of the satellite remote sensing image dataset enhancement process affecting the accuracy of model training.

[0202] It should be noted that other corresponding descriptions of the functional units involved in the satellite remote sensing image dataset enhancement system provided in this application embodiment can be found in the corresponding descriptions in the satellite remote sensing image dataset enhancement method provided in the above embodiments, and will not be repeated here.

[0203] This application also provides a computer device, specifically a personal computer, server, network device, etc. The computer device includes a bus, processor, memory, and communication interface, and may also include input / output interfaces and a display device. The processor of the computer device provides computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores an operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The database of the computer device stores location information. The network interface of the computer device is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements the steps in the various method embodiments.

[0204] Those skilled in the art will understand that the structure of the computer device described above is only a partial structure related to the solution of this application, and does not constitute a limitation on the computer device to which the solution of this application is applied. A specific computer device may include more or fewer components, or combine certain components, or have different component arrangements.

[0205] In one embodiment, a computer-readable storage medium is also provided, which may be non-volatile or volatile, and a computer program is stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0206] In one embodiment, a computer program product is also provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0207] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0208] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods.

[0209] Any references to memory, database, or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc.

[0210] Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can take many forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0211] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchain. The processors involved in the embodiments provided in this application may be, but are not limited to, general-purpose processors, graphics processors, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc.

[0212] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0213] The embodiments described above are merely examples of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these modifications and improvements all fall within the protection scope of this application.

Claims

1. A method for enhancing satellite remote sensing image datasets, characterized in that, The method includes: Acquire a satellite remote sensing image dataset, which includes multiple satellite remote sensing images; Semantic segmentation is performed on the satellite remote sensing image to obtain a semantic segmentation result, the semantic segmentation result including image regions divided from the satellite remote sensing image, the image regions including target regions; Based on the semantic segmentation results and combined with the anchor box size distribution characteristics of the target detection model, differential enhancement rules are formulated for the image regions respectively; the differential enhancement rules include the enhancement rules for the target regions; the enhancement rules for the target regions are used to calculate the adaptation coefficient through the adaptation degree analysis between the target scale and the reference anchor box, and to perform differential scaling on the reference anchor box based on the adaptation coefficient; In formulating differentiated enhancement rules, a target scale determination threshold is set, which includes a single target pixel percentage threshold and a target bounding rectangle side length threshold. Based on the target scale determination threshold, the targets to be detected are divided into small targets and medium-to-large targets. Small targets are those whose single target pixel percentage is lower than the single target pixel percentage threshold and whose bounding rectangle side length is smaller than the target bounding rectangle side length threshold. Medium-to-large targets are those other than small targets. The default anchor frame parameters of the target detection model are read. Based on the semantic hierarchical label library, the scale features of all targets to be detected in the current batch of satellite remote sensing images are extracted. The semantic hierarchical label library is used to record target information according to the semantic segmentation results. The scale features include the average bounding rectangle size and the scale standard deviation. The adaptation coefficient is calculated based on the default anchor frame parameters and the scale features. The adaptation coefficient includes... The method includes: obtaining the adaptation coefficients for the small target and the medium-to-large target; and performing differential scaling on the baseline anchor frame based on the adaptation coefficients to generate a dynamic anchor frame candidate set, including: generating an initial candidate set based on the adaptation coefficients and the scale standard deviation, the initial candidate set including adjustment functions for the small target and the medium-to-large target; the adjustment functions being used to adjust the anchor frame size for the small target and the medium-to-large target; setting an adjustment step size, the adjustment step size being used to continuously adapt the anchor frame size to the target scale distribution; obtaining target characteristic information of the target to be detected, the target characteristic information including the shape characteristics of the target to be detected; setting constraint factors according to the target characteristic information, the constraint factors including a proportional constraint factor set based on the shape characteristics; and performing cluster optimization on the initial candidate set based on the adjustment step size and the constraint factors to generate the dynamic anchor frame candidate set. The satellite remote sensing image dataset is subjected to hierarchical enhancement using the differential enhancement rules to generate an enhanced dataset; the enhanced dataset includes multiple enhanced images that are synchronously updated through target detection model-linked sample selection and annotation; the enhanced dataset balances the class distribution by weighted sampling of multiple enhanced images; Specifically, during the hierarchical enhancement process, the average intersection-union ratio (IU) between the dynamic anchor boxes in the candidate dynamic anchor boxes set and the small target is calculated. The number of iterations for the enhanced super-resolution generative adversarial network (GAN) is set based on the IU, and the number of iterations is positively correlated with the IU. A scaling factor is calculated based on the fit coefficient of the small target, and the scaling range of the dynamic anchor boxes is adjusted according to the scaling factor. Based on the number of iterations and the scaling range, the enhanced super-resolution GAN is used to perform image enhancement on the small target to obtain an enhanced image containing the small target.

2. The method according to claim 1, characterized in that, Obtain a dataset of satellite remote sensing images, including: Acquire satellite information and raw images captured by the satellite; the satellite information includes the radiation response characteristics of the satellite sensors, satellite orbital parameters, attitude angle data, and ground control point information; Perform radiometric correction on the original image based on the radiometric response characteristics information; Based on the satellite orbit parameters, the attitude angle data, and the ground control point information, a polynomial correction model is used to perform geometric distortion correction on the original image; The original image is subjected to noise removal and size adjustment to generate the satellite remote sensing image.

3. The method according to claim 1, characterized in that, Perform semantic segmentation on the satellite remote sensing image to obtain semantic segmentation results, including: The satellite remote sensing image is input into a semantic segmentation model, which optimizes the feature extraction process by introducing an attention mechanism. The semantic segmentation model is used to perform semantic segmentation on the satellite remote sensing image to divide the image region from the satellite remote sensing image; the image region includes the target region, the background region and the interference region; the target region is an image region containing the target to be detected; the background region is an image region without the target to be detected; the interference region is an image region containing interference content, the interference content including at least one of clouds, shadows and fog spots; The semantic segmentation result is generated by aggregating the target region, the background region, and the interference region. Record target information based on semantic segmentation results, including target scale, target location, and target category; A semantic hierarchical tag library is constructed based on the target information.

4. The method according to claim 1, characterized in that, The satellite remote sensing image dataset is subjected to hierarchical enhancement using the differential enhancement rules to generate an enhanced dataset, including: The coverage of the dynamic anchor frame with the medium-to-large target at different rotation angles is statistically analyzed. The coverage is the ratio of the overlapping area of ​​the dynamic anchor frame and the medium-to-large target to the total area of ​​the medium-to-large target. Based on the coverage, a candidate enhancement angle is selected, wherein the candidate enhancement angle is a rotation angle where the coverage is greater than or equal to a coverage threshold; The optimal enhancement angle is randomly selected from the candidate enhancement angles according to the satellite imaging azimuth angle weight; A shearing ratio constraint is set based on the outer rectangle size of the dynamic anchor frame, and the shearing ratio constraint includes an upper limit for the shearing ratio; the upper limit for the shearing ratio is set according to the overlap area between the dynamic anchor frame and the medium-to-large target and the maximum side length of the medium-to-large target. The target shape distortion parameters are adjusted according to the aspect ratio of the dynamic anchor frame. The target shape distortion parameters include the occlusion area ratio and the attitude offset angle. The occlusion area ratio and the attitude offset angle are calculated based on the average intersection-to-union ratio of the dynamic anchor frame and the medium-to-large target. Image enhancement is performed on the satellite remote sensing image according to the optimal enhancement angle, the shearing ratio constraint, and the target morphology distortion parameters to obtain an enhanced image containing the medium-to-large target.

5. The method according to claim 1, characterized in that, The satellite remote sensing image dataset is subjected to hierarchical enhancement using the differential enhancement rules to generate an enhanced dataset, including: Based on the aforementioned differential enhancement rules, hierarchical enhancement is performed on the satellite remote sensing image dataset to obtain hierarchically enhanced data; The hierarchical augmented data is input into a pre-trained target detection model to calculate target detection evaluation parameters, which include target detection accuracy and target detection recall. Based on the target detection evaluation parameters, high-quality sample data is selected from the hierarchical augmented data. The high-quality sample data includes augmented images with target detection accuracy not lower than the original sample detection accuracy and recall rate not lower than the recall rate threshold. A label synchronization mapping function is established based on the affine transformation matrix, and the target position label coordinates are updated using the label synchronization mapping function; A sample weighted fusion strategy is adopted to balance the category distribution of the hierarchical augmented data through weighted sampling to generate the augmented dataset; the sample weighted fusion strategy is used to increase the weight of augmented samples for scarce category targets and small targets.

6. The method according to claim 1, characterized in that, The method further includes: Build a lightweight enhancement framework; Configure iteration parameters, which include parameter update period, enhancement scaling adjustment step size, and disturbance simulation intensity adjustment step size; Based on the lightweight enhancement framework and the iteration parameters, the differentiated enhancement strategy is dynamically and iteratively adjusted using the gradient descent algorithm. The satellite remote sensing image dataset is subjected to hierarchical enhancement using the adjusted differential enhancement strategy; When the iterative adjustment process reaches the set iteration termination condition, the optimal augmented dataset and augmentation parameter configuration information are output.

7. A satellite remote sensing image dataset enhancement system, characterized in that, The system is applied to the method according to any one of claims 1-6; the system comprises: The data acquisition module is used to acquire a satellite remote sensing image dataset, which includes multiple satellite remote sensing images. A semantic segmentation module is used to perform semantic segmentation on the satellite remote sensing image to obtain a semantic segmentation result, the semantic segmentation result including image regions divided from the satellite remote sensing image, the image regions including target regions; The rule generation module is used to formulate differentiated enhancement rules for the image regions based on the semantic segmentation results and the anchor box size distribution characteristics of the target detection model. The differentiated enhancement rules include enhancement rules for the target regions. The enhancement rules for the target regions are used to calculate the adaptation coefficient through the adaptation degree analysis between the target scale and the reference anchor box, and to perform differentiated scaling on the reference anchor box based on the adaptation coefficient. An image enhancement module is used to perform hierarchical enhancement on the satellite remote sensing image dataset using the differential enhancement rules to generate an enhanced dataset; the enhanced dataset includes multiple enhanced images that are synchronously updated through target detection model-linked sample selection and annotation; the enhanced dataset balances the class distribution by weighted sampling of multiple enhanced images.

Images