Automatic color correction method for mass remote sensing images of sub-meter satellite

Abstract

The invention discloses an automatic color correction method for mass remote sensing images of a sub-meter satellite. Belongs to the technical field of automatic color correction of mass sub-meter satellite remote sensing images, and particularly relates to the technical field of an automatic color correction method for mass remote sensing images of sub-meter satellites. Mass sub-meter high-resolution pairing data are obtained by training an improved ResNet50 model, and then a histogram matching method is combined to obtain the color of the mass remote sensing images of the sub-meter satellites. And automatic color correction of remote sensing images of massive sub-meter satellites is realized. The method comprises the following steps: obtaining a sub-meter remote sensing image before correction, and carrying out histogram matching on the sub-meter remote sensing image before correction; pre-processing the sub-meter remote sensing image before correction and the corresponding RGB histogram; constructing and improving a ResNet50 model: adding a full connection module after a full connection layer of the ResNet50 model; and correcting a sub-meter remote sensing image through a ResNet50 improved model.

Description

Technical Field

[0001] The present invention relates to the technical field of automated color correction for massive sub-meter satellite remote sensing images, and specifically to a technical field of an automated color correction method for massive remote sensing images of sub-meter satellites. Background Art

[0002] Sub-meter satellite remote sensing image products have important application values in the fields of map application, disaster and emergency management, precision agriculture and forestry. The occurrence of color problems in remote sensing image products is often the result of the combined action of multiple factors.

[0003] In addition, the reflection characteristics of the surface itself also affect the color accuracy. Different ground objects (such as vegetation, water bodies, urban buildings, etc.) reflect light of different wavelengths according to their inherent physical and chemical properties, and this difference is reflected as a color difference in remote sensing images. Without proper correction and calibration, these natural color changes may be misinterpreted or misrepresented. In the process of processing remote sensing data, the selection and application of color correction algorithms are also crucial. Improper image processing techniques such as abnormal radiation correction may further distort the original color information. Especially when processing a large amount of data automatically, algorithms that are not optimized for specific scenarios or conditions may lead to inconsistent color output.

[0004] In remote sensing image processing, radiation correction is a basic and crucial step, and its purpose is to eliminate the effects caused by the systematic errors of sensors, atmospheric interference and the characteristics of the surface itself, so that the image is closer to the spectral reflectance of the real surface. However, the radiation correction process itself may also introduce new color problems, especially when the correction method is not accurate enough or there are quality problems in the original data itself.

[0005] Radiation correction usually needs to adjust the image data according to the radiation response function of the sensor. If these parameters (such as the gain and offset of the sensor) are not accurately measured or estimated, it may lead to the true color in the image being wrongly enhanced or suppressed. For example, in the visible light band, if the correction of the red band is excessive, it may cause the image to present a reddish visual effect while ignoring the balance of other colors. An important link in radiation correction is atmospheric correction, that is, removing the influence of the atmosphere on light propagation. Different atmospheric correction models are applicable to different types of atmospheric conditions, such as sunny, cloudy or foggy days. Incorrect selection of the atmospheric correction model may lead to color distortion of the corrected data. For example, correcting a foggy day image according to the sunny day model may over-enhance certain bands, resulting in unnatural colors. The diversity of surface materials and conditions requires radiation correction to adapt to the characteristics of different ground objects. For areas with complex surface cover, such as areas where cities and natural vegetation are mixed, a unified correction strategy may not be able to effectively process all types of ground objects, resulting in overcorrection or undercorrection of the colors of some ground objects.

[0006] For the color correction methods of remote sensing images, they can be mainly divided into two categories, namely unsupervised methods and supervised methods. Unsupervised color correction methods are independent of any external standard images and rely on processing the internal information of the original data.

[0007] Supervised color correction methods rely on known reference images or ground truth data to correct the remote sensing images to be processed. The histogram matching technique adjusts the histogram of the image to be corrected to make it consistent with the histogram of the selected reference image, so as to achieve the purpose of color correction. However, this method can only be carried out one-to-one in the case of having a reference image. With the development of artificial intelligence technology, image color correction methods based on deep learning have begun to be popular. These methods use a large amount of training data to train neural networks, automatically learn the complex mapping relationship between the images and the ground truth, and show strong capabilities in dealing with complex and heterogeneous data. However, for the color correction of a large number of sub-meter satellite images, how to obtain a large number of high-resolution paired datasets is still one of the unsolved problems. Figure 1

[0008] Therefore, the automatic color correction of a large number of remote sensing images of sub-meter satellites still faces many challenges. One of the main challenges is to obtain a large number of high-resolution paired data of sub-meter level, covering a rich variety of surface types, different times and lighting conditions. In addition, for the processing of a large amount of remote sensing data, it is crucial to develop efficient algorithms, and it is also essential to enhance the robustness and generalization ability of the algorithms. Summary of the Invention

[0009] Aiming at the problem that it is difficult to obtain a large number of high-resolution paired data of sub-meter level in the existing color correction methods, the present invention provides an automatic color correction method for a large number of remote sensing images of sub-meter satellites. By training an improved ResNet50 model, a large number of high-resolution paired data of sub-meter level are obtained, and then combined with the histogram matching method, the automatic color correction of remote sensing images of a large number of sub-meter satellites is realized.

[0010] The method includes the following steps:

[0011] S1. Obtain N sub-meter remote sensing images before correction, and perform histogram matching on the sub-meter remote sensing images before correction to obtain the RGB histograms of the N sub-meter remote sensing images before correction;

[0012] S2. Preprocess the N sub-meter remote sensing images before correction and the corresponding RGB histograms to obtain M pairs of image block datasets;

[0013] S3. Divide the M pairs of image block datasets into a training set and a validation set according to 8:2;

[0014] S4. Construct and improve the ResNet50 model to obtain an improved ResNet50 model:

[0015] S41. After the fully connected layer of the ResNet50 model, add a fully connected module;

[0016] S5. Train the improved ResNet50 model with the training set and the validation set. When the convergence of the loss function tends to be stable, the training is completed, and the trained improved ResNet50 model is obtained;

[0017] S6. Calibrate the sub-meter-level remote sensing image with the trained improved ResNet50 model.

[0018] Furthermore, the preprocessing is specifically: crop according to the size of 224×224 pixels.

[0019] Furthermore, the fully connected module passes through the first fully connected layer, the second fully connected layer, and the third fully connected layer in parallel from input to output. The first fully connected layer, the second fully connected layer, and the third fully connected layer all use the Softmax activation function.

[0020] Furthermore, the functions of the first fully connected layer, the second fully connected layer, and the third fully connected layer are to predict the probability distributions of the R, G, and B channels respectively according to the spatial dimensions of the R, G, and B channels.

[0021] Furthermore, the specific process of calibrating the sub-meter-level remote sensing image with the trained improved ResNet50 model is as follows:

[0022] S51. Input the sub-meter-level remote sensing image to be calibrated into the trained improved ResNet50 model to obtain the probability distributions of the RGB channels after calibration of the sub-meter-level remote sensing image;

[0023] S52. Perform histogram matching on the probability distributions of the RGB channels after calibration of the sub-meter-level remote sensing image and the sub-meter-level remote sensing image to be calibrated to obtain the calibrated sub-meter-level remote sensing image.

[0024] Furthermore, when the processor executes the computer program, the steps of the above method are implemented.

[0025] Furthermore, when the computer instructions are executed by the processor, the steps of the above method are implemented.

[0026] The beneficial effects of the present invention are:

[0027] (1) After the fully connected layer of the existing ResNet50 model, a fully connected module (the first fully connected layer, the second fully connected layer, and the third fully connected layer) is introduced in parallel. The single spatial dimension aggregation value output by the fully connected layer is divided into probability distributions of the R, G, and B channels, so that the RGB channel distribution information can be extracted, improving the robustness and generalization ability of the ResNet50 basic model.

[0028] (2) First, this invention obtains a large number of high-precision sub-meter remote sensing image paired datasets through histogram matching and cropping methods, covering rich surface types, different times, and lighting conditions. The trained improved ResNet50 model can automatically correct remote sensing images without reference to obtain the RGB channel probability distribution after the correction of sub-meter remote sensing images.

[0029] (3) By combining histogram matching and the improved ResNet50 model, this invention can perform automatic color correction of a large number of sub-meter satellite remote sensing images, improving the delivery efficiency and quality of remote sensing image products, and greatly reducing labor costs, which has important applications in fields such as precision agriculture management, disaster assessment, and environmental monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a schematic diagram of the structure of the improved ResNet50 model described in the embodiment of the present invention;

[0031] Figure 2 It is a schematic diagram of the structure of the Bottleneck module described in the embodiment of the present invention;

[0032] Figure 3 It is a flowchart of the method described in the embodiment of the present invention;

[0033] Figure 4 (a) It is the Landsat9 surface reflectance image described in the embodiment of the present invention; (b) It is the image before color correction; (c) It is the image after color correction. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0034] Next, the technical solutions of the present invention will be clearly and completely described in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of them. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0035] This embodiment provides a method for automatic color correction of a large number of remote sensing images of sub-meter satellites. The flowchart of the method is as Figure 3As shown in the figure, this embodiment relies on the "Jilin-1" series of sub-meter remote sensing satellite images to obtain a large number of sub-meter remote sensing image data sources. In this embodiment, 100 sub-meter remote sensing images of the "Jilin-1" high-resolution series before calibration are collected, which cover different landforms and ground objects around the world, thereby improving the generalization ability of the model. In order to obtain the true surface reflectance of ground objects, this embodiment uses the official surface reflectance data of Landsat8 / 9 as a reference (data source: Google Earth Engine platform), and uses programming to download the data of the same region corresponding to the "Jilin-1" remote sensing images, obtaining 100 corresponding surface reflectance images with a resolution of 30m. The 100 sub-meter remote sensing images of the "Jilin-1" high-resolution series before calibration and the 100 corresponding surface reflectance images with a resolution of 30m are subjected to histogram matching, and the colors of the "Jilin-1" images before calibration are mapped to the colors of the surface reflectance, thereby obtaining the RGB histograms of 100 sub-meter remote sensing images before calibration.

[0036] Subsequently, the 100 sub-meter remote sensing images before calibration and the corresponding RGB histograms are cropped into multiple image patches with a size of 224×224 pixels, and a total of about 30,000 pairs of image patches are obtained. Each pair of image patches can be divided into before matching and after matching;

[0037] The 30,000 pairs of image patches are divided into a training set and a validation set according to a ratio of 8:2.

[0038] Construct and improve the ResNet50 model to obtain the ResNet50 improved model (deep neural network model):

[0039] As Figure 1 shown, after the fully connected layer of the ResNet50 model, a fully connected module is added;

[0040] From input to output, the fully connected module passes through the first fully connected layer, the second fully connected layer, and the third fully connected layer in parallel. The first fully connected layer, the second fully connected layer, and the third fully connected layer all use the Sofmax activation function. The functions of the first fully connected layer, the second fully connected layer, and the third fully connected layer are to predict the probability distributions of the R, G, and B channels respectively according to the spatial dimensions of the R, G, and B channels.

[0041] The fully connected module divides the single spatial dimension aggregation value output by the fully connected layer into the probability distributions of the R, G, and B channels, so that the distribution information of the RGB channels can be extracted, improving the robustness and generalization ability of the ResNet50 basic model.

[0042] The improved ResNet-50 model starts from the preliminary processing layer of the input image. This part consists of a large 7×7 convolutional kernel (ConV7×7) for quickly extracting low-level features and expanding the receptive field. Subsequently, it is followed by a 3×3 max pooling layer (MaXPool2d3×3) to further reduce the spatial resolution of the feature map. Through this combination, the network effectively compresses the computational amount while retaining key visual information.

[0043] The following main body of the network consists of four stages, corresponding to the four convolutional parts of conv2_x, conv3_x, conv4_x, and conv5_x respectively. The number of residual modules in each convolutional part is 3, 4, 6, and 3 in sequence, totaling 16 Bottleneck residual blocks.

[0044] As Figure 2 shown, each Bottleneck structure consists of three layers of convolution: First, a 1×1 convolution is used for channel compression, then a 3×3 convolution for spatial feature extraction, and finally a 1×1 convolution to raise the number of channels back to the original dimension. This "compression-convolution-expansion" structural design can significantly reduce the computational amount and the number of parameters while maintaining the expressive ability.

[0045] In each residual block, while the input features are processed through the three-layer convolution of the main path, they are also directly passed to the output end through a bypass and added to the output of the main path. This way of cross-layer direct connection effectively alleviates the problems of gradient disappearance and performance degradation that occur in the training process of deep networks, enabling the network depth to be further increased without damaging the learning ability.

[0046] In addition, batch normalization (BN, Batch Normalization) and ReLU activation functions are equipped after each layer of convolution operation to improve the training stability and non-linear modeling ability. After all the residual modules are completed, the improved ResNet-50 model aggregates the spatial dimensions of each channel into a single value through a global average pooling layer and a fully connected layer, and then through the fully connected module, divides the single spatial dimension aggregation value into the probability distributions of the R, G, and B channels.

[0047] The ResNet50 improved model is trained through the training set and the validation set. The learning rate is set to 5e-5, the weight decay is set to 5e-4, the batch size is set to 256, and the number of training times traverses the entire dataset 200 times. When the convergence of the loss function tends to be stable, the training is completed, and the trained ResNet50 improved model is obtained.

[0048] As Figure 3 shown, the sub-meter-level remote sensing images are corrected through the trained ResNet50 improved model.

[0049] S51. Input the sub-meter remote sensing image to be corrected into the trained improved ResNet50 model to obtain the probability distribution of the RGB channels of the corrected sub-meter remote sensing image.

[0050] S52. Perform histogram matching on the probability distribution of the RGB channels of the corrected sub-meter remote sensing image and the sub-meter remote sensing image to be corrected to obtain the corrected sub-meter remote sensing image.

[0051] In this embodiment, some regions are selected to test the method of the present invention, and the tested images are sourced from the "Jilin-1" satellite data source. Taking the region near Beijing Capital International Airport as an example, as Figure 4 shown, (a) represents the Landsat9 surface reflectance image, (b) represents the image before correction (the original image of the "Jilin-1" series), and (c) represents the image after correction (the "Jilin-1" satellite image after color correction). Here, the colors of the airport and the trees are compared. Before correction, the airport in the image is yellowish, and the color of the trees is relatively dark. After color correction, these problems can be well solved. From the above experimental results, it can be seen that the method of the present invention can effectively perform color correction on sub-meter remote sensing satellite images and achieve an ideal effect, reduce the cost of manual delineation, and improve the efficiency and quality of image delivery.

Claims

1. An automated color correction method for a large amount of remote sensing images of sub-meter satellites, characterized in that, The method includes the following steps: S1. Obtain N sub-meter-level remote sensing images before calibration, perform histogram matching on the sub-meter-level remote sensing images before calibration, and obtain the RGB histograms of the N sub-meter-level remote sensing images before calibration; S2. Preprocess the N sub-meter-level remote sensing images and the corresponding RGB histograms to obtain M pairs of map block data sets; S3. Divide the M pairs of map block data sets into a training set and a validation set according to 8:2; S4. Construct and improve the ResNet50 model to obtain an improved ResNet50 model: S41. After the fully connected layer of the ResNet50 model, add a fully connected module; S5. Train the improved ResNet50 model with the training set and the validation set. When the convergence of the loss function tends to be stable, the training is completed, and the trained improved ResNet50 model is obtained; S6. Calibrate the sub-meter-level remote sensing images through the trained improved ResNet50 model.

2. The automatic color correction method for mass remote sensing images of a sub-meter satellite according to claim 1, wherein The specific preprocessing is: crop according to the size of 224×224 pixels.

3. An automated color correction method for mass remote sensing images of sub-meter satellites according to claim 2, characterized in that The fully connected module passes through the first fully connected layer, the second fully connected layer, and the third fully connected layer in parallel from input to output. The first fully connected layer, the second fully connected layer, and the third fully connected layer all use the Sofmax activation function.

4. The automated color correction method for mass remote sensing images of sub-meter satellites according to claim 3, characterized in that, The functions of the first fully connected layer, the second fully connected layer, and the third fully connected layer are to predict the probability distributions of the R, G, and B channels respectively according to the spatial dimensions of the R, G, and B channels.

5. A method for automatically color correcting a large amount of remote sensing images of a sub-meter satellite according to claim 4, characterized in that, The specific process of calibrating the sub-meter-level remote sensing images through the trained improved ResNet50 model is as follows: S51. Input the sub-meter-level remote sensing image to be calibrated into the trained improved ResNet50 model to obtain the probability distributions of the RGB channels after calibration of the sub-meter-level remote sensing image; S52. Perform histogram matching on the probability distributions of the RGB channels after calibration of the sub-meter-level remote sensing image and the sub-meter-level remote sensing image to be calibrated to obtain the sub-meter-level remote sensing image after calibration.

6. An electronic device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1-5.

7. A computer-readable storage medium for storing computer instructions, characterized in that, When the computer instructions are executed by the processor, it implements the steps of the method according to any one of claims 1-5.

Images