A method for generating a remote sensing multispectral image defogging dataset

Abstract

The application provides a method for generating a remote sensing multispectral image defogging dataset. The method is based on a preset number of cirrus channel images without fog and remote sensing multispectral images, and combines an atmospheric scattering model, so that a relatively realistic remote sensing multispectral image defogging dataset with fog and without fog can be generated. The method can effectively alleviate the condition that a remote sensing multispectral image defogging algorithm based on deep learning lacks a large number of remote sensing multispectral image datasets with fog and without fog.

Description

Technical Field

[0001] The present invention relates to the fields of atmospheric optics, image processing, and more particularly to a method for generating a remote sensing multispectral image defogging dataset. Background Art

[0002] Remote sensing multispectral imagery is widely used in industry and commerce, but it is susceptible to weather conditions, which can blur or even lose image detail, reduce contrast, and hinder its effective utilization. Clouds and fog are the most common weather conditions encountered in remote sensing imaging. Much of the Earth is permanently obscured by clouds and fog, and with the increasing atmospheric pollution brought on by industrial development, the global probability of foggy weather is increasing, significantly reducing the efficiency of remote sensing multispectral imagery.

[0003] The goal of remote sensing multispectral image dehazing algorithms is to restore degraded remote sensing multispectral images affected by fog and cloud, thereby improving the utilization efficiency of remote sensing multispectral images. Traditional remote sensing multispectral image dehazing algorithms often rely on artificial priors or statistical laws. The main problems of such algorithms are weak adaptability, unstable dehazing performance, and overly complex parameter adjustment. With the emergence of deep learning, remote sensing multispectral image dehazing algorithms based on deep learning have achieved performance far exceeding that of traditional algorithms on some public datasets. However, the development of such algorithms is still limited by the lack of a large amount of high-quality, fog-free and fog-involved remote sensing multispectral image data. In particular, when using supervised learning methods, the foggy and fog-free images in the training data must be paired.

[0004] However, in natural conditions, it is impossible to simultaneously obtain remote sensing multispectral images of an area with and without fog. Therefore, the vast majority of large-scale image dehazing datasets are currently produced using synthetic methods. Although synthetic foggy images differ from real foggy images, these synthetic image dehazing datasets have effectively promoted the research of deep learning image dehazing algorithms, and deep learning algorithms trained on synthetic datasets still have a certain dehazing effect on real foggy images.

[0005] Currently, synthetic remote sensing multispectral image dehazing datasets mainly face two problems: (1) the number of images is insufficient; (2) the synthesis method of foggy images is relatively simple. Summary of the Invention

[0006] In view of the shortcomings of the existing technology, the present invention proposes a new method for generating a remote sensing multispectral image defogging dataset, which includes the following steps:

[0007] A method for generating a remote sensing multispectral image defogging dataset comprises the following steps:

[0008] S1: Collect a preset number of cirrus channel images and remote sensing multispectral images without fog, and construct a remote sensing multispectral image source dataset;

[0009] S2: Generate a multispectral transmittance map based on a cirrus channel image and a fog thickness parameter;

[0010] S3: Generate a multispectral atmospheric light based on a remote sensing multispectral image without fog;

[0011] S4: Generate a remote sensing multispectral image with fog based on the multispectral transmittance map, the remote sensing multispectral image without fog, and the multispectral atmospheric light;

[0012] S5: By limiting the value range of the fog thickness parameter, the fog thickness is divided into a preset number of grades. For each remote sensing multispectral image without fog in the source dataset, a number of different fog thickness parameters and the same number of different cirrus channel images are sampled at each fog grade, steps S2, S3 and S4 are performed, a preset number of controlled remote sensing multispectral images with fog are synthesized, and all remote sensing multispectral images without fog and controlled remote sensing multispectral images with fog are collected as a remote sensing multispectral image defogging dataset.

[0013] Further, step S2 specifically includes:

[0014] S201: For a cirrus channel image, eliminate the dark level in the image to obtain a fog mask;

[0015] S202: Based on the fog mask and the fog thickness parameter, generate a reference channel reflectance map and a reference channel transmittance map;

[0016] S203: Based on the scatter plot data pair of the index and the reflectance, use a third-order polynomial to fit the curve relationship between the index and the reflectance, the reference channel reflectance map, and generate a corresponding index map;

[0017] S204: Based on the reference channel transmittance map and the index map, use the derived formula of the atmospheric scattering model to generate the transmittance maps of the remaining channels to obtain a multispectral transmittance map.

[0018] Further, in step S201, the cirrus channel image is normalized and the dark level is eliminated by a linear stretching method, which avoids distortion of the synthesized fog image caused by the dark level and introduces a significant bias to the target dataset.

[0019] Further, step S3 specifically includes:

[0020] S301: For a remote sensing multispectral image without fog, estimate the atmospheric light value of the corresponding channel using the brightest part of the pixels in each channel;

[0021] S302: for each non-fog remote sensing multispectral image in the source data set, the above steps are performed to obtain the atmospheric light estimation value of each channel of the non-fog remote sensing multispectral image, and the mean and variance of the atmospheric light estimation value are calculated in each channel, and the channel with the smallest variance is taken as the atmospheric light reference channel;

[0022] S303: based on the atmospheric light estimation value of each channel of the single non-fog remote sensing multispectral image obtained in step S301 and the mean and atmospheric light reference channel of the atmospheric light estimation value of each channel of the non-fog remote sensing multispectral image obtained in step S302, the atmospheric light estimation value of each channel of the single image is corrected to obtain the multispectral atmospheric light.

[0023] Further, in steps S302 and S303, the mean calculation formula and the correction formula of the atmospheric light estimation value are as follows:

[0024]

[0025]

[0026] A′ i = α · A i + (1-α) · R i A r , i, r ∈ {1, 2, …, n}

[0027] Wherein, N J represents the total number of non-fog remote sensing multispectral images in the source data set, r represents the serial number of the atmospheric light reference channel, represents the atmospheric light estimation value of channel i estimated by the kth remote sensing multispectral image in the source data set, represents the mean of the atmospheric light estimation value of channel i, R i represents the ratio of the mean of the atmospheric light estimation value of channel i to the mean of the atmospheric light estimation value of the atmospheric light reference channel, α represents the fusion weight, and the value is between 0.1 and 0.2, A i represents the atmospheric light estimation value of channel i estimated by a non-specific non-fog remote sensing multispectral image, A′ i represents the corrected atmospheric light estimation value of channel i.

[0028] Further, step S4 specifically includes:

[0029] S401: according to the truncation coefficient, the multispectral transmittance map obtained in step S2 is truncated;

[0030] S402: according to the multispectral transmittance map of step S2, the non-fog remote sensing multispectral image and the multispectral atmospheric light of step S3, and the truncated multispectral transmittance map of step S401, the improved atmospheric scattering model is combined to synthesize the foggy remote sensing multispectral image.

[0031] Furthermore, in step S401, the multi-spectral transmittance graph is truncated according to the truncation coefficient, and the transmittance graph of each channel after truncation is t′. i (x) is obtained according to the following formula:

[0032] t′ i (x)=max(0,1-w·(1-t i (x))),i∈{1,2,…,n}

[0033] Where x represents the pixel position in the image, (·) represents the value of the corresponding pixel position in the image, max(a,b) represents the larger value of a or b, w represents the truncation coefficient, which ranges from 1 to 1.5, and n represents the total number of channels in the multispectral image.

[0034] Furthermore, in step S402, the improved atmospheric scattering model is as follows:

[0035] I i (x) = J i (x)t′ i (x)+A′ i (1-t i (x)),i∈{1,2,…,n}

[0036] Among them, t i represents the transmittance map of channel i, x represents the pixel position in the image, (·) represents the value of the corresponding pixel position in the image, and J i represents the channel i image of the fog-free remote sensing multispectral image, t i (x) and t′ i (x) represents the transmittance diagram of channel i before and after truncation, A′ i represents the atmospheric light estimation value of channel i after correction, I i The channel i image represents the synthetic foggy remote sensing multispectral image.

[0037] The beneficial effects of the present invention are:

[0038] The present invention can synthesize foggy remote sensing multispectral images by using fog-free remote sensing multispectral images and cirrus cloud channel images. At the same time, a strategy for synthesizing defogging datasets of remote sensing multispectral images with and without fog is provided. The synthesized foggy remote sensing multispectral images are more realistic and take into account the physical phenomenon that fog has different effects on different channels. They are suitable for generating corresponding datasets in the research of remote sensing multispectral image defogging algorithms based on deep learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a flow chart of the remote sensing multispectral image defogging dataset synthesis method proposed in the present invention;

[0040] Figure 2 This is a detailed flow chart of the remote sensing multispectral image defogging dataset synthesis method proposed in the present invention (taking Landsat-8 satellite remote sensing multispectral image as an example);

[0041] Figure 3 This figure shows samples from a remote sensing multispectral image dehazing dataset synthesized using the method proposed in this invention. DETAILED DESCRIPTION

[0042] The input of the method proposed in the present invention is a remote sensing multispectral image source dataset, which includes two parts: (1) a certain amount of fog-free remote sensing multispectral image data; and (2) a certain amount of cirrus channel image data. After processing by the method proposed in the present invention, the output remote sensing multispectral image defogging dataset includes a certain amount of foggy and fog-free remote sensing multispectral image data. In addition, in order to meet the data loading capacity of most hardware platforms, the embodiment of the present invention uniformly crops the large remote sensing multispectral images into small images with no overlap and a size of 512×512 pixels.

[0043] For the sake of convenience, the parameters and image labels in the remote sensing multispectral image source dataset and the remote sensing multispectral image defogging dataset are defined as follows in this embodiment: the number of cirrus channel images is N c , the kth cirrus channel image is A non-specific cirrus channel image is ρ c The number of fog-free remote sensing multispectral images is N J , the kth fog-free remote sensing multispectral image is J k , a non-specific fog-free remote sensing multispectral image is J, where the images of each channel are represented by J i ,i∈{1,2,…,n}, the wavelength distribution corresponding to each channel is expressed as λ i ,i∈{1,2,…,n}, n represents the total number of channels of the multispectral image. The fog thickness is divided into L levels, and M fog thickness parameters are sampled at each level to synthesize the foggy remote sensing multispectral image. The number of foggy remote sensing multispectral images is N I =N J ×L×M, the remote sensing multispectral image with fog compared with the k-th fog-free remote sensing multispectral image is I k,l,m ,k∈{1,2,…,N J},l∈{1,2,…,L},m∈{1,2,…,M}, a non-specific foggy remote sensing multispectral image is I, where the images of each channel are represented by I i,i∈{1,2,…,n}. For simplicity, it is assumed that the value range of all input and output images is normalized to between 0 and 1, that is, ρ c ,J,I∈[0,1].

[0044] This paper proposes a method for generating a remote sensing multispectral image defogging dataset. Figure 1 A flow chart showing the method is shown, Figure 2 This is a detailed flowchart using Landsat-8 satellite remote sensing multispectral imagery (a fog-free remote sensing multispectral image contains 7 channels) as an example. It specifically includes the following steps:

[0045] In step S1, a certain number of cirrus cloud channel images and fog-free remote sensing multispectral image data are collected as source datasets to construct a remote sensing multispectral image source dataset. The cirrus clouds in the cirrus cloud channel images are required to be morphologically similar to fog, without distinct boundaries or shapes, to improve the realism of the fog in the composite image. The fog-free remote sensing multispectral images are also required to contain a rich variety of seasonal and landform variations to avoid bias in the dataset towards certain landforms.

[0046] In sub-step S201 of step S2, for a cirrus channel image ρ c , eliminate the dark level by linear stretching and obtain the fog mask ρ′ c Avoid the distortion of synthetic foggy images caused by dark levels, which introduces a significant bias to the remote sensing multispectral image defogging dataset. The linear stretching formula is as follows:

[0047] ρ min =percentile(ρ c ,p)#(1)

[0048] ρ max =percentile(ρ c ,100-p)#(2)

[0049]

[0050] Where x represents the pixel position in the image, (·) represents the value of the corresponding pixel position in the image, percentile(ρ,p) represents sorting all pixels in image ρ in ascending order of pixel value and taking the value corresponding to the pixel at position p% from lowest to highest, and min / max(a,b) represents taking the smaller / larger value of a or b.

[0051] In sub-step S202 of step S2, the channel with the shortest wavelength is used as the reference channel, and the fog mask ρ′ obtained in sub-step S201 is used as the reference channel. c And the input fog thickness parameter ω, generate the reflectivity map ρ1 and transmittance map t1 of the reference channel, and the generation formula is as follows:

[0052] ρ1=ω·ρ′ c #(4)

[0053] t1=1-ρ1#(5)

[0054] In sub-step S203 of step S2, based on 6 scattered data pairs of reflectivity and index: (0.108, 4.000), (0.255, 2.000), (0.334, 1.000), (0.412, 0.700), (0.490, 0.500), (0.765, 0.000), the above data are from Table 1 of the literature CHAVEZ JR P S. An improved dark-object subtraction technique for atmospheric scattering correction of multispectral data [J]. Remote sensing of environment, 1988, 24 (3): 459-47, a third-order polynomial is used for fitting to obtain a curve relationship between index and reflectivity. Combined with the reference channel reflectivity map ρ1 obtained in sub-step S202, the index map γ is generated, and the index map γ has the following functional relationship with the reference channel reflectivity map ρ1:

[0055]

[0056] Among them, a0=6.537, a1=-27.465, a2=41.224, a3=-21.547, ρ l =0.108,ρ h =0.765.

[0057] In sub-step S204 of step S2, based on the transmittance map t1 of the reference channel obtained in sub-step S202 and the index map γ obtained in sub-step S203, the transmittance maps t1 of the remaining channels are synthesized using the derived formula of the atmospheric scattering model. iThe synthesis formula is derived from formula (10) of QIN M, XIE F, LI W, et al. Dehazing for multispectral remote sensing images based on a convolutional neural network with the residual architecture[J]. IEEE journal of selected topics in applied earth observations and remote sensing, 2018, 11(5): 1645-1655, as follows:

[0058]

[0059] Here, x represents the pixel position in the image, and (·) represents the value of the corresponding pixel position in the image.

[0060] In sub-step S301 of step S3, for a fog-free remote sensing multispectral image, the fog-free channel images J i , the brightest part of pixels in i∈{1,2,…,n} is used to estimate the atmospheric light value A of the corresponding channel i , the estimation formula is as follows:

[0061] Ω i (x) = top(J i ,p),i∈{1,2,…,n}#(8)

[0062]

[0063] Where x represents the pixel position in the image, (·) represents the value of the corresponding pixel position in the image, top(ρ,p) represents sorting all pixels in the image ρ in ascending order of value and taking the set of positions corresponding to the top p% of pixels from low to high, and K = |Ω i (x)|, represents the set Ω i The number of elements in (x). The recommended value of p is between 0.01 and 0.1.

[0064] In sub-step S302 of step S3, sub-step S301 is performed for each fog-free remote sensing multispectral image in the source data set to obtain atmospheric light estimation values ​​for each channel of each fog-free remote sensing multispectral image. The mean and variance of the atmospheric light estimation values ​​are calculated for each channel, and the channel with the smallest variance is used as the atmospheric light reference channel.

[0065] In the substep S303 of the step S3, the atmospheric light value estimated in the substep S301 is corrected, the mean and variance of the atmospheric light estimation value of each channel obtained in the substep S302 are used to weaken the deviation of the atmospheric light estimation value caused by the remote sensing image containing a large amount of specific landscape or topography (such as desert, forest), the ratio of the mean of the atmospheric light estimation value of each channel to the mean of the atmospheric light estimation value of the atmospheric light reference channel is taken as a correction factor, and the atmospheric light estimation value of each channel of the current image is corrected to obtain the corrected atmospheric light value A' of each channel i , i ∈ {1, 2, …, n}, and the whole correction process is described by the following formula:

[0066]

[0067]

[0068] A' i = α · A i + (1 - α) · R i A r , i ∈ {1, 2, …, n}#(12)

[0069] wherein, represents the atmospheric light value of channel i estimated from the k-th remote sensing multispectral image in the source data set, represents the mean of the atmospheric light estimation value of channel i, A i represents the atmospheric light estimation value of channel i of a non-specific remote sensing multispectral image, R i represents the ratio of the mean of the atmospheric light estimation value of channel i to the mean of the atmospheric light estimation value of the atmospheric light reference channel, and α represents the fusion weight, which is suggested to be between 0.1 and 0.2, A' i represents the corrected atmospheric light estimation value of channel i of a non-specific remote sensing multispectral image.

[0070] In the substep S401 of the step S4, in order to simulate the case that the reflected light of the ground is completely attenuated when thick fog blocks, the truncated coefficient w is used to truncate the transmittance map t i obtained in the substep S204 to obtain the truncated transmittance map t' i , and the truncation formula is as follows:

[0071] t' i (x) = max(0, 1 - w · (1 - t i (x))), i ∈ {1, 2, …, n}#(13)

[0072] Where x represents the pixel position in the image, (·) represents the value at the corresponding pixel position in the image, max(a,b) represents the larger value of a or b, and w represents the truncation coefficient, which is recommended to be between 1 and 1.5.

[0073] In sub-step S402 of step S4, a contrasting foggy remote sensing multispectral image I is synthesized based on the multispectral transmittance map t of step S2, the fog-free remote sensing multispectral image J and the multispectral atmospheric light A′ of step S3, and the truncated multispectral transmittance map t′ of sub-step S401, combined with an improved atmospheric scattering model. The original atmospheric scattering model is as shown in formula (5) in NARASIMHAN SG, NAYAR SK. Contrast restoration of weather degraded images[J]. IEEE transactions on pattern analysis and machine intelligence, 2003, 25(6): 713-724: E = I·ρ·e -βd +I·(1-e -βd ). The improved atmospheric scattering model is as follows:

[0074] I i (x) = J i (x)t′ i (x)+A′ i (1-t i (x)),i∈{1,2,…,n}#(14)

[0075] Where x represents the pixel position in the image, (·) represents the value of the corresponding pixel position in the image, and i represents the channel number.

[0076] In step S5, a strategy for synthesizing a remote sensing multispectral image defogging dataset based on a remote sensing multispectral source dataset is given. For the kth fog-free remote sensing multispectral image J in the source dataset, k ,k∈{1,2,…,N J}, the fog thickness in the composite image is controlled by limiting the value range of the fog thickness parameter. The fog thickness parameter is divided into L levels. In each level, the value range of the fog thickness parameter ω is [a0,a1), [a1,a2),…, [a L-1 ,a L ), where 0≤a i i+1 ≤1,i∈{0,1,…,L-1}, randomly sample M fog thickness parameters within the value range of each level, and obtain ω k,l,m ,k∈{1,2,…,N J ​},l∈{1,2,…,L},m∈{1,2,…,M}. Then randomly select L×M cirrus cloud channel images from the source dataset. k∈{1,2,…,N J}, l∈{1,2,…,L}, m∈{1,2,…,M}, and perform random rotation (0°, 90°, 180° or 270°) and left-right flipping. Based on the k-th fog-free remote sensing multispectral image J k , fog level parameter ω k,l,m and Cirrus Channel images Perform steps S2, S3, and S4 L×M times to synthesize L×M contrasting foggy remote sensing multispectral images I k,l,m ,k∈{1,2,…,N J},l∈{1,2,…,L},m∈{1,2,…,M}, all fog-free remote sensing multispectral images and control foggy remote sensing multispectral images are collected as the remote sensing multispectral image dehazing dataset.

[0077] The above embodiments are only intended to help understand the method and core concept of the present invention. It should be noted that, without departing from the principles of the present invention, a number of improvements and modifications may be made to the present invention by those skilled in the art, and such improvements and modifications also fall within the scope of protection of the claims of the present invention.

Claims

1. A method for generating a remote sensing multispectral image defogging dataset, characterized in that: The steps include: S1: Collect a preset number of cirrus channel images and fog-free remote sensing multispectral images to construct a remote sensing multispectral image source dataset; S2: Generate a multispectral transmittance map based on a cirrus channel image and fog thickness parameters; S3: Generate multispectral atmospheric light based on a fog-free remote sensing multispectral image; S4: Generating a remote sensing multispectral image with fog based on the multispectral transmittance map, the fog-free remote sensing multispectral image, and the multispectral atmospheric light, specifically comprising: S401: According to the truncation coefficient, the multi-spectral transmittance map obtained in step S2 is truncated, and the transmittance map of each channel after truncation is According to the following formula: in, represents the pixel position in the image, Indicates taking the value of the corresponding pixel position in the image, Indicates taking The larger value of Indicates the cutoff coefficient, which ranges from 1 to 1.

5. Represents the total number of channels of the multispectral image; S402: Based on the multispectral transmittance map of step S2, the fog-free remote sensing multispectral image and the multispectral atmospheric light of step S3, and the truncated multispectral transmittance map of step S401, a foggy remote sensing multispectral image is synthesized in combination with an improved atmospheric scattering model, wherein the improved atmospheric scattering model is as follows: in, Indicates channel The transmittance diagram, represents the pixel position in the image, Indicates taking the value of the corresponding pixel position in the image, Channels representing fog-free remote sensing multispectral images image, and Respectively represent the transmittance graph of each channel before and after truncation, Indicates the corrected channel The estimated atmospheric light value, Channels representing synthetic foggy remote sensing multispectral images image; S5: Divide the fog thickness into a preset number of levels by limiting the value range of the fog thickness parameter. For each fog-free remote sensing multispectral image in the source dataset, sample several different fog thickness parameters and the same number of different cirrus channel images at each fog level, perform steps S2, S3 and S4, and synthesize a preset number of control foggy remote sensing multispectral images. Collect all fog-free remote sensing multispectral images and control foggy remote sensing multispectral images as the remote sensing multispectral image defogging dataset.

2. The method for generating a remote sensing multispectral image defogging dataset according to claim 1, characterized in that: Step S2 specifically includes: S201: for a cirrus cloud channel image, eliminate the dark level in the image to obtain a fog mask; S202: Generate a reference channel reflectivity map and a reference channel transmittance map based on the fog mask and the fog thickness parameter; S203: Based on the scattered data pairs of index and reflectivity, using a third-order polynomial to fit the curve relationship diagram of index and reflectivity and the reference channel reflectivity diagram, a corresponding index diagram is generated; S204: Based on the transmittance map and index map of the reference channel, transmittance maps of the remaining channels are generated using a derived formula of the atmospheric scattering model to obtain a multispectral transmittance map.

3. The method for generating a remote sensing multispectral image defogging dataset according to claim 2, characterized in that: In step S201 , the cirrus cloud channel image is normalized and the dark level is eliminated by a linear stretching method.

4. The method for generating a remote sensing multispectral image defogging dataset according to claim 1, wherein: Step S3 specifically includes: S301: For a fog-free remote sensing multispectral image, the atmospheric light value of the corresponding channel is estimated using the brightest part of the pixels in each channel; S302: Perform the above steps for each fog-free remote sensing multispectral image in the source dataset to obtain atmospheric light estimation values ​​for each channel of each fog-free remote sensing multispectral image. Calculate the mean and variance of the atmospheric light estimation values ​​for each channel, and use the channel with the smallest variance as the atmospheric light reference channel. S303: Based on the atmospheric light estimation values ​​of each channel of the single fog-free remote sensing multispectral image obtained in step S301 and the average of the atmospheric light estimation values ​​of each channel of the fog-free remote sensing multispectral image obtained in step S302 and the atmospheric light reference channel, the atmospheric light estimation values ​​of each channel of the single image are corrected to obtain multispectral atmospheric light.

5. The method for generating a remote sensing multispectral image defogging dataset according to claim 4, characterized in that: In steps S302 and S303, the mean calculation formula and correction formula of the atmospheric light estimation value are as follows: in, Represents the total number of fog-free remote sensing multispectral images in the source dataset, Indicates the serial number of the atmospheric light reference channel, Indicates the source data set Channels estimated from remote sensing multispectral images The estimated atmospheric light value, Indicates channel The mean of the atmospheric light estimates, Indicates channel The ratio of the mean of the atmospheric light estimation value of the atmospheric light reference channel to the mean of the atmospheric light estimation value of the atmospheric light reference channel, Indicates the fusion weight, the value is between 0.1 and 0.2, Represents the channel estimated from a non-specific fog-free remote sensing multispectral image The estimated atmospheric light value, Indicates the corrected channel Estimated atmospheric light.

Images