A 16-bit remote sensing image to 8-bit image mapping method for deep learning

Abstract

The application discloses a kind of 16-bit remote sensing image to 8-bit image mapping method for deep learning.The method can exclude the influence of abnormal pixel value on the actual effective value range of image, can ensure that the converted image is closer to the original image while improving image brightness, contrast and definition;Help to improve the visualization effect and human-computer interaction processing efficiency of remote sensing image, promote the cross-sensor migration application of deep learning model based on remote sensing image.The application can adaptively realize the efficient mapping of 16-bit remote sensing image to 8-bit image, and the mapping result is more consistent with the brightness, contrast and definition of the original image, and has good compatibility for existing deep learning model, providing important technical support for the rapid conversion of massive heterogeneous remote sensing data and its application in the field of deep learning, and having important reference significance for the cross-fusion development of deep learning and remote sensing field.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computer science, and particularly relates to a 16-bit remote sensing image to 8-bit image mapping method for deep learning. BACKGROUND

[0002] With the rapid development of artificial intelligence technology, deep learning algorithms have gradually been popularized in the field of remote sensing. However, remote sensing images are usually stored based on 16-bit depth, and there are many inconveniences in subsequent processing such as visual interpretation, rapid annotation and efficient calculation. Moreover, the actual effective value range of remote sensing images taken by different sensors will be distributed in different range intervals. This limits the portability of the model, that is, the model trained based on data of a specific sensor is difficult to be applied to other sensor images. Natural images (RGB three-channel images) commonly used in deep learning are generally stored based on 8-bit depth, and the value range of pixels in each channel is 0-255. Compared with 16-bit images with a value range of 0-65535, 8-bit images can effectively save storage space, facilitate visualization, be easy for human-computer interaction processing, and be suitable for a wide range of deep learning models. In the context of the rapid integration of remote sensing and deep learning, mapping 16-bit remote sensing images to 8-bit storage has become a mainstream way.

[0003] At present, many open-source datasets based on remote sensing images use 8-bit images for storage, such as the remote sensing image fine-grained target recognition dataset FAIR1M and the DOTA dataset. Common image mapping methods for existing remote sensing image datasets include percentage linear stretching, segmented stretching, logarithmic transformation, exponential transformation, histogram-based color mapping, etc. However, the bit depth conversion from 16-bit to 8-bit is a mapping operation from high precision to low precision, which inevitably causes information loss. Existing methods have shown many problems in the mapping of 16-bit to 8-bit images, such as overexposure in bright areas, abnormal pixel values in dark areas, and reduced contrast, etc. Therefore, how to ensure that the brightness, contrast and clarity of the converted image are within a reasonable range is still a problem to be improved. SUMMARY

[0004] In view of the deficiencies of the prior art, the present application provides a 16-bit remote sensing image to 8-bit image mapping method for deep learning. The method adaptively obtains the maximum and minimum values of the image based on the mean and standard deviation of the original image to exclude outliers, thereby redefining the pixel value range of the image, so as to realize the mapping of 16-bit images to 8-bit images under the guidance of the value range. At the same time, in view of the common phenomenon of image contrast loss, the mapped image is contrast corrected based on the combination of histogram specification and image dehazing method, and finally an 8-bit image with a visual effect closer to the original image is obtained.

[0005] In order to achieve the above object, the technical scheme provided by the present application is a 16-bit remote sensing image to 8-bit image mapping method for deep learning, comprising the following steps:

[0006] Step 1, pre-processing the remote sensing image and exporting it as a 16-bit image;

[0007] Step 2, calculating the mean and standard deviation of the 16-bit image;

[0008] Step 3, calculating the maximum and minimum values adapted to the 16-bit image;

[0009] Step 4, mapping the 16-bit image to an 8-bit image according to the maximum and minimum values obtained in step 3;

[0010] Step 5, performing histogram specification processing on the 8-bit image obtained in step 4;

[0011] Step 6, performing post-processing based on dark channel dehazing on the image processed in step 5;

[0012] Step 7, writing geographic information and applying it to a deep learning task.

[0013] Moreover, in step 1, the collected remote sensing image is pre-processed to remove clouds, the red, green and blue bands of the pre-processed image are combined to form a true color image, and the true color image is exported as a 16-bit image in GeoTIFF format and downloaded to a local computer.

[0014] Moreover, in step 2, the mean and standard deviation are calculated as follows:

[0015]

[0016]

[0017] wherein, represents the mean of the reflectivity of all pixels of the image, σ represents the standard deviation of the reflectivity of all pixels of the image, N represents the number of pixels of the image, x i represents the reflectivity value of the i-th pixel of the image, i = [1, 2, 3, …, N].

[0018] Moreover, in step 3, the maximum and minimum values are calculated as follows:

[0019]

[0020]

[0021] wherein, Max adapt represents the maximum value adapted to the 16-bit image exported in step 1; Min adaptThis represents the minimum value that fits the 16-bit image derived in step 1; σ represents the mean reflectance of all pixels in the image; N represents the number of pixels in the image; and rate is a stretching parameter, which is set according to different sensor images.

[0022] Furthermore, the image mapping method in step 4 is as follows:

[0023]

[0024] In the formula, Image 8bit This represents the values ​​of all pixels in the 8-bit image after image mapping. 16bit Max represents all pixel values ​​of the 16-bit image before image mapping is performed. adapt Min represents the maximum value adapted to the 16-bit image derived in step 1. adapt This represents the minimum value that fits the 16-bit image derived in step 1.

[0025] Furthermore, in step 5, the maximum value max of the 8-bit image is first calculated. raw and minimum value min raw Then set the specified interval [minn olm ,max nolm Substitute the maximum and minimum values ​​of the 8-bit image, the maximum and minimum values ​​of the specified interval, and all pixel values ​​of the 8-bit image into the histogram specification formula to obtain the histogram specification image;

[0026] The histogram standardization calculation method is as follows:

[0027]

[0028] In the formula, Image nolm Image represents all pixel values ​​of a standardized image. 8bit Represents all pixel values ​​of an 8-bit image, max nolm min represents the maximum value of the defined interval. nolm Max represents the minimum value of the defined interval. 8bit min represents the maximum value of an 8-bit image. 8bit This represents the minimum value of an 8-bit image.

[0029] Furthermore, in step 6, the defined image obtained in step 5 is first traversed, and the pixel value with the lowest brightness in the three channels is selected at each pixel position to obtain the dark channel image. darkThen, in the dark channel image, the pixels with the top 0.1% brightness are selected, and the corresponding highest brightness value in the normalized image is found according to the positions of the pixels, as the atmospheric light value A. The dark channel pixel value and the atmospheric light value A are substituted into the defogging formula to obtain the 8-bit image after defogging.

[0030] The defogging formula is as follows:

[0031]

[0032] In the formula, Image defog represents the image after defogging, Image nolm represents the normalized image, Image dark represents the dark channel image, and A represents the atmospheric light value of the normalized image.

[0033] Compared with the prior art, the present application has the following advantages:

[0034] 1) By obtaining the maximum value and the minimum value adaptive to the given 16-bit image, the influence of the abnormal pixel values that are too large or too small in the image on the actual effective value range of the image is excluded; 2) By introducing the post-processing steps of image normalization and dark channel defogging, the brightness, contrast and definition of the image are improved, and the converted image is closer to the visualization effect of the original image; 3) The efficient mapping from the 16-bit remote sensing image to the 8-bit image can be adaptively realized, the mapping result is more consistent with the brightness, contrast and definition of the original image, and the existing deep learning model has good compatibility. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 The flowchart of the embodiment of the present application.

[0036] Figure 2 The process diagram of obtaining the maximum value and the minimum value adaptive to the given 16-bit image of the present application.

[0037] Figure 3 The visualization effects of the 8-bit images and the original image obtained by different methods in the embodiment 1 of the present application, wherein Figure 3 (a) is the visualization effect diagram of the 8-bit image obtained by the traditional method, Figure 3 (b) is the visualization effect diagram of the 8-bit image obtained by the method proposed in the present application, Figure 3 (c) is the visualization effect diagram of the given original 16-bit remote sensing image.

[0038] Figure 4 The visualization effects of the 8-bit images and the original image obtained by different methods in the embodiment 2 of the present application, wherein Figure 4 (a) is the visualization effect diagram of the 8-bit image obtained by the traditional method,Figure 4 (b) is a visualization of the 8-bit image obtained by the method of the present invention. Figure 4 (c) is a visualization of the given original 16-bit remote sensing image. Detailed Implementation

[0039] This invention provides a method for mapping 16-bit remote sensing images to 8-bit images for deep learning. The technical solution of this invention will be further described below with reference to the accompanying drawings.

[0040] like Figure 1 As shown, the process of this embodiment of the invention includes the following steps:

[0041] In Example 1, the present invention achieves 16-bit image to 8-bit image mapping on remote sensing images containing large-scale farmland and rivers, mainly including the following steps:

[0042] Step 1: Image acquisition and preprocessing.

[0043] First, a Sentinel-2 16-bit multispectral remote sensing image containing large-scale farmland and rivers was collected from Google Earth Engine (GEE). This image came from Sentinel-2 Level-1C products and had undergone geometric correction and other preprocessing. Then, cloud removal preprocessing was performed on the collected remote sensing image. The red, green and blue bands of the preprocessed image were combined into a true-color image and exported as a 16-bit GeoTIFF image, which was then downloaded to the local computer.

[0044] Step 2: Calculate the mean and standard deviation of the 16-bit image.

[0045] The reflectance value of each pixel in the 16-bit image exported in step 1 is read by computer programming, and the mean and standard deviation of the reflectance of all pixels in the current image are calculated as follows:

[0046]

[0047]

[0048] In the formula, Let σ represent the mean reflectance of all pixels in the image, σ represent the standard deviation of the reflectance of all pixels in the image, and N represent the number of pixels in the image. i This represents the reflectance value of the i-th pixel in the image, where i = [1, 2, 3, ..., N].

[0049] Step 3: Find the maximum and minimum values ​​suitable for 16-bit images.

[0050] Find the maximum value Max that fits the 16-bit image derived in step 1. adaptand the minimum value Min adapt The maximum and minimum values re-define the value range of the 16-bit image derived in step 1, eliminating the influence of abnormal pixel values that are too large or too small on the actual effective value range of the image.

[0051] The maximum and minimum values are calculated as follows:

[0052]

[0053]

[0054] where Max adapt represents the maximum value of the 16-bit image derived in step 1; Min adapt represents the minimum value of the 16-bit image derived in step 1. represents the mean value of the reflectivity of all pixels of the image; σ represents the standard deviation of the reflectivity of all pixels of the image; N represents the number of pixels of the image; and rate is an adjustable stretching parameter, which is taken as 4.5 in this embodiment, and can be adjusted according to different sensors in actual application.

[0055] Step 4: mapping the 16-bit image into an 8-bit image according to the maximum and minimum values obtained in step 3.

[0056] Substitute the maximum and minimum values obtained in step 3 into the image mapping expression to map the numerical value of each pixel of the 16-bit image into 8 bits.

[0057] The image mapping method is shown as follows:

[0058]

[0059] where Image 8bit represents the value of all pixels of the 8-bit image after image mapping, Image 16bit represents the value of all pixels of the 16-bit image before image mapping, Max adapt represents the maximum value of the 16-bit image derived in step 1, Min adapt represents the minimum value of the 16-bit image derived in step 1.

[0060] Step 5: post-processing based on histogram specification.

[0061] Perform histogram specification processing on the 8-bit image obtained in step 4 to improve the contrast and brightness of the image. First, calculate the maximum value max raw and the minimum value min raw of the 8-bit image, and then give a specification interval [min nolm , maxnolm ], the default setting is [0, 255], the maximum and minimum values of the 8-bit image, the maximum and minimum values of the specified interval, and all pixel values of the 8-bit image are substituted into the histogram specification formula to obtain a histogram specification image.

[0062] The histogram specification calculation is as follows:

[0063]

[0064] In the formula, Image nolm represents all pixel values of the specification image, Image 8bit represents all pixel values of the 8-bit image, max nolm represents the maximum value of the specification interval, min nolm represents the minimum value of the specification interval, max 8bit represents the maximum value of the 8-bit image, min 8bit represents the minimum value of the 8-bit image.

[0065] Step 6, post-processing based on dark channel dehazing.

[0066] After the histogram specification processing, the classic dark channel dehazing algorithm is used to further eliminate the blurring phenomenon and abnormal value noise of the image, and to improve the clarity of the image. First, traverse the specification image obtained in step 5, and select the lowest pixel value of the three-channel brightness at each pixel position to obtain a dark channel image Image dark Then, in the dark channel image, take the top 0.1% of pixels according to brightness, and find the corresponding highest brightness value in the specification image according to the positions of these pixels, as the atmospheric illumination value A. Substitute the dark channel pixel value and the atmospheric illumination value A into the dehazing formula to obtain the dehazed 8-bit image.

[0067] The dehazing formula is as follows:

[0068]

[0069] In the formula, Image defog represents the image after dehazing, Image nolm represents the specification image, Image dark represents the dark channel image, and A represents the atmospheric illumination value of the specification image.

[0070] Step 7, write geographic information and apply to deep learning tasks.

[0071] The projection, coordinates and other geographic information data of the original remote sensing image are read using the GDAL library, and these data are written into the 8-bit image after fog removal obtained in step 6. The image with geographic information is saved as a GeoTIFF format file, and finally an 8-bit image more similar to the original remote sensing image is obtained. The 8-bit image with geographic information is used as input data of the deep learning model to realize the training and inference of the deep learning model on the remote sensing image.

[0072] In embodiment 2, the present application realizes the mapping of 16-bit images to 8-bit images on remote sensing images containing large-scale mountains and smoke and other ground objects, and the specific implementation steps are as follows:

[0073] Step 1, image collection and preprocessing.

[0074] First, a scene of 16-bit multispectral remote sensing image of Sentinel-2 of large-scale mountains and smoke and other ground objects is collected from Google Earth Engine (GEE). The image comes from Sentinel-2 Level-1C product and has been preprocessed, such as geometric precision correction. Then, the collected remote sensing image is preprocessed to remove clouds. The red, green and blue bands of the preprocessed image are synthesized into a true color image, which is exported as a GeoTIFF format 16-bit image and downloaded to a local computer.

[0075] The processing procedures of steps 2 to 7 are consistent with those of embodiment 1, which will not be described here.

[0076] Figure 3 The visualization effects of the 8-bit images obtained by different methods in embodiment 1 of the present application and the original image are compared. From the comparison chart, it can be seen that the 8-bit image obtained by the traditional method shows a serious loss of brightness, the overall image is obviously dark, and the contrast is reduced, resulting in the difference between the dark and bright parts being weakened. Figure 3 It can be seen that the 8-bit image obtained by the method of the present application has a closer image brightness to the original remote sensing image, which is significantly improved compared with the traditional method, and does not cause overexposure in the bright area.

[0077] Figure 4 The visualization effects of the 8-bit images obtained by different methods in embodiment 2 of the present application and the original image are compared. From the comparison chart, it can be seen that the 8-bit image obtained by the traditional method has a significant color difference compared with the original image, and the contrast is obviously reduced, resulting in unclear image texture and outline. The 8-bit image obtained by the method of the present application has a contrast and clarity state extremely close to the original remote sensing image, and even further optimization is realized in the bright area.

[0078] The specific embodiments described herein are merely illustrative of the spirit of the application. Various modifications or changes in the specific embodiments described herein can occur to those skilled in the art to which the application pertains without departing from the spirit of the application, and it is understood that such modifications or changes are to be considered as within the scope of the application as defined by the appended claims.

Claims

1. A method for mapping 16-bit remote sensing images to 8-bit images for deep learning, the method comprising: Comprising the following steps: Step 1, pre-process the remote sensing image and export it as a 16-bit image; Step 2, calculate the mean and standard deviation of the 16-bit image; Step 3, calculate the maximum and minimum values suitable for the 16-bit image; The maximum and minimum values are calculated as follows: （3） （4） wherein represents the maximum value of the 16-bit image derived in step 1 ; represents the minimum value of the 16-bit image derived in step 1 ; represents the mean value of the reflectance of all pixels of the image; represents the standard deviation of the reflectance of all pixels of the image; represents the number of pixels of the image; is a stretching parameter, which is set according to the different sensor images; Step 4, map the 16-bit image to an 8-bit image based on the maximum and minimum values obtained in step 3; The image mapping method is as follows: （5） wherein represents all pixel values of the 8-bit image after performing the image mapping, represents all pixel values of the 16-bit image before performing the image mapping, represents the maximum value adapted to the 16-bit image derived in step 1, represents the minimum value adapted to the 16-bit image derived in step 1, Step 5, perform histogram specification processing on the 8-bit image obtained in step 4; Step 6, perform post-processing based on dark channel dehazing on the image processed in step 5; Step 7, write geographic information and apply it to deep learning tasks.

2. The 16-bit remote sensing image to 8-bit image mapping method for deep learning according to claim 1, wherein: In step 1, perform cloud removal preprocessing on the collected remote sensing image, combine the red, green, and blue bands of the preprocessed image into a true color image, and export it as a GeoTIFF format 16-bit image, and download it to the local computer.

3. The 16-bit remote sensing image to 8-bit image mapping method for deep learning according to claim 1, wherein: The calculation method of mean and standard deviation in step 2 is as follows: （1） （2） wherein, represents the mean of the reflectance values of all pixels of the image, represents the standard deviation of the reflectance values of all pixels of the image, represents the number of pixels of the image, represents the reflectance value of the i-th pixel of the image, represents the reflectance value of the i-th pixel of the image, =[1, 2, 3, …, ].

4. The 16-bit remote sensing image to 8-bit image mapping method for deep learning according to claim 1, wherein: The maximum value of the 8-bit image is first calculated in Step 5 and the minimum value , and then the specified interval [ , ] is set, the maximum and minimum values of the 8-bit image, the maximum and minimum values of the specified interval, and all pixel values of the 8-bit image are substituted into the histogram specification formula to obtain a histogram specification image; The histogram specification calculation method is as follows: （6） wherein denotes all pixel values of the specified image, denotes all pixel values of the 8-bit image, denotes the maximum value of the specified interval, denotes the minimum value of the specified interval, denotes the maximum value of the 8-bit image, denotes the minimum value of the 8-bit image.

5. The 16-bit remote sensing image to 8-bit image mapping method for deep learning according to claim 1, wherein: In step 6, the specified image obtained in step 5 is first traversed, and the lowest pixel value of the three channels is selected at each pixel position to obtain a dark channel image Then, the top 0.1% of pixels in the dark channel image are selected according to the brightness, the corresponding highest brightness value in the specified image is found according to the positions of the pixels, taken as the atmospheric light value A, and the dark channel pixel value and the atmospheric light value A are substituted into the defogging formula to obtain an 8-bit image after defogging. The dehazing formula is as follows: （7） In the formula, denotes the image after fog removal, denotes the specified image, denotes the dark channel image, and A denotes the atmospheric light value of the specified image.

6. The 16-bit remote sensing image to 8-bit image mapping method for deep learning according to claim 1, wherein: In step 7, first read the projection, coordinate, and geographic information data of the original remote sensing image, and write these data into the dehazed 8-bit image. The 8-bit image with geographic information is used as input data for the deep learning model to realize the training and inference of the deep learning model on the remote sensing image.

Images