A method for acquiring high dynamic remote sensing images

Abstract

The present invention relates to a method for acquiring high-dynamic remote sensing images, which belongs to the field of remote sensing image processing and aerospace remote sensing technology. The method for acquiring high-dynamic remote sensing images of the present invention comprises the following steps: Step 1: using a bright image Img light (i, j) obtains the information of the dark area in the scene; Step 2, use the dark image Img dark (i, j) obtains the information of the bright area in the scene; Step 3, fuses the two LDR images in steps 1 and 2 to synthesize the HDR image Img HDR The high-dynamic remote sensing image acquisition method of the present invention only requires two images, one dark and one bright, without the need for specialized calibration equipment or parameters such as camera exposure time, scene illumination, and brightness. It can accurately estimate the brightness mapping function between the two images, thereby obtaining an HDR image. Due to the accuracy of the mapping function, artifacts at the interface of the images are effectively avoided. This method is simple, effective, and suitable for engineering applications.

Description

Technical Field

[0001] The present invention relates to the field of remote sensing image processing and aerospace remote sensing technology, and in particular to a method for acquiring high dynamic range (HDR) remote sensing images. Background Art

[0002] In the field of space remote sensing, the wide variation in reflectivity of ground targets can easily cause the radiance span of the scene detected by a space camera to exceed the camera's dynamic range. This can cause low-brightness targets to be drowned out by noise or cause high-brightness targets to become oversaturated, resulting in poor contrast and loss of texture detail in remote sensing images. This hinders tasks such as target segmentation, detection, and recognition. Furthermore, target brightness is low at low solar angles or during twilight. High-speed push-broom modes and attitude and orbit control restrict the integration time and number of integration stages of space cameras, limiting the dynamic range of remote sensing images. Simply increasing the gain will increase the noise in the remote sensing images. Therefore, acquiring remote sensing images with high signal-to-noise ratio and high dynamic range is of great theoretical and practical significance.

[0003] HDR image acquisition can be achieved through both hardware and software approaches. The former requires modifications to the camera's hardware system, which is expensive and rarely adopted. Common approaches to the latter include single-exposure HDR image acquisition methods and multi-exposure HDR image generation methods. Of course, with the development of deep learning technology, many deep learning-based HDR image generation algorithms have emerged. However, deep learning methods require large image datasets and are time-consuming to process, making them unsuitable for engineering applications. Therefore, they are beyond the scope of this invention.

[0004] Single-exposure high-dynamic range (HDR) image acquisition methods use image processing to expand and stretch the dynamic range of a single-frame low dynamic range (LDR) image, thereby restoring the highlight and shadow details missing from the LDR image. Single-exposure HDR imaging, which recovers an HDR image from a single LDR image, is simple and practical, but due to the limited image information available, it cannot accurately restore the true HDR scene information. Multi-exposure HDR image acquisition methods continuously acquire multiple frames of images at different exposures and use image fusion algorithms to combine information from different brightness regions of the images into a single HDR image. This method can capture true scene information under different light levels and can generate HDR images with more accurate details and richer information. It is suitable for a variety of application scenarios and has been widely used. However, currently used multi-exposure HDR acquisition methods require establishing a functional relationship between exposure time, scene illumination, and brightness, and even require specialized calibration equipment to solve the camera response curve to recover the actual scene illumination value. Summary of the Invention

[0005] The present invention aims to solve the technical problem in the prior art that when obtaining HDR images from multiple frame exposure images, the process of solving the camera response function is complicated, which leads to limited engineering applications, and provides a high dynamic remote sensing image acquisition method.

[0006] In order to solve the above technical problems, the technical solutions of the present invention are as follows:

[0007] A method for acquiring high-dynamic remote sensing images comprises the following steps:

[0008] Step 1: Use bright image Img light (i, j) obtains information about the dark area in the scene;

[0009] Step 2: Use dark image Img dark (i, j) obtains the information of the bright area in the scene;

[0010] Step 3: Fuse the two LDR images from steps 1 and 2 to create a synthetic HDR image Img HDR (i,j).

[0011] In the above technical solution, step 3 is: first, by calculating the brightness mapping function between the two LDR images in steps 1 and 2, the fused image Img is obtained. fus (i, j), the image Img is transformed into fus (i, j) is mapped to the maximum grayscale value of the corresponding image, and finally the HDR image Img is obtained. HDR (i,j).

[0012] In the above technical solution, the fused image Img is obtained by the brightness mapping function fus The steps for (i,j) are as follows:

[0013] Assume that the dark image Img dark (i,j) is mapped to the bright image Img light The brightness mapping function of (i, j) is f(.), then:

[0014] Img light (i,j)=f(Img dark (i,j)) (1)

[0015] Find the uniform area of ​​each pixel value x from the dark image, and calculate the pixel mean of the uniform area, which is recorded as x;

[0016] And find the corresponding uniform area on the bright image, calculate the pixel mean of the uniform area, and record it as f(x);

[0017] By traversing the entire image, we can obtain pairs of data points (x, f(x)) corresponding to different pixel values. f(x) is linearly related to x. Assuming the brightness mapping function is a linear function, the relationship is as follows:

[0018] f(x)=k*x+b (2)

[0019] The values ​​of coefficients k and b can be obtained through linear fitting;

[0020] Given a threshold TH, when the pixel value is less than or equal to the threshold, let the pixel value Img of the fused image at point (i, j) be fus (i,j) equals Img light (i, j), otherwise according to the brightness mapping function relationship as shown in formula (1), Img dark (i,j) is mapped to obtain Img light (i,j) and then assigned to Img fus (i, j), the final fused image is shown in formula (3):

[0021]

[0022] In the above technical solution, the final HDR image Img HDR (i, j), its expression (4) is as follows:

[0023]

[0024] Where n is the number of bits of the imaging sensor.

[0025] In the above technical solution, the acquisition method uses a camera with three modes: global exposure mode, low-light mode, and weak-light mode to acquire bright images and dark images.

[0026] In the above technical solution, the acquisition method uses a long-exposure bright image and a short-exposure dark image in a global exposure imaging mode to acquire an HDR image.

[0027] In the above technical solution, the acquisition method adopts the low-light mode to acquire bright images, adopts the weak-light mode to acquire dark images, and acquires HDR images.

[0028] The beneficial effects of the present invention are:

[0029] The high dynamic remote sensing image acquisition method of the present invention only requires two images, dark and bright, without professional calibration equipment and parameters such as camera exposure time, scene illumination and brightness, and can accurately estimate the brightness mapping function between the two images to obtain an HDR image. Due to the accuracy of the mapping function, artifacts at the junction of the images are effectively avoided. The method is simple, effective and suitable for engineering applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0031] Figure 1 is a linear graph of f(x) and x.

[0032] Figure 2 HDR acquisition and corresponding histogram in global exposure mode, where (a) is the dark image, (b) is the bright image, and (c) is the HDR image.

[0033] Figure 3 HDR acquisition and corresponding histograms in low-light and low-light imaging modes, where (a) is the dark image, (b) is the bright image, and (c) is the HDR image. DETAILED DESCRIPTION

[0034] The present invention will be described in detail below with reference to the accompanying drawings.

[0035] The high dynamic remote sensing image acquisition method of the present invention comprises the following steps:

[0036] Step 1: Use bright image Img light (i, j) obtains information about the dark area in the scene;

[0037] Step 2: Use dark image Img dark (i, j) obtains the information of the bright area in the scene;

[0038] Step 3: Finally, the two LDR images of steps 1 and 2 are fused to form the HDR image Img. HDR (i, j), the HDR image Img HDR (i, j) contains all the details of the dark and bright areas in the scene.

[0039] The specific steps of step 3 are as follows: first, the fusion image Img is obtained by calculating the brightness mapping function between the two LDR images in steps 1 and 2. fus (i, j), the image Img is transformed into fus (i, j) is mapped to the maximum grayscale value of the corresponding image, and finally the HDR image Img is obtained. HDR (i, j). The following is a detailed introduction to the acquisition of brightness mapping function and grayscale mapping.

[0040] 1. Get the brightness mapping function

[0041] Assume that the dark image Img dark (i,j) is mapped to the bright image Img light The brightness mapping function of (i, j) is f(.), then:

[0042] Img light (i,j)=f(Img dark (i,j)) (1)

[0043] We find a uniform region (e.g., 3*3 in size) of each pixel value x in the dark image, calculate the pixel mean of the uniform region, and record it as x; and find the corresponding uniform region in the bright image, calculate the pixel mean of the uniform region, and record it as f(x). By traversing the entire image, we can obtain data point sets (x, f(x)) corresponding to different pixel values. We plot these data point sets as follows Figure 1 As shown by the blue dots in the .

[0044] It can be seen that f(x) is linearly related to x. We assume that the brightness mapping function is a linear function, and the relationship is as follows:

[0045] f(x)=k*x+b (2)

[0046] The values ​​of coefficients k and b can be obtained by linear fitting, such as Figure 1 As shown by the straight line in .

[0047] We give a threshold TH. When the pixel value is less than or equal to the threshold, let the pixel value Img of the fused image at point (i, j) be fus (i,j) equals Img light (i, j), otherwise according to the brightness mapping function relationship as shown in formula (1), Img dark (i,j) is mapped to obtain Img light (i,j) and then assigned to Img fus (i,j). Finally, the fused image Img is obtained fus (i, j) is as shown in formula (3):

[0048]

[0049] 2. Grayscale mapping

[0050] The image Img obtained by the above fusion fus (i, j), the grayscale value of the bright area obtained by the mapping function is greater than the maximum grayscale value of the imaging sensor (such as the maximum grayscale value of 10-bit image is 4095), so we need to use the grayscale mapping method to convert Img fus (i, j) is mapped to the maximum grayscale value of the corresponding image, and finally the HDR image Img is obtained. HDR (i,j), as shown below.

[0051]

[0052] Here, n is the number of bits of the imaging sensor, for example, n=10.

[0053] In addition, the present invention can not only use image pairs with different exposure times to obtain HDR images, but also is applicable to bright and dark image pairs obtained in different imaging modes. The camera used in the present invention has three modes: global exposure mode, low light mode, and low light mode. In the global exposure imaging mode, we can use long exposure bright image and short exposure dark image to obtain HDR image and its corresponding histogram as shown in the following figure: Figure 2 In addition, under the same exposure time, the low light mode, i.e., high gain rolling shutter mode, is used to obtain a bright image, and the low light mode, i.e., low gain rolling shutter mode, is used to obtain a dark image. The HDR image and its histogram are shown as follows: Figure 3 shown.

[0054] It can be seen that whether it is the HDR image obtained by the dark and bright image pair in the global exposure mode or the HDR image obtained by the dark and bright image pair in the low-light and weak-light imaging modes, its dynamic range has been well expanded, and the details in both the dark and bright areas of the image have been well displayed.

[0055] The high dynamic remote sensing image acquisition method of the present invention only requires two images, dark and bright, without professional calibration equipment and parameters such as camera exposure time, scene illumination and brightness, and can accurately estimate the brightness mapping function between the two images to obtain an HDR image. Due to the accuracy of the mapping function, artifacts at the junction of the images are effectively avoided. The method is simple, effective and suitable for engineering applications.

[0056] Obviously, the above embodiments are merely examples for clarity of explanation and are not intended to limit the implementation methods. Those skilled in the art will readily appreciate that other variations or modifications based on the above descriptions are possible. It is not necessary and impossible to enumerate all implementation methods here. Obvious variations or modifications arising therefrom remain within the scope of protection of the present invention.

Claims

1. A method for acquiring high dynamic remote sensing images, characterized in that: The following steps are involved: Step 1: Use bright image Img light (i, j) obtains information about the dark area in the scene; Step 2: Use dark image Img dark (i, j) obtains the information of the bright area in the scene; Step 3: Fuse the two LDR images from steps 1 and 2 to create a synthetic HDR image Img HDR (i,j); Step 3 is as follows: First, the fusion image Img is obtained by calculating the brightness mapping function between the two LDR images in steps 1 and 2. fus (i, j), the image Img is transformed into fus (i, j) is mapped to the maximum grayscale value of the corresponding image, and finally the HDR image Img is obtained. HDR (i,j); The fused image Img is obtained through the brightness mapping function fus The steps for (i,j) are as follows: Assume that the dark image Img dark (i,j) is mapped to the bright image Img light The brightness mapping function of (i, j) is f(.), then: Img light (i,j)=f(Img dark (i,j)) (1) Find the uniform area of ​​each pixel value x from the dark image, and calculate the pixel mean of the uniform area, which is recorded as x; And find the corresponding uniform area on the bright image, calculate the pixel mean of the uniform area, and record it as f(x); By traversing the entire image, we can obtain pairs of data points (x, f(x)) corresponding to different pixel values. f(x) is linearly related to x. Assuming the brightness mapping function is a linear function, the relationship is as follows: f(x)=k*x+b (2) The values ​​of coefficients k and b can be obtained through linear fitting; Given a threshold TH, when the pixel value is less than or equal to the threshold, let the pixel value Img of the fused image at point (i, j) be fus (i,j) equals Img light (i, j), otherwise according to the brightness mapping function relationship as shown in formula (1), Img dark (i,j) is mapped to obtain Img light (i,j) and then assigned to Img fus (i, j), the final fused image is shown in formula (3):

2. The high dynamic remote sensing image acquisition method according to claim 1, characterized in that: The final HDR image Img HDR (i, j), its expression (4) is as follows: Where n is the number of bits of the imaging sensor.

3. The high dynamic remote sensing image acquisition method according to claim 1, characterized in that: The acquisition method acquires bright images and dark images using a camera having three modes: global exposure mode, low-light mode, and weak-light mode.

4. The high dynamic remote sensing image acquisition method according to claim 3, characterized in that: The acquisition method uses a long-exposure bright image and a short-exposure dark image in a global exposure mode to acquire an HDR image.

5. The high dynamic remote sensing image acquisition method according to claim 3, characterized in that: The acquisition method adopts a low-light mode to acquire a bright image, adopts a weak-light mode to acquire a dark image, and acquires an HDR image.

Images