High-dynamic remote sensing image acquisition method

By fusing bright and dark images using a brightness mapping function, the problem of complex camera response function calculation in existing technologies is solved, achieving efficient HDR image acquisition that is suitable for engineering applications.

CN119012019AActive Publication Date: 2024-11-22CHANGCHUN INST OF OPTICS FINE MECHANICS & PHYSICS CHINESE ACAD OF SCI
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202410957446.7
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-17
Publication Date
2024-11-22
Estimated Expiration
2044-07-17

AI Technical Summary

Technical Problem

When acquiring HDR images from existing multi-frame exposure images, the process of solving the camera response function is complex, which limits its engineering applications.

Method used

By acquiring bright and dark images, image fusion is performed using a brightness mapping function to synthesize a high dynamic range remote sensing image. A grayscale mapping method is used to extend the dynamic range of the image and avoid artifacts.

Benefits of technology

It simplifies the HDR image acquisition process, accurately estimates the brightness mapping function, requires no special equipment, is suitable for engineering applications, and avoids artifacts at image boundaries.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119012019A_ABST
    Figure CN119012019A_ABST
Patent Text Reader

Abstract

The invention relates to a high-dynamic remote sensing image acquisition method, and belongs to the technical field of remote sensing image processing and space remote sensing. The invention discloses a high-dynamic remote sensing image acquisition method, which comprises the following steps of: 1, acquiring information of a dark area in a scene by using a bright image Imglight (i, j); step 2, acquiring information of a bright area in the scene by using the dark image Imgdark (i, j); and step 3, fusing the two LDR images in the step 1 and the step 2, and synthesizing an HDR image ImgHDR (i, j). According to the high dynamic remote sensing image acquisition method, only the dark image and the bright image are needed, professional calibration equipment and parameters such as exposure time, scene illumination and brightness of a camera are not needed, the brightness mapping function between the two images can be accurately estimated, and therefore the HDR image is obtained; the artifact phenomenon at the junction of the images is effectively avoided, and the method is simple, effective and suitable for engineering application.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the field of remote sensing image processing and aerospace remote sensing technology, and in particular to a high dynamic range (HDR) remote sensing image acquisition method. Background Art

[0002] In the field of aerospace remote sensing, due to the large difference in reflectivity of ground targets, it is easy to cause the span of the scene radiance detected by the space camera to exceed the dynamic range of the camera, so that low-brightness targets are drowned by noise, or high-brightness targets are oversaturated, resulting in poor contrast and loss of texture details in remote sensing images, which is not conducive to the segmentation, detection and recognition of targets of interest. In addition, the brightness of targets is low under low solar angles or twilight conditions, and the high-speed push-broom mode and attitude and orbit control conditions limit the integration time and integration series of the space camera, resulting in a limited dynamic range of remote sensing images. If the gain is simply increased, the noise of the remote sensing image will increase. Therefore, obtaining remote sensing images with high signal-to-noise ratio and high dynamics has important theoretical value and application significance.

[0003] The acquisition of HDR images can be started from two aspects: hardware and software. The former requires changing the hardware system of the camera, which is expensive and rarely used. The latter common methods include HDR image acquisition methods based on single exposure and HDR image generation methods based on multiple exposures. Of course, with the development of deep learning technology, many HDR image generation algorithms based on deep learning have emerged. Since the deep learning method requires a huge image data set and the processing is time-consuming, it is not conducive to engineering applications and is not within the scope of discussion of this invention.

[0004] The high dynamic image acquisition method based on single exposure is to expand and stretch the dynamic range of a single frame of low dynamic range (LDR) image by image processing, so as to restore the highlight details and dark details missing in the LDR image. Single exposure HDR imaging recovers the HDR image from a single frame of LDR image. This method is simple and practical, but due to the small amount of image information that can be used, it cannot accurately restore the real HDR scene information. The HDR image acquisition method based on multi-exposure is to continuously obtain multiple frames of images with different exposures for the scene, and synthesize the information of different brightness areas of the image into a frame of high dynamic image through the image fusion algorithm. This method can obtain the real scene information under different light levels, and can generate HDR images with more accurate details and richer information. It is suitable for a variety of application scenarios and is widely used. However, the currently used multi-exposure HDR acquisition method requires the establishment of a functional relationship between exposure time, scene illumination and brightness, and even requires professional calibration equipment to solve the camera response curve to restore the illumination value of the actual scene. Summary of the invention

[0005] The present invention aims to solve the technical problem that when obtaining HDR images from multiple exposure frames in the prior art, the process of solving the camera response function is complicated, resulting in limited engineering applications, and provides a high dynamic remote sensing image acquisition method.

[0006] In order to solve the above technical problems, the technical solutions of the present invention are as follows:

[0007] A high dynamic remote sensing image acquisition method comprises the following steps:

[0008] Step 1: Use bright image Img light (i, j) obtain information about the dark area in the scene;

[0009] Step 2: Use the dark image Img dark (i, j) obtain the information of the bright area in the scene;

[0010] Step 3: Fuse the two LDR images in steps 1 and 2 to synthesize the HDR image Img HDR (i,j).

[0011] In the above technical solution, step 3 is: first, by calculating the brightness mapping function between the two LDR images in steps 1 and 2, a fused image Img is obtained. fus (i, j), the image Img is transformed using the grayscale mapping method fus (i, j) is mapped to the maximum grayscale value of the corresponding image, and finally the HDR image Img is obtained. HDR (i,j).

[0012] In the above technical solution, the fused image Img is obtained by the brightness mapping function. fus The steps for (i,j) are as follows:

[0013] Assume that the dark image Img dark (i,j) is mapped to the bright image Img light The brightness mapping function of (i,j) is f(.), then:

[0014] Img light (i,j)=f(Img dark (i,j)) (1)

[0015] Find the uniform area of ​​each pixel value x from the dark image, and calculate the pixel mean of the uniform area, denoted as x;

[0016] And find the corresponding uniform area on the bright image, calculate the pixel mean of the uniform area, and record it as f(x);

[0017] By traversing the entire image, we get the data point pairs (x, f(x)) corresponding to different pixel values. f(x) is linearly related to x. Assuming the brightness mapping function is a linear function, the relationship is as follows:

[0018] f(x)=k*x+b (2)

[0019] The values ​​of coefficients k and b can be obtained through linear fitting;

[0020] Given a threshold TH, when the pixel value is less than or equal to the threshold, let the pixel value Img of the fused image at point (i, j) be fus (i,j) equals Img light (i, j), otherwise according to the brightness mapping function relationship as shown in formula (1), Img dark (i,j) is mapped to get Img light (i,j) and then assign it to Img fus (i, j), the final fused image is shown in formula (3):

[0021]

[0022] In the above technical solution, the final HDR image Img HDR (i,j), its expression (4) is as follows:

[0023]

[0024] Where n is the number of bits of the imaging sensor.

[0025] In the above technical solution, the acquisition method uses a camera with three modes: global exposure mode, low light mode and weak light mode to acquire bright images and dark images.

[0026] In the above technical solution, the acquisition method, in the global exposure imaging mode, uses a long exposure bright image and a short exposure dark image to acquire an HDR image.

[0027] In the above technical solution, the acquisition method adopts the low light mode to acquire the bright image, adopts the weak light mode to acquire the dark image, and acquires the HDR image.

[0028] The beneficial effects of the present invention are:

[0029] The high dynamic remote sensing image acquisition method of the present invention only needs two images, a dark image and a bright image, and does not require professional calibration equipment and parameters such as camera exposure time, scene illumination and brightness. The brightness mapping function between the two images can be accurately estimated to obtain an HDR image. Due to the accuracy of the mapping function, the artifact phenomenon at the junction of the images is effectively avoided. The method is simple and effective and suitable for engineering applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0031] Figure 1 is a linear graph of f(x) and x.

[0032] Figure 2 HDR acquisition and corresponding histogram in global exposure mode, where (a) is the dark image, (b) is the bright image, and (c) is the HDR image.

[0033] Figure 3 HDR acquisition and corresponding histogram in low light and weak light imaging modes, where (a) is a dark image, (b) is a bright image, and (c) is an HDR image. DETAILED DESCRIPTION

[0034] The present invention is described in detail below with reference to the accompanying drawings.

[0035] The high dynamic remote sensing image acquisition method of the present invention comprises the following steps:

[0036] Step 1: Use bright image Img light (i, j) obtain information about the dark area in the scene;

[0037] Step 2: Use the dark image Img dark (i, j) obtain the information of the bright area in the scene;

[0038] Step 3: Finally, the two LDR images of steps 1 and 2 are fused to form the HDR image Img. HDR (i, j), the HDR image Img HDR (i,j) contains all the details of the dark and bright areas in the scene.

[0039] The specific steps of step 3 are as follows: first, the fusion image Img is obtained by calculating the brightness mapping function between the two LDR images in steps 1 and 2. fus (i, j), the image Img is transformed using the grayscale mapping method fus (i, j) is mapped to the maximum grayscale value of the corresponding image, and finally the HDR image Img is obtained. HDR (i, j). The following is a detailed introduction to the acquisition of brightness mapping function and grayscale mapping.

[0040] 1. Get the brightness mapping function

[0041] Assume that the dark image Img dark (i,j) is mapped to the bright image Img light The brightness mapping function of (i,j) is f(.), then:

[0042] Img light (i,j)=f(Img dark (i,j)) (1)

[0043] We find the uniform region (e.g. 3*3 in size) of each pixel value x in the dark image, calculate the pixel mean of the uniform region, denoted as x; and find the corresponding uniform region in the bright image, calculate the pixel mean of the uniform region, denoted as f(x). By traversing the entire image, we can get the data point set pairs (x, f(x)) corresponding to different pixel values. We plot these data point sets as Figure 1 As shown by the blue dots in the figure.

[0044] It can be seen that f(x) is linearly related to x. We assume that the brightness mapping function is a linear function, and the relationship is as follows:

[0045] f(x)=k*x+b (2)

[0046] The values ​​of coefficients k and b can be obtained by linear fitting, such as Figure 1 As shown by the straight line in .

[0047] We give a threshold TH. When the pixel value is less than or equal to the threshold, let the pixel value Img of the fused image at point (i, j) be fus (i,j) equals Img light (i, j), otherwise according to the brightness mapping function relationship as shown in formula (1), Img dark (i,j) is mapped to get Img light (i,j) and then assign it to Img fus (i,j). Finally, the fused image Img is obtained fus (i, j) is as shown in formula (3):

[0048]

[0049] 2. Grayscale mapping

[0050] The image Img obtained by the above fusion fus (i, j), the grayscale value of the bright area obtained by the mapping function is greater than the maximum grayscale value of the imaging sensor (such as the maximum grayscale value of the 10-bit image is 4095), so we need to use the grayscale mapping method to convert Img fus (i, j) is mapped to the maximum grayscale value of the corresponding image, and finally the HDR image Img is obtained. HDR (i,j), as shown in the following formula.

[0051]

[0052] Wherein, n is the number of bits of the imaging sensor, for example, n=10.

[0053] In addition, the present invention can not only use image pairs with different exposure times to obtain HDR images, but also is applicable to bright and dark image pairs obtained in different imaging modes. The camera used in the present invention has three modes: global exposure mode, low light mode, and weak light mode. In the global exposure imaging mode, we can use long exposure bright images and short exposure dark images to obtain HDR images and their corresponding histograms, as shown in the following figure. Figure 2 In addition, under the same exposure time, the low light mode, i.e., high gain rolling shutter mode, is used to obtain a bright image, and the weak light mode, i.e., low gain rolling shutter mode, is used to obtain a dark image. The HDR image and its histogram are shown in FIG. Figure 3 shown.

[0054] It can be seen that whether it is the HDR image obtained by the dark and bright image pair in the global exposure mode, or the HDR image obtained by the dark and bright image pair in the low-light and weak-light imaging mode, its dynamic range has been well expanded, and the details in both the dark and bright areas of the image have been well displayed.

[0055] The high dynamic remote sensing image acquisition method of the present invention only needs two images, a dark image and a bright image, and does not require professional calibration equipment and parameters such as camera exposure time, scene illumination and brightness. The brightness mapping function between the two images can be accurately estimated to obtain an HDR image. Due to the accuracy of the mapping function, the artifact phenomenon at the junction of the images is effectively avoided. The method is simple and effective and suitable for engineering applications.

[0056] Obviously, the above embodiments are merely examples for the purpose of clear explanation, and are not intended to limit the implementation methods. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the implementation methods here. The obvious changes or modifications derived therefrom are still within the scope of protection of the invention.

Claims

1. A high dynamic remote sensing image acquisition method, characterized in that: The following steps are involved: Step 1: Use bright image Img light (i, j) obtain information about the dark area in the scene; Step 2: Use the dark image Img dark (i, j) obtain the information of the bright area in the scene; Step 3: Fuse the two LDR images in steps 1 and 2 to synthesize the HDR image Img HDR (i,j).

2. The high dynamic remote sensing image acquisition method according to claim 1, characterized in that: Step 3 is: first, by calculating the brightness mapping function between the two LDR images in steps 1 and 2, the fused image Img is obtained. fus (i, j), the image Img is transformed using the grayscale mapping method fus (i, j) is mapped to the maximum grayscale value of the corresponding image, and finally the HDR image Img is obtained. HDR (i,j).

3. The high dynamic remote sensing image acquisition method according to claim 2, characterized in that: The fused image Img is obtained through the brightness mapping function fus The steps for (i,j) are as follows: Assume that the dark image Img dark (i,j) is mapped to the bright image Img light The brightness mapping function of (i,j) is f(.), then: Img light (i,j)=f(Img dark (i,j)) (1) Find the uniform area of ​​each pixel value x from the dark image, and calculate the pixel mean of the uniform area, denoted as x; And find the corresponding uniform area on the bright image, calculate the pixel mean of the uniform area, and record it as f(x); By traversing the entire image, we get the data point pairs (x, f(x)) corresponding to different pixel values. f(x) is linearly related to x. Assuming the brightness mapping function is a linear function, the relationship is as follows: f(x)=k*x+b (2) The values ​​of coefficients k and b can be obtained through linear fitting; Given a threshold TH, when the pixel value is less than or equal to the threshold, let the pixel value Img of the fused image at point (i, j) be fus (i,j) equals Img light (i, j), otherwise according to the brightness mapping function relationship as shown in formula (1), Img dark (i,j) is mapped to get Img light (i,j) and then assign it to Img fus (i, j), the final fused image is shown in formula (3):

4. The high dynamic remote sensing image acquisition method according to claim 2, characterized in that: The final HDR image Img HDR (i,j), its expression (4) is as follows: Where n is the number of bits of the imaging sensor.

5. The high dynamic remote sensing image acquisition method according to claim 1, characterized in that: The acquisition method acquires bright images and dark images using a camera having three modes: a global exposure mode, a low light mode, and a weak light mode.

6. The high dynamic remote sensing image acquisition method according to claim 5, characterized in that: The acquisition method uses a long-exposure bright image and a short-exposure dark image to acquire an HDR image in a global exposure imaging mode.

7. The high dynamic remote sensing image acquisition method according to claim 5, characterized in that: The acquisition method adopts a low light mode to acquire a bright image, adopts a weak light mode to acquire a dark image, and acquires an HDR image.

Citation Information

Patent Citations

  • Image synthesizing method and system

    CN101394487A

  • Video pipeline processing method and device

    CN111131719A

  • Joint auto-exposure tone-mapping system

    US20230114798A1