A method for selecting exposure time for high dynamic range imaging

Abstract

The present invention discloses a method for selecting exposure time for high dynamic range imaging, which relates to the field of high dynamic range (HDR) imaging. First, the camera parameters required for calculating the brightness signal-to-noise ratio are calibrated; then, a scene brightness distribution histogram is obtained, and after selecting the brightness range of interest, the signal-to-noise ratio of the target after encoding is calculated according to the photoelectric conversion function; finally, the signal-to-noise ratio of the target after encoding and the calculation formula of the brightness signal-to-noise ratio are used to calculate the shortest exposure time sequence that meets the signal-to-noise ratio requirements, and this is output as the final selected exposure time sequence. The present invention combines HDR imaging exposure time selection with HDR encoding quantization interval, providing a practical optimization target for reducing noise and improving the efficiency of HDR image acquisition and reconstruction quality.

Description

Technical Field

[0001] The present invention belongs to the field of imaging, and in particular relates to a method for selecting exposure time for high dynamic range imaging. Background Art

[0002] High dynamic range imaging (HDRI) technology automatically adjusts exposure parameters based on the dynamic range of the scene, capturing multiple images to capture details at various brightness levels and fusing them into a single HDR image. Compared to traditional low dynamic range (LDR) images, it records information across a wider brightness range, providing a more realistic visual experience. Traditional SDR digital images encode the linear response values ​​of the sensor using the gamma opto-electrical transfer function (OETF) to improve information storage efficiency. With the development of HDR technology, international organizations have introduced two OETFs based on visual characteristics: the perception quantizer (PQ) curve and the hybrid log-gamma curve (HLG).

[0003] High dynamic range imaging technologies based on multi-frame synthesis are mainly divided into radiometric reconstruction and exposure fusion. Radiometric reconstruction normalizes pixel values ​​to quantities proportional to the scene's radiance based on the exposure parameters of each image. The images are then fused according to weights to produce HDR images that are correlated with the scene's true radiance. Exposure fusion directly fuses LDR images based on information such as saturation and contrast of corresponding pixels in each image, producing low dynamic range images that retain the scene's high dynamic range details.

[0004] The exposure time set used to capture HDR images significantly impacts their image quality. Prior art methods for selecting exposure times in radiometric reconstruction methods primarily fall into two categories: predefined exposure time and optimized image quality. The former typically suffers from low acquisition efficiency, often failing to capture the full dynamic range of a scene and resulting in a low signal-to-noise ratio. The latter typically requires iterative optimization, which is time-consuming and lacks visual consistency due to the target signal-to-noise ratio setting not taking into account the HDR encoding function. Summary of the Invention

[0005] The purpose of the present invention is to solve the problems existing in the prior art and provide a method for selecting exposure time for high dynamic range imaging, so as to obtain image quality that meets the visual perception of the human eye with the shortest possible exposure time to improve the efficiency of HDR image acquisition.

[0006] In order to achieve the above-mentioned object of the invention, the present invention specifically adopts the following technical solutions:

[0007] A method for selecting exposure time for high dynamic range imaging comprises the following steps:

[0008] S1. Calibrate the camera parameters required to calculate the brightness signal-to-noise ratio, the camera parameters including the camera quantum efficiency coefficient and the normalized pixel value corresponding to the middle gray;

[0009] S2. In the scene to be captured, calculate the camera gain coefficient based on the normalized pixel value corresponding to the middle gray calibrated in S1 and the ISO sensitivity parameter corresponding to the SDR image, obtain a first SDR image set in the original image encoding format, normalize each SDR image in the first SDR image set, and calculate the brightness of each pixel in the scene to be captured based on each normalized first SDR image in the first SDR image set, the camera's aperture F number, the exposure time of the SDR image in each exposure, and the camera's gain coefficient. Perform a weighted average of the brightness calculation results using the exposure time of the SDR image in each exposure as a weight to obtain a brightness image and construct a scene brightness distribution histogram.

[0010] S3. Select the brightness range of interest from the scene brightness distribution histogram obtained in S2, and calculate the signal-to-noise ratio of the target after encoding based on the photoelectric conversion function;

[0011] S4: Calculate the shortest exposure time sequence that meets the signal-to-noise ratio requirement according to the target encoded signal-to-noise ratio and the calculation formula of the luminance signal-to-noise ratio obtained in S3, and output the shortest exposure time sequence as the final selected exposure time sequence.

[0012] Based on the above solution, each step can be implemented in the following preferred specific manner.

[0013] Preferably, the specific process of step S1 is as follows:

[0014] S11 pre-acquire the same moment of the original image encoding format of the first calibration image and the second calibration image in JPG format;

[0015] S12. Extract the green channel of the first calibration image and the green channel of the second calibration image and normalize them separately. Search the normalized green channel of the second calibration image for pixels whose pixel values ​​are within a preset pixel interval as second mid-gray pixels. Obtain the coordinates of the second mid-gray pixels and construct a coordinate set. Pixels in the normalized green channel of the first calibration image whose coordinates are within the coordinate set are defined as first mid-gray pixels. The average pixel value of the first mid-gray pixels is used as the normalized pixel value corresponding to mid-gray. Calculate the camera gain coefficient g according to the following formula:

[0016]

[0017] Where Ystd is the normalized pixel value corresponding to middle gray; ISO is the ISO sensitivity parameter corresponding to the calibration image;

[0018] S13. Low-pass filtering is performed on the green channel of the first calibration image, and the green channel after low-pass filtering constitutes a calibration image with noise removed;

[0019] S14. In the denoised calibration image, a pixel with a grayscale value of V is obtained as a first reference pixel. A first pixel coordinate set is constructed using the coordinates of the first reference pixel. A pixel with a green channel coordinate in the first pixel coordinate set is used as a first test pixel. The pixel value mean and pixel value variance of the first test pixel are calculated. Starting from grayscale value V = 0, the pixel value mean and pixel value variance are continuously traversed until grayscale value V reaches a maximum grayscale value. The slope K of the pixel value variance of all grayscale values ​​with respect to the pixel value mean of all grayscale values ​​is fitted using the least squares method. The camera quantum efficiency coefficient k is calculated as follows:

[0020]

[0021] Preferably, in step S12, the pixel interval is a symmetrical closed interval with 0.5 as the interval center value.

[0022] Preferably, the specific process of step S2 is as follows: normalize each SDR image in the first SDR image set, convert all pixel values ​​of each first SDR image in the first SDR image set into brightness normalized values ​​according to the following formula, use the exposure time of the SDR image at each exposure as a weight to perform weighted averaging on the brightness calculation results to obtain a brightness image, and calculate the scene brightness distribution histogram:

[0023]

[0024] Where Y represents the normalized pixel value of each first SDR image in the first SDR image set; F represents the aperture F number of the camera; t i represents the exposure time of the i-th exposure.

[0025] Preferably, the specific process of step S3 is as follows:

[0026] S31. Obtain a 10-bit code value array and normalize it to the range of 0 to 1, and transform each code value in the normalized code value array according to the electro-optical conversion function to obtain the linear normalized brightness Y corresponding to each gray level of the photoelectric conversion function. oe And perform the difference to get the brightness interval corresponding to the coding value The target signal-to-noise ratio of the coding value space is calculated as follows:

[0027]

[0028] S32. Select the brightness range of interest from the scene brightness distribution histogram obtained in S2 [L roil ,L roiu ], according to the photoelectric conversion function, the brightness within the brightness range of interest is transformed to obtain the brightness distribution histogram in the coding value space; where L roil Indicates the lower bound of the brightness range of interest, L roiu Indicates the upper bound of the brightness range of interest;

[0029] S33. Preset the signal-to-noise ratio modulation coefficient c, and calculate the target signal-to-noise ratio SNR after encoding according to the target signal-to-noise ratio of the encoding value space and the brightness distribution histogram in the encoding value space by the following formula: obj :

[0030]

[0031] Where counts represents the frequency of the brightness distribution histogram in the coding value space in each quantization interval; counts max Represents the brightness distribution histogram in the encoding value space at the maximum frequency value.

[0032] Preferably, the specific process of step S4 is as follows:

[0033] S401. Preset the aperture F number of the camera and the ISO parameter corresponding to the image to be collected, and obtain all available exposure time vectors of the camera, wherein the available exposure time vector consists of available exposure times;

[0034] S402. Generate a quantization point sequence with equal spacing within the brightness range of interest as needed, wherein the quantization point sequence is composed of quantization points;

[0035] S403. The linear normalized brightness Y corresponding to each gray level of the camera quantum efficiency coefficient and the photoelectric conversion function calibrated in S1 oe The luminance signal-to-noise ratio (SNR) at each quantization point for each exposure time is calculated as follows:

[0036]

[0037] Where, ∑t i represents the sum of all available exposure times at a brightness of L, Y o ' e Represents Y oe The derivative of

[0038] S404: constructing a luminance signal-to-noise ratio curve using the luminance signal-to-noise ratios at all quantization points for each available exposure time, sequentially taking the quantization points in each luminance signal-to-noise ratio curve whose values ​​are lower than the target signal-to-noise ratio after encoding as inferior reference points, taking the difference between the luminance signal-to-noise ratio of the inferior reference point in each luminance signal-to-noise ratio curve and the target signal-to-noise ratio after encoding as the insufficient signal-to-noise ratio, and summing the insufficient signal-to-noise ratios in each luminance signal-to-noise ratio curve to obtain a signal-to-noise ratio difference value for each luminance signal-to-noise ratio curve;

[0039] S405: taking the minimum non-zero value of the signal-to-noise ratio difference among all the signal-to-noise ratio difference amounts as the selected signal-to-noise ratio difference amount, and taking the exposure time corresponding to the selected signal-to-noise ratio difference amount as the first item of the exposure time sequence;

[0040] S406: Using the luminance signal-to-noise ratio curve corresponding to the exposure time corresponding to the selected signal-to-noise ratio difference as the selected luminance signal-to-noise ratio curve, using the position of the first quantization point where the luminance signal-to-noise ratio is 0 in the selected luminance signal-to-noise ratio curve as the selected quantization point, using each luminance signal-to-noise ratio curve other than the luminance signal-to-noise ratio curve corresponding to the exposure time corresponding to the selected signal-to-noise ratio difference as the luminance signal-to-noise ratio curve to be processed, obtaining the luminance signal-to-noise ratio value of each luminance signal-to-noise ratio curve to be processed at the selected quantization point and generating a luminance signal-to-noise ratio value set, searching the luminance signal-to-noise ratio value set for the luminance signal-to-noise ratio value having the smallest absolute value of the difference with the target encoded signal-to-noise ratio as the optimal luminance signal-to-noise ratio, and adding the exposure time corresponding to the optimal luminance signal-to-noise ratio to the exposure time sequence;

[0041] S407: replacing the exposure time corresponding to the selected signal-to-noise ratio difference with the most recently added item in the shortest exposure time sequence, repeating S406 to generate a new exposure time corresponding to the optimal brightness signal-to-noise ratio and setting it as the next item in the exposure time sequence;

[0042] S408: until the luminance signal-to-noise ratio of the upper limit of the luminance range of interest is greater than the signal-to-noise ratio of the target after encoding of the upper limit of the luminance range of interest, a complete exposure time sequence is obtained;

[0043] S409: Arrange all available exposure times in descending order in an available exposure time sequence, and combine the complete exposure time sequence in S408 with elements in the available exposure time sequence to form an optimized exposure time sequence;

[0044] S410: Using the optimized exposure time sequence as the initial value of the second optimization sequence, first setting the optimization element number j=1, and combining the current exposure time as a first optimization sequence with a lower update frequency;

[0045] S411: The second optimized sequence obtained in the kth iteration of the small loop As the third optimized sequence obtained in the kth iteration of the small loop The initial value of The jth item is modified to the available exposure time sequence T avi id(j)+1th element, calculate The luminance signal-to-noise ratio under the condition is obtained to obtain the optimized luminance signal-to-noise ratio, and the difference between the optimized luminance signal-to-noise ratio and the target encoded signal-to-noise ratio is calculated to obtain the signal-to-noise ratio difference; where id(j) represents the element index of the available exposure time sequence;

[0046] S412: Determine whether there is an item less than zero in the signal-to-noise ratio difference: If not, The value of is used as the second optimization sequence in the k+1th iteration of the small loop The value of , and update the available exposure time sequence number vector, return to S411 to get the signal-to-noise ratio difference again, until there is an item less than zero in the signal-to-noise ratio difference; if there is, then The value of is used as the second optimization sequence in the k+1th iteration of the small loop The value of , updates the optimized element sequence number;

[0047] S413: Determine whether the updated optimized element number has exceeded the number of elements in the optimized exposure time sequence. If not, return to S411 to perform the k+1th iteration of the small loop; if more, execute S414;

[0048] S414: Determine whether the first optimization sequence is equal to the second optimization sequence: If not, reset the optimization element number j to 1, use the second optimization sequence to update the first optimization sequence and return to S411 to restart the small loop iteration; if equal, use the first optimization sequence as the final selected exposure time sequence and output it, ending the calculation.

[0049] Compared with the prior art, the present invention has the following beneficial effects:

[0050] This paper uses camera noise and gain parameter calibration results to establish a mapping relationship between real-world scene radiance and digital image pixel values. Combined with the quantization interval of HDR encoding, this method proposes a method for selecting exposure time for high-dynamic-range imaging. This method fully utilizes the potential of the camera system hardware, improving the efficiency of HDR image acquisition while maintaining the signal-to-noise ratio. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 is a flow chart of the steps of the method of the present invention;

[0052] Figure 2 Flowchart of the method of the present invention when obtaining the final selected exposure time sequence;

[0053] Figure 3Schematic diagram of an image used to calibrate camera parameters in an embodiment of the present invention;

[0054] Figure 4 Schematic diagram of a high dynamic range scene to be captured in an embodiment of the present invention;

[0055] Figure 5 Schematic diagram of a distribution histogram of the brightness of a high dynamic range scene to be captured in a logarithmic domain according to an embodiment of the present invention;

[0056] Figure 6 Schematic diagram of a signal-to-noise ratio curve after target encoding in an embodiment of the present invention;

[0057] Figure 7 Schematic diagram of the signal-to-noise ratio curve after PQ encoding and the luminance signal-to-noise ratio curve after target encoding for a single image at each exposure time in an embodiment of the present invention;

[0058] Figure 8 Schematic diagram of the post-encoding signal-to-noise ratio and the target post-encoding signal-to-noise ratio curves after reconstructing the HDR image using the final exposure time sequence in an embodiment of the present invention. DETAILED DESCRIPTION

[0059] In order to make the above-mentioned objects, features and advantages of the present invention more clearly understood, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways than those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. The technical features in the various embodiments of the present invention can be combined accordingly without conflicting with each other.

[0060] In the description of the present invention, it should be understood that the terms "first" and "second" are used solely for descriptive purposes and are not to be construed as indicating or implying relative importance or implicitly specifying the number of technical features being described. Therefore, features defined as "first" or "second" may explicitly or implicitly include at least one of such features.

[0061] The principles used in the method of the present invention are first introduced below.

[0062] 1) HDR imaging based on radiometric reconstruction first divides each image by its exposure time to obtain the same brightness scale and then obtains the HDR image through weighted averaging. This process can be expressed as follows:

[0063]

[0064] Where H(x) represents the pixel value of the HDR image at pixel coordinate x; Ne is the number of SDR images required to synthesize the HDR image; I i (x) represents the linear pixel value at the i-th exposure; w(I i (x)) represents the weight function with the linear pixel value at the i-th exposure as the independent variable; t i is the exposure time of the i-th exposure.

[0065] 2) The HDR image synthesis algorithm based on the photon Poisson noise estimator (PPNE) takes the exposure time t of the i-th exposure as i As a weight function, it is widely used in industry. The present invention defines the ratio of the pixel expected value to the standard deviation as the output reference signal-to-noise ratio SNR PPNE , which can be expressed by the following formula:

[0066]

[0067] Where, represents the expected value of the pixel of the HDR image; σ(H(x)) represents the standard deviation of the pixel of the HDR image.

[0068] 3) Based on the Photon Poisson Noise Estimator (PPNE), the luminance signal-to-noise ratio of the synthesized HDR image at each brightness after PQ encoding can be derived, which can be expressed as follows:

[0069]

[0070] Where SNR(L) represents the luminance signal-to-noise ratio at luminance L; L is the luminance at each pixel in the scene in candela per square meter; ∑t i represents the sum of all available exposure times at a brightness of L; k is the camera quantum efficiency coefficient; F represents the aperture F number of the camera; Y oe Y is the linear normalized brightness corresponding to each gray level of the photoelectric conversion function (OETF); o ' e Y oe The derivative of .

[0071] In this embodiment, the available exposure time is defined as the exposure time corresponding to a normalized pixel value at a brightness of L being less than 0.95.

[0072] 4) According to the International Organization for Standardization (ISO) definition of the sensitivity of imaging devices, the brightness L at each pixel in a scene measured in candela per square meter can be approximately calculated using the following formula:

[0073]

[0074]

[0075] Where, Y is the normalized pixel value; F is the aperture F number of the camera; g is the gain coefficient of the camera; Y std is the normalized pixel value corresponding to middle gray; ISO is the ISO sensitivity parameter corresponding to the calibration image.

[0076] Based on the above principles, the signal-to-noise ratio of HDR images under any combination of exposure parameters can be estimated, thereby achieving efficient image acquisition.

[0077] like Figure 1 As shown, in a preferred implementation of the present invention, the above-mentioned method for selecting exposure time for high dynamic range imaging includes the following steps S1 to S4. The specific implementation process is described in detail below.

[0078] S1: Calibrate the camera parameters required to calculate the brightness signal-to-noise ratio, which include the camera quantum efficiency coefficient k and the normalized pixel value Y corresponding to the middle gray std .

[0079] It should be noted that in step S1, in order to use the calculation formula of the brightness signal-to-noise ratio and the brightness calculation formula at each pixel to predict the brightness signal-to-noise ratio of the image, it is first necessary to calculate the camera quantum efficiency coefficient k and the normalized pixel value Y corresponding to the middle gray. std For calibration, the camera's aperture number F, exposure time for each exposure, and ISO parameters corresponding to the calibration image are provided by the EXIF ​​information stored in the calibration image.

[0080] In the present invention, the specific process of step S1 is as follows:

[0081] S11: pre-acquire a first calibration image R1 in a raw image encoding format (RAW format) and a second calibration image J1 in a JPG format at the same moment.

[0082] It should be noted that in step S11 of this embodiment, a camera is used to capture a calibration image. During the camera capture process, the ISO sensitivity parameter corresponding to the calibration image is set to ISO clb , which corresponds to a range of 100 to 3200, the exposure compensation value is set to 0, and the metering mode is set to automatic. These calibration images should be distributed at each grayscale level of the histogram and should be out of focus or contain uniform areas.

[0083] S12: Extract the green channel of the first calibration image R1 and the green channel of the second calibration image J1 and normalize them respectively, search for pixels whose pixel values ​​are within a preset pixel interval in the normalized green channel of the second calibration image J1 as second medium-gray pixels, obtain the coordinates of the second medium-gray pixels and construct a coordinate set P, take the pixels in the normalized green channel of the first calibration image R1 whose coordinates are in the coordinate set P as first medium-gray pixels, and take the average pixel value of the first medium-gray pixels as the normalized pixel value Y corresponding to the medium-gray std And calculate the camera gain coefficient g according to the following formula:

[0084]

[0085] It should be noted that in step S12 of this embodiment, the pixel interval is a symmetrical closed interval with 0.5 as the center value of the interval. Exemplarily, the pixel interval is set to [0.49, 0.51] and pixels whose pixel values ​​are within this interval are selected.

[0086] S13: performing low-pass filtering on the green channel of the first calibration image R1, and forming a calibration image R2 with noise removed from the green channel after the low-pass filtering.

[0087] S14: Obtain a pixel with a grayscale value of V in the denoised calibration image R2 as a first reference pixel, construct a first pixel coordinate set P2(V) based on the coordinates of the first reference pixel, use the pixel with the coordinates of the green channel of the first calibration image R1 in the first pixel coordinate set P2(V) as a first test pixel, and calculate the pixel value mean Y of the first test pixel mean (V) and pixel value variance Y var (V), starting from the grayscale value V = 0, it continues to traverse until the grayscale value V reaches the maximum grayscale value, and the pixel value variance Y of all grayscale values ​​is fitted by the least squares method var The pixel value mean Y of all gray values mean The slope K of the camera is calculated according to the following formula:

[0088]

[0089] It should be noted that in step S14 of this embodiment, for the non-defocused image containing uniform areas mentioned in S11 (i.e., the second calibration image J1 and the first calibration image R1), the uniform areas are manually selected or automatically segmented to calculate the Y of each area. var With Y mean , use the least squares method to fit Y var About Y mean The slope of the camera is calculated according to the above formula to calculate the camera quantum efficiency coefficient.

[0090] S2: In the scene to be shot, the normalized pixel value Y corresponding to the middle gray calibrated in S1 std and the sensitivity ISO parameter corresponding to the SDR image, calculate the camera's gain coefficient g, obtain a first SDR image set in the original image encoding format (RAW format), normalize each SDR image in the first SDR image set, and calculate the brightness L of each pixel in the scene to be photographed according to each normalized first SDR image in the first SDR image set, the camera's aperture F number, the exposure time of the SDR image in each exposure, and the camera's gain coefficient g, perform weighted averaging on the brightness calculation results using the exposure time of the SDR image in each exposure as a weight, obtain a brightness image, and construct a scene brightness distribution histogram.

[0091] It should be noted that the main purpose of step S2 of the present invention is to calculate the brightness distribution of the scene to be photographed according to the calculation formula of the brightness at each pixel, so as to be used in the subsequent steps to predict the brightness signal-to-noise ratio of each exposure time combination according to the calculation formula of the brightness signal-to-noise ratio. In step S2 of the present invention, in the scene to be photographed, the aperture F number of the camera selected by the image collector is set to F pre , the ISO sensitivity parameter value corresponding to the SDR image is ISO pre The camera’s automatic bracketing program obtains the initial exposure time sequence, and uses the initial exposure time sequence to acquire a series of SDR images of the scene to be shot. The exposure time parameters corresponding to each SDR image are recorded as When capturing a series of SDR images, these SDR images can be captured by manually modifying the exposure time or by using an automatic exposure program. However, regardless of the capture method, it is necessary to ensure that at least one image has no overexposed pixels in order to obtain an accurate scene brightness distribution.

[0092] In the present invention, the specific process of step S2 is as follows: normalize each SDR image in the first SDR image set, convert all pixel values ​​of each first SDR image in the first SDR image set into brightness normalized values ​​according to the following formula, use the exposure time of the SDR image at each exposure as a weight to perform weighted averaging on the brightness calculation results to obtain a brightness image, and calculate the scene brightness distribution histogram:

[0093]

[0094] Where Y represents the normalized pixel value of each first SDR image in the first SDR image set; F represents the aperture F number of the camera; t i represents the exposure time of the i-th exposure.

[0095] S3: Select the brightness range of interest from the scene brightness distribution histogram obtained in S2 [Lroil ,L roiu ], calculate the signal-to-noise ratio (SNR) of the target after encoding based on the Electrical-Optical Transfer Function (EOTF) obj .

[0096] It should be noted that, in step S3 of the present invention, the photoelectric conversion function may adopt PQ (perceptual quantizer) or HLG (Hybrid-Log Gamma).

[0097] In the present invention, the specific process of step S3 is as follows:

[0098] S31: Get a 10-bit code value array and normalize it to the range of 0 to 1, and transform each code value in the normalized code value array according to the electro-optical conversion function to obtain the linear normalized brightness Y corresponding to each gray level of the photoelectric conversion function (OETF) oe And perform the difference to get the brightness interval corresponding to the coding value The target signal-to-noise ratio of the coding value space is calculated as follows:

[0099]

[0100] S32: Select the brightness range of interest from the scene brightness distribution histogram obtained in S2 [L roil ,L roiu ], according to the photoelectric conversion function, the brightness within the brightness range of interest is transformed to obtain the brightness distribution histogram in the coding value space; where L roil Indicates the lower bound of the brightness range of interest, L roiu Indicates the upper bound of the brightness range of interest.

[0101] S33: Preset the signal-to-noise ratio modulation coefficient c, and calculate the target signal-to-noise ratio SNR after encoding according to the target signal-to-noise ratio of the encoding value space and the brightness distribution histogram in the encoding value space by the following formula: obj :

[0102]

[0103] Where counts represents the frequency of the brightness distribution histogram in the coding value space in each quantization interval; counts max Represents the brightness distribution histogram in the encoding value space at the maximum frequency value.

[0104] S4: The target signal-to-noise ratio (SNR) after encoding obtained from S3 objThe shortest exposure time sequence that meets the signal-to-noise ratio requirement is calculated using the calculation formula of the brightness signal-to-noise ratio, and the shortest exposure time sequence is used as the final selected exposure time sequence. And output.

[0105] In the present invention, Figure 2 As shown, the specific process of step S4 is as follows:

[0106] S401: Preset the camera's aperture F number and the ISO parameter corresponding to the image to be captured, and obtain all available exposure time vectors T of the camera. avi , the available exposure time vector consists of available exposure times.

[0107] S402: As needed, in the brightness range of interest [L roil ,L roiu ] A quantization point sequence L(j) is generated with medium spacing, and the quantization point sequence is composed of quantization points.

[0108] S403. The linear normalized brightness Y corresponding to each gray level of the camera quantum efficiency coefficient k and the photoelectric conversion function (OETF) calibrated in S1 oe The brightness signal-to-noise ratio at each quantization point for each exposure time is calculated as follows:

[0109]

[0110] S404: A luminance signal-to-noise ratio curve is formed by the luminance signal-to-noise ratio at all quantization points of each available exposure time, and the signal-to-noise ratio SNR after each luminance signal-to-noise ratio curve whose value is lower than the target encoding is sequentially divided. obj The quantization point of the luminance signal-to-noise ratio curve is taken as the inferior reference point, the difference between the luminance signal-to-noise ratio of the inferior reference point in each luminance signal-to-noise ratio curve and the target signal-to-noise ratio after encoding is taken as the insufficient signal-to-noise ratio, and the insufficient signal-to-noise ratios in each luminance signal-to-noise ratio curve are added together to obtain the signal-to-noise ratio difference of each luminance signal-to-noise ratio curve.

[0111] S405: taking the non-zero minimum value of the signal-to-noise ratio difference among all the signal-to-noise ratio difference amounts as the selected signal-to-noise ratio difference amount, and taking the exposure time t1 corresponding to the selected signal-to-noise ratio difference amount as the first item of the exposure time sequence.

[0112] S406: The luminance signal-to-noise ratio curve corresponding to the exposure time t1 corresponding to the selected signal-to-noise ratio difference is used as the selected luminance signal-to-noise ratio curve, the position of the first quantization point where the luminance signal-to-noise ratio is 0 in the selected luminance signal-to-noise ratio curve is used as the selected quantization point, each luminance signal-to-noise ratio curve other than the luminance signal-to-noise ratio curve corresponding to the exposure time t1 corresponding to the selected signal-to-noise ratio difference is used as the luminance signal-to-noise ratio curve to be processed, the luminance signal-to-noise ratio value of each luminance signal-to-noise ratio curve to be processed at the selected quantization point is obtained and a luminance signal-to-noise ratio value set is generated, and the luminance signal-to-noise ratio value set is searched for the signal-to-noise ratio SNR after encoding that matches the target luminance signal-to-noise ratio in the luminance signal-to-noise ratio value set. obj The brightness signal-to-noise ratio value with the smallest absolute value of the difference is taken as the optimal brightness signal-to-noise ratio, and the exposure time corresponding to the optimal brightness signal-to-noise ratio is added to the exposure time sequence.

[0113] S407: Replace the exposure time corresponding to the selected signal-to-noise ratio difference with the latest item added in the shortest exposure time sequence, repeat S406, generate a new exposure time corresponding to the optimal brightness signal-to-noise ratio and use it as the next item in the exposure time sequence.

[0114] S408: until the luminance signal-to-noise ratio of the upper limit of the luminance range of interest is greater than the target encoded signal-to-noise ratio of the upper limit of the luminance range of interest, and a complete exposure time sequence is obtained.

[0115] S409: Arrange all available exposure times in descending order in the available exposure time sequence T avi In T avi The elements in S408 are combined to form the complete exposure time sequence to form the optimized exposure time sequence t opt .

[0116] In this embodiment S409, for example, the exposure time vector can be in the form of T avi =[2 1 0.5 0.25 0.125 0.1 0.05 0.001 0.0005 0.0001], the available exposure times with serial numbers 2, 5 and 7 constitute the optimized exposure time sequence, that is, the optimized exposure time sequence t opt The first term is the available exposure time vector T avi The id(1)th element of , the optimized exposure time series can be expressed as t opt =[1,0.125,0.05]=[T avi (2),T avi (5),T avi (7)], id = [2, 5, 7] represents the selected available exposure time sequence number vector.

[0117] S410: Optimizing the exposure time sequence as the second optimization sequence Initial value, first let the optimization element number j = 1, and take the current exposure time combination as a first optimization sequence t with a lower update frequency opt1 .

[0118] S411: The second optimized sequence obtained in the kth iteration of the small loop As the third optimized sequence obtained in the kth iteration of the small loop The initial value of The jth item is modified to the available exposure time sequence T avi The id(j)+1th element T avi (id(j)+1), calculate The luminance signal-to-noise ratio SNR(L) under the condition is used to obtain the optimized luminance signal-to-noise ratio, and the optimized luminance signal-to-noise ratio and the target encoded signal-to-noise ratio SNR are calculated. obj The difference between them is the signal-to-noise ratio difference SNRdif.

[0119] In step S411 of this embodiment, The jth item is modified to the available exposure time sequence T avi The id(j)+1th element T avi (id(j)+1), for example, After modification, it becomes Then after the modification

[0120] S412: Determine whether there is an item less than zero in the signal-to-noise ratio difference: If not, The value of is used as the second optimization sequence in the k+1th iteration of the small loop The value of , and update the available exposure time sequence number vector, return to S411 to get the signal-to-noise ratio difference again, until there is an item less than zero in the signal-to-noise ratio difference; if there is, then The value of is used as the second optimization sequence in the k+1th iteration of the small loop The value of (i.e. not updating ), update the optimization element number (i.e. start optimizing the second optimization sequence in the subsequent cycle (j+1th item of ).

[0121] In step S412 of this embodiment, the available exposure time sequence number vector is updated, that is, the original id = [2, 5, 7] is updated to id = [3, 5, 7] and then continued to update The jth item of (perform the k+1th iteration of the small loop).

[0122] S413: Determine whether the updated optimized element number has exceeded the optimized exposure time sequence t optIf the number of elements in does not exceed, return to S411 to perform the k+1th iteration of the small loop; if it exceeds, execute S414.

[0123] In this embodiment, the number of elements in the optimized exposure time sequence is expressed as length(t opt ) calculation, the length function is a function that calculates the number of elements in a one-dimensional vector.

[0124] S414: Determine the first optimized sequence t opt1 Is it equal to the second optimized sequence (i.e., determine whether the exposure time sequence has not changed after a large cycle): If not, reset the optimization element number j to 1, and use the second optimization sequence to update the first optimization sequence t opt1 And return to S411 to restart the small loop iteration; if it is equal, it proves that the exposure time sequence has converged and no longer changes, then the first optimized sequence is used as the final selected exposure time sequence End calculation.

[0125] In order to better demonstrate the specific implementation and technical effects of the present invention, a high dynamic range imaging exposure time selection method shown in steps S1 to S4 in the above preferred implementation is applied to a specific example.

[0126] Example

[0127] The specific implementation process of the high dynamic range imaging exposure time selection method used in this embodiment is as described above and will not be repeated here. The following describes the key steps in implementing the above method using a Sony ILCE-7RM2 mirrorless camera equipped with a Sigma 24-70mm F2.8 DG DN|Art lens as an example, with the green channel noise level as a benchmark.

[0128] In this embodiment, the camera can store 14-bit RAW images and 8-bit JPG images with linear pixel value distribution. The camera parameters required for noise estimation are gain g and quantum efficiency coefficient k. Parameter acquisition includes the following steps:

[0129] Place the color card in a standard light box simulating D65 light source, fix the digital camera on a tripod, set ISO to 100, exposure compensation to 0, and automatic metering mode to capture RAW and JPG images at the same time. Figure 3 As shown. Perform basic ISP (image signal process) operations on the RAW image and normalize it to [0,1]. Normalize the JPG image. Calculate the linear grayscale value Y of the pixels in the RAW image whose grayscale values ​​are in the range [0.49,0.51] in the JPG image. std, for this embodiment Y std =0.09, at the same time, calculate the camera gain g = (Y std ×100) / 14=0.643. Obtain the mask of the central area of ​​each neutral color block in the image and calculate the average pixel value Y at each mask. mean and the variance Y var , fit Y by the least squares method var About Y mean The slope K of the camera is used to calculate the camera quantum efficiency coefficient.

[0130] In such Figure 4 Capture HDR images in the specific scene shown, set the aperture number F pre =16, ISO value ISO pre = 100; Use the camera's continuous exposure setting to shoot a total of 5 RAW images at 3EV (exposure value) intervals, and obtain the exposure time of this group of images.

[0131] Assume that the pixel value is Y, and convert the set of images into a brightness estimation image in candela per square meter according to the brightness calculation formula at each pixel, obtain the HDR reconstructed brightness image, and obtain the scene brightness distribution histogram (such as Figure 5 shown).

[0132] According to the obtained scene brightness distribution histogram, set the upper and lower bounds of the brightness range of interest [3,342701] (unit cd / m 2 ), create a 10-bit code value array, normalize it to the range of 0 to 1, and transform the code value according to the PQ EOTF formula to obtain Y eo , for Y eo After difference, we get Then according to the formula Get the target signal-to-noise ratio in the encoding value space Then adjust the SNR according to the signal-to-noise ratio modulation coefficient c obj , the signal-to-noise ratio after target encoding corresponding to different c is as follows Figure 6 In this embodiment, the signal-to-noise ratio modulation coefficient c=0.

[0133] Keep the aperture number F and ISO value used in the previous pre-collection unchanged, and set the total available exposure time vector of the camera to T avi ; Select quantization points in the interval [3,342701] at equal intervals as needed, and calculate the brightness signal-to-noise ratio of each exposure time series at each quantization point according to the brightness signal-to-noise ratio calculation formula. The calculation results are as follows Figure 7 shown.

[0134] For each exposure time corresponding to the SNR, calculate its value lower than the SNR obj The sum of the parts and find the exposure time t1 corresponding to the smallest non-zero number as the first item of the exposure time sequence. In this embodiment, t1 is 0.126s. At the position where the SNR corresponding to t1 drops to 0, find the SNR and SNR obj The closest exposure time, which is 0.0061 s in this embodiment, is added to the exposure time sequence. The above steps are repeated until the highest brightness meets the SNR requirement. At this time, the exposure time sequence is [0.126 0.0061 0.0002].

[0135] Use T avi The exposure time series is expressed as: t opt =[T avi (13)T avi (26)T avi (42)], modify the largest item in the exposure time sequence and replace it with the item with the second longest exposure time in the available exposure time sequence, and calculate the brightness signal-to-noise ratio (SNR) corresponding to the modified sequence according to the formula new , such as SNR at each brightness level new >SNR obj , then the replacement is applied; repeat the above operation until no replacement can be performed, then modify the second long time item in the exposure time sequence, and repeat until the last item in the exposure time sequence is replaced. Check whether the exposure time sequence obtained by the small loop is the same as that before the loop. In this embodiment, the result is the same, and the final selected exposure time sequence is obtained. The SNR curve of the HDR image reconstructed by the final selected exposure time sequence and the target SNR curve is as follows: Figure 8 shown.

[0136] The embodiment described above is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Persons skilled in the art may make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, any technical solution obtained by equivalent substitution or equivalent transformation falls within the scope of protection of the present invention.

Claims

1. A method for selecting exposure time for high dynamic range imaging, characterized in that: The following steps are involved: S1. Calibrate the camera parameters required to calculate the brightness signal-to-noise ratio, the camera parameters including the camera quantum efficiency coefficient and the normalized pixel value corresponding to the middle gray; S2. In the scene to be captured, calculate the camera gain coefficient based on the normalized pixel value corresponding to the middle gray calibrated in S1 and the ISO sensitivity parameter corresponding to the SDR image, obtain a first SDR image set in the original image encoding format, normalize each SDR image in the first SDR image set, and calculate the brightness of each pixel in the scene to be captured based on each normalized first SDR image in the first SDR image set, the camera's aperture F number, the exposure time of the SDR image in each exposure, and the camera's gain coefficient. Perform a weighted average of the brightness calculation results using the exposure time of the SDR image in each exposure as a weight to obtain a brightness image and construct a scene brightness distribution histogram. S3. Select the brightness range of interest from the scene brightness distribution histogram obtained in S2, and calculate the signal-to-noise ratio of the target after encoding based on the photoelectric conversion function; S4: Calculate the shortest exposure time sequence that meets the signal-to-noise ratio requirement according to the target encoded signal-to-noise ratio and the calculation formula of the luminance signal-to-noise ratio obtained in S3, and output the shortest exposure time sequence as the final selected exposure time sequence.

2. The method for selecting exposure time for high dynamic range imaging according to claim 1, wherein: The specific process of step S1 is as follows: S11 pre-acquire the same moment of the original image encoding format of the first calibration image and the second calibration image in JPG format; S12. Extract the green channel of the first calibration image and the green channel of the second calibration image and normalize them separately. Search the normalized green channel of the second calibration image for pixels whose pixel values ​​are within a preset pixel interval as second mid-gray pixels. Obtain the coordinates of the second mid-gray pixels and construct a coordinate set. Pixels in the normalized green channel of the first calibration image whose coordinates are within the coordinate set are defined as first mid-gray pixels. The average pixel value of the first mid-gray pixels is used as the normalized pixel value corresponding to mid-gray. Calculate the camera gain coefficient g according to the following formula: Where Y std is the normalized pixel value corresponding to middle gray; ISO is the ISO sensitivity parameter corresponding to the calibration image; S13. Low-pass filtering is performed on the green channel of the first calibration image, and the green channel after low-pass filtering constitutes a calibration image with noise removed; S14. In the denoised calibration image, a pixel with a grayscale value of V is obtained as a first reference pixel. A first pixel coordinate set is constructed using the coordinates of the first reference pixel. A pixel with a green channel coordinate in the first pixel coordinate set is used as a first test pixel. The pixel value mean and pixel value variance of the first test pixel are calculated. Starting from grayscale value V = 0, the pixel value mean and pixel value variance are continuously traversed until grayscale value V reaches a maximum grayscale value. The slope K of the pixel value variance of all grayscale values ​​with respect to the pixel value mean of all grayscale values ​​is fitted using the least squares method. The camera quantum efficiency coefficient k is calculated as follows:

3. The method for selecting exposure time for high dynamic range imaging according to claim 2, wherein: In step S12, the pixel interval is a symmetrical closed interval with 0.5 as the interval center value.

4. The method for selecting exposure time for high dynamic range imaging according to claim 2, wherein: The specific process of step S2 is as follows: normalize each SDR image in the first SDR image set, convert all pixel values ​​of each first SDR image in the first SDR image set into brightness normalized values ​​according to the following formula, use the exposure time of the SDR image at each exposure as a weight to perform weighted averaging on the brightness calculation results to obtain a brightness image, and calculate the scene brightness distribution histogram: Where Y represents the normalized pixel value of each first SDR image in the first SDR image set; F represents the aperture F number of the camera; t i represents the exposure time of the i-th exposure.

5. The method for selecting exposure time for high dynamic range imaging according to claim 1, wherein: The specific process of step S3 is as follows: S31. Obtain a 10-bit code value array and normalize it to the range of 0 to 1, and transform each code value in the normalized code value array according to the electro-optical conversion function to obtain the linear normalized brightness Y corresponding to each gray level of the photoelectric conversion function. oe And perform the difference to get the brightness interval corresponding to the coding value The target signal-to-noise ratio of the coding value space is calculated as follows: S32. Select the brightness range of interest from the scene brightness distribution histogram obtained in S2 [L roil ,L roiu ], according to the photoelectric conversion function, the brightness within the brightness range of interest is transformed to obtain the brightness distribution histogram in the coding value space; where L roil Indicates the lower bound of the brightness range of interest, L roiu Indicates the upper bound of the brightness range of interest; S33. Preset the signal-to-noise ratio modulation coefficient c, and calculate the target signal-to-noise ratio SNR after encoding according to the target signal-to-noise ratio of the encoding value space and the brightness distribution histogram in the encoding value space by the following formula: obj : Where counts represents the frequency of the brightness distribution histogram in the coding value space in each quantization interval; counts max Represents the brightness distribution histogram in the encoding value space at the maximum frequency value.

6. The method for selecting exposure time for high dynamic range imaging according to claim 4, wherein: The specific process of step S4 is as follows: S401. Preset the aperture F number of the camera and the ISO parameter corresponding to the image to be collected, and obtain all available exposure time vectors of the camera, wherein the available exposure time vector consists of available exposure times; S402. Generate a quantization point sequence with equal spacing within the brightness range of interest as needed, wherein the quantization point sequence is composed of quantization points; S403. The linear normalized brightness Y corresponding to each gray level of the camera quantum efficiency coefficient and the photoelectric conversion function calibrated in S1 oe The luminance signal-to-noise ratio (SNR) at each quantization point for each exposure time is calculated as follows: Where, ∑t i represents the sum of all available exposure times at a brightness of L, Y o ′ e Represents Y oe The derivative of S404: constructing a luminance signal-to-noise ratio curve using the luminance signal-to-noise ratios at all quantization points for each available exposure time, sequentially taking the quantization points in each luminance signal-to-noise ratio curve whose values ​​are lower than the target signal-to-noise ratio after encoding as inferior reference points, taking the difference between the luminance signal-to-noise ratio of the inferior reference point in each luminance signal-to-noise ratio curve and the target signal-to-noise ratio after encoding as the insufficient signal-to-noise ratio, and summing the insufficient signal-to-noise ratios in each luminance signal-to-noise ratio curve to obtain a signal-to-noise ratio difference value for each luminance signal-to-noise ratio curve; S405: taking the minimum non-zero value of the signal-to-noise ratio difference among all the signal-to-noise ratio difference amounts as the selected signal-to-noise ratio difference amount, and taking the exposure time corresponding to the selected signal-to-noise ratio difference amount as the first item of the exposure time sequence; S406: Using the luminance signal-to-noise ratio curve corresponding to the exposure time corresponding to the selected signal-to-noise ratio difference as the selected luminance signal-to-noise ratio curve, using the position of the first quantization point where the luminance signal-to-noise ratio is 0 in the selected luminance signal-to-noise ratio curve as the selected quantization point, using each luminance signal-to-noise ratio curve other than the luminance signal-to-noise ratio curve corresponding to the exposure time corresponding to the selected signal-to-noise ratio difference as the luminance signal-to-noise ratio curve to be processed, obtaining the luminance signal-to-noise ratio value of each luminance signal-to-noise ratio curve to be processed at the selected quantization point and generating a luminance signal-to-noise ratio value set, searching the luminance signal-to-noise ratio value set for the luminance signal-to-noise ratio value having the smallest absolute value of the difference with the target encoded signal-to-noise ratio as the optimal luminance signal-to-noise ratio, and adding the exposure time corresponding to the optimal luminance signal-to-noise ratio to the exposure time sequence; S407: replacing the exposure time corresponding to the selected signal-to-noise ratio difference with the most recently added item in the shortest exposure time sequence, repeating S406 to generate a new exposure time corresponding to the optimal brightness signal-to-noise ratio and setting it as the next item in the exposure time sequence; S408: until the luminance signal-to-noise ratio of the upper limit of the luminance range of interest is greater than the signal-to-noise ratio of the target after encoding of the upper limit of the luminance range of interest, a complete exposure time sequence is obtained; S409: Arrange all available exposure times in descending order in an available exposure time sequence, and combine the complete exposure time sequence in S408 with elements in the available exposure time sequence to form an optimized exposure time sequence; S410: Using the optimized exposure time sequence as the initial value of the second optimization sequence, first setting the optimization element number j=1, and combining the current exposure time as a first optimization sequence with a lower update frequency; S411: The second optimized sequence obtained in the kth iteration of the small loop As the third optimized sequence obtained in the kth iteration of the small loop The initial value of The jth item is modified to the available exposure time sequence T avi id(j)+1th element, calculate The luminance signal-to-noise ratio under the condition is obtained to obtain the optimized luminance signal-to-noise ratio, and the difference between the optimized luminance signal-to-noise ratio and the target encoded signal-to-noise ratio is calculated to obtain the signal-to-noise ratio difference; where id(j) represents the element index of the available exposure time sequence; S412: Determine whether there is an item less than zero in the signal-to-noise ratio difference: If not, The value of is used as the second optimization sequence in the k+1th iteration of the small loop The value of , and update the available exposure time sequence number vector, return to S411 to get the signal-to-noise ratio difference again, until there is an item less than zero in the signal-to-noise ratio difference; if there is, then The value of is used as the second optimization sequence in the k+1th iteration of the small loop The value of , updates the optimized element sequence number; S413: Determine whether the updated optimized element number has exceeded the number of elements in the optimized exposure time sequence. If not, return to S411 to perform the k+1th iteration of the small loop; if more, execute S414; S414: Determine whether the first optimization sequence is equal to the second optimization sequence: If not, reset the optimization element number j to 1, use the second optimization sequence to update the first optimization sequence and return to S411 to restart the small loop iteration; if equal, use the first optimization sequence as the final selected exposure time sequence and output it, ending the calculation.

Images