High dynamic range image generation methods, apparatus and computer equipment

Abstract

This application relates to a method, apparatus, and computer device for generating high dynamic range (HDR) images. The method includes: acquiring multiple original images corresponding to different exposure parameters; adaptively fusing the multiple original images to obtain an intermediate image; performing dynamic range compression on the intermediate image to obtain a brightness-optimized image; and performing desaturation on the brightness-optimized image to obtain a HDR image. By performing dynamic range compression on the fused intermediate image followed by desaturation, the problem of uncontrolled saturation is solved, the realism of image colors is improved, and the image quality of the HDR image is further enhanced.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a method, apparatus and computer device for generating high dynamic range images. Background Technology

[0002] A magnifying observation device is an optical or optoelectronic instrument system used for high-magnification, high-resolution imaging of tiny objects or fine structures. During the imaging process of a magnifying observation device, there are usually significant differences in reflectivity on the surface of the observed sample; that is, different regions of the same observed sample have vastly different reflective capabilities to incident light. Examples include areas where metallic highlights coexist with micrometer-level grooves, or areas where wafer copper lines and passivation layers coexist. These reflectivity differences make it difficult for a single exposure image to simultaneously preserve both bright textures and dark structures; in other words, a magnifying observation device cannot clearly record detailed information from both high-reflectivity and low-reflectivity areas simultaneously under single-exposure conditions.

[0003] Current high dynamic range (HDR) imaging techniques typically rely on multiple frames with different exposures to obtain HDR images. However, during HDR compression, saturation loss can occur, causing image colors to deviate from reality. In other words, the HDR images generated by current techniques have poor image quality. Summary of the Invention

[0004] Therefore, it is necessary to provide a high dynamic range image generation method, apparatus, and computer equipment to address the aforementioned technical problems.

[0005] In a first aspect, this application provides a high dynamic range image generation method, the method comprising: acquiring multiple original images corresponding to different exposure parameters; adaptively fusing the multiple original images to obtain an intermediate image; dynamically compressing the intermediate image to obtain a brightness-optimized image; determining the initial red grayscale value, initial green grayscale value, and initial blue grayscale value of each pixel based on the intermediate image and the brightness-optimized image, and performing desaturation operation on the initial red grayscale value, initial green grayscale value, and initial blue grayscale value of each pixel to obtain a high dynamic range image.

[0006] In one embodiment, the plurality of original images include a first exposure image corresponding to a first exposure parameter and a second exposure image corresponding to a second exposure parameter; the adaptive fusion of the plurality of original images to obtain an intermediate image includes: determining, based on the first exposure image and the second exposure image, a first local brightness value of each pixel in the first exposure image, a first fusion weight value corresponding to the first local brightness value of each pixel, a second local brightness value of each pixel in the second exposure image, and a second fusion weight value corresponding to the second local brightness value of each pixel; determining the exposure ratio based on the first exposure parameter and the second exposure parameter; and determining the brightness value of each pixel in the intermediate image based on the first local brightness value of each pixel in the first exposure image, the first fusion weight value corresponding to the first local brightness value of each pixel, the second local brightness value of each pixel in the second exposure image, the second fusion weight value corresponding to the second local brightness value of each pixel, and the exposure ratio.

[0007] In one embodiment, determining the first local brightness value of each pixel in the first exposed image, the first fusion weight value corresponding to the first local brightness value of each pixel, the second local brightness value of each pixel in the second exposed image, and the second fusion weight value corresponding to the second local brightness value of each pixel based on the first exposed image and the second exposed image includes: determining the first local brightness value of each pixel in the first exposed image based on the brightness value of each pixel in the first exposed image and the local range of each pixel; determining the second local brightness value of each pixel in the second exposed image based on the brightness value of each pixel in the second exposed image and the local range of each pixel; determining the second fusion weight value corresponding to the second local brightness value of each pixel and the first fusion weight value corresponding to the first local brightness value of each pixel based on the second local brightness value of each pixel and a first lookup table; the first lookup table is constructed based on the Sigmoid function.

[0008] In one embodiment, the step of dynamically compressing the intermediate image to obtain a brightness-optimized image includes: determining the brightness value of each pixel in the intermediate image based on the intermediate image; and determining the brightness value of each pixel in the brightness-optimized image based on the brightness value of each pixel in the intermediate image and a second lookup table.

[0009] In one embodiment, the method further includes: constructing a low-light enhancement curve based on the ACES mapping algorithm; dividing the low-light enhancement curve into a first curve and a second curve based on a suppression range threshold; adjusting the second curve using an exponential function to obtain a highlight suppression curve; and constructing a second lookup table based on the first curve and the highlight suppression curve.

[0010] In one embodiment, the step of determining the initial red, initial green, and initial blue gray values ​​of each pixel based on the intermediate image and the brightness-optimized image, and performing desaturation operations on the initial red, initial green, and initial blue gray values ​​of each pixel to obtain a high dynamic range image includes: determining the initial red, initial green, and initial blue gray values ​​of each pixel in the high dynamic range image based on the brightness values ​​of each pixel in the brightness-optimized image and the brightness values ​​of each pixel in the intermediate image; and performing desaturation operations on the initial red, initial green, and initial blue gray values ​​of each pixel in the high dynamic range image to obtain the final red, final green, and final blue gray values ​​of each pixel in the high dynamic range image.

[0011] In one embodiment, determining the initial red, initial green, and initial blue grayscale values ​​of each pixel in the high dynamic range image based on the brightness values ​​of each pixel in the brightness-optimized image and the brightness values ​​of each pixel in the intermediate image includes: determining the pixel group corresponding to each pixel in the intermediate image; each pixel group includes four pixels, corresponding to the red channel grayscale value, the first green channel grayscale value, the second green channel grayscale value, and the blue channel grayscale value, respectively; determining the total grayscale value of each pixel group based on the brightness values ​​of each pixel in the intermediate image; and determining the total grayscale value of each pixel group based on the brightness values ​​of the target pixel in the brightness-optimized image and the corresponding pixel values ​​of the target pixel. The initial red grayscale value of the target pixel in the high dynamic range image is determined by the total grayscale value of the group and the red channel grayscale value of the pixel group corresponding to the target pixel. The initial green grayscale value of the target pixel in the high dynamic range image is determined by the brightness value of the target pixel in the brightness-optimized image, the total grayscale value of the pixel group corresponding to the target pixel, the first green channel grayscale value of the pixel group corresponding to the target pixel, and the second green channel grayscale value of the pixel group corresponding to the target pixel. The initial blue grayscale value of the target pixel in the high dynamic range image is determined by the brightness value of the target pixel in the brightness-optimized image, the total grayscale value of the pixel group corresponding to the target pixel, and the blue channel grayscale value of the pixel group corresponding to the target pixel.

[0012] In one embodiment, the step of performing desaturation operations on the initial red, green, and blue grayscale values ​​of each pixel in the high dynamic range image to obtain the final red, green, and blue grayscale values ​​of each pixel in the high dynamic range image includes: determining the weighted grayscale value of the target pixel based on the initial red, green, and blue grayscale values ​​of the target pixel in the high dynamic range image; determining the mixing parameters of the target pixel based on the brightness value of the target pixel in the brightness optimization image and a preset desaturation parameter; and determining the weighted grayscale value of the target pixel based on the high dynamic range image. The final red grayscale value of the target pixel in the high dynamic range image is determined by using the initial red grayscale value, the weighted grayscale value, and the mixing parameters of the target pixel. The final green grayscale value of the target pixel in the high dynamic range image is also determined by using the initial green grayscale value, the weighted grayscale value, and the mixing parameters of the target pixel. Finally, the final blue grayscale value of the target pixel in the high dynamic range image is determined by using the initial blue grayscale value, the weighted grayscale value, and the mixing parameters of the target pixel.

[0013] Secondly, this application also provides a high dynamic range image generation apparatus, the apparatus comprising: an acquisition module for acquiring multiple original images corresponding to different exposure parameters; a fusion module for adaptively fusion of the multiple original images to obtain an intermediate image; a compression module for dynamically compressing the intermediate image to obtain a brightness-optimized image; and a desaturation module for determining the initial red grayscale value, initial green grayscale value, and initial blue grayscale value of each pixel based on the intermediate image and the brightness-optimized image, and performing desaturation operations on the initial red grayscale value, initial green grayscale value, and initial blue grayscale value of each pixel to obtain the high dynamic range image.

[0014] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement any of the high dynamic range image generation methods described in the first aspect.

[0015] The aforementioned high dynamic range (HMR) image generation method, apparatus, and computer equipment acquire multiple original images corresponding to different exposure parameters and adaptively fuse these images to obtain an intermediate image. Dynamic range compression is then applied to the intermediate image to obtain a highlight-optimized image, followed by desaturation to obtain the HMR image. By performing dynamic range compression and desaturation on the fused intermediate image, the problem of saturation imbalance is solved, the realism of image colors is improved, and the image quality of the HMR image is further enhanced. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating a high dynamic range image generation method in one embodiment;

[0017] Figure 2 This is a flowchart illustrating an intermediate image generation method in one embodiment;

[0018] Figure 3 This is a flowchart illustrating a brightness-optimized image generation method in one embodiment;

[0019] Figure 4 This is a flowchart illustrating a desaturation operation method in one embodiment;

[0020] Figure 5 This is a graph of the Sigmoid function in one embodiment;

[0021] Figure 6 A graph of the ACES mapping algorithm in one embodiment;

[0022] Figure 7 This is a structural block diagram of a high dynamic range image generation device in one embodiment;

[0023] Figure 8 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0025] A magnifying observation device is an optical or optoelectronic instrument system used for high-magnification, high-resolution imaging of tiny objects or fine structures. During the imaging process of a magnifying observation device, there are usually significant differences in reflectivity on the surface of the observed sample; that is, different regions of the same observed sample have vastly different reflective capabilities to incident light. Examples include areas where metallic highlights coexist with micrometer-level grooves, or areas where wafer copper lines and passivation layers coexist. These reflectivity differences make it difficult for a single exposure image to simultaneously preserve both bright textures and dark structures; in other words, a magnifying observation device cannot clearly record detailed information from both high-reflectivity and low-reflectivity areas simultaneously under single-exposure conditions.

[0026] Current high dynamic range (HDR) imaging techniques typically rely on multiple frames of images with different exposures to obtain HDR images. However, these techniques suffer from the following technical problems:

[0027] 1. When performing high dynamic range (HDR) image synthesis, the computational load for efficient deployment is large due to the algorithm's reliance on floating-point operations or global optimization. This makes it difficult to run on the FPGA built into the magnification observation device, requiring the original image to be transmitted to an external device for HDR image generation. However, this image transmission results in high latency and cannot support real-time preview and autofocus on the magnification observation device.

[0028] 2. Most high dynamic range image synthesis algorithms operate in the RGB domain, ignoring the characteristics of the original image, which can lead to interpolation artifacts and blurred edges.

[0029] 3. When generating high dynamic range images, highly reflective areas such as metals are prone to overexposure and whitening, resulting in the loss of microstructure information and affecting the accuracy of 3D reconstruction.

[0030] 4. During high dynamic range compression, there may be a problem of uncontrolled saturation, which will cause the image colors to deviate from reality.

[0031] Based on the above, current related technologies cannot generate high dynamic range (HDR) images in real time at the magnifying observation device, and the generated HDR images have poor image quality. Therefore, there is an urgent need to provide a HDR image generation method that can generate high dynamic range (HDR) images in real time at the magnifying observation device and produce high-quality images.

[0032] In one embodiment, such as Figure 1 As shown, a high dynamic range image generation method is provided. This method is applied to a magnifying observation device, which can be any optical or optoelectronic instrument system capable of high-magnification, high-resolution imaging of tiny objects or fine structures. Examples include optical microscopes, 3D ultra-depth-of-field imaging systems, semiconductor defect detection equipment, precision metal surface analyzers, microelectronic packaging inspection devices, and various industrial endoscopes. The high dynamic range image generation method includes the following steps:

[0033] Step 101: Obtain multiple original images corresponding to different exposure parameters.

[0034] When imaging a sample, the magnifying observation device acquires multiple raw images with different exposure parameters within the same frame. The sample being observed can be any object capable of being imaged in the magnifying observation device; this embodiment does not impose specific limitations, such as a wafer. Exposure parameters can be the light input conditions controlling the image sensor's acquisition, such as exposure time, gain, or aperture size; this embodiment does not impose specific limitations, but specifically, the exposure parameter can be exposure time. Raw images are the original images acquired under the corresponding exposure parameters. There can be multiple raw images, such as two, three, or five; this embodiment does not impose specific limitations on the number of raw images. For example, when the exposure parameters include a first exposure parameter, a second exposure parameter, and a third exposure parameter, the multiple raw images can be the first exposure image corresponding to the first exposure parameter, the second exposure image corresponding to the second exposure parameter, and the third exposure image corresponding to the third exposure parameter. Since the multiple raw images are acquired within the same frame, differing only in exposure parameters, the sizes of the multiple raw images are identical, meaning that the same pixel positions in the multiple raw images can correspond one-to-one. The raw images can be acquired Bayer RAW data.

[0035] Step 102: Adaptively fuse multiple original images to obtain an intermediate image.

[0036] Adaptive fusion is a multi-image fusion strategy based on dynamically assigning weights to local image features. Specifically, for each original image, the local image features of each pixel in the original image are calculated separately, and then weights are dynamically assigned to each pixel in the original image. Based on the local image features and dynamic weights of each pixel, image fusion is performed to obtain an intermediate image. The local image features of each pixel can be statistical brightness information within its local neighborhood, i.e., the local brightness value. By adaptively fusion of the original images, multiple original images can be merged without entering the RGB domain, preserving edge sharpness and suppressing interpolation artifacts.

[0037] Step 103: Perform dynamic range compression on the intermediate image to obtain a brightness-optimized image.

[0038] Dynamic range compression (DLL) is the operation of mapping an intermediate image to a limited brightness range suitable for the output of a display device. DLL can suppress highlights in high-brightness areas and enhance shadows in low-brightness areas of the intermediate image, resulting in a brightness-optimized image. Specifically, a non-linear mapping curve is pre-generated based on the ACES mapping algorithm and a highlight suppression algorithm. A second lookup table is then generated based on the non-linear mapping curve. This second lookup table is used to perform highlight suppression or shadow enhancement on the brightness value of each pixel in the intermediate image, thus obtaining a brightness-optimized image. The second lookup table represents the mapping relationship between the brightness values ​​before and after DLL compression. It is understandable that the pre-generation of the non-linear mapping curve based on the ACES mapping algorithm and the highlight suppression algorithm, and the generation of the second lookup table based on the non-linear mapping curve, can be performed on an external computer device. After generating the second lookup table, it is stored in a magnifying observation device. When performing DLL, the magnifying observation device only needs to look up the second lookup table based on the brightness value of each pixel in the intermediate image to obtain the brightness value of each pixel after DLL compression, thus obtaining a brightness-optimized image. By using a second lookup table for dynamic range compression, the computational load can be reduced, further ensuring that the high dynamic range image generation method runs on the FPGA built into the magnification and observation device.

[0039] Step 104: Based on the intermediate image and the brightness-optimized image, determine the initial red grayscale value, initial green grayscale value, and initial blue grayscale value of each pixel, and perform desaturation operation on the initial red grayscale value, initial green grayscale value, and initial blue grayscale value of each pixel to obtain a high dynamic range image.

[0040] Because highlight suppression is applied to high-brightness areas in the brightness-optimized image, the color saturation in these areas increases due to the reduced brightness. Therefore, a desaturation operation is needed to obtain a high dynamic range (HDR) image. The HDR image is the final output image that exhibits high dynamic range visual performance and color fidelity. HDR images can clearly display the texture of high-brightness areas and the structure of low-brightness areas.

[0041] This embodiment acquires multiple original images corresponding to different exposure parameters and adaptively fuses them to obtain an intermediate image. Dynamic range compression is then applied to the intermediate image to obtain a highlight-optimized image, followed by desaturation to obtain a high dynamic range image. By performing dynamic range compression and desaturation on the fused intermediate image, the problem of saturation imbalance is solved, improving the realism of image colors and further enhancing the image quality of the high dynamic range image. The use of a second lookup table for dynamic range compression reduces computation, enabling high dynamic range image generation to run efficiently on the FPGA built into the magnification and observation device with low latency, thus supporting real-time preview.

[0042] In one embodiment, such as Figure 2 As shown, an intermediate image generation method is provided, which specifically includes the following steps:

[0043] This embodiment uses multiple original images, including a first exposure image corresponding to a first exposure parameter and a second exposure image corresponding to a second exposure parameter, as an example for illustration. The exposure parameter can be the exposure time; the first exposure parameter is less than the second exposure parameter, meaning the first exposure time is less than the second exposure time. The first exposure image acquired using the first exposure parameter is a short exposure image, and the second exposure image acquired using the second exposure parameter is a long exposure image.

[0044] Step 201: Based on the first exposure image and the second exposure image, determine the first local brightness value of each pixel in the first exposure image, the first fusion weight value corresponding to the first local brightness value of each pixel, the second local brightness value of each pixel in the second exposure image, and the second fusion weight value corresponding to the second local brightness value of each pixel.

[0045] The first local brightness value of a pixel is statistical brightness information obtained from the local neighborhood centered on that pixel in the first exposure image. The second local brightness value of a pixel is statistical brightness information obtained from the local neighborhood centered on that pixel in the second exposure image. The second fusion weight value corresponding to the second local brightness value of each pixel can be determined based on the Sigmoid function and the second local brightness value of each pixel. The first fusion weight value corresponding to the first local brightness value of each pixel can be determined based on the second fusion weight value corresponding to the second local brightness value of each pixel. More specifically, it may include the following steps:

[0046] Step 1: Determine the first local brightness value of each pixel in the first exposure image based on the brightness value of each pixel and the local range of each pixel.

[0047] The local range of each pixel is a spatial neighborhood region centered on the corresponding pixel. For example, the local range can be a 3×3 pixel region centered on the pixel; the local range can be a 7×7 pixel region centered on the pixel. This embodiment does not specifically limit the size of the local range. Since the first exposure image is Bayer RAW data, Bayer RAW data contains red channel grayscale values, first green channel grayscale values, second green channel grayscale values, and blue channel grayscale values ​​within a 2×2 pixel region. Specifically, within the 2×2 pixel region, the pixel in the upper left corner has the red channel grayscale value, the pixel in the upper right corner has the first green channel grayscale value, the pixel in the lower left corner has the second green channel grayscale value, and the pixel in the lower right corner has the blue channel grayscale value. The brightness value of each pixel in the first exposure image is the corresponding channel grayscale value of that pixel. For example, if the current pixel has a red channel grayscale value, then the brightness value of that pixel is the red channel grayscale value; if the current pixel has a blue channel grayscale value, then the brightness value of that pixel is the blue channel grayscale value. When determining the first local brightness value, for any given pixel, the average brightness of all pixels within the corresponding local area is calculated, and this average brightness value is used as the first local brightness value for that pixel. The first local brightness value needs to be calculated for every pixel in the first exposed image.

[0048] For example, taking a local area of ​​3×3 pixels as an example, the target pixel is surrounded by a 3×3 pixel area, plus itself, totaling 9 pixels. The average brightness of these 9 pixels is calculated. The first local brightness value of the target pixel is this average brightness value. The first local brightness value is calculated for each pixel in the first exposure image. The specific calculation formula is as follows:

[0049] (1)

[0050] Where L1(x,y) represents the first local brightness value of a pixel; N represents the number of pixels within the local range; and Is(i,j) represents the brightness value of a pixel.

[0051] Step 2: Determine the second local brightness value of each pixel in the second exposure image based on the brightness value of each pixel and the local range of each pixel.

[0052] The local extent of each pixel is a spatial neighborhood region centered on that pixel. For example, the local extent can be a 3×3 pixel region centered on the pixel; the local extent can be a 7×7 pixel region centered on the pixel. This embodiment does not specifically limit the size of the local extent. Since the second exposure image is Bayer RAW data, Bayer RAW data contains red channel grayscale values, first green channel grayscale values, second green channel grayscale values, and blue channel grayscale values ​​within a 2×2 pixel region. Specifically, within the 2×2 pixel region, the pixel in the upper left corner has the red channel grayscale value, the pixel in the upper right corner has the first green channel grayscale value, the pixel in the lower left corner has the second green channel grayscale value, and the pixel in the lower right corner has the blue channel grayscale value. The brightness value of each pixel in the second exposure image is the corresponding channel grayscale value. For example, if the current pixel has a red channel grayscale value, then the brightness value of that pixel is the red channel grayscale value; if the current pixel has a blue channel grayscale value, then the brightness value of that pixel is the blue channel grayscale value. When determining the second local brightness value, for any given pixel, the average brightness of all pixels within the corresponding local area is calculated, and this average brightness value is used as the second local brightness value for that pixel. The second local brightness value needs to be calculated for every pixel in the second exposure image.

[0053] For example, taking a 3×3 pixel region as an example, the target pixel is surrounded by a 3×3 pixel region, plus itself, totaling 9 pixels. The average brightness of these 9 pixels is calculated. The second local brightness value of the target pixel is this average brightness value. The second local brightness value is calculated for each pixel in the second exposure image. The specific calculation formula is as follows:

[0054] (2)

[0055] Where L2(x,y) represents the second local brightness value of a pixel; N represents the number of pixels within the local range; and Is(i,j) represents the brightness value of a pixel.

[0056] Step 3: Based on the second local brightness value of each pixel and the first lookup table, determine the second fusion weight value corresponding to the second local brightness value of each pixel and the first fusion weight value corresponding to the first local brightness value of each pixel; the first lookup table is constructed based on the Sigmoid function.

[0057] The first lookup table can be a pre-built mapping table structure used to directly convert input local brightness values ​​into corresponding fusion weight values. Implementing weight mapping through the first lookup table reduces the computational load on the FPGA and improves processing speed. The construction of the first lookup table can be performed on an external computer device. After the first lookup table is built, the external computer device sends it to the magnification and observation device, which stores the first lookup table. Specifically, the first lookup table can be pre-calculated based on the Sigmoid function or the inverse Sigmoid function, and the different brightness values ​​and their corresponding fusion weight values ​​are stored as a one-dimensional array to obtain the first lookup table. Specifically, the Sigmoid function is shown below:

[0058] (3)

[0059] Where f(x) represents the second fusion weight value, ranging from 0 to 1; x represents the brightness value; center represents the median brightness; offset represents the horizontal offset of the brightness; and k represents the steepness of the transition region. The median brightness is a constant, while offset and k are manually set according to the actual scene. When constructing the first lookup table, each brightness value is substituted into the above formula to obtain the fusion weight value corresponding to each brightness value, thus obtaining the first lookup table.

[0060] When determining the fusion weight, the magnifying observation device first determines the second fusion weight value corresponding to the second local brightness value of each pixel based on the second local brightness value of each pixel and a first lookup table. Specifically, the second local brightness value of each pixel is looked up in the first lookup table to obtain the second fusion weight value corresponding to the second local brightness value of each pixel. For example, for the first pixel, the second fusion weight value corresponding to the second local brightness value of the first pixel is obtained by looking up the first lookup table; for the second pixel, the second fusion weight value corresponding to the second local brightness value of the second pixel is obtained by looking up the first lookup table.

[0061] After obtaining the second fusion weight value, the first fusion weight value corresponding to the first local brightness value of each pixel is determined based on the second fusion weight value. Specifically, 1 minus the second fusion weight value yields the first fusion weight value corresponding to the first local brightness value of the corresponding pixel. For example, for the first pixel, 1 minus the second fusion weight value of the first pixel yields the first fusion weight value corresponding to the first local brightness value of the first pixel; for the second pixel, 1 minus the second fusion weight value of the second pixel yields the first fusion weight value corresponding to the first local brightness value of the second pixel.

[0062] Step 202: Determine the exposure ratio based on the first exposure parameter and the second exposure parameter.

[0063] The first exposure parameter is the exposure parameter corresponding to the acquisition of the first exposure image, and the second exposure parameter is the exposure parameter corresponding to the acquisition of the second exposure image. The first exposure image is a short exposure image, and the second exposure image is a long exposure image. The ratio of the second exposure parameter to the first exposure parameter is used as the exposure ratio. Specifically, the first exposure parameter and the second exposure parameter are the exposure durations. In the example of calculating the exposure ratio, if the first exposure parameter is 10ms and the second exposure parameter is 40ms, then the exposure ratio is 40 / 10 = 4.

[0064] Step 203: Determine the brightness value of each pixel in the intermediate image based on the first local brightness value of each pixel in the first exposure image, the first fusion weight value corresponding to the first local brightness value of each pixel, the second local brightness value of each pixel in the second exposure image, the second fusion weight value corresponding to the second local brightness value of each pixel, and the exposure ratio.

[0065] The specific formula for determining the brightness value of each pixel in the intermediate image is as follows:

[0066] (4)

[0067] Where fusedVal represents the brightness value of a pixel in the intermediate image; L represents the second local brightness value of a pixel in the second exposure image; weight L represents the second fusion weight value corresponding to the second local brightness value of the pixel; S represents the first local brightness value of a pixel in the first exposure image; weight S represents the first fusion weight value corresponding to the first local brightness value of the pixel; and D represents the exposure ratio.

[0068] Specifically, for any given pixel, the first local brightness value of the pixel in the first exposed image, the first fusion weight value corresponding to the first local brightness value of the pixel, the second local brightness value of the pixel in the second exposed image, the second fusion weight value corresponding to the second local brightness value of the pixel, and the exposure ratio are substituted into the above formula to obtain the brightness value of the corresponding pixel in the intermediate image. The brightness value of each pixel in the intermediate image is calculated separately to obtain the intermediate image.

[0069] This embodiment utilizes the Sigmoid function to pre-construct a first lookup table. During actual image processing, weight mapping is achieved through this first lookup table, which reduces the computational load on the FPGA and improves processing speed. Furthermore, during adaptive fusion, adaptive fusion is performed in the Bayer domain, avoiding RGB interpolation artifacts. This can be executed in real-time on the FPGA, reducing transmission latency and supporting real-time preview.

[0070] In one embodiment, such as Figure 3 As shown, a brightness-optimized image generation method is provided, which specifically includes the following steps:

[0071] Before performing dynamic range compression on the intermediate image, a second lookup table needs to be pre-constructed. This second lookup table can be a pre-built luminance mapping table, representing the mapping relationship between luminance values ​​before and after dynamic range compression. Implementing dynamic range compression using this second lookup table reduces the computational load on the FPGA and improves processing speed. The construction of the second lookup table can be performed on an external computer. After construction, the external computer sends the second lookup table to the magnification and observation device, which then stores it.

[0072] Constructing the second lookup table specifically includes the following steps:

[0073] Step 1: Construct a low-light enhancement curve based on the ACES mapping algorithm.

[0074] The ACES mapping algorithm is a non-linear color mapping standard used to convert high dynamic range brightness into a display-appropriate color mapping, offering good tonal continuity and visual fidelity. Specifically, the ACES mapping algorithm is as follows:

[0075] (5)

[0076] (6)

[0077] Where x represents the luminance value before dynamic range compression; x' represents the luminance value after input limiting; f(x) represents the luminance value after mapping by the ACES mapping algorithm; A, B, C, D, E, F, and G are parameters of the ACES mapping algorithm, which can be set according to human experience. For example, A = 10.51; B = 0.01; C = 1.50; D = 2.45; E = 0.05; F = 0.05; G = 0.0625.

[0078] Based on the above f(x), a curve without specular suppression can be obtained. This function is monotonically increasing from 0 to ∞. As f(x) approaches ∞, the increment decreases, remaining almost constant. Therefore, it is necessary to restrict the input to the function, i.e., by calculating x'. The obtained f(x) curve is then normalized to obtain the dark-light enhancement curve. The range of f(x) for the dark-light enhancement curve is 0-1.

[0079] Step 2: Based on the suppression range threshold, the dark light enhancement curve is divided into a first curve and a second curve.

[0080] After obtaining the low-light enhancement curve, it is necessary to use an exponential function to suppress highlights. However, using an exponential function for highlight suppression will also affect the mid-light and low-light regions. Therefore, it is necessary to limit the range of the curve for highlight suppression by using a suppression range threshold. The suppression range threshold can be set according to actual usage requirements; this embodiment does not impose a specific limitation. Preferably, the suppression range threshold is 0.5, and the value range of f(x) of the low-light enhancement curve is 0-1. By using a suppression range threshold of 0.5, the low-light enhancement curve can be divided into two segments: 0-0.5 represents the first curve, indicating the mid-light and low-light regions; 0.5-1 represents the second curve, indicating the highlight region. After dividing the first and second curves, highlight suppression can be performed only on the second curve, thereby avoiding the impact on the mid-light and low-light regions.

[0081] Step 3: Adjust the second curve using an exponential function to obtain the highlight suppression curve.

[0082] The exponential function can be the POW function. The second curve can be adjusted using the exponential function as follows:

[0083] (7)

[0084] Where thresh is the suppression range threshold. When f(x) is greater than 0.5, substituting f(x) into the above formula yields f(x)', which is the highlight suppression curve obtained after suppressing the highlight of the second curve. Within the range of 0.5-1, as f(x) increases, f(x) is gradually compressed, and the highlight is effectively suppressed.

[0085] Step 4: Based on the first curve and the highlight suppression curve, construct the second lookup table.

[0086] After obtaining the first curve and the highlight suppression curve, they are combined to obtain the complete curve. The second lookup table can then be derived from the complete curve. The second lookup table represents the mapping relationship between the luminance values ​​before dynamic range compression and the luminance values ​​after dynamic range compression.

[0087] This embodiment constructs a low-light enhancement curve based on the ACES mapping algorithm, generating a visually natural base response curve to ensure the continuity and naturalness of low-light enhancement. The low-light enhancement curve is divided into a first curve and a second curve based on a suppression range threshold, achieving separation between the low-to-mid-brightness and high-brightness regions. An exponential function is used to adjust the second curve to obtain a highlight suppression curve, effectively suppressing overexposure in strong reflection areas and preserving key microstructural details. A second lookup table is constructed based on the first curve and the highlight suppression curve, integrating the dark enhancement and highlight suppression logic into an efficient integer quantization mapping table, supporting low-latency real-time processing on resource-constrained platforms such as FPGAs. This significantly reduces processing complexity and improves high dynamic range image quality while maintaining color stability and detail preservation.

[0088] Step 301: Determine the brightness value of each pixel in the intermediate image based on the intermediate image.

[0089] In practical use, the second lookup table has already been stored in the magnified observation device. At this point, it is necessary to determine the brightness value of each pixel in the intermediate image based on the intermediate image.

[0090] Step 302: Determine the brightness value of each pixel in the brightness-optimized image based on the brightness value of each pixel in the intermediate image and the second lookup table.

[0091] After obtaining the brightness value of each pixel in the intermediate image, the brightness value of each pixel in the brightness-optimized image is determined by looking up a second lookup table. Specifically, for any given pixel, the brightness value of that pixel in the intermediate image is used to look up the second lookup table to obtain the brightness value of the corresponding pixel in the brightness-optimized image. The brightness value of each pixel in the brightness-optimized image is then calculated to obtain the brightness-optimized image.

[0092] In one embodiment, such as Figure 4 As shown, a desaturation operation method is provided, which specifically includes the following steps:

[0093] Step 401: Based on the brightness values ​​of each pixel in the brightness-optimized image and the brightness values ​​of each pixel in the intermediate image, determine the initial red grayscale value, initial green grayscale value, and initial blue grayscale value of each pixel in the high dynamic range image.

[0094] Determining the initial red, green, and blue gray values ​​for each pixel in a high dynamic range image specifically includes the following steps:

[0095] Step 1: Determine the pixel group corresponding to each pixel in the intermediate image.

[0096] Since both the intermediate image and the brightness-optimized image are processed in the BayerRAW data domain, the resulting intermediate image and brightness-optimized image are also BayerRAW data. For the intermediate image, each 2×2 pixel region is considered a pixel group, meaning each pixel group includes four pixels. Each pixel group contains grayscale values ​​for the red channel, the first green channel, the second green channel, and the blue channel. In other words, the four pixels within the same pixel group correspond to the grayscale values ​​for the red channel, the first green channel, the second green channel, and the blue channel, respectively.

[0097] Step 2: Determine the total grayscale value of each pixel group based on the brightness value of each pixel in the intermediate image.

[0098] After determining the pixel groups, for each pixel group, first determine the brightness values ​​of the four pixels within that group, namely the grayscale values ​​of the red channel, the first green channel, the second green channel, and the blue channel. Summate these grayscale values ​​to obtain the total grayscale value for the corresponding pixel group. The total grayscale value needs to be calculated for each pixel group. The specific calculation method is as follows:

[0099] (8)

[0100] Where R[i] represents the gray value of the red channel in the pixel group; G1[i] represents the gray value of the first green channel in the pixel group; G2[i] represents the gray value of the second green channel in the pixel group; B[i] represents the gray value of the blue channel in the pixel group; and total represents the total gray value of the pixel group.

[0101] Step 3: Determine the initial red gray value of the target pixel in the high dynamic range image based on the brightness value of the target pixel in the brightness-optimized image, the total gray value of the pixel group corresponding to the target pixel, and the red channel gray value of the pixel group corresponding to the target pixel.

[0102] When determining the initial red grayscale value of a target pixel in a high dynamic range image, it is first necessary to determine the brightness value of the target pixel in the brightness optimization image. Since the first and second exposure images are the same size, corresponding pixel positions can be mapped one-to-one. Therefore, the size of the intermediate image obtained from the first and second exposure images is also the same as the first and second exposure images. The size of the brightness optimization image obtained from the intermediate image is also the same as the first and second exposure images. The target pixel is any pixel at the same position in the brightness optimization image, intermediate image, and high dynamic range image. First, determine the brightness value of the target pixel in the brightness optimization image. Then, based on the position of the target pixel in the intermediate image, determine the pixel group to which the target pixel belongs. Based on this pixel group, determine the total grayscale value of the pixel group and the red channel grayscale value within that pixel group. The specific calculation method for the initial red grayscale value of the target pixel is as follows:

[0103] (9)

[0104] Where R represents the initial red grayscale value of the target pixel in the high dynamic range image; R[i] represents the red channel grayscale value in the pixel group; total represents the total grayscale value of the pixel group; and f(x)' represents the brightness value of the target pixel in the brightness-optimized image. Based on the above formula, the initial red grayscale value of each pixel is calculated to obtain the initial red grayscale value of each pixel in the high dynamic range image.

[0105] Step 4: Determine the initial green gray value of the target pixel in the high dynamic range image based on the brightness value of the target pixel in the brightness-optimized image, the total gray value of the pixel group corresponding to the target pixel, the first green channel gray value of the pixel group corresponding to the target pixel, and the second green channel gray value of the pixel group corresponding to the target pixel.

[0106] First, determine the brightness value of the target pixel in the brightness-optimized image. Then, based on the target pixel's position in the intermediate image, determine the pixel group to which the target pixel belongs. Based on this pixel group, determine the total grayscale value of the pixel group, as well as the grayscale values ​​of the first and second green channels within that pixel group. The specific calculation method for the initial green grayscale value of the target pixel is as follows:

[0107] (10)

[0108] Where G represents the initial green grayscale value of the target pixel in the high dynamic range image; G1[i] represents the grayscale value of the first green channel in the pixel group; G2[i] represents the grayscale value of the second green channel in the pixel group; total represents the total grayscale value of the pixel group; and f(x)' represents the brightness value of the target pixel in the brightness-optimized image. Based on the above formula, the initial green grayscale value of each pixel is calculated to obtain the initial green grayscale value of each pixel in the high dynamic range image.

[0109] Step 5: Determine the initial blue grayscale value of the target pixel in the high dynamic range image based on the brightness value of the target pixel in the brightness-optimized image, the total grayscale value of the pixel group corresponding to the target pixel, and the blue channel grayscale value of the pixel group corresponding to the target pixel.

[0110] First, determine the brightness value of the target pixel in the brightness-optimized image. Then, based on the target pixel's position in the intermediate image, determine the pixel group to which the target pixel belongs. Based on this pixel group, determine the total grayscale value of that pixel group and the blue channel grayscale value within that pixel group. The specific calculation method for the initial blue grayscale value of the target pixel is as follows:

[0111] (11)

[0112] Where B represents the initial blue grayscale value of the target pixel in the high dynamic range image; B[i] represents the blue channel grayscale value in the pixel group; total represents the total grayscale value of the pixel group; and f(x)' represents the brightness value of the target pixel in the brightness-optimized image. Based on the above formula, the initial blue grayscale value of each pixel is calculated to obtain the initial blue grayscale value of each pixel in the high dynamic range image.

[0113] Step 402: Perform desaturation operations on the initial red, green, and blue gray values ​​of each pixel in the high dynamic range image to obtain the final red, green, and blue gray values ​​of each pixel in the high dynamic range image.

[0114] The desaturation operation specifically includes the following steps:

[0115] Step 1: Determine the weighted gray value of the target pixel based on the initial red, green, and blue gray values ​​of the target pixel in the high dynamic range image.

[0116] The weighted gray value of the target pixel is obtained by averaging the initial red, green, and blue gray values ​​of the target pixel. The target pixel is any pixel at the same location in the brightness-optimized image, intermediate image, and high dynamic range image. The specific calculation formula is as follows:

[0117] (12)

[0118] Where R represents the initial red grayscale value of the target pixel in the high dynamic range image; G represents the initial green grayscale value of the target pixel in the high dynamic range image; B represents the initial blue grayscale value of the target pixel in the high dynamic range image; and meanRGB represents the weighted grayscale value of the target pixel.

[0119] Step 2: Determine the blending parameters of the target pixel based on the brightness value of the target pixel in the brightness optimization image and the preset desaturation parameters.

[0120] The specific calculation formula is as follows:

[0121] (13)

[0122] Where f(x)' represents the brightness value of the target pixel in the brightness-optimized image; H and I represent preset desaturation parameters, which can be set according to actual usage requirements. Preferably, H=0.5 and I=0.6; blend represents the blending parameter of the target pixel.

[0123] Step 3: Determine the final red gray value of the target pixel in the high dynamic range image based on the initial red gray value of the target pixel, the weighted gray value of the target pixel, and the mixing parameters of the target pixel.

[0124] The specific calculation formula is as follows:

[0125] (14)

[0126] Where R' represents the final red grayscale value of the target pixel in the high dynamic range image; R represents the initial red grayscale value of the target pixel in the high dynamic range image; blend represents the blending parameter of the target pixel; and meanRGB represents the weighted grayscale value of the target pixel.

[0127] Step 4: Determine the final green gray value of the target pixel in the high dynamic range image based on the initial green gray value of the target pixel, the weighted gray value of the target pixel, and the mixing parameters of the target pixel.

[0128] The specific calculation formula is as follows:

[0129] (15)

[0130] Where G' represents the final green grayscale value of the target pixel in the high dynamic range image; G represents the initial green grayscale value of the target pixel in the high dynamic range image; blend represents the blending parameter of the target pixel; and meanRGB represents the weighted grayscale value of the target pixel.

[0131] Step 5: Determine the final blue gray value of the target pixel in the high dynamic range image based on the initial blue gray value of the target pixel, the weighted gray value of the target pixel, and the mixing parameters of the target pixel.

[0132] The specific calculation formula is as follows:

[0133] (16)

[0134] Where B' represents the final blue grayscale value of the target pixel in the high dynamic range image; B represents the initial blue grayscale value of the target pixel in the high dynamic range image; blend represents the blending parameter of the target pixel; and meanRGB represents the weighted grayscale value of the target pixel.

[0135] This embodiment performs desaturation operations on the initial red, green, and blue grayscale values ​​of each pixel in the high dynamic range image to obtain the final red, green, and blue grayscale values ​​of each pixel in the high dynamic range image. By introducing the Bayer domain multi-channel information of the original image in the desaturation operation stage, the light response characteristics of the sensor's lowest layer are preserved, avoiding color artifacts caused by RGB domain interpolation. Combined with the brightness value of each pixel in the brightness optimization image as a guiding signal, the initial red, green, and blue grayscale values ​​are reconstructed, realizing accurate color mapping from extended dynamic range to display range. Pixel-level desaturation operations are performed on the initial grayscale values ​​of each color channel to maintain overall brightness perception while preventing color shift caused by oversaturation of a certain channel, thereby comprehensively improving image quality.

[0136] In one specific embodiment, a method is provided for magnifying high dynamic range (HDR) images directly from a camera in an observation device. This method enables end-to-end HDR image generation, i.e., high dynamic range image generation, within the raw data domain of the image sensor, directly outputting high-quality HDR images without relying on a backend computing platform. The method specifically includes the following steps:

[0137] Step 1: Adaptive fusion of multi-exposure images.

[0138] At least two original images with different exposure durations for the same scene are acquired. Based on the local brightness features of the images, spatially adaptive fusion weights are dynamically generated. According to the fusion weights and exposure ratios, pixel-level weighted fusion is performed on the original images of the multiple exposures to generate an intermediate image with an extended dynamic range. Among them, local brightness features refer to the statistical brightness information in the local neighborhood centered on the current pixel, which is used to characterize the exposure state of the region. Specifically, for the Bayer RAW data of the short exposure image and the long exposure image, the local brightness value of each pixel is calculated. For the specific calculation method, please refer to formulas (1)-(2).

[0139] The local brightness values ​​of each pixel in the short-exposure image and the local brightness values ​​of each pixel in the long-exposure image can then be used to generate spatially adaptive fusion weights. Specifically, the fusion weight of each pixel in the short-exposure image and the fusion weight of each pixel in the long-exposure image can be determined through a first lookup table, and weight fusion is performed to obtain an intermediate image. This achieves an adaptive fusion strategy that favors short-exposure areas in highlight areas and long-exposure areas in shadow areas. For specific weight fusion details, please refer to formula (4).

[0140] When constructing the first lookup table, the fusion weights of the long-exposure image are calculated based on the Sigmoid function to obtain the first lookup table. See formula (3) for details. Figure 5 , Figure 5 The curve is the sigmoid function curve.

[0141] Step 2: Tone mapping of facial visual perception.

[0142] A configurable second lookup table is constructed, which is used to perform global or local dynamic range compression on the intermediate image to obtain a brightness-optimized image. The second lookup table includes suppression strategies for high-brightness areas and enhancement strategies for low-brightness areas to optimize the overall image contrast and detail visibility.

[0143] A second lookup table is constructed based on the ACES mapping algorithm and the specular suppression algorithm. See formulas (5)-(7) for details. First, based on the ACES mapping algorithm, i.e., formula (5), a curve without specular suppression is constructed, as shown in the figure. Figure 6 As shown. This function is monotonically increasing from 0 to ∞. As the function approaches ∞, the increment decreases, remaining almost constant. Therefore, it is necessary to restrict the input of the function. Please refer to formula (6) for details. The above formulas (5) and (6) are for 14-bit RAW data. For other bit-level RAW data, the parameters in the formulas need to be adjusted. Normalize the function to obtain the dark light enhancement curve.

[0144] The value range of the dark enhancement curve is [0-1]. POW processing on it can effectively suppress highlights, but it will also affect the mid-light and low-light areas. Therefore, it is necessary to limit the range of the curve for highlight suppression by using a suppression range threshold. Please refer to formula (7) for details. This yields the second lookup table. The highlight suppression process is smooth and will not produce halos or other phenomena at the critical positions. In addition, in order to avoid the impact of contrast enhancement on the suppression of dark areas, a dark enhancement processing can be performed on the curve. The idea is similar to highlight suppression, so it will not be elaborated here. Finally, based on the above operations, a curve with highlight suppression and dark enhancement can be obtained, and the second lookup table can be obtained based on this curve.

[0145] Step 3: Maintain color consistency and optimize naturalness.

[0146] If the original RAW data is directly mapped, the image will become noticeably gray and the saturation will decrease. To avoid these negative effects, the input 14-bit RAW data needs to be preprocessed, i.e., the Bayer (e.g., RGGB) grayscale of the RAW image needs to be extracted and weighted. See formulas (8)-(11) for details. Since the curve of the highlight part is suppressed, the color of the highlight part will have reduced brightness and increased saturation, so desaturation processing is required to obtain a high dynamic range image. See formulas (12)-(16) for details. Based on the above formulas, the grayscale values ​​of the current red, green and blue channels will be brought closer to the average value, and the saturation will be reduced. If you want to map the high dynamic range image back to the RAW data, simply multiply the high dynamic range image by the RAW bit depth.

[0147] Step 4: Efficient implementation in embedded hardware.

[0148] The first lookup table and the second lookup table in the above steps are stored in the FPGA of the magnification observation device. A pipelined processing architecture is built on a programmable logic device (such as an FPGA) to realize end-to-end low-latency processing from raw image input to high dynamic range image output, which meets the stringent real-time requirements of industrial microscopy systems.

[0149] The method provided in this embodiment is not limited to a specific image format, sensor type, or exposure quantity, and is applicable to imaging systems containing Bayer, Quad Bayer, monochrome, or other raw data formats. Furthermore, the method can be extended to multispectral, ultraviolet, or infrared imaging scenarios, as long as there is a dynamic range limitation issue.

[0150] This embodiment discloses a method for directly outputting high dynamic range (HDR) images from a camera in a magnifying observation device. Within the Bayer RAW raw data domain output by the image sensor, this method achieves end-to-end generation of HDR images through a three-stage processing approach: multi-exposure adaptive fusion, modulated tone mapping, and color fidelity reconstruction. Based on dynamically generated spatial adaptive fusion weights using local brightness features, combined with nonlinear mapping for highlight suppression and shadow enhancement, and incorporating channel coordination and desaturation mechanisms, the method effectively preserves the bright details and dark structures of highly reflective samples (such as metals and wafers) while ensuring color consistency. The entire process employs lookup table (LUT) driven fixed-point arithmetic, achieving millisecond-level low-latency processing on embedded platforms such as FPGAs, requiring no backend host intervention, and directly outputting HDR images suitable for 3D depth-of-field synthesis or AI detection. This embodiment is applicable to various magnifying observation devices such as microscopes, endoscopes, and industrial inspection equipment, significantly improving the real-time imaging quality of high-contrast samples.

[0151] In one specific application scenario, industrial microscopy is used for 3D ultra-depth-of-field surface inspection of metal parts. The scenario involves using a 3D ultra-depth-of-field microscope to acquire images of the surface of an aluminum alloy part. The microscope has a Z-axis step of 2μm and a total of 30 layers. A 5-megapixel industrial camera with Bayer RAW12 raw image output is used. Exposure is achieved with a long exposure of 1 / 50s (capturing the bottom of the scratch) and a short exposure of 1 / 400s (preserving the metallic highlight texture). The results achieved using this application are: the frosted texture of the metal surface is clearly visible without overexposure; details in the scratch and recessed areas are enhanced; the output image, used as input for 3D height reconstruction, exhibits good edge continuity with a reconstruction error of <1%. Image output performance is 1.9ms / frame, meeting the requirements for real-time preview and autofocus.

[0152] In one specific application scenario, such as semiconductor wafer defect detection, the sample is a 12-inch silicon wafer containing copper wiring layers and passivation layers. The exposure strategy during image acquisition is dynamic Exposure_Ratio (center bright area Ratio=8, edge dark area Ratio=4). The results achieved using this application are: no overflow at the metal line edges, a 30% improvement in the contrast of micropores in the passivation layer, facilitating AI defect classification. The method is deployed on the microscope's built-in FPGA acquisition card, requiring no external host.

[0153] In one embodiment, such as a general scenario, consumer-grade HDR photography, a mixed indoor / outdoor lighting scene is captured by a mobile phone with an exposure ratio of 4:1. The effect achieved using this application is that the details of the clouds outside the window and the textures of the indoor furniture are both clear, with natural color transitions.

[0154] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0155] Based on the same inventive concept, this application also provides a high dynamic range image generation apparatus for implementing the high dynamic range image generation method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more embodiments of the high dynamic range image generation apparatus provided below can be found in the limitations of the high dynamic range image generation method described above, and will not be repeated here.

[0156] In one embodiment, such as Figure 7 As shown, a high dynamic range image generation apparatus is provided, comprising: an acquisition module 100, a fusion module 200, a compression module 300, and a desaturation module 400, wherein:

[0157] The acquisition module 100 is used to acquire multiple raw images corresponding to different exposure parameters.

[0158] The fusion module 200 is used to adaptively fuse multiple original images to obtain an intermediate image.

[0159] Compression module 300 is used to perform dynamic range compression on the intermediate image to obtain a brightness-optimized image.

[0160] The desaturation module 400 is used to determine the initial red grayscale value, initial green grayscale value, and initial blue grayscale value of each pixel based on the intermediate image and the brightness optimized image, and to perform desaturation operation on the initial red grayscale value, initial green grayscale value, and initial blue grayscale value of each pixel to obtain a high dynamic range image.

[0161] The fusion module 200 is further configured to: determine, based on the first exposure image and the second exposure image, a first local brightness value of each pixel in the first exposure image, a first fusion weight value corresponding to the first local brightness value of each pixel, a second local brightness value of each pixel in the second exposure image, and a second fusion weight value corresponding to the second local brightness value of each pixel; determine the exposure ratio based on the first exposure parameters and the second exposure parameters; and determine the brightness value of each pixel in the intermediate image based on the first local brightness value of each pixel in the first exposure image, the first fusion weight value corresponding to the first local brightness value of each pixel, the second local brightness value of each pixel in the second exposure image, the second fusion weight value corresponding to the second local brightness value of each pixel, and the exposure ratio.

[0162] The fusion module 200 is further configured to: determine a first local brightness value of each pixel in the first exposed image based on the brightness value of each pixel in the first exposed image and the local range of each pixel; determine a second local brightness value of each pixel in the second exposed image based on the brightness value of each pixel in the second exposed image and the local range of each pixel; and determine a second fusion weight value corresponding to the second local brightness value of each pixel and a first fusion weight value corresponding to the first local brightness value of each pixel based on the second local brightness value of each pixel and a first lookup table; wherein the first lookup table is constructed based on the Sigmoid function.

[0163] The compression module 300 is further configured to determine the brightness value of each pixel in the intermediate image based on the intermediate image; and to determine the brightness value of each pixel in the brightness-optimized image based on the brightness value of each pixel in the intermediate image and the second lookup table.

[0164] The compression module 300 is also used to construct a low-light enhancement curve based on the ACES mapping algorithm; divide the low-light enhancement curve into a first curve and a second curve based on the suppression range threshold; adjust the second curve through an exponential function to obtain a highlight suppression curve; and construct a second lookup table based on the first curve and the highlight suppression curve.

[0165] The desaturation module 400 is further configured to determine the initial red grayscale value, initial green grayscale value, and initial blue grayscale value of each pixel in the high dynamic range image based on the brightness value of each pixel in the brightness-optimized image and the brightness value of each pixel in the intermediate image; and to perform desaturation operations on the initial red grayscale value, initial green grayscale value, and initial blue grayscale value of each pixel in the high dynamic range image to obtain the final red grayscale value, final green grayscale value, and final blue grayscale value of each pixel in the high dynamic range image.

[0166] The desaturation module 400 is further configured to determine the pixel group corresponding to each pixel in the intermediate image; each pixel group includes four pixels, corresponding to the grayscale values ​​of the red channel, the first green channel, the second green channel, and the blue channel, respectively; the total grayscale value of each pixel group is determined based on the brightness value of each pixel in the intermediate image; the initial red grayscale value of the target pixel in the high dynamic range image is determined based on the brightness value of the target pixel in the brightness-optimized image, the total grayscale value of the pixel group corresponding to the target pixel, and the red channel grayscale value of the pixel group corresponding to the target pixel; the initial green grayscale value of the target pixel in the high dynamic range image is determined based on the brightness value of the target pixel in the brightness-optimized image, the total grayscale value of the pixel group corresponding to the target pixel, the first green channel grayscale value of the pixel group corresponding to the target pixel, and the second green channel grayscale value of the pixel group corresponding to the target pixel; and the initial blue grayscale value of the target pixel in the high dynamic range image is determined based on the brightness value of the target pixel in the brightness-optimized image, the total grayscale value of the pixel group corresponding to the target pixel, and the blue channel grayscale value of the pixel group corresponding to the target pixel.

[0167] The desaturation module 400 is further configured to: determine a weighted grayscale value of a target pixel based on its initial red, initial green, and initial blue grayscale values ​​in the high dynamic range image; determine a mixing parameter of a target pixel based on its brightness value in the brightness-optimized image and a preset desaturation parameter; determine a final red grayscale value of a target pixel in the high dynamic range image based on its initial red, weighted grayscale value, and mixing parameter; determine a final green grayscale value of a target pixel in the high dynamic range image based on its initial green, weighted grayscale value, and mixing parameter; and determine a final blue grayscale value of a target pixel in the high dynamic range image based on its initial blue, weighted grayscale value, and mixing parameter.

[0168] Each module in the aforementioned high dynamic range image generation device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0169] In one embodiment, a computer device is provided, the internal structure of which can be shown as follows: Figure 8As shown, the computer device can be an FPGA in a magnifying observation device. The computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores data for running a high dynamic range image generation method. The network interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements a high dynamic range image generation method.

[0170] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0171] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement any of the high dynamic range image generation methods described above.

[0172] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements any of the high dynamic range image generation methods described above.

[0173] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0174] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0175] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for generating high dynamic range images, characterized in that, The high dynamic range image generation method is applied to a magnifying observation device, and the method includes: Acquire multiple raw images corresponding to different exposure parameters; Adaptive fusion is performed on multiple original images to obtain an intermediate image; The intermediate image is subjected to dynamic range compression to obtain a brightness-optimized image; Based on the intermediate image and the brightness-optimized image, the initial red grayscale value, initial green grayscale value, and initial blue grayscale value of each pixel are determined, and desaturation operation is performed on the initial red grayscale value, initial green grayscale value, and initial blue grayscale value of each pixel to obtain a high dynamic range image; The plurality of original images include a first exposure image corresponding to a first exposure parameter and a second exposure image corresponding to a second exposure parameter; the adaptive fusion of the plurality of original images to obtain an intermediate image includes: determining, based on the first exposure image and the second exposure image, a first local brightness value of each pixel in the first exposure image, a first fusion weight value corresponding to the first local brightness value of each pixel, a second local brightness value of each pixel in the second exposure image, and a second fusion weight value corresponding to the second local brightness value of each pixel; determining the exposure ratio based on the first exposure parameter and the second exposure parameter; and determining the brightness value of each pixel in the intermediate image based on the first local brightness value of each pixel in the first exposure image, the first fusion weight value corresponding to the first local brightness value of each pixel, the second local brightness value of each pixel in the second exposure image, the second fusion weight value corresponding to the second local brightness value of each pixel, and the exposure ratio.

2. The method according to claim 1, characterized in that, The step of determining, based on the first exposure image and the second exposure image, a first local brightness value for each pixel in the first exposure image, a first fusion weight value corresponding to the first local brightness value of each pixel, a second local brightness value for each pixel in the second exposure image, and a second fusion weight value corresponding to the second local brightness value of each pixel, includes: Based on the brightness value of each pixel in the first exposed image and the local range of each pixel, determine the first local brightness value of each pixel in the first exposed image; Based on the brightness value of each pixel in the second exposed image and the local range of each pixel, determine the second local brightness value of each pixel in the second exposed image; Based on the second local brightness value of each pixel and the first lookup table, the second fusion weight value corresponding to the second local brightness value of each pixel and the first fusion weight value corresponding to the first local brightness value of each pixel are determined; the first lookup table is constructed based on the Sigmoid function.

3. The method according to claim 1, characterized in that, The step of performing dynamic range compression on the intermediate image to obtain a brightness-optimized image includes: Based on the intermediate image, determine the brightness value of each pixel in the intermediate image; The brightness value of each pixel in the brightness-optimized image is determined based on the brightness value of each pixel in the intermediate image and the second lookup table.

4. The method according to claim 3, characterized in that, The method further includes: Construct a low-light enhancement curve based on the ACES mapping algorithm; Based on the suppression range threshold, the dark light enhancement curve is divided into a first curve and a second curve; The highlight suppression curve is obtained by adjusting the second curve using an exponential function; Based on the first curve and the highlight suppression curve, a second lookup table is constructed.

5. The method according to claim 1, characterized in that, The step of determining the initial red, green, and blue grayscale values ​​of each pixel based on the intermediate image and the brightness-optimized image, and performing desaturation operations on the initial red, green, and blue grayscale values ​​of each pixel to obtain a high dynamic range image includes: Based on the brightness values ​​of each pixel in the brightness-optimized image and the brightness values ​​of each pixel in the intermediate image, the initial red grayscale value, the initial green grayscale value, and the initial blue grayscale value of each pixel in the high dynamic range image are determined. Desaturation operations are performed on the initial red, green, and blue gray values ​​of each pixel in the high dynamic range image to obtain the final red, green, and blue gray values ​​of each pixel in the high dynamic range image.

6. The method according to claim 5, characterized in that, The step of determining the initial red, initial green, and initial blue gray values ​​of each pixel in the high dynamic range image based on the brightness values ​​of each pixel in the brightness-optimized image and the brightness values ​​of each pixel in the intermediate image includes: Determine the pixel group corresponding to each pixel in the intermediate image; each pixel group includes: four pixels, corresponding to the gray values ​​of the red channel, the first green channel, the second green channel, and the blue channel, respectively; The total grayscale value of each pixel group is determined based on the brightness value of each pixel in the intermediate image. Based on the brightness value of the target pixel in the brightness-optimized image, the total gray value of the pixel group corresponding to the target pixel, and the red channel gray value of the pixel group corresponding to the target pixel, the initial red gray value of the target pixel in the high dynamic range image is determined. Based on the brightness value of the target pixel in the brightness-optimized image, the total gray value of the pixel group corresponding to the target pixel, the first green channel gray value of the pixel group corresponding to the target pixel, and the second green channel gray value of the pixel group corresponding to the target pixel, the initial green gray value of the target pixel in the high dynamic range image is determined. Based on the brightness value of the target pixel in the brightness-optimized image, the total gray value of the pixel group corresponding to the target pixel, and the blue channel gray value of the pixel group corresponding to the target pixel, the initial blue gray value of the target pixel in the high dynamic range image is determined.

7. The method according to claim 6, characterized in that, The step of performing desaturation operations on the initial red, green, and blue gray values ​​of each pixel in the high dynamic range image to obtain the final red, green, and blue gray values ​​of each pixel in the high dynamic range image includes: The weighted gray value of the target pixel is determined based on the initial red, initial green, and initial blue gray values ​​of the target pixel in the high dynamic range image. The mixing parameters of the target pixel are determined based on the brightness value of the target pixel in the brightness-optimized image and the preset desaturation parameters; The final red gray value of the target pixel in the high dynamic range image is determined based on the initial red gray value of the target pixel, the weighted gray value of the target pixel, and the mixing parameters of the target pixel. The final green gray value of the target pixel in the high dynamic range image is determined based on the initial green gray value of the target pixel, the weighted gray value of the target pixel, and the mixing parameters of the target pixel. The final blue gray value of the target pixel in the high dynamic range image is determined based on the initial blue gray value of the target pixel, the weighted gray value of the target pixel, and the mixing parameters of the target pixel.

8. A high dynamic range image generation apparatus, characterized in that, The device includes: The acquisition module is used to acquire multiple raw images corresponding to different exposure parameters; The fusion module is used to adaptively fuse multiple original images to obtain an intermediate image; The compression module is used to perform dynamic range compression on the intermediate image to obtain a brightness-optimized image; The desaturation module is used to determine the initial red, initial green, and initial blue gray values ​​of each pixel based on the intermediate image and the brightness-optimized image, and to perform desaturation operations on the initial red, initial green, and initial blue gray values ​​of each pixel to obtain a high dynamic range image. The plurality of original images include a first exposure image corresponding to a first exposure parameter and a second exposure image corresponding to a second exposure parameter; the adaptive fusion of the plurality of original images to obtain an intermediate image includes: determining, based on the first exposure image and the second exposure image, a first local brightness value of each pixel in the first exposure image, a first fusion weight value corresponding to the first local brightness value of each pixel, a second local brightness value of each pixel in the second exposure image, and a second fusion weight value corresponding to the second local brightness value of each pixel; determining the exposure ratio based on the first exposure parameter and the second exposure parameter; and determining the brightness value of each pixel in the intermediate image based on the first local brightness value of each pixel in the first exposure image, the first fusion weight value corresponding to the first local brightness value of each pixel, the second local brightness value of each pixel in the second exposure image, the second fusion weight value corresponding to the second local brightness value of each pixel, and the exposure ratio.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

Images