A high dynamic range image processing method and apparatus

Abstract

This invention provides a high dynamic range (HDR) image processing method and apparatus. The method includes: adjusting the exposure time and gain parameters of an image acquisition module so that long frames capture details in dark areas, short frames capture details in bright areas, and medium frames are positioned between the long and short frames to achieve balance; acquiring Bayer image data of the long, medium, and short frames using the image acquisition module; inputting the image data into an LED flicker detection device, which performs flicker detection and removes flickering areas; and inputting flicker-free areas into an HDR fusion module, which synthesizes different frames to generate a HDR image. This invention, by introducing LED flicker detection, effectively reduces the negative impact of LED flicker on image quality and improves the quality of HDR images by detecting flicker in the image data and removing flickering areas.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a high dynamic range image processing method and apparatus. Background Technology

[0002] Light-emitting diodes (LEDs) are widely used as a light source in various fields such as traffic lights, signs, and automotive lights. The driving method of LEDs typically involves rapid on / off switching, but due to the persistence of vision in the human eye, this switching frequency often exceeds the range of human perception. When a sensor performs imaging, if the exposure time is set shorter than the LED's on / off cycle, the sensor may only capture the LED in its off state. This can lead to misidentification of the LED's state, thereby misleading the judgment of advanced driver assistance systems (ADAS) and reducing its accuracy.

[0003] High Dynamic Range (HDR) imaging technology plays a crucial role in automotive image sensors. In certain dynamic scenes, such as when a vehicle enters or exits a tunnel, there are simultaneously distinct bright and dark areas, making it difficult to clearly capture both regions with a single exposure time. HDR technology combines multiple image frames with different exposure parameters to cover a wider range of brightness. It uses short exposure times to capture details in bright areas and long exposure times to capture details in dark areas, ultimately synthesizing an image with a wider dynamic range and richer details.

[0004] Traditional methods for suppressing flicker typically require setting the image sensor's frame rate to a value divisible by the LED flicker frequency or setting the exposure time to an integer multiple of the LED flicker cycle. However, in reality, the frequencies of LED light sources used in vehicle lights, roads, and traffic signs are not fixed. Different frequencies of LED light sources may exist in the same scene, making it impossible to avoid flickering simply by setting appropriate frequencies and exposure times. Furthermore, when multiple LED light sources with different flicker frequencies exist in a scene, it is also impossible to simultaneously prevent flickering.

[0005] Furthermore, to avoid capturing the LED off state, an increased exposure time is required, but this leads to overexposure of bright areas in the image; conversely, a shorter exposure time increases the likelihood of capturing the LED off state. Overexposure in long frames combined with capturing the LED off state in short frames results in a reduced dynamic range. Therefore, avoiding flicker in HDR imaging is a challenging task and a crucial performance indicator for automotive image sensors. Summary of the Invention

[0006] The purpose of this invention is to provide a high dynamic range image processing method and apparatus. By introducing an LED flicker detection device, flickering areas are detected and removed, thereby effectively reducing the negative impact of LED flicker on image quality and improving the quality of high dynamic range images.

[0007] To address the aforementioned technical problems, according to a first aspect of the present invention, a high dynamic range image processing method is provided, comprising the following steps:

[0008] The exposure time and gain parameters of the image acquisition module are adjusted so that long frames capture details in dark areas, short frames capture details in bright areas, and medium frames are located between the long frames and the short frames to achieve a balance.

[0009] The image acquisition module is used to acquire Bayer image data in long, medium, and short frames;

[0010] The image data is input into an LED flicker detection device, which performs flicker detection and removes areas where flickering occurs; and

[0011] The area without flicker is input into the HDR fusion module, which synthesizes different frames to generate a high dynamic range image.

[0012] Optionally, the method for acquiring three frames of image data (long frame, medium frame, and short frame) using the image acquisition module includes: acquiring four Bayer images, and acquiring Bayer image data of the long frame, medium frame, and short frame at the same time based on the four Bayer images.

[0013] Optionally, after adjusting the exposure time and gain parameters of the image acquisition module and before completing the detection of the flickering area, the processing method further includes:

[0014] Calculate the exposure values ​​of the long frame, the medium frame, and the short frame;

[0015] Calculate the exposure value difference between different frames and set an exposure value difference threshold; and

[0016] Set the saturation threshold for the pixels of the long frame and the medium frame.

[0017] Optionally, the image data is input into an LED flicker detection device, and the method by which the LED flicker detection device performs flicker detection and removes flickering areas includes:

[0018] Determine whether the difference in first exposure value between the long frame and the short frame is greater than an exposure value difference threshold;

[0019] If the first exposure value difference is greater than the exposure value difference threshold, it is determined that the pixel data of the short frame is flickering, the pixel data of the short frame is discarded, and it is determined whether the second exposure value difference between the long frame and the medium frame is greater than the exposure value difference threshold.

[0020] If the second exposure value difference is greater than the exposure value difference threshold, it is determined that the pixel data of the middle frame is flickering, the pixel data of the middle frame is discarded, and the pixel data of the long frame is used for subsequent operations.

[0021] If the second exposure value difference is less than or equal to the exposure value difference threshold, it is determined that the pixel data of the intermediate frame does not flicker, and the pixel data of the intermediate frame is used for subsequent operations.

[0022] If the first exposure value difference is less than or equal to the exposure value difference threshold, it is determined that the pixel data of the short frame does not flicker, the pixel data of the short frame is retained, and it is determined whether the pixel data of the long frame is saturated.

[0023] If the pixel data of the long frame is not saturated, then the pixel data of the long frame is used for subsequent operations;

[0024] If the pixel data of the long frame is saturated, then determine whether the difference in the third exposure value between the medium frame and the short frame is greater than the exposure value difference threshold.

[0025] If the third exposure value difference is greater than the exposure value difference threshold, it is determined that the pixel data of the short frame is flickering, the pixel data of the short frame is discarded, and the pixel data of the medium frame is used for subsequent operations.

[0026] If the third exposure value difference is less than or equal to the exposure value difference threshold, it is determined that the pixel data of the short frame does not flicker, the pixel data of the short frame is retained, and it is determined whether the pixel data of the medium frame has reached saturation.

[0027] If the pixel data of the intermediate frame is not saturated, then the pixel data of the intermediate frame is used for subsequent operations;

[0028] If the pixel data of the intermediate frame is saturated, then the fused pixels of the intermediate frame and the short frame are used for subsequent operations.

[0029] Optionally, the image data is input into an LED flicker detection device, and the method by which the LED flicker detection device performs flicker detection and removes flickering areas includes:

[0030] Determine whether the difference in first exposure value between the long frame and the short frame is greater than an exposure value difference threshold;

[0031] If the first exposure value difference is greater than the exposure value difference threshold, it is determined that the pixel data of the short frame is flickering, the pixel data of the short frame is discarded, and it is determined whether the second exposure value difference between the long frame and the medium frame is greater than the exposure value difference threshold.

[0032] If the second exposure value difference is greater than the exposure value difference threshold, it is determined that the pixel data of the middle frame is flickering, the pixel data of the middle frame is discarded, and the pixel data of the long frame is used for subsequent operations.

[0033] If the second exposure value difference is less than or equal to the exposure value difference threshold, it is determined that the pixel data of the intermediate frame does not flicker, and the pixel data of the intermediate frame is used for subsequent operations.

[0034] If the first exposure value difference is less than or equal to the exposure value difference threshold, it is determined that the pixel data of the short frame does not flicker, the pixel data of the short frame is retained, and it is determined whether the pixel data of the long frame is saturated.

[0035] If the pixel data of the long frame is not saturated, then the pixel data of the long frame is used for subsequent operations;

[0036] If the pixel data of the long frame is saturated, then determine whether the pixel data of the medium frame has reached saturation;

[0037] If the pixel data of the intermediate frame is not saturated, then the pixel data of the intermediate frame is used for subsequent operations;

[0038] If the pixel data of the intermediate frame is saturated, then determine whether the difference in the third exposure value between the intermediate frame and the short frame is greater than the exposure value difference threshold.

[0039] If the third exposure value difference is greater than the exposure value difference threshold, it is determined that the pixel data of the short frame is flickering, the pixel data of the short frame is discarded, and the pixel data of the medium frame is used for subsequent operations.

[0040] If the third exposure value difference is less than or equal to the exposure value difference threshold, it is determined that the pixel data of the short frame does not flicker, and the fused pixels of the medium frame and the short frame are used for subsequent operations.

[0041] Optionally, the image data is input into an LED flicker detection device, and the method by which the LED flicker detection device performs flicker detection and removes flickering areas includes:

[0042] Determine whether the pixel data of the long frame is saturated;

[0043] If the pixel data of the long frame is not saturated, then the pixel data of the long frame is used for subsequent operations;

[0044] If the pixel data of the long frame is saturated, then determine whether the difference in the first exposure value between the long frame and the short frame is greater than the exposure value difference threshold.

[0045] If the first exposure value difference is greater than the exposure value difference threshold, it is determined that the pixel data of the short frame is flickering, the pixel data of the short frame is discarded, and it is determined whether the second exposure value difference between the long frame and the medium frame is greater than the exposure value difference threshold.

[0046] If the second exposure value difference is greater than the exposure value difference threshold, it is determined that the pixel data of the middle frame is flickering, the pixel data of the middle frame is discarded, and the pixel data of the long frame is used for subsequent operations.

[0047] If the second exposure value difference is less than or equal to the exposure value difference threshold, it is determined that the pixel data of the intermediate frame does not flicker, and the pixel data of the intermediate frame is used for subsequent operations.

[0048] If the first exposure value difference is less than or equal to the exposure value difference threshold, it is determined that the pixel data of the short frame does not flicker, the pixel data of the short frame is retained, and it is determined whether the third exposure value difference between the medium frame and the short frame is greater than the exposure value difference threshold.

[0049] If the third exposure value difference is greater than the exposure value difference threshold, it is determined that the pixel data of the short frame is flickering, the pixel data of the short frame is discarded, and the pixel data of the medium frame is used for subsequent operations.

[0050] If the third exposure value difference is less than or equal to the exposure value difference threshold, it is determined that the pixel data of the short frame does not flicker, the pixel data of the short frame is retained, and it is determined whether the pixel data of the medium frame has reached saturation.

[0051] If the pixel data of the intermediate frame is not saturated, then the pixel data of the intermediate frame is used for subsequent operations;

[0052] If the pixel data of the intermediate frame is saturated, then the fused pixels of the intermediate frame and the short frame are used for subsequent operations.

[0053] Optionally, the image data is input into an LED flicker detection device, and the method by which the LED flicker detection device performs flicker detection and removes flickering areas includes:

[0054] Determine whether the pixel data of the long frame is saturated;

[0055] If the pixel data of the long frame is not saturated, then the pixel data of the long frame is used for subsequent operations;

[0056] If the pixel data of the long frame is saturated, then determine whether the difference in the first exposure value between the long frame and the short frame is greater than the exposure value difference threshold.

[0057] If the first exposure value difference is greater than the exposure value difference threshold, it is determined that the pixel data of the short frame is flickering, the pixel data of the short frame is discarded, and it is determined whether the second exposure value difference between the long frame and the medium frame is greater than the exposure value difference threshold.

[0058] If the second exposure value difference is greater than the exposure value difference threshold, it is determined that the pixel data of the middle frame is flickering, the pixel data of the middle frame is discarded, and the pixel data of the long frame is used for subsequent operations.

[0059] If the second exposure value difference is less than or equal to the exposure value difference threshold, it is determined that the pixel data of the intermediate frame does not flicker, and the pixel data of the intermediate frame is used for subsequent operations.

[0060] If the first exposure value difference is less than or equal to the exposure value difference threshold, it is determined that the pixel data of the short frame does not flicker, the pixel data of the short frame is retained, and it is determined whether the pixel data of the medium frame has reached saturation.

[0061] If the pixel data of the intermediate frame is not saturated, then the pixel data of the intermediate frame is used for subsequent operations;

[0062] If the pixel data of the intermediate frame is saturated, then determine whether the difference in the third exposure value between the intermediate frame and the short frame is greater than the exposure value difference threshold.

[0063] If the third exposure value difference is greater than the exposure value difference threshold, it is determined that the pixel data of the short frame is flickering, the pixel data of the short frame is discarded, and the pixel data of the medium frame is used for subsequent operations.

[0064] If the third exposure value difference is less than or equal to the exposure value difference threshold, it is determined that the pixel data of the short frame does not flicker, the pixel data of the short frame is retained, and the fused pixels of the medium frame and the short frame are used for subsequent operations.

[0065] Optionally, the method for determining whether a pixel is saturated is: when the pixel brightness value is greater than the saturation threshold, the pixel is determined to be saturated; wherein, the saturation threshold is 90% of the full-well capacity of the image sensor.

[0066] Optionally, the flicker-free region is input into the HDR fusion module, and the method by which the HDR fusion module synthesizes different frames to generate a high dynamic range image includes:

[0067] Calculate the inter-frame fusion weights;

[0068] The image data is subjected to wide dynamic range image merging, transforming the image from low bit width to high bit width; and

[0069] The high bit width image is tone-mapped back to its original bit width.

[0070] Optionally, methods for calculating inter-frame fusion weights include: calculating the gradient information of the image, calculating the saturation of the image, and calculating the brightness of the image.

[0071] Optionally, after generating the high dynamic range image, the method further includes processing the high dynamic range image using an image processing pipeline.

[0072] Optionally, the image processing pipeline includes processing methods such as: demosaicing, color correction matrix, gamma correction, and color space conversion.

[0073] Optionally, the exposure time of the long frame is greater than or equal to the reciprocal of the LED flashing frequency.

[0074] To solve the above-mentioned technical problems, according to a second aspect of the present invention, a high dynamic range image processing apparatus is provided for implementing the high dynamic range image processing method as described above, the high dynamic range image processing apparatus comprising:

[0075] An adjustment device is used to adjust the exposure time and gain parameters of the image acquisition module so that long frames capture details in dark areas, short frames capture details in bright areas, and medium frames are located between the long frames and the short frames to achieve a balance.

[0076] Image acquisition module, used to acquire Bayer image data in long, medium and short frames;

[0077] An LED flicker detection device receives the image data and is used to perform flicker detection and remove areas where flickering occurs; and

[0078] The HDR fusion module receives flicker-free areas and uses them to synthesize different frames to generate high dynamic range images.

[0079] In summary, the high dynamic range image processing method and apparatus provided by this invention first adjusts the exposure time and gain parameters of the image acquisition module so that long frames capture details in dark areas, short frames capture details in bright areas, and medium frames are positioned between the long and short frames to achieve balance. Then, the image acquisition module acquires Bayer image data for long, medium, and short frames. Next, the image data is input into an LED flicker detection device, which performs flicker detection and removes flickering areas. Finally, flicker-free areas are input into an HDR fusion module, which synthesizes different frames to generate a high dynamic range image. This invention, by introducing LED flicker detection, completes flicker detection of image data and removes flickering areas, thereby effectively mitigating the negative impact of LED flicker on image quality and improving the quality of high dynamic range images. Furthermore, the LED flicker detection of this invention can directly process Bayer image data. Once Bayer image data for long, medium, and short frames is acquired, the flickering area detection process can be started immediately without storing previous image data, significantly reducing the need for buffer space and thus optimizing storage efficiency.

[0080] Furthermore, the method provided by this invention does not rely on the pre-known flicker frequency of the LED light source to be recovered. By setting the minimum exposure time limit of the long frame, the minimum detection frequency can be flexibly set, enabling the method to adapt to and handle LED light sources of different frequencies that may exist in the same scene.

[0081] Furthermore, the flicker detection method of the present invention mainly involves basic addition and multiplication operations, thus requiring less computation. This characteristic not only simplifies the complexity of the algorithm but also reduces the cost of hardware implementation, making the present invention particularly suitable for resource-constrained systems. Attached Figure Description

[0082] Those skilled in the art will understand that the accompanying drawings are provided to better understand the invention and do not constitute any limitation on the scope of the invention. Wherein:

[0083] Figure 1 This is a flowchart of a high dynamic range image processing method provided in an embodiment of the present invention.

[0084] Figure 2 This is a schematic diagram of a four-bayer image acquired according to an embodiment of the present invention.

[0085] Figure 3 It is by Figure 2 The diagram shows a long frame obtained from the four Bayer images.

[0086] Figure 4 It is by Figure 2A schematic diagram of the medium frame obtained from the four Bayer images shown.

[0087] Figure 5 It is by Figure 2 A schematic diagram of a short frame obtained from the four Bayer images shown.

[0088] Figure 6 This is a flowchart of completing flicker detection and removing flickering areas provided in Embodiment 1 of the present invention.

[0089] Figure 7 This is a flowchart of completing flicker detection and removing flickering areas provided in Embodiment 2 of the present invention.

[0090] Figure 8 This is a flowchart of completing flicker detection and removing flickering areas provided in Embodiment 3 of the present invention.

[0091] Figure 9 This is a flowchart of completing flicker detection and removing flickering areas provided in Embodiment 4 of the present invention. Detailed Implementation

[0092] To make the objectives, advantages, and features of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the drawings are all in a very simplified form and are not drawn to scale, and are only used to facilitate and clarify the explanation of the embodiments of this invention. Furthermore, the structures shown in the drawings are often part of the actual structures. In particular, different figures may emphasize different aspects and may sometimes use different scales.

[0093] As used herein, the singular forms “a,” “an,” and “the” include plural objects unless otherwise expressly indicated. As used herein, the term “or” is generally used to include “and / or” unless otherwise expressly indicated. As used herein, the term “a number” is generally used to include “at least one” unless otherwise expressly indicated. As used herein, the term “at least two” is generally used to include “two or more” unless otherwise expressly indicated. Furthermore, the terms “first,” “second,” and “third” are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as “first,” “second,” or “third” may explicitly or implicitly include one or at least two of that feature.

[0094] Figure 1 This is a flowchart of a high dynamic range image processing method provided in an embodiment of the present invention. Please refer to it. Figure 1As shown, the high dynamic range image processing method provided in this embodiment includes the following steps:

[0095] Step S1: Adjust the exposure time and gain parameters of the image acquisition module so that the long frame captures details in the dark area, the short frame captures details in the bright area, and the medium frame is located between the long frame and the short frame to achieve a balance.

[0096] Step S2: Use the image acquisition module to acquire Bayer image data in long, medium and short frames;

[0097] Step S3: The image data is input into the LED flicker detection device, which performs flicker detection and removes areas where flickering occurs; and

[0098] Step S4: Input the area without flicker into the HDR fusion module, which synthesizes different frames to generate a high dynamic range image.

[0099] This invention introduces LED flicker detection to detect flicker in image data and remove flickering areas, thereby effectively mitigating the negative impact of LED flicker on image quality and improving the quality of high dynamic range images. Furthermore, the LED flicker detection of this invention can directly process Bayer image data. Once long, medium, and short frame Bayer image data are acquired, the flicker detection process can be started immediately without storing previous image data, significantly reducing the need for buffer space and thus optimizing storage efficiency.

[0100] The following describes each step of the high dynamic range image processing method provided in this embodiment.

[0101] In step S1, the exposure time and gain parameters of the image acquisition module are adjusted so that long frames capture details in dark areas, short frames capture details in bright areas, and medium frames are located between the long frames and the short frames to achieve a balance.

[0102] In this embodiment, a long frame refers to a frame with a long exposure time and gain, a short frame refers to a frame with a short exposure time and gain, and a medium frame refers to a frame with a medium exposure time and gain. By adjusting the exposure time and gain parameters of the image acquisition module, the long frame can capture details in dark areas, the short frame can capture details in bright areas, and the medium frame achieves a balance between the long frame and the short frame to retain an appropriate amount of details in both dark and bright areas.

[0103] To ensure flicker-free capture of the LED light source using long frames, the exposure time must be at least one LED flicker cycle. However, this may lead to image saturation in the LED light source area. Short frames, with their shorter exposure times, are more prone to flicker. LED flicker detection relies primarily on long frames, while dynamic range expansion depends on short frames. In this scenario, pixel data from both long and short frames may be lost, making it difficult to achieve a balance between flicker recovery and dynamic range expansion. To address this issue, this invention introduces a medium frame, which serves as an exposure frame between the long and short frames, providing an additional data reference point. The data from the medium frame not only helps recover details in the flickering area but also assists in expanding the dynamic range, thereby achieving a better balance between flicker recovery and dynamic range.

[0104] In one embodiment of the present invention, the exposure time and gain of the long frame, the medium frame, and the short frame are calculated using an automatic exposure control algorithm. In another embodiment of the present invention, the exposure time and gain of the long frame, the medium frame, and the short frame can be controlled by manual setting.

[0105] In one embodiment of the present invention, to ensure that the long frame does not flicker when capturing the LED light source, the exposure time of the long frame must be greater than or equal to the LED flicker period, or greater than or equal to the reciprocal of the LED flicker frequency. This avoids flickering on the long frame and provides reliable image data for the recovery of flickering areas. Taking a 90Hz LED flicker frequency as an example, the exposure time of the long frame should be set to at least 11.1 milliseconds. In this embodiment, the exposure time and gain of the long frame, medium frame, and short frame need to be recorded simultaneously. This recorded data will be used to calculate the interrelationship of pixel values ​​between different frames, thereby accurately determining the flickering status of the LED light source.

[0106] In this embodiment, the exposure time is set first. If the exposure requirement cannot be met even if the maximum exposure time is reached, the gain can also be set. The final exposure value is represented by the product of the exposure time and the gain.

[0107] In step S2, the image acquisition module is used to acquire Bayer image data in long, medium and short frames.

[0108] The long frame has the highest exposure value, the medium frame has the next highest exposure value, and the short frame has the lowest exposure value. It can be acquired synchronously or serially.

[0109] In one embodiment of the present invention, the method for acquiring three frames of image data (long frame, medium frame, and short frame) using the image acquisition module includes: acquiring a quad Bayer image, and acquiring Bayer image data of the long frame, medium frame, and short frame at the same time based on the quad Bayer image. That is, frames with different exposure values ​​are obtained through quad Bayer technology. LED flicker detection depends on the relationship between the exposure values ​​of the long frame, medium frame, and short frame, which includes, but is not limited to, the ratio of brightness values ​​between frames and the absolute and relative differences in brightness values ​​between frames. The ratio between each exposure is not fixed and needs to be reasonably set according to hardware conditions to achieve a better fusion effect.

[0110] Figure 2 This is a schematic diagram of a four-bayer image acquired according to an embodiment of the present invention. Please refer to... Figure 2 As shown, the four Bayer images are 4×4 images in BGGR format, with adjacent four pixels being color filters of the same color. Three exposure levels are set within a 2×2 pixel area to obtain long-frame, medium-frame, and short-frame Bayer images, where... Figure 3 It is by Figure 2 The diagram shown is a long frame obtained from the four Bayer images. Figure 4 It is by Figure 2 The diagram shown is a schematic of the medium frame obtained from the four Bayer images. Figure 5 It is by Figure 2 The diagram shows a short frame obtained from a four-Bayer image. Please refer to... Figures 3 to 5 As shown, the four Bayer images can be used to obtain Bayer image data of long frames, medium frames, and short frames at the same time.

[0111] In one embodiment of the present invention, after adjusting the exposure time and gain parameters of the image acquisition module (i.e., step S1) and before completing the detection of the flickering area (i.e., step S3), the method further includes: calculating the exposure values ​​of the long frame, the medium frame, and the short frame; calculating the exposure value difference between different frames and setting an exposure value difference threshold; and setting a saturation threshold for pixel saturation of the long frame and the medium frame.

[0112] First, the exposure values ​​of the long frame, the medium frame, and the short frame are calculated. The exposure value is determined by the exposure time and gain, ExpVal = ExpTime * Gain, where ExpVal represents the exposure value, ExpTime represents the exposure time, and Gain represents the gain. The ratio of the exposure values ​​of the long frame to the medium frame, and the ratio of the exposure values ​​of the medium frame to the short frame, are calculated from the exposure values ​​of the three frames. The ratio of the exposure values ​​represents the relationship between the pixel brightness values ​​of the three frames in the unsaturated region.

[0113] Next, the exposure value difference between different frames is calculated. By comparing the pixel brightness values ​​of long, medium, and short frames, the relative brightness changes between them can be quantified. To quantify these relative brightness changes, an exposure value difference threshold is set to assess whether LED flickering exists between medium and short frames. Because medium or short frames have shorter exposure times, they are more likely to capture the off state of the LED light source. In this case, the exposure value difference between long frames and medium or short frames will be significantly greater than the preset exposure value difference threshold. Therefore, if the calculated exposure value difference exceeds the exposure value difference threshold, the system will determine that flickering exists.

[0114] Furthermore, a saturation threshold is set for the pixel saturation of the long frame and the medium frame. Theoretically, pixel brightness value has a linear relationship with exposure time. Determining whether a pixel is saturated depends on the full-well capacity of the image sensor. In this embodiment, 90% of the maximum full-well capacity is set as the saturation threshold. When the pixel brightness value is greater than this saturation threshold, the pixel is determined to be saturated; when the pixel brightness value is less than or equal to the saturation threshold, the pixel is determined not to be saturated. In practice, the saturation threshold needs to be adjusted based on the linear relationship between the image sensor's exposure time and pixel brightness value.

[0115] Subsequently, if the pixel data of a long frame reaches saturation, the long frame data is discarded; if the pixel data of the long frame does not reach saturation, the long frame data is used; if the medium frame reaches saturation, the medium frame and short frame data are fused; if the medium frame does not reach saturation, the medium frame data is used.

[0116] In step S3, the image data is input into the LED flicker detection device, which performs flicker detection and removes areas where flickering occurs.

[0117] In this embodiment, the presence of flickering can be determined by comparing the exposure value difference between different frames with the exposure value difference threshold. Frames exhibiting flickering are discarded. Simultaneously, pixel saturation is assessed to determine whether to use the pixel data of that frame. Exposure value difference and saturation can be cross-checked. The following four embodiments illustrate this, but the order of judgment described in these embodiments is not limited to those examples.

[0118] Figure 6 This is a flowchart illustrating the process of detecting and removing flickering areas according to Embodiment 1 of the present invention. Please refer to... Figure 6 As shown, the method by which the LED flicker detection device completes flicker detection and removes flickering areas includes the following steps:

[0119] In step S301, it is determined whether the first exposure value difference between the long frame and the short frame is greater than the exposure value difference threshold. In this embodiment, the exposure value difference between the long frame and the short frame is used as the first exposure value difference to distinguish it from the exposure value difference between the long frame and the medium frame, and the exposure value difference between the medium frame and the short frame.

[0120] If the first exposure value difference is greater than the exposure value difference threshold, then it is determined that the pixel data of the short frame is flickering, the pixel data of the short frame is discarded, and step S302 is executed.

[0121] In step S302, it is determined whether the difference in second exposure values ​​between the long frame and the medium frame is greater than the exposure value difference threshold. In this embodiment, the difference in exposure values ​​between the long frame and the medium frame is used as the second exposure value difference.

[0122] If the second exposure value difference is greater than the exposure value difference threshold, then it is determined that the pixel data of the middle frame is flickering, the pixel data of the middle frame is discarded, step S303 is executed, and the pixel data of the long frame is used for subsequent operations.

[0123] If the second exposure value difference is less than or equal to the exposure value difference threshold, that is, if not, then it is determined that the pixel data of the intermediate frame does not flicker, and step S304 is executed to use the pixel data of the intermediate frame for subsequent operations.

[0124] In step S301, if the first exposure value difference is less than or equal to the exposure value difference threshold, that is, if not, then it is determined that the pixel data of the short frame does not flicker, the pixel data of the short frame is retained, and step S305 is executed.

[0125] In step S305, it is determined whether the pixel data of the long frame is saturated. If the pixel data of the long frame is not saturated, i.e., if not, then step S303 is executed, and the pixel data of the long frame is used for subsequent operations. If the pixel data of the long frame is saturated, i.e., if so, then step S306 is executed.

[0126] In step S306, it is determined whether the difference in third exposure value between the medium frame and the short frame is greater than the exposure value difference threshold. In this embodiment, the difference in exposure value between the medium frame and the short frame is used as the third exposure value difference.

[0127] If the third exposure value difference is greater than the exposure value difference threshold, then it is determined that the pixel data of the short frame flickers, the pixel data of the short frame is discarded, and step S304 is executed, using the pixel data of the medium frame for subsequent operations. If the third exposure value difference is less than or equal to the exposure value difference threshold, then it is determined that the pixel data of the short frame does not flicker, the pixel data of the short frame is retained, and step S307 is executed.

[0128] In step S307, it is determined whether the pixel data of the intermediate frame has reached saturation. If the pixel data of the intermediate frame is not saturated, i.e., if not, then step S304 is executed, and the pixel data of the intermediate frame is used for subsequent operations. If the pixel data of the intermediate frame is saturated, i.e., if yes, then step S308 is executed.

[0129] In step S308, the fused pixels of the medium frame and the short frame are used for subsequent operations.

[0130] Figure 7 This is a flowchart illustrating the process of detecting and removing flickering areas according to Embodiment 2 of the present invention. Please refer to... Figure 7 As shown, the method by which the LED flicker detection device completes flicker detection and removes flickering areas includes the following steps:

[0131] First, determine whether the difference in the first exposure value between the long frame and the short frame is greater than the exposure value difference threshold, that is, execute step S301.

[0132] If the first exposure value difference is greater than the exposure value difference threshold, then it is determined that the pixel data of the short frame is flickering, the pixel data of the short frame is discarded, and step S302 is executed.

[0133] In step S302, it is determined whether the difference in the second exposure value between the long frame and the medium frame is greater than the exposure value difference threshold.

[0134] If the second exposure value difference is greater than the exposure value difference threshold, then it is determined that the pixel data of the middle frame is flickering, the pixel data of the middle frame is discarded, step S303 is executed, and the pixel data of the long frame is used for subsequent operations.

[0135] If the second exposure value difference is less than or equal to the exposure value difference threshold, that is, if not, then it is determined that the pixel data of the intermediate frame does not flicker, and step S304 is executed to use the pixel data of the intermediate frame for subsequent operations.

[0136] In step S301, if the first exposure value difference is less than or equal to the exposure value difference threshold, that is, if not, then it is determined that the pixel data of the short frame does not flicker, the pixel data of the short frame is retained, and step S305 is executed.

[0137] In step S305, it is determined whether the pixel data of the long frame is saturated. If the pixel data of the long frame is not saturated, i.e., if not, then step S303 is executed, and the pixel data of the long frame is used for subsequent operations. If the pixel data of the long frame is saturated, i.e., if so, then step S307 is executed.

[0138] In step S307, it is determined whether the pixel data of the intermediate frame has reached saturation. If the pixel data of the intermediate frame is not saturated, i.e., if not, then step S304 is executed, and the pixel data of the intermediate frame is used for subsequent operations. If the pixel data of the intermediate frame is saturated, i.e., if yes, then step S306 is executed.

[0139] In step S306, it is determined whether the difference in the third exposure value between the medium frame and the short frame is greater than the exposure value difference threshold.

[0140] If the third exposure value difference is greater than the exposure value difference threshold, then it is determined that the pixel data of the short frame flickers, the pixel data of the short frame is discarded, and step S304 is executed, using the pixel data of the medium frame for subsequent operations. If the third exposure value difference is less than or equal to the exposure value difference threshold, then it is determined that the pixel data of the short frame does not flicker, the pixel data of the short frame is retained, and step S308 is executed.

[0141] In step S308, the fused pixels of the medium frame and the short frame are used for subsequent operations.

[0142] Figure 8 This is a flowchart illustrating the process of detecting and removing flickering areas, provided in Embodiment 3 of the present invention. Please refer to... Figure 8 As shown, the method by which the LED flicker detection device completes flicker detection and removes flickering areas includes the following steps:

[0143] First, determine whether the long frame is saturated, i.e., execute step S305.

[0144] If the pixel data of the long frame is not saturated, i.e., if not, then step S303 is executed, and the pixel data of the long frame is used for subsequent operations. If the pixel data of the long frame is saturated, i.e., if so, then step S301 is executed.

[0145] In step S301, it is determined whether the difference in the first exposure value between the long frame and the short frame is greater than the exposure value difference threshold.

[0146] If the first exposure value difference is greater than the exposure value difference threshold, then it is determined that the pixel data of the short frame is flickering, the pixel data of the short frame is discarded, and step S302 is executed.

[0147] In step S302, it is determined whether the difference in the second exposure value between the long frame and the medium frame is greater than the exposure value difference threshold.

[0148] If the second exposure value difference is greater than the exposure value difference threshold, then it is determined that the pixel data of the middle frame is flickering, the pixel data of the middle frame is discarded, step S303 is executed, and the pixel data of the long frame is used for subsequent operations.

[0149] If the second exposure value difference is less than or equal to the exposure value difference threshold, that is, if not, then it is determined that the pixel data of the intermediate frame does not flicker, and step S304 is executed to use the pixel data of the intermediate frame for subsequent operations.

[0150] In step S301, if the first exposure value difference is less than or equal to the exposure value difference threshold, that is, if not, then it is determined that the pixel data of the short frame does not flicker, the pixel data of the short frame is retained, and step S306 is executed.

[0151] In step S306, it is determined whether the difference in the third exposure value between the medium frame and the short frame is greater than the exposure value difference threshold.

[0152] If the third exposure value difference is greater than the exposure value difference threshold, then it is determined that the pixel data of the short frame flickers, the pixel data of the short frame is discarded, and step S304 is executed, using the pixel data of the medium frame for subsequent operations. If the third exposure value difference is less than or equal to the exposure value difference threshold, then it is determined that the pixel data of the short frame does not flicker, the pixel data of the short frame is retained, and step S307 is executed.

[0153] In step S307, it is determined whether the pixel data of the intermediate frame has reached saturation. If the pixel data of the intermediate frame is not saturated, i.e., if not, then step S304 is executed, and the pixel data of the intermediate frame is used for subsequent operations. If the pixel data of the intermediate frame is saturated, i.e., if yes, then step S308 is executed.

[0154] In step S308, the fused pixels of the medium frame and the short frame are used for subsequent operations.

[0155] Figure 9 This is a flowchart illustrating the process of completing flicker detection and removing flickering areas, as provided in Embodiment 4 of the present invention. Please refer to... Figure 9 As shown, the method by which the LED flicker detection device completes flicker detection and removes flickering areas includes the following steps:

[0156] First, determine whether the long frame is saturated, i.e., execute step S305.

[0157] If the pixel data of the long frame is not saturated, i.e., if not, then step S303 is executed, and the pixel data of the long frame is used for subsequent operations. If the pixel data of the long frame is saturated, i.e., if so, then step S301 is executed.

[0158] In step S301, it is determined whether the difference in the first exposure value between the long frame and the short frame is greater than the exposure value difference threshold.

[0159] If the first exposure value difference is greater than the exposure value difference threshold, then it is determined that the pixel data of the short frame is flickering, the pixel data of the short frame is discarded, and step S302 is executed.

[0160] In step S302, it is determined whether the difference in the second exposure value between the long frame and the medium frame is greater than the exposure value difference threshold.

[0161] If the second exposure value difference is greater than the exposure value difference threshold, then it is determined that the pixel data of the middle frame is flickering, the pixel data of the middle frame is discarded, step S303 is executed, and the pixel data of the long frame is used for subsequent operations.

[0162] If the second exposure value difference is less than or equal to the exposure value difference threshold, that is, if not, then it is determined that the pixel data of the intermediate frame does not flicker, and step S304 is executed to use the pixel data of the intermediate frame for subsequent operations.

[0163] In step S301, if the first exposure value difference is less than or equal to the exposure value difference threshold, that is, if not, then it is determined that the pixel data of the short frame does not flicker, the pixel data of the short frame is retained, and step S307 is executed.

[0164] In step S307, it is determined whether the pixel data of the intermediate frame has reached saturation. If the pixel data of the intermediate frame is not saturated, i.e., if not, then step S304 is executed, and the pixel data of the intermediate frame is used for subsequent operations. If the pixel data of the intermediate frame is saturated, i.e., if yes, then step S306 is executed.

[0165] In step S306, it is determined whether the difference in the third exposure value between the medium frame and the short frame is greater than the exposure value difference threshold.

[0166] If the third exposure value difference is greater than the exposure value difference threshold, then it is determined that the pixel data of the short frame flickers, the pixel data of the short frame is discarded, and step S304 is executed, using the pixel data of the medium frame for subsequent operations. If the third exposure value difference is less than or equal to the exposure value difference threshold, then it is determined that the pixel data of the short frame does not flicker, the pixel data of the short frame is retained, and step S308 is executed.

[0167] In step S308, the fused pixels of the medium frame and the short frame are used for subsequent operations.

[0168] The LED flicker detection of the present invention can directly process Bayer image data. Once Bayer image data of long frames, medium frames and short frames are acquired, the detection process of flickering areas can be started immediately without storing previous image data, which significantly reduces the demand for buffer space and thus optimizes storage efficiency.

[0169] In addition, the method provided by the present invention does not rely on the pre-known flicker frequency of the LED light source to be recovered. By setting the minimum exposure time limit of the long frame, the minimum detection frequency can be flexibly set, so that the method can adapt to and handle LED light sources of different frequencies that may exist in the same scene.

[0170] Furthermore, the flicker detection method of the present invention mainly involves basic addition and multiplication operations, thus requiring less computation. This characteristic not only simplifies the complexity of the algorithm but also reduces the cost of hardware implementation, making the present invention particularly suitable for resource-constrained systems.

[0171] In step S4, the area without flicker is input into the HDR fusion module, which synthesizes different frames to generate a high dynamic range image.

[0172] In one embodiment of the present invention, the method by which the HDR fusion module synthesizes different frames to generate a high dynamic range image includes:

[0173] First, the inter-frame fusion weights are calculated. Methods for calculating these weights include, but are not limited to, calculating the image's gradient information, saturation, and brightness. During frame fusion, the gradient between saturated and unsaturated regions is typically small, saturation is close to 0, and brightness values ​​tend towards the image's extremes; for example, in an 8-bit image, brightness values ​​might be close to 0 or 255. In these cases, due to limited color information or extreme brightness, the fusion weights are designed to be relatively small to avoid overemphasizing these regions during image fusion.

[0174] Next, wide dynamic range image merging is performed on the image data to transform the image from low bit width to high bit width.

[0175] Then, the high bit width image is tone-mapped back to its original bit width. This process ensures that the image maintains its extended dynamic range while guaranteeing compatibility and facilitating presentation on standard display devices. Tone mapping maps a wide range of HDR colors to a wide range of LDR (Low Dynamic Range) colors. Mapping algorithms include, but are not limited to, Reinhard Tone Mapping and Filmic Tone Mapping.

[0176] In one embodiment of the present invention, after generating a high dynamic range image, the high dynamic range image can be further processed by the image processing pipeline. In this embodiment, exemplary methods of image processing pipeline processing include, but are not limited to: demosaicing, color correction matrix (CCM), gamma correction, and color space conversion. After processing by each module of the image processing pipeline, the original Bayer format image will be converted into a format suitable for display or further image processing, such as RGB or YUV domain.

[0177] In practical applications such as vehicle lighting, road lighting, and traffic signs, the flicker frequency of LED light sources is often unknown. The method of this invention does not require prior knowledge of the LED flicker frequency. Instead, by setting a sufficiently long exposure time for long frames, it ensures that no flickering occurs within those long frames, thus satisfying the conditions for flicker detection. This method avoids the need to precisely set the frame rate and exposure time to prevent flicker. This design not only simplifies the LED flicker detection process but also improves the applicability and practicality of HDR image processing, especially under dynamic and uncertain lighting conditions.

[0178] Accordingly, the present invention also provides a high dynamic range image processing apparatus for implementing the high dynamic range image processing method described above, the high dynamic range image processing apparatus comprising:

[0179] An adjustment device is used to adjust the exposure time and gain parameters of the image acquisition module so that long frames capture details in dark areas, short frames capture details in bright areas, and medium frames are located between the long frames and the short frames to achieve a balance.

[0180] Image acquisition module, used to acquire Bayer image data in long, medium and short frames;

[0181] An LED flicker detection device receives the image data and is used to perform flicker detection and remove areas where flickering occurs; and

[0182] The HDR fusion module receives flicker-free areas and uses them to synthesize different frames to generate high dynamic range images.

[0183] In summary, the high dynamic range image processing method and apparatus provided by this invention first adjusts the exposure time and gain parameters of the image acquisition module so that long frames capture details in dark areas, short frames capture details in bright areas, and medium frames are positioned between the long and short frames to achieve balance. Then, the image acquisition module acquires Bayer image data for long, medium, and short frames. Next, the image data is input into an LED flicker detection device, which performs flicker detection and removes flickering areas. Finally, flicker-free areas are input into an HDR fusion module, which synthesizes different frames to generate a high dynamic range image. This invention, by introducing LED flicker detection, completes flicker detection of image data and removes flickering areas, thereby effectively mitigating the negative impact of LED flicker on image quality and improving the quality of high dynamic range images. Furthermore, the LED flicker detection of this invention can directly process Bayer image data. Once Bayer image data for long, medium, and short frames is acquired, the flickering area detection process can be started immediately without storing previous image data, significantly reducing the need for buffer space and thus optimizing storage efficiency.

[0184] Furthermore, the method provided by this invention does not rely on the pre-known flicker frequency of the LED light source to be recovered. By setting the minimum exposure time limit of the long frame, the minimum detection frequency can be flexibly set, enabling the method to adapt to and handle LED light sources of different frequencies that may exist in the same scene.

[0185] Furthermore, the flicker detection method of the present invention mainly involves basic addition and multiplication operations, thus requiring less computation. This characteristic not only simplifies the complexity of the algorithm but also reduces the cost of hardware implementation, making the present invention particularly suitable for resource-constrained systems.

[0186] The above description is merely a description of preferred embodiments of the present invention and is not intended to limit the scope of the present invention in any way. Any changes or modifications made by those skilled in the art based on the above disclosure shall fall within the protection scope of the claims.

Claims

1. A high dynamic range image processing method, characterized in that, Includes the following steps: The exposure time and gain parameters of the image acquisition module are adjusted so that long frames capture details in dark areas, short frames capture details in bright areas, and medium frames are located between the long frames and the short frames to achieve a balance. The image acquisition module is used to acquire Bayer image data in long, medium, and short frames; The image data is input into the LED flicker detection device, which performs flicker detection and removes areas where flickering occurs. The methods include: Determine whether the difference in first exposure value between the long frame and the short frame is greater than an exposure value difference threshold; If the first exposure value difference is greater than the exposure value difference threshold, it is determined that the pixel data of the short frame is flickering, the pixel data of the short frame is discarded, and it is determined whether the second exposure value difference between the long frame and the medium frame is greater than the exposure value difference threshold. If the second exposure value difference is greater than the exposure value difference threshold, it is determined that the pixel data of the middle frame is flickering, the pixel data of the middle frame is discarded, and the pixel data of the long frame is used. If the second exposure value difference is less than or equal to the exposure value difference threshold, then it is determined that the pixel data of the intermediate frame does not flicker, and the pixel data of the intermediate frame is adopted. If the first exposure value difference is less than or equal to the exposure value difference threshold, it is determined that the pixel data of the short frame does not flicker, the pixel data of the short frame is retained, and it is determined whether the pixel data of the long frame is saturated. If the pixel data of the long frame is not saturated, then the pixel data of the long frame shall be used; If the pixel data of the long frame is saturated, then determine whether the difference in the third exposure value between the medium frame and the short frame is greater than the exposure value difference threshold. If the third exposure value difference is greater than the exposure value difference threshold, it is determined that the pixel data of the short frame is flickering, the pixel data of the short frame is discarded, and the pixel data of the medium frame is used. If the third exposure value difference is less than or equal to the exposure value difference threshold, it is determined that the pixel data of the short frame does not flicker, the pixel data of the short frame is retained, and it is determined whether the pixel data of the medium frame has reached saturation. If the pixel data of the intermediate frame is not saturated, then the pixel data of the intermediate frame shall be used; If the pixel data of the intermediate frame is saturated, then the merged pixels of the intermediate frame and the short frame are used; or Determine whether the difference in first exposure value between the long frame and the short frame is greater than an exposure value difference threshold; If the first exposure value difference is greater than the exposure value difference threshold, it is determined that the pixel data of the short frame is flickering, the pixel data of the short frame is discarded, and it is determined whether the second exposure value difference between the long frame and the medium frame is greater than the exposure value difference threshold. If the second exposure value difference is greater than the exposure value difference threshold, it is determined that the pixel data of the middle frame is flickering, the pixel data of the middle frame is discarded, and the pixel data of the long frame is used. If the second exposure value difference is less than or equal to the exposure value difference threshold, then it is determined that the pixel data of the intermediate frame does not flicker, and the pixel data of the intermediate frame is adopted. If the first exposure value difference is less than or equal to the exposure value difference threshold, it is determined that the pixel data of the short frame does not flicker, the pixel data of the short frame is retained, and it is determined whether the pixel data of the long frame is saturated. If the pixel data of the long frame is not saturated, then the pixel data of the long frame shall be used; If the pixel data of the long frame is saturated, then determine whether the pixel data of the medium frame has reached saturation; If the pixel data of the intermediate frame is not saturated, then the pixel data of the intermediate frame shall be used; If the pixel data of the intermediate frame is saturated, then determine whether the difference in the third exposure value between the intermediate frame and the short frame is greater than the exposure value difference threshold. If the third exposure value difference is greater than the exposure value difference threshold, it is determined that the pixel data of the short frame is flickering, the pixel data of the short frame is discarded, and the pixel data of the medium frame is used. If the third exposure value difference is less than or equal to the exposure value difference threshold, then it is determined that the pixel data of the short frame does not flicker, and the fused pixels of the medium frame and the short frame are used; or Determine whether the pixel data of the long frame is saturated; If the pixel data of the long frame is not saturated, then the pixel data of the long frame shall be used; If the pixel data of the long frame is saturated, then determine whether the difference in the first exposure value between the long frame and the short frame is greater than the exposure value difference threshold. If the first exposure value difference is greater than the exposure value difference threshold, it is determined that the pixel data of the short frame is flickering, the pixel data of the short frame is discarded, and it is determined whether the second exposure value difference between the long frame and the medium frame is greater than the exposure value difference threshold. If the second exposure value difference is greater than the exposure value difference threshold, it is determined that the pixel data of the middle frame is flickering, the pixel data of the middle frame is discarded, and the pixel data of the long frame is used. If the second exposure value difference is less than or equal to the exposure value difference threshold, then it is determined that the pixel data of the intermediate frame does not flicker, and the pixel data of the intermediate frame is adopted. If the first exposure value difference is less than or equal to the exposure value difference threshold, it is determined that the pixel data of the short frame does not flicker, the pixel data of the short frame is retained, and it is determined whether the third exposure value difference between the medium frame and the short frame is greater than the exposure value difference threshold. If the third exposure value difference is greater than the exposure value difference threshold, it is determined that the pixel data of the short frame is flickering, the pixel data of the short frame is discarded, and the pixel data of the medium frame is used. If the third exposure value difference is less than or equal to the exposure value difference threshold, it is determined that the pixel data of the short frame does not flicker, the pixel data of the short frame is retained, and it is determined whether the pixel data of the medium frame has reached saturation. If the pixel data of the intermediate frame is not saturated, then the pixel data of the intermediate frame shall be used; If the pixel data of the intermediate frame is saturated, then the merged pixels of the intermediate frame and the short frame are used; or, Determine whether the pixel data of the long frame is saturated; If the pixel data of the long frame is not saturated, then the pixel data of the long frame shall be used; If the pixel data of the long frame is saturated, then determine whether the difference in the first exposure value between the long frame and the short frame is greater than the exposure value difference threshold. If the first exposure value difference is greater than the exposure value difference threshold, it is determined that the pixel data of the short frame is flickering, the pixel data of the short frame is discarded, and it is determined whether the second exposure value difference between the long frame and the medium frame is greater than the exposure value difference threshold. If the second exposure value difference is greater than the exposure value difference threshold, it is determined that the pixel data of the middle frame is flickering, the pixel data of the middle frame is discarded, and the pixel data of the long frame is used. If the second exposure value difference is less than or equal to the exposure value difference threshold, then it is determined that the pixel data of the intermediate frame does not flicker, and the pixel data of the intermediate frame is adopted. If the first exposure value difference is less than or equal to the exposure value difference threshold, it is determined that the pixel data of the short frame does not flicker, the pixel data of the short frame is retained, and it is determined whether the pixel data of the medium frame has reached saturation. If the pixel data of the intermediate frame is not saturated, then the pixel data of the intermediate frame shall be used; If the pixel data of the intermediate frame is saturated, then determine whether the difference in the third exposure value between the intermediate frame and the short frame is greater than the exposure value difference threshold. If the third exposure value difference is greater than the exposure value difference threshold, it is determined that the pixel data of the short frame is flickering, the pixel data of the short frame is discarded, and the pixel data of the medium frame is used. If the third exposure value difference is less than or equal to the exposure value difference threshold, then it is determined that the pixel data of the short frame does not flicker, the pixel data of the short frame is retained, and the fused pixels of the medium frame and the short frame are used; and The region without flicker is input into the HDR fusion module, which calculates the fusion weights between frames and synthesizes different frames to generate a high dynamic range image.

2. The high dynamic range image processing method according to claim 1, characterized in that, The method for acquiring three frames of image data (long frame, medium frame, and short frame) using the image acquisition module includes: acquiring four Bayer images, and acquiring Bayer image data of the long frame, medium frame, and short frame at the same time based on the four Bayer images.

3. The high dynamic range image processing method according to claim 1, characterized in that, After adjusting the exposure time and gain parameters of the image acquisition module and before completing the detection of the flickering area, the processing method further includes: Calculate the exposure values ​​of the long frame, the medium frame, and the short frame; Calculate the exposure value difference between different frames and set an exposure value difference threshold; and Set the saturation threshold for the pixels of the long frame and the medium frame.

4. The high dynamic range image processing method according to claim 1, characterized in that, The method for determining whether a pixel is saturated is as follows: when the pixel brightness value is greater than the saturation threshold, the pixel is determined to be saturated; wherein, the saturation threshold is 90% of the full-well capacity of the image sensor.

5. The high dynamic range image processing method according to claim 1, characterized in that, The method by which flicker-free regions are input into the HDR fusion module, and the HDR fusion module synthesizes different frames to generate a high dynamic range image, includes: Calculate the inter-frame fusion weights; The image data is subjected to wide dynamic range image merging, transforming the image from low bit width to high bit width; and The high bit width image is tone-mapped back to its original bit width.

6. The high dynamic range image processing method according to claim 5, characterized in that, Methods for calculating inter-frame fusion weights include: calculating the gradient information of the image, calculating the saturation of the image, and calculating the brightness of the image.

7. The high dynamic range image processing method according to claim 1, characterized in that, After generating the high dynamic range image, the process further includes processing the high dynamic range image using an image processing pipeline.

8. The high dynamic range image processing method according to claim 7, characterized in that, The image processing pipeline includes the following processing methods: demosaicing, color correction matrix, gamma correction, and color space conversion.

9. The high dynamic range image processing method according to claim 1, characterized in that, The exposure time of the long frame is greater than or equal to the reciprocal of the LED flicker frequency.

10. A high dynamic range image processing apparatus, characterized in that, For implementing the high dynamic range image processing method as described in any one of claims 1 to 9, the high dynamic range image processing apparatus comprises: An adjustment device is used to adjust the exposure time and gain parameters of the image acquisition module so that long frames capture details in dark areas, short frames capture details in bright areas, and medium frames are located between the long frames and the short frames to achieve a balance. Image acquisition module, used to acquire Bayer image data in long, medium and short frames; An LED flicker detection device receives the image data and performs flicker detection to remove areas where flickering occurs; and The HDR fusion module receives flicker-free areas and uses them to synthesize different frames to generate high dynamic range images.

Images