Anti-overexposure image processing method and system under strong light based on high dynamic range

Abstract

The invention relates to the technical field of image shooting and processing, and discloses an anti-overexposure image processing method and system under strong light based on a high dynamic range, and the method comprises the steps: collecting a plurality of frames with different exposures for the same scene through an extremely weak light full-color night vision device, carrying out the multi-frame fusion, and generating a high dynamic range image; the method comprises the following steps: mapping an image with a high dynamic range to display equipment with a low dynamic range, and readjusting the brightness and contrast of the image to realize tone mapping; different areas in the image are independently processed, an overexposure area is identified, the brightness of the overexposure area is independently reduced, and the original brightness and contrast of other areas are kept; histogram equalization parameters are dynamically adjusted according to the local brightness distribution of the image, the image contrast and details are improved, and image color correction is achieved. The problem of overexposure of the ultra-weak light full-color night vision device in a strong light environment is solved through a high dynamic range algorithm, and the image quality of the night vision device under different illumination conditions is remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image shooting and processing, and in particular to a method and system for preventing overexposure of an image under strong light based on a high dynamic range. Background Art

[0002] When dealing with strong light environments, traditional cameras usually use methods such as reducing gain, installing optical filters, or simple brightness adjustment to solve the overexposure problem. However, these methods have many limitations. Reducing gain may result in insufficient overall image brightness and loss of dark details; optical filters increase costs and have limited applicable scenarios; simple brightness adjustment is difficult to accurately handle complex lighting conditions, and it is easy to leave dark or overexposed areas.

[0003] Extremely low-light full-color night vision devices are devices that can generate color images under extremely low-light conditions and are widely used in night monitoring, security, military reconnaissance and other fields. However, under extreme lighting conditions (such as strong light environments), the image quality of extremely low-light full-color night vision devices is easily affected, and high dynamic range technology is required to avoid overexposure problems. Summary of the invention

[0004] The purpose of the present invention is to provide an image processing method and system for preventing overexposure under strong light based on high dynamic range, and to solve the overexposure problem of extremely weak light full-color night vision goggles in strong light environments through a high dynamic range (HDR) algorithm, so as to significantly improve the image quality of night vision goggles under different lighting conditions.

[0005] To achieve the above object, the present invention provides an image processing method for preventing overexposure under strong light based on high dynamic range, comprising the following steps:

[0006] S1, multi-frame fusion: In a strong light environment, the extremely low-light full-color night vision device collects multiple frames of different exposures for the same scene, and fuses the multiple frames of different exposures at the pixel level to generate a high dynamic range image;

[0007] S2, tone mapping: mapping a high dynamic range image to a low dynamic range display device, readjusting the brightness and contrast of the image to achieve tone mapping;

[0008] S3, local adjustment: Based on the tone-mapped image, different areas in the image are processed independently, overexposed areas are identified, and the brightness of the overexposed areas is reduced by 30% to 70% respectively, while maintaining the original brightness and contrast of other areas;

[0009] S4, color correction: for the locally adjusted image, dynamically adjust the histogram equalization parameters according to the local brightness distribution of the image to improve the image contrast and details and achieve image color correction.

[0010] Furthermore, the frames with different exposures include short exposure frames for capturing details of highlight parts and long exposure frames for capturing details of shadow parts.

[0011] Furthermore, in S1, multiple frames with different exposures are fused at the pixel level to generate an image with a high dynamic range, which specifically includes:

[0012] Frame alignment: Using an algorithm based on feature point matching, the SIFT algorithm is used to extract key feature points in each frame of the image, and the relative displacement between different frames is calculated to ensure accurate spatial alignment of the frames, with the error controlled within 0.1 pixels.

[0013] Weight assignment: Assign weights to each pixel based on the exposure and image content of each frame. The highlight part in the short exposure frame has a higher weight, and the weight calculation is based on the comparison of the pixel brightness with the preset highlight threshold; the shadow part in the long exposure frame has a higher weight, and the weight calculation is based on the comparison of the pixel brightness with the preset shadow threshold; the pixels in the brightness range between the preset highlight threshold and the preset shadow threshold are weighted by linear interpolation or more complex brightness-related functions to ensure that the sum of all pixel weights is 1.

[0014] Pixel value fusion: weighted average calculation is performed on the pixel values ​​of corresponding pixels in different frames according to the assigned weights to obtain the fused pixel values ​​and generate the fused image;

[0015] Denoising and smoothing: The fused image is denoised using median filtering, and the filter window size is set to 3×3 pixels. At the same time, bilinear interpolation is used for smoothing to make the transition between different exposure areas natural and the brightness gradient change in the transition area is controlled within 5%.

[0016] Furthermore, in S1, generating a high dynamic range image further includes:

[0017] Detail enhancement: The Laplacian operator is used for edge detection to enhance the contour and texture details of objects in the fused image. The template coefficient of the Laplacian operator is set to [0, -1, 0; -1, 4, -1; 0, -1, 0] to highlight the detail information in the image.

[0018] Furthermore, in S2, the brightness and contrast of the image are readjusted to achieve tone mapping, which specifically includes:

[0019] According to the content and brightness distribution of the image, the overly bright areas are compressed. The compression ratio is dynamically determined according to the brightness value and can be up to 50%. At the same time, the brightness of the dark areas is enhanced by up to 200% to maintain the overall visual balance.

[0020] Furthermore, in S3, the method for identifying the overexposed area is:

[0021] By judging the brightness threshold, the brightness of the image is higher than 1000cd / m 2 The area is judged as overexposed area.

[0022] Furthermore, the method further comprises:

[0023] Use deep learning-based machine learning algorithms to identify and classify the scene types of images;

[0024] Based on the identified scene, the overexposed area is determined, and the brightness of the overexposed area is adjusted by training the convolutional neural network.

[0025] The present invention further provides an image processing system for preventing overexposure under strong light based on high dynamic range, which is applied to the above-mentioned image processing method for preventing overexposure under strong light based on high dynamic range, comprising:

[0026] The multi-frame fusion module is used to fuse multiple frames with different exposures at the pixel level to generate a high dynamic range image; multiple frames with different exposures are collected for the same scene in a strong light environment through an extremely weak light full-color night vision device;

[0027] A tone mapping module is used to map a high dynamic range image to a low dynamic range display device, readjust the brightness and contrast of the image to achieve tone mapping;

[0028] The local adjustment module is used to process different areas of the image independently based on the tone-mapped image, identify the overexposed areas, and reduce the brightness of the overexposed areas by 30% to 70% while maintaining the original brightness and contrast of other areas.

[0029] The color correction module is used to dynamically adjust the histogram equalization parameters of the locally adjusted image according to the local brightness distribution of the image, so as to improve the image contrast and details and realize the image color correction.

[0030] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the computer program implements the above-mentioned method for preventing overexposure of images under strong light based on high dynamic range.

[0031] Compared with the prior art, the technical effects of the present invention are:

[0032] The present invention proposes an image processing method for preventing overexposure under strong light based on high dynamic range. Through multi-frame fusion, tone mapping, local adjustment, color correction, etc., the overexposure problem of extremely weak light full-color night vision device in strong light environment is effectively solved, and the image quality and adaptability are significantly improved. Among them, high dynamic range (HDR) technology is specifically adopted. In the multi-frame fusion process, under different lighting conditions, an extremely weak light full-color night vision device is used to shoot multiple frames with different exposures, including short exposure frames (to avoid overexposure) and long exposure frames (to ensure dark details). Multi-frame fusion makes each frame image maintain spatial consistency. Multiple frames with different exposures are fused at the pixel level to generate a high dynamic range image, compress the high dynamic range image to the range that the display can display, and maintain the details and contrast of the image at the same time. The naturalness and authenticity of the color are ensured through local adjustment and color correction, and the image quality is further improved. The extremely weak light full-color night vision device can effectively solve the overexposure problem in a strong light environment through the image processing method based on the high dynamic range (HDR) algorithm proposed by the present invention, so as to significantly improve the image quality of the full-color night vision device under different lighting conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0034] Figure 1 4 is a flow chart of a method for processing an image under strong light to prevent overexposure based on a high dynamic range in an embodiment of the present invention. DETAILED DESCRIPTION

[0035] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0036] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0037] The purpose of the present invention is to propose a method for solving the overexposure problem of an extremely weak light full-color night vision device in a strong light environment by using a high dynamic range (HDR) algorithm, so as to significantly improve the image quality of the night vision device under different lighting conditions.

[0038] like Figure 1 As shown, the method for processing an image under strong light to prevent overexposure based on a high dynamic range provided by the present invention comprises the following steps:

[0039] S1, multi-frame fusion: In a strong light environment, the extremely low-light full-color night vision device collects multiple frames of different exposures for the same scene, and fuses the multiple frames of different exposures at the pixel level to generate a high dynamic range image.

[0040] S2, tone mapping: mapping a high dynamic range image to a low dynamic range display device, readjusting the brightness and contrast of the image to achieve tone mapping;

[0041] S3, local adjustment: Based on the tone-mapped image, different areas in the image are processed independently, overexposed areas are identified, and the brightness of the overexposed areas is reduced by 30% to 70% respectively, while maintaining the original brightness and contrast of other areas;

[0042] S4, color correction: for the locally adjusted image, dynamically adjust the histogram equalization parameters according to the local brightness distribution of the image to improve the image contrast and details and achieve image color correction.

[0043] Specifically, in combination with the embodiment, the image processing method for preventing overexposure under strong light based on high dynamic range is introduced in detail as follows:

[0044] 1. Multi-frame fusion

[0045] Multiple frames of different exposures of the same scene are collected, including short-exposure frames (for example, exposure time of 0.001 seconds) for capturing details of highlight parts and long-exposure frames (for example, exposure time of 0.1 seconds) for capturing details of shadow parts.

[0046] The specific steps of multi-frame fusion are as follows:

[0047] (1) Frame alignment: Using an algorithm based on feature point matching, the SIFT (Scale-Invariant Feature Transform) algorithm is used to extract key feature points in the image and calculate the relative displacement between different frames to ensure accurate spatial alignment of the frames with an error of less than 0.1 pixels. For example, when shooting branches swaying in the wind, the algorithm can accurately identify key feature points such as the bifurcation points on the branches and achieve precise alignment. For moving objects or scenes, motion compensation is performed to reduce ghosting and blurring.

[0048] (2) Weight assignment: Weights are assigned to each pixel based on the exposure and image content of each frame. The highlight part in the short exposure frame has a higher weight, and the weight calculation is based on the comparison of the pixel brightness with the preset highlight threshold; the shadow part in the long exposure frame has a higher weight, and the weight calculation is based on the comparison of the pixel brightness with the preset shadow threshold. For example, for pixels with brightness higher than 800, the weight of the short exposure frame is set to 0.8, and the weight of the long exposure frame is 0.2; for pixels with brightness lower than 100, the weight of the short exposure frame is 0.2, and the weight of the long exposure frame is 0.8.

[0049] Pixels in the middle brightness range are weighted using linear interpolation or more complex brightness-related functions to ensure that the sum of all pixel weights is 1.

[0050] When the pixel brightness is between the preset highlight threshold and the preset shadow threshold, linear interpolation or more complex brightness-related functions are used to calculate the weight to ensure the rationality and accuracy of the weight distribution, so that frames with different exposures can better retain image details when fused and generate high-quality high dynamic range images. At the same time, the specific weight distribution will be adjusted according to the actual scene and image characteristics to adapt to different lighting conditions and image content.

[0051] (3) Pixel value fusion: The pixel values ​​of corresponding pixels in different frames are weighted averaged according to the assigned weights. Assume that the pixel value in the short exposure frame is 200 and the weight is 0.7; the same pixel value in the long exposure frame is 100 and the weight is 0.3. The fused pixel value is 200×0.7+100×0.3=170.

[0052] (4) Denoising and smoothing: The fused image is denoised using a median filter, with the filter window size set to 3×3 pixels. Bilinear interpolation is used for smoothing to ensure a natural transition between different exposure areas, with the brightness gradient change in the transition area controlled within 5%. For example, in the transition area from highlight to shadow, processing is performed to avoid obvious boundaries or mutations, making the transition smoother and more natural.

[0053] (5) Detail enhancement: Use the Laplacian operator for edge detection to enhance the contour and texture details of the object. The template coefficient of the Laplacian operator is set to [0, -1, 0; -1, 4, -1; 0, -1, 0] to highlight the details in the image. Taking the night scene of the city as an example, the short exposure frame can clearly show the top of the high-rise building with bright lights, and the long exposure frame can show the details of the dim street. Through the above fusion steps, a high-quality night scene image with no overexposure of the top of the high-rise building and rich street details is finally obtained.

[0054] To verify the effect of multi-frame fusion, the following experiment was conducted: 100 sets of scene images in strong light environments were selected, and the traditional single-frame shooting and the multi-frame fusion method of the present invention were used respectively. The comparison results showed that 70% of the images in the traditional method had obvious overexposed areas, while only 10% of the images in the multi-frame fusion method were slightly overexposed, and the image detail clarity was improved by an average of 30%.

[0055] 2. Tone Mapping

[0056] Map high dynamic range scenes to low dynamic range display devices and readjust the brightness and contrast of the image.

[0057] According to the content and brightness distribution of the image, the overly bright areas are compressed, and the compression ratio is dynamically determined according to the brightness value, up to 50%. At the same time, the brightness of the dark areas is enhanced, and the enhancement range can be up to 200% to maintain the overall visual balance. For example, when there is strong light in extremely weak light conditions, the brightness of the surrounding strong light areas is appropriately compressed to retain color and details and avoid overexposure of a white area.

[0058] Through actual testing, in 50 sets of scenes with extremely weak light and strong light contrast, the contrast uniformity of the images after tone mapping was improved by 40% and the color fidelity was improved by 25%.

[0059] 3. Local Adjustment

[0060] Different areas of the image are processed independently to identify overexposed areas (judged by brightness threshold, for example, brightness higher than 1000cd / m 2 The HDR algorithm detects overexposure in certain areas and reduces their brightness by 30% to 70% while maintaining the original brightness and contrast of other areas. For example, in extremely low light conditions, the lights of oncoming cars cause some areas to be overexposed. The HDR algorithm adjusts the brightness of these overexposed light areas without affecting the normal brightness of other road details.

[0061] Experiments show that in 80 sets of scene images containing car light interference, after using local adjustment, the brightness of the overexposed area is effectively controlled and the overall visibility of the image is improved by 45%.

[0062] 4. Color Correction

[0063] Overexposure may cause color distortion. The HDR algorithm corrects the color and restores the true color.

[0064] Dynamically adjust the histogram equalization parameters according to the local brightness distribution of the image to improve the image contrast and details. For example, in extremely weak light conditions, the color of objects under strong light is too bright or distorted. Through color correction, the objects can present more natural and realistic colors.

[0065] The actual effect evaluation shows that the average color deviation of the 60 sets of images that have undergone color correction has been reduced from 15% to less than 5%, and the color reproduction has been significantly improved.

[0066] 5. Scenario-based analysis

[0067] Using deep learning-based machine learning algorithms, such as convolutional neural networks (CNNs), we can identify and classify common scene types (such as indoor, outdoor, night scenes, etc.). The training data contains 10,000 images of different scenes, and the recognition accuracy is improved by continuously optimizing network parameters, reaching an accuracy of more than 95%.

[0068] Optimize the overexposure problem according to the characteristics of the scene. For example, when shooting a scene with strong light coming in from the window indoors, identify the situation where the indoor-outdoor light ratio is large and deal with the overexposed window area in a targeted manner. The CNN architecture uses a combination of multiple convolutional layers and pooling layers to effectively extract image features. The training algorithm uses the stochastic gradient descent method with an adaptive learning rate, combined with L2 regularization to prevent overfitting, and improves the model's generalization ability through data enhancement (such as random cropping and flipping).

[0069] In recent years, deep learning-based methods have made significant progress in the field of image processing. These methods can automatically adjust the brightness of highlight areas by training neural networks.

[0070] In summary, the present invention can effectively solve the overexposure problem of extremely low-light full-color night vision devices in strong light environments through the comprehensive application of the above-mentioned high dynamic range (HDR) algorithm, and significantly improve image quality and adaptability.

[0071] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the computer program implements the above-mentioned method for preventing overexposure of images under strong light based on high dynamic range.

[0072] The above is only a preferred specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for processing an image to prevent overexposure under strong light based on high dynamic range, characterized in that: The following steps are involved: S1, multi-frame fusion: In a strong light environment, the extremely low-light full-color night vision device collects multiple frames of different exposures for the same scene, and fuses the multiple frames of different exposures at the pixel level to generate a high dynamic range image; S2, tone mapping: mapping a high dynamic range image to a low dynamic range display device, readjusting the brightness and contrast of the image to achieve tone mapping; S3, local adjustment: Based on the tone-mapped image, different areas in the image are processed independently, overexposed areas are identified, and the brightness of the overexposed areas is reduced by 30% to 70% respectively, while maintaining the original brightness and contrast of other areas; S4, color correction: for the locally adjusted image, dynamically adjust the histogram equalization parameters according to the local brightness distribution of the image to improve the image contrast and details and achieve image color correction.

2. The method for preventing overexposure in strong light based on high dynamic range according to claim 1, characterized in that: The frames with different exposures include short exposure frames for capturing details of a highlight portion and long exposure frames for capturing details of a shadow portion.

3. The method for preventing overexposure in strong light based on high dynamic range according to claim 1, characterized in that: In S1, multiple frames with different exposures are fused at the pixel level to generate a high dynamic range image, which specifically includes: Frame alignment: Using an algorithm based on feature point matching, the SIFT algorithm is used to extract key feature points in each frame of the image, and the relative displacement between different frames is calculated to ensure accurate spatial alignment of the frames, with the error controlled within 0.1 pixels. Weight assignment: assign weights to each pixel according to the exposure and pixel brightness of each frame, wherein the brightness of each pixel is compared with a preset highlight threshold and a preset shadow threshold to determine the weights of pixels with different exposures; Pixel value fusion: weighted average calculation is performed on the pixel values ​​of corresponding pixels in different frames according to the assigned weights to obtain the fused pixel values ​​and generate the fused image; Denoising and smoothing: The fused image is denoised using median filtering, and the filter window size is set to 3×3 pixels. At the same time, bilinear interpolation is used for smoothing to make the transition between different exposure areas natural and the brightness gradient change in the transition area is controlled within 5%.

4. The method for preventing overexposure in strong light based on high dynamic range according to claim 3, characterized in that: In S1, generating a high dynamic range image also includes: Detail enhancement: The Laplacian operator is used for edge detection to enhance the contour and texture details of objects in the fused image. The template coefficient of the Laplacian operator is set to [0, -1, 0; -1, 4, -1; 0, -1, 0] to highlight the detail information in the image.

5. The method for preventing overexposure in strong light based on high dynamic range according to claim 1, characterized in that: In S2, the brightness and contrast of the image are readjusted to achieve tone mapping, which specifically includes: According to the content and brightness distribution of the image, the overly bright areas are compressed. The compression ratio is dynamically determined according to the brightness value and can be up to 50%. At the same time, the brightness of the dark areas is enhanced by up to 200% to maintain the overall visual balance.

6. The method for preventing overexposure in strong light based on high dynamic range according to claim 1, characterized in that: In S3, the method for identifying the overexposed area is: By judging the brightness threshold, the brightness of the image is higher than 1000cd / m 2 The area is judged as overexposed area.

7. The method for preventing overexposure in strong light based on high dynamic range according to claim 1, characterized in that: The method further comprises: Use deep learning-based machine learning algorithms to identify and classify the scene types of images; Based on the identified scene, the overexposed area is determined, and the brightness of the overexposed area is adjusted by training the convolutional neural network.

8. A high dynamic range based anti-overexposure image processing system under strong light, applied to the high dynamic range based anti-overexposure image processing method under strong light according to any one of claims 1 to 7, characterized in that: include: The multi-frame fusion module is used to fuse multiple frames with different exposures at the pixel level to generate a high dynamic range image; Multiple frames with different exposures are collected from the same scene in a strong light environment using an extremely low-light full-color night vision device; A tone mapping module is used to map a high dynamic range image to a low dynamic range display device, readjust the brightness and contrast of the image to achieve tone mapping; The local adjustment module is used to process different areas of the image independently based on the tone-mapped image, identify the overexposed areas, and reduce the brightness of the overexposed areas by 30% to 70% while maintaining the original brightness and contrast of other areas. The color correction module is used to dynamically adjust the histogram equalization parameters of the locally adjusted image according to the local brightness distribution of the image, so as to improve the image contrast and details and realize the image color correction.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for preventing overexposure of an image under strong light based on a high dynamic range is implemented.

Images