Image-based Halation Processing Method, Device, and Storage Medium

Abstract

The present application discloses an image-based halo processing method, device, and storage medium. The image-based halo processing method includes: obtaining an initial dark frame image collected according to current dark frame parameters and an initial bright frame image collected according to current bright frame parameters. By analyzing whether the brightness information of the target area in the initial dark frame image reaches the brightness requirement, the current dark frame parameters can be adjusted according to the brightness information of the target area to obtain target dark frame parameters, so that the target dark frame image collected according to the target dark frame parameters meets the brightness requirement and avoids overexposure and underexposure of the image; performing image fusion processing on the target dark frame image and the initial bright frame image according to the height information of the target area, and then a de-halo image can be obtained. The above solution can effectively reduce the halo phenomenon in the image.

Description

Technical Field

[0001] This application relates to the field of image processing technologies, and particularly to an image-based halo processing method, device, and storage medium. Background Art

[0002] In the field of image processing, a halo refers to a blurred effect that spreads out from the edges of an image after image development. This phenomenon is usually caused by reasons such as light scattering or the characteristics of an optical lens. Especially when observing bright objects in a dark scene, the halo phenomenon is very likely to occur.

[0003] For example, in a traffic application scenario, if the environment where the image acquisition device is located is relatively dark, the image acquisition device will set relatively high exposure parameters to acquire a bright image. However, when there are luminous objects such as vehicle lights and traffic signals in the environment, it may cause an obvious halo phenomenon in the image.

[0004] When the halo phenomenon is severe, it will affect the accurate analysis of the image. For example, it is impossible to distinguish information such as the direction and color of the signal light in the image. Summary of the Invention

[0005] This application provides at least an image-based halo processing method, device, equipment, and computer-readable storage medium.

[0006] In a first aspect of this application, an image-based halo processing method is provided, including: obtaining an initial dark frame image collected according to current dark frame parameters and an initial bright frame image collected according to current bright frame parameters; adjusting the current dark frame parameters according to the brightness information of a target area in the initial dark frame image to obtain target dark frame parameters; obtaining a target dark frame image collected according to the target dark frame parameters; performing image fusion processing on the target dark frame image and the initial bright frame image according to the height information of the target area to obtain a dehaloed image.

[0007] In an embodiment, the adjusting the current dark frame parameters according to the brightness information of a target area in the initial dark frame image to obtain target dark frame parameters includes: obtaining a target brightness interval; adjusting the current dark frame parameters according to the comparison result between the brightness value of the target area in the initial dark frame image and the left and right boundaries of the target brightness interval to obtain the target dark frame parameters.

[0008] In one embodiment, the current dark frame parameter includes a dark frame exposure parameter. Adjusting the current dark frame parameter according to the comparison result between the brightness value of the target area in the initial dark frame image and the left and right boundaries of the target brightness interval to obtain the target dark frame parameter includes: if the brightness value is less than the left boundary of the target brightness interval, increasing the dark frame exposure parameter to obtain the target dark frame parameter; if the brightness value is greater than the right boundary of the target brightness interval, decreasing the dark frame exposure parameter to obtain the target dark frame parameter.

[0009] In one embodiment, obtaining the target dark frame image acquired according to the target dark frame parameter includes: acquiring a to-be-processed dark frame image according to the target dark frame parameter; comparing the original brightness of the to-be-processed dark frame image with a preset brightness threshold to obtain a brightness comparison result; in response to the brightness comparison result indicating that the original brightness is less than the preset brightness threshold, increasing the original brightness to obtain the target dark frame image; in response to the brightness comparison result indicating that the original brightness is greater than the preset brightness threshold, decreasing the original brightness to obtain the target dark frame image.

[0010] In one embodiment, obtaining the target dark frame image acquired according to the target dark frame parameter includes: acquiring a to-be-processed dark frame image according to the target dark frame parameter; in response to the brightness information of the target area in the to-be-processed dark frame image being less than the left boundary of the target brightness interval, performing a brightening process on the to-be-processed dark frame image using a first brightening parameter to obtain the target dark frame image; in response to the brightness information of the target area in the to-be-processed dark frame image being greater than the right boundary of the target brightness interval, performing a linear mapping process on the to-be-processed dark frame image to obtain the target dark frame image; in response to the brightness information of the target area in the to-be-processed dark frame image being within the target brightness interval, performing a brightness transition process on the to-be-processed dark frame image to obtain the target dark frame image.

[0011] In one embodiment, the target area includes a traffic signal light area. Before adjusting the current dark frame parameter according to the brightness information of the target area in the initial dark frame image to obtain the target dark frame parameter, the method further includes: performing target detection processing on the initial dark frame image to obtain the traffic signal light area in the initial dark frame image; or, in response to a received area selection instruction, determining the traffic signal light area in the initial dark frame image according to the area selection instruction; determining the average red brightness of the traffic signal light area as the brightness information.

[0012] In one embodiment, the image fusion processing of the target dark frame image and the initial bright frame image according to the height information of the target area to obtain a de-halo image includes:

[0013] Determine the height information of the target area as the fusion height information; alternatively, determine the fusion height information according to the height information of the target area and a preset fusion parameter, where the fusion height information is greater than or equal to the height information of the target area; determine the target fusion area of the target dark frame image and the initial bright frame image according to the fusion height information; and perform fusion processing on the target dark frame image and the initial bright frame image according to the target fusion area to obtain the de-halo image.

[0014] In one embodiment, the performing fusion processing on the target dark frame image and the initial bright frame image according to the target fusion area to obtain the de-halo image includes: determining, according to the height information and the fusion height information, a dark frame fusion weight of the target dark frame image corresponding to each pixel row and a bright frame fusion weight of the initial bright frame image in the target fusion area; and performing fusion processing on the target dark frame image and the initial bright frame image according to the dark frame fusion weight and the bright frame fusion weight to obtain the de-halo image.

[0015] A second aspect of the present application provides an image-based halo processing device, including: a first acquisition module, configured to acquire an initial dark frame image acquired according to a current dark frame parameter and an initial bright frame image acquired according to a current bright frame parameter; a parameter adjustment module, configured to adjust the current dark frame parameter according to the brightness information of a target area in the initial dark frame image to obtain a target dark frame parameter; a second acquisition module, configured to acquire a target dark frame image acquired according to the target dark frame parameter; and an image fusion module, configured to perform image fusion processing on the target dark frame image and the initial bright frame image according to the height information of the target area to obtain a de-halo image.

[0016] A third aspect of the present application provides an electronic device, including a memory and a processor, where the processor is configured to execute program instructions stored in the memory to implement the above-mentioned image-based halo processing method.

[0017] A fourth aspect of the present application provides a computer-readable storage medium, on which program instructions are stored, and when the program instructions are executed by a processor, the above-mentioned image-based halo processing method is implemented.

[0018] In the above solution, an initial dark frame image collected according to the current dark frame parameters and an initial bright frame image collected according to the current bright frame parameters are obtained. By analyzing whether the brightness information of the target area in the initial dark frame image reaches the brightness requirement, the current dark frame parameters can be adjusted according to the brightness information of the target area to obtain target dark frame parameters, so that the target dark frame image collected according to the target dark frame parameters meets the brightness requirement, avoiding overexposure and underexposure of the image. Image fusion processing is performed on the target dark frame image and the initial bright frame image according to the height information of the target area, and then a de-halo image can be obtained, thereby effectively reducing the halo phenomenon in the image.

[0019] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and do not limit the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings herein are incorporated into the specification and constitute a part of this specification. These drawings illustrate embodiments consistent with the present application and, together with the specification, are used to explain the technical solutions of the present application.

[0021] Figure 1 is a schematic flowchart of an exemplary embodiment of the image-based halo processing method of the present application;

[0022] Figure 2 is a schematic diagram of an exemplary traffic signal light area in the image-based halo processing method of the present application;

[0023] Figure 3 is a schematic diagram of an exemplary image fusion in the image-based halo processing method of the present application;

[0024] Figure 4 is a block diagram of an image-based halo processing device shown in an exemplary embodiment of the present application;

[0025] Figure 5 is a schematic structural diagram of an embodiment of an electronic device of the present application;

[0026] Figure 6 is a schematic structural diagram of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0027] The solutions of the embodiments of the present application will be described in detail below with reference to the accompanying drawings of the specification.

[0028] In the following description, specific details such as specific system structures, interfaces, and technologies are presented for the purpose of illustration rather than limitation, so as to thoroughly understand the present application.

[0029] As used herein, the term "and / or" is merely a description of the relationship between associated objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship. Furthermore, "plurality" in this article means two or more than two. In addition, the term "at least one" in this article means any one of a plurality or any combination of at least two of a plurality. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set composed of A, B, and C.

[0030] In the field of image processing, halation refers to the blurred effect that spreads out from the image edge after the image is developed. This phenomenon is usually caused by reasons such as light scattering, light reflection, or the characteristics of the optical lens. Especially when observing bright objects in a dark scene, the halation phenomenon is very likely to occur. For example, in a traffic application scenario, if the environment where the image acquisition device is located is relatively dark, the image acquisition device will set a higher exposure parameter to acquire a bright image. However, when there are luminous objects such as vehicle lights and traffic lights in the environment, it may cause an obvious halation phenomenon in the image.

[0031] It should be noted that the halation processing method of the present application can be used in traffic application scenarios or other image acquisition application scenarios, which is not limited here. For the sake of understanding, the traffic application scenario is mainly used as an example in the present application for illustration.

[0032] Please refer to Figure 1 , Figure 1 is a schematic flowchart of an exemplary embodiment of the image-based halation processing method of the present application. Specifically, it can include the following steps:

[0033] Step S110, obtaining an initial dark frame image acquired according to the current dark frame parameter and an initial bright frame image acquired according to the current bright frame parameter.

[0034] It should be noted that in the image acquisition scenario, the image quality is often optimized by collecting bright and dark frames. For example, technologies such as Wide Dynamic Range (WDR) and High Dynamic Range (HDR) are not elaborated here.

[0035] Among them, the overall brightness of the bright-frame image is greater than that of the dark-frame image. The bright frame is obtained based on the bright-frame acquisition parameters (the initial bright-frame image is acquired based on the current bright-frame parameters), and the dark frame is obtained based on the dark-frame acquisition parameters (the initial dark-frame image is acquired based on the current dark-frame parameters). The acquisition parameters can include but are not limited to exposure parameters, brightening parameters, etc. By adjusting these acquisition parameters, the adjustment of the image brightness can be achieved.

[0036] It should also be noted that in the image acquisition process of this application, the bright frame and the dark frame can be obtained periodically or randomly, and no limitation is made here. For example, it can be to acquire a dark frame after acquiring a bright frame, or to acquire a dark frame after acquiring two bright frames, or to acquire two dark frames after acquiring two bright frames, or it can also be to randomly acquire some dark frames after randomly acquiring some bright frames, etc., which will not be elaborated here. And this application also does not limit the acquisition order of the initial dark-frame image and the initial bright-frame image.

[0037] Thus, all the image frames acquired by the image acquisition device can be classified into bright frames and dark frames. The bright-frame image can be used for tasks such as image detection that require analyzing the image content, and the dark-frame image can be used for image processing tasks such as image quality optimization.

[0038] Step S120: Adjust the current dark-frame parameters according to the brightness information of the target area in the initial dark-frame image to obtain the target dark-frame parameters.

[0039] The target area is the area in the initial dark-frame image that needs to analyze whether it is underexposed or overexposed, usually the light source area.

[0040] Exemplarily, the target area in the initial dark-frame image can be obtained, and its brightness information of the target area is compared with the obtained target brightness range, and then whether to adjust the current dark-frame parameters is determined according to the comparison result to obtain the target dark-frame parameters. Among them, the target brightness range is used to judge whether the target area is underexposed or overexposed according to the brightness information of the target area. The target brightness range can be preset and fixed, or preset and support dynamic adjustment, and no limitation is made here.

[0041] For example, if the brightness information of the target area is within the target brightness range [a, b], it indicates that the brightness of the target area meets the brightness requirement, and the current dark-frame parameters can be determined as the target dark-frame parameters. If the brightness information of the target area is outside the target brightness range [a, b], it indicates that the brightness of the target area does not meet the brightness requirement, and the current dark-frame parameters need to be adjusted accordingly, and the adjusted dark-frame parameters are used as the target dark-frame parameters.

[0042] Step S130: Obtain the target dark-frame image acquired according to the target dark-frame parameters.

[0043] The description is made in combination with the foregoing steps. If it is determined in the foregoing steps that the current dark frame parameter does not meet the brightness requirement, after adjusting the current dark frame parameter to obtain the target dark frame parameter, the target dark frame image collected through the target dark frame parameter should meet the brightness requirement. If it is determined in the foregoing steps that the current dark frame parameter meets the brightness requirement, the current dark frame parameter can be determined as the target dark frame parameter, and image acquisition can continue with the same parameter. In this case, the target dark frame image obtained can be a newly collected dark frame image according to the target dark frame parameter, or the initial dark frame image can be used as the target dark frame image, which is not limited herein.

[0044] Step S140: Perform image fusion processing on the target dark frame image and the initial bright frame image according to the height information of the target area to obtain a de-halo image.

[0045] The description is made in combination with the foregoing steps. The target dark frame image is an image that meets the brightness requirement. By performing image fusion processing on the target dark frame image and the initial bright frame image, the reduction of the halo phenomenon in the bright frame image can be achieved, and a de-halo image can be obtained.

[0046] It should be noted that in the specific implementation process of this application, in addition to fusing the target dark frame image and the initial bright frame image, it is also possible to collect a new bright frame image (the current bright frame image) with the current bright frame parameter (or the parameter after adjusting the current bright frame parameter) after collecting the target dark frame image, and fuse the target dark frame image and the current bright frame image to obtain a de-halo image.

[0047] It should also be noted that in the image fusion processing process of this application, it is possible to perform global fusion on the target dark frame image and the initial bright frame image. That is, each pixel point in the de-halo image is obtained through image fusion processing. Although the global fusion method can reduce the halo phenomenon in the image, it may also cause the overall brightness of the image to decrease, affecting subsequent processing such as image detection of the image.

[0048] Therefore, the image fusion processing process of this application can also be local fusion, that is, a part of the pixel points in the de-halo image are obtained through image fusion processing. For example, the target area of the target dark frame image and the target area of the initial bright frame image are determined as the fusion area for fusion, and the image content in the initial bright frame image can be retained in other unfused areas, so that the halo is reduced in the target area without affecting the image content in other areas. In the traffic application scenario, it is equivalent to performing image fusion processing on the area where the traffic signal is located to reduce the halo phenomenon generated by the traffic signal, but the image fusion processing is not performed on the area outside the area where the traffic signal is located (such as the road area), thereby avoiding the impact on traffic road detection.

[0049] Optionally, in order to make the bright and dark regions in the obtained anti-halo image natural, the target region may also be enlarged to obtain a fusion region, and then the fusion region of the target dark frame image and the fusion region of the initial bright frame image are fused.

[0050] It can be seen that in this application, by obtaining the initial dark frame image collected according to the current dark frame parameters and the initial bright frame image collected according to the current bright frame parameters, analyzing whether the brightness information of the target region in the initial dark frame image meets the brightness requirement, and then adjusting the current dark frame parameters according to the brightness information of the target region to obtain the target dark frame parameters, so that the target dark frame image collected according to the target dark frame parameters meets the brightness requirement and avoids overexposure and underexposure of the image; by performing image fusion processing on the target dark frame image and the initial bright frame image according to the height information of the target region, an anti-halo image can be obtained, thereby effectively reducing the halo phenomenon in the image.

[0051] Based on the above embodiments, the embodiment of this application describes the step of adjusting the current dark frame parameters according to the brightness information of the target region in the initial dark frame image to obtain the target dark frame parameters. Specifically, the method of this embodiment includes the following steps:

[0052] Obtain the target brightness interval; adjust the current dark frame parameters according to the comparison result between the brightness value of the target region in the initial dark frame image and the left and right boundaries of the target brightness interval to obtain the target dark frame parameters.

[0053] Illustrated in combination with the foregoing embodiments, the target brightness region in this embodiment may refer to a pre-set brightness interval, or may be a brightness interval obtained after some adjustment processing, which is not limited herein. The brightness information may include a brightness value, and there are various methods for obtaining the brightness value, which are not limited herein. For example, the brightness value in the raw domain of the image may be directly obtained, or the Y value in the YUV information of the image may be used as the brightness value, etc. In this embodiment, the brightness in the raw domain of the image is mainly used as an example for illustration. The brightness value of the target region may be a statistical value determined according to the brightness values of each pixel in the target region (such as average value, weighted average value, mode, median, etc., which are not limited herein).

[0054] Exemplarily, the target brightness interval [a, b] has a left boundary a and a right boundary b. Through the comparison result between the brightness value of the target region and the left and right boundaries of the target brightness interval, it can be determined whether the brightness value of the target region meets the brightness requirement. If it does not meet the brightness requirement, the current dark frame parameters need to be adjusted to obtain the target dark frame parameters, so that the brightness value of the target region in the target dark frame image collected by the image acquisition device according to the target dark frame parameters meets the brightness requirement.

[0055] Based on the above embodiments, the embodiments of the present application will describe the step of adjusting the current dark frame parameter according to the comparison result between the brightness value of the target area in the initial dark frame image and the left and right boundaries of the target brightness interval to obtain the target dark frame parameter. Among them, the current dark frame parameter includes the dark frame exposure parameter. Specifically, the method of this embodiment includes the following steps:

[0056] If the brightness value is less than the left boundary of the target brightness interval, the dark frame exposure parameter is increased to obtain the target dark frame parameter; if the brightness value is greater than the right boundary of the target brightness interval, the dark frame exposure parameter is decreased to obtain the target dark frame parameter.

[0057] Combined with the foregoing embodiments, the current dark frame parameter includes the dark frame exposure parameter, and the dark frame exposure parameter is a parameter used to control the exposure of the image acquisition device when acquiring a dark frame image. When the brightness of the target area in the dark frame image does not meet the brightness requirement, the brightness of the subsequent acquired dark frame image can be changed by adjusting the dark frame exposure parameter.

[0058] Exemplarily, if the brightness value of the target area in the initial dark frame image is less than the left boundary of the target brightness interval, it indicates that the target area is underexposed, and the dark frame exposure parameter needs to be increased to obtain the target dark frame parameter, so that the brightness value of the target area of the target dark frame image acquired subsequently according to the target dark frame parameter can be within the preset brightness interval. Similarly, if the brightness value of the target area in the initial dark frame image is greater than the right boundary of the target brightness interval, it indicates that the target area is overexposed, and the dark frame exposure parameter needs to be decreased to obtain the target dark frame parameter.

[0059] Another exemplarily, if the brightness value of the target area in the initial dark frame image is greater than or equal to the left boundary of the target brightness interval and less than or equal to the right boundary of the target brightness interval, that is, within the target brightness interval of the brightness value, it indicates that the brightness value of the target area meets the brightness requirement, and the current dark frame parameter can be determined as the target dark frame parameter without adjusting the dark frame exposure parameter.

[0060] Based on the above embodiments, the embodiments of the present application will describe the step of obtaining the target dark frame image acquired according to the target dark frame parameter. Specifically, the method of this embodiment includes the following steps:

[0061] Acquire the dark frame image to be processed according to the target dark frame parameter; compare the original brightness of the dark frame image to be processed with the preset brightness threshold to obtain a brightness comparison result; in response to the brightness comparison result indicating that the original brightness is less than the preset brightness threshold, increase the original brightness to obtain the target dark frame image; in response to the brightness comparison result indicating that the original brightness is greater than the preset brightness threshold, decrease the original brightness to obtain the target dark frame image.

[0062] Combined with the foregoing embodiments, in the present application, the brightness range of the dark frame image is restricted by setting the target brightness interval. However, the required brightness ranges of the dark frame images in different application scenarios are not the same. Therefore, in the specific implementation manner of the present application, a method is also provided to open upper-layer parameters: a preset brightness threshold (such as the target brightness that a signal lamp needs to reach), link the original brightness in the raw domain of the image (such as the actual brightness of the signal lamp), and adaptively adjust the brightness on the raw.

[0063] Exemplarily, in a specific application scenario, the preset brightness threshold can be fixed or variable. For example, it can be adjusted after receiving a setting instruction input by the user, and no limitation is made here. In addition, the preset brightness threshold can be a numerical value or a range value, and no limitation is made here either.

[0064] Compare the original brightness (raw brightness) of the target area in the dark frame image with the preset brightness threshold to obtain a brightness comparison result. In response to the brightness comparison result indicating that the original brightness is less than the preset brightness threshold, increase the original brightness; in response to the brightness comparison result indicating that the original brightness is greater than the preset brightness threshold, decrease the original brightness. Thus, the brightness of the target area on the raw can be directly controlled by the preset brightness threshold. When a brighter brightness of the target area on the raw is required, the preset brightness threshold can be increased, otherwise it can be decreased. Finally, the larger the value of the preset brightness threshold, the brighter the target area on the raw; the smaller the value of the preset brightness threshold, the darker the target area on the raw.

[0065] Based on the above embodiments, the embodiments of the present application describe the steps of obtaining a target dark frame image collected according to target dark frame parameters. Specifically, the method of this embodiment includes the following steps:

[0066] Collect a dark frame image to be processed according to the target dark frame parameters; in response to the brightness information of the target area in the dark frame image to be processed being less than the left boundary of the target brightness interval, use a first brightening parameter to brighten the dark frame image to be processed to obtain a target dark frame image; in response to the brightness information of the target area in the dark frame image to be processed being greater than the right boundary of the target brightness interval, perform a linear mapping process on the dark frame image to be processed to obtain a target dark frame image; in response to the brightness information of the target area in the dark frame image to be processed being within the target brightness interval, perform a brightness transition process on the dark frame image to be processed to obtain a target dark frame image.

[0067] Combined with the foregoing embodiments, the exposure parameters act on the RAW Domain. The implementation process of the foregoing embodiments mainly focuses on the raw data directly output by the image sensor (raw data without image processing). These raw data contain the most original image signal information, can truly reflect the performance of the sensor and the optical system, and avoid the deviation brought by the Image Signal Processor (ISP) algorithm. Usually, it contains information of three components: red (R), green (G), and blue (B).

[0068] During the ISP processing process that exists in the image acquisition process, it is mainly responsible for receiving and processing the raw data of the photosensitive element, which is an important link in the image quality of the entire photo-taking and video-recording. ISP processing is divided into Raw domain processing, RGB domain processing, and YUV domain processing. Raw domain processing includes bad pixel correction, black level correction, lens shading correction, automatic white balance, demosaicing, etc.; RGB domain processing includes Gamma curve correction, color correction matrix, color space conversion; YUV domain processing includes luminance noise reduction, color noise reduction, edge enhancement, hue saturation control, contrast and brightness adjustment.

[0069] Therefore, in addition to adjusting the brightness of the dark frame by adjusting the exposure parameters acting on the raw domain in the foregoing embodiments, the brightness of the dark frame can also be adjusted by the brightening parameters in other domains during the ISP processing process. The method of adjusting the brightness of the raw domain in the foregoing embodiments of the present application and the method of adjusting the ISP brightness in this embodiment can be executed alternatively or all, and no limitation is made here. For example, it can be after comparing and adjusting the original brightness with the preset brightness threshold, and then referring to the ISP processing process in this embodiment for brightness adjustment.

[0070] Exemplarily, since the brightness on the dark frame is relatively dark and there is a large difference compared with the bright frame, if the non-brightened dark frame and bright frame are directly used for fusion processing, although the display effect of the target area is optimized, other areas may appear darker and the picture is not natural. The present application can perform further ISP brightening processing on the dark frame collected according to the target dark frame parameters to ensure that there are details in the dark area of the image and the halo phenomenon in the target area is acceptable.

[0071] Specifically, in the ISP, modules with different brightening intensities (i.e., brightening templates corresponding to different brightening parameters) can be set according to different brightness intervals to perform special ISP brightening processing on dark frames. For example, the Gamma brightening module (adjusting the brightness of the image by adjusting the Gamma curve) can be referred to. Combining the description of the foregoing embodiments, the brightness interval of the target area can be [a, b]. Map the interval values to the brightening module. If the brightness value of the target area is lower than a, a stronger brightening intensity (the first brightening parameter) is used; if the brightness value of the target area is higher than b, a linear mapping process is performed on its brightness value without brightening processing, and the original brightening parameter is maintained; if the brightness value of the target area is within the interval [a, b], a brightness transition process can be performed on the target area so that the brightness of the target area is gradually brightened. Among them, the first brightening parameter should be greater than the original brightening parameter and should also be greater than the brightening parameter used in the brightness transition process. The specific method of the brightness transition process can refer to the Gamma correction technology. By adjusting the brightness through the Gamma curve, the brightness change can be made not abrupt.

[0072] Based on the foregoing embodiments, the embodiments of the present application describe the steps before adjusting the current dark frame parameters according to the brightness information of the target area in the initial dark frame image to obtain the target dark frame parameters. Among them, the target area includes the traffic signal light area. Specifically, the method of this embodiment includes the following steps:

[0073] Perform target detection processing on the initial dark frame image to obtain the traffic signal light area in the initial dark frame image; or, in response to the received area selection instruction, determine the traffic signal light area in the initial dark frame image according to the area selection instruction; determine the average red brightness of the traffic signal light area as the brightness information.

[0074] Combined with the foregoing embodiments, an example is now given in conjunction with one of the specific scenarios to which the present application can be applied. The halo processing method of the present application can be applied to a traffic scene, especially when the traffic scene includes traffic signal lights. When collecting images in a traffic scene, it is easy to be affected by traffic signal lights, resulting in a halo phenomenon in the collected images. The target area of the present application is also the area where the traffic signal light is located.

[0075] Exemplarily, reference can be made to Figure 2 as shown in Figure 2 is an exemplary schematic diagram of a traffic signal light area in the image-based halo processing method of the present application. Before adjusting the current dark frame parameters of the initial dark frame image, it is necessary to first determine the traffic signal light area in the initial dark frame image, and then judge whether it is necessary to adjust the dark frame acquisition parameters according to the brightness information of the traffic signal light area.

[0076] Among them, the method for determining the traffic signal light area in the initial dark frame image may include, but is not limited to, performing object detection processing on the initial dark frame image according to methods such as image template matching and neural network processing to obtain the traffic signal light area in the initial dark frame image; in addition, it may also be to display the initial dark frame image through a display interface such as a web, and in response to the received area selection instruction input by the user (such as drawing a region box), determine the traffic signal light area in the initial dark frame image according to the area selection instruction. It should be noted that there may be one or more traffic signal light areas in the same image, which is not limited here.

[0077] For example, in this application, a traffic signal light area recognition module can be set. After receiving the signal light area drawn by the user on the web, the traffic signal light area recognition module will process this area and output information such as the color state of the signal light and the circumscribed rectangle area of the signal light. In the dark frame, the color state of the signal light and the circumscribed rectangle area of the signal light output by the signal light recognition module can be obtained first. Inside the circumscribed rectangle area of the signal light, the foreground points of the signal light are segmented. Since in this rectangle area, except for the signal light, it is the lamp panel area, the brightness histogram distribution of the image is basically in the shape of two peaks. Considering the real-time requirement of brightness statistics and fully considering the time consumption of the processing process in this application, the global threshold segmentation method is selected to segment the foreground and background of the traffic signal light to accurately obtain relevant information of the signal light (such as color information, brightness information, etc.).

[0078] It should be noted that in this application, considering that in the traffic scene, the red light of the traffic signal light is more likely to be overexposed compared to lights of other colors (such as green lights and yellow lights). Therefore, this application analyzes based on the red brightness, and determines the average red brightness of the traffic signal light area as the brightness information. That is, in the specific implementation process, by statistically calculating the brightness of each red light on the raw data of the initial dark frame image, the average red brightness is obtained, and then the current dark frame parameters are adaptively adjusted to ensure that the brightness of the dark frame image is within a reasonable range. Assume that for the image collected with x bit data, the brightness interval is [a, b], and this interval can also be adjusted according to the value of bit (such as 8 bit, 10 bit, etc., which will not be elaborated here). At this time, the brightness of most signal lights is moderate. Inside the circumscribed rectangle frame of the signal light, the red brightness of the target area is statistically calculated based on the segmented foreground points, and then the average red brightness is obtained. If the average red brightness is relatively high compared to the brightness interval, the exposure parameter is reduced; if the average red brightness is relatively low compared to the brightness interval, the exposure parameter is increased; if the average red brightness is within the brightness interval, the subsequent judgment process is carried out with reference to the foregoing embodiments.

[0079] Based on the above embodiments, the embodiments of the present application will describe the step of performing image fusion processing on the target dark frame image and the initial bright frame image according to the height information of the target area to obtain a dehalo image. Specifically, the method of this embodiment includes the following steps:

[0080] Determine the height information of the target area as the fusion height information; or, determine the fusion height information according to the height information of the target area and a preset fusion parameter, where the fusion height information is greater than or equal to the height information of the target area; determine the target fusion area of the target dark frame image and the initial bright frame image according to the fusion height information; perform fusion processing on the target dark frame image and the initial bright frame image according to the target fusion area to obtain a dehalo image.

[0081] Exemplarily, reference can be made to Figure 3 as shown in Figure 3 is an exemplary image fusion schematic diagram in the image-based halo processing method of the present application.

[0082] Combined with the foregoing embodiments, in the specific implementation process of the present application, according to actual requirements, the target dark frame image and the initial bright frame image can be globally fused, or the target dark frame image and the initial bright frame image can be locally fused. Among them, global fusion is to use all image areas of the target dark frame image and all image areas of the initial bright frame image as the fusion area (usually, the initial dark frame image, the target dark frame image, and the initial bright frame image, etc. are all of the same size), and local fusion is to use local image areas in the target dark frame image and local image areas in the initial bright frame image as the fusion area.

[0083] Therefore, the present application can determine the fusion height information of the fusion area according to the height information of the target area. Specifically, the height information of the target area can be directly determined as the fusion height information, but there may be obvious brightness mutation phenomena in the image obtained by this fusion. Therefore, in order to make the transition area fuse naturally, the fusion height information can also be determined according to the height information of the target area and a preset fusion parameter, and the preset fusion parameter is used to expand the fusion area.

[0084] For example, if there are multiple signal light areas in the image, first compare the height coordinates of each signal light area to obtain the maximum height value h1 (that is, the height information of the target area). In order to make the transition of the target fusion area natural, therefore, the maximum height of the target fusion area can be n times of h1. Furthermore, obtain the pixel row number range of the target fusion area: the minimum row number can be 0, and the maximum row number can be , thereby determining the target fusion area.

[0085] Based on the above embodiments, the embodiments of the present application will describe the step of fusing the target dark frame image and the initial bright frame image according to the target fusion region to obtain the dehaloed image. Specifically, the method of this embodiment includes the following steps:

[0086] According to the height information and the fusion height information, determine the dark frame fusion weight of the target dark frame image corresponding to each pixel row and the bright frame fusion weight of the initial bright frame image in the target fusion region; fuse the target dark frame image and the initial bright frame image according to the dark frame fusion weight and the bright frame fusion weight to obtain the dehaloed image.

[0087] Combined with the foregoing embodiments for description, within the target fusion region, that is, from pixel row 0 to row of the image, set a specific fusion ratio. For the image where all of row 0 can use the dark frame, for rows and above (until reaching the maximum row number of the image), the effect of all using the bright frame can be achieved. For the middle rows of the target fusion region [0, , they can be fused according to linear transition, and finally the fused image is obtained. Its mathematical expression can be:

[0088]

[0089] Wherein, refers to the target dark frame image in yuv format, refers to the initial bright frame image in yuv format, refers to the fused image in yuv format (that is, equivalent to the dehaloed image). This formula represents that when fusing the pixels from row 0 to row in the fused image, according to the height information h1 and the fusion height information , determine the dark frame fusion weight , and the bright frame fusion weight , and then perform image fusion. For the pixels from row to row , the bright frame image can be directly used. It is equivalent that the image in the target fusion region is obtained by fusing the dark frame and the bright frame, while the image outside the target fusion region is obtained from the bright frame. The dehaloed image can be obtained by splicing the fused image in the target fusion region and the bright frame image outside the target fusion region.

[0090] Further, in the present application, the anti-halo intensity can also be set to control the fusion weight of the bright and dark frames. Combining with the foregoing embodiments, it can be known that the preset fusion parameter n value determines the fusion weight of the bright and dark frames. The larger the n, the larger the proportion of the dark frame and the smaller the proportion of the bright frame; the smaller the n, the smaller the proportion of the dark frame and the larger the proportion of the bright frame. Therefore, the upper layer opens the anti-halo intensity parameter and links it with the n value, which is equivalent to using the n value as the anti-halo intensity parameter. The stronger the set anti-halo intensity, the larger the n and the stronger the anti-halo effect; the weaker the set anti-halo intensity, the smaller the n and the weaker the anti-halo effect.

[0091] Exemplarily, the process of determining the anti-halo intensity can be determined according to the received anti-halo instruction input by the user. For example, the stronger the anti-halo intensity represented by the anti-halo instruction input by the user, the larger the n value increases positively. Or it can be through technologies such as artificial intelligence, neural networks, and image processing to detect the image halo in the dark frame image and / or bright frame image, thereby determining the halo intensity in the image, and then determining the anti-halo intensity according to its halo intensity (the halo intensity is positively correlated with the anti-halo intensity), and the n value during image fusion can be adaptively obtained.

[0092] In summary, combining with the foregoing embodiments, it can be known that the halo processing method of the present application is based on the problem of the existence of halo in the image, and proposes a method for image fusion based on the bright frame and the dark frame to achieve anti-halo processing. Among them, the brightness of the dark frame raw can be adaptively adjusted according to the brightness of the target area on the dark frame raw to ensure that the brightness of the target area on the dark frame raw is reasonable and the shape is clear. Then, specific ISP brightening processing can be performed on the dark frame to ensure that there are details in the dark area of the image and the halo effect is acceptable. Finally, according to the selected target area, the fusion range of the bright and dark frames is determined, and the bright and dark frames are fused.

[0093] In addition, the upper layer linkage parameters can also be opened: the preset brightness threshold (the target brightness required for the target area), the ISP brightening parameter, and the anti-halo intensity, etc., to improve the adaptability of the present solution.

[0094] It should be further noted that the execution subject of the halo processing method based on the image can be a halo processing device based on the image. For example, the halo processing method based on the image can be executed by a terminal device, a server, or other processing devices. Among them, the terminal device can be a user equipment (UE), a computer, a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementation manners, the halo processing method based on the image can be implemented by the processor calling the computer-readable instructions stored in the memory.

[0095] Figure 4 is a block diagram of an image-based halo processing device shown in an exemplary embodiment of the present application. As Figure 4 shown, the exemplary image-based halo processing device 400 includes: a first acquisition module 410, a parameter adjustment module 420, a second acquisition module 430, and an image fusion module 440. Specifically:

[0096] The first acquisition module 410 is configured to acquire an initial dark frame image acquired according to the current dark frame parameters and an initial bright frame image acquired according to the current bright frame parameters.

[0097] The parameter adjustment module 420 is configured to adjust the current dark frame parameters according to the brightness information of the target area in the initial dark frame image to obtain target dark frame parameters.

[0098] The second acquisition module 430 is configured to acquire a target dark frame image acquired according to the target dark frame parameters.

[0099] The image fusion module 440 is configured to perform image fusion processing on the target dark frame image and the initial bright frame image according to the height information of the target area to obtain a de-halo image.

[0100] In this exemplary image-based halo processing device, by acquiring the initial dark frame image acquired according to the current dark frame parameters and the initial bright frame image acquired according to the current bright frame parameters, analyzing whether the brightness information of the target area in the initial dark frame image reaches the brightness requirement, and then adjusting the current dark frame parameters according to the brightness information of the target area to obtain target dark frame parameters, so that the target dark frame image acquired according to the target dark frame parameters meets the brightness requirement and avoids overexposure and underexposure of the image; performing image fusion processing on the target dark frame image and the initial bright frame image according to the height information of the target area can obtain a de-halo image, thereby effectively reducing the halo phenomenon in the image.

[0101] It should be noted that the device provided in the above embodiment and the method provided in the above embodiment belong to the same concept. The specific manners in which each module and unit perform operations have been described in detail in the method embodiment and will not be repeated here. In practical applications, the device provided in the above embodiment can, according to needs, allocate the above functions to different functional modules, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above. This is not limited here.

[0102] Among them, the functions of each module can be referred to in the embodiment of the image-based halo processing method and will not be repeated here.

[0103] Please refer to Figure 5 , Figure 5It is a schematic structural diagram of an embodiment of the electronic device of the present application. The electronic device 100 includes a memory 101 and a processor 102. The processor 102 is used to execute the program instructions stored in the memory 101 to implement the steps in any of the above embodiments of the image-based halo processing method. In a specific implementation scenario, the electronic device 100 may include, but is not limited to: a microcomputer, a server. In addition, the electronic device 100 may also include mobile devices such as a laptop computer, a tablet computer, etc., which are not limited herein.

[0104] Specifically, the processor 102 is used to control itself and the memory 101 to implement the steps in any of the above embodiments of the image-based halo processing method. The processor 102 may also be referred to as a CPU (Central Processing Unit, central processing unit). The processor 102 may be an integrated circuit chip with signal processing capabilities. The processor 102 may also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. In addition, the processor 102 may be implemented jointly by integrated circuit chips.

[0105] In this exemplary electronic device, an initial dark frame image acquired according to the current dark frame parameters and an initial bright frame image acquired according to the current bright frame parameters are obtained. Analyze whether the brightness information of the target area in the initial dark frame image reaches the brightness requirement. Furthermore, the current dark frame parameters can be adjusted according to the brightness information of the target area to obtain the target dark frame parameters, so that the target dark frame image acquired according to the target dark frame parameters meets the brightness requirement, avoiding overexposure and underexposure of the image; performing image fusion processing on the target dark frame image and the initial bright frame image according to the height information of the target area, a dehalo image can be obtained, thereby effectively reducing the halo phenomenon in the image.

[0106] Please refer to Figure 6 , Figure 6 It is a schematic structural diagram of an embodiment of the computer-readable storage medium of the present application. The computer-readable storage medium 110 stores program instructions 111 that can be run by a processor. The program instructions 111 are used to implement the steps in any of the above embodiments of the image-based halo processing method.

[0107] In this exemplary storage medium, by running the program instructions in the storage medium, an initial dark frame image acquired according to the current dark frame parameters and an initial bright frame image acquired according to the current bright frame parameters are obtained. The brightness information of the target area in the initial dark frame image is analyzed to determine whether it meets the brightness requirement. Furthermore, the current dark frame parameters can be adjusted based on the brightness information of the target area to obtain target dark frame parameters, so that the target dark frame image acquired according to the target dark frame parameters meets the brightness requirement, avoiding overexposure and underexposure of the image. Image fusion processing is performed on the target dark frame image and the initial bright frame image according to the height information of the target area, and then a de-halo image can be obtained, thereby effectively reducing the halo phenomenon in the image.

[0108] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the methods described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0109] The above descriptions of the various embodiments tend to emphasize the differences between the various embodiments. Their similarities or similarities can be referred to each other. For the sake of brevity, they will not be repeated in this article.

[0110] In several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in an electrical, mechanical or other form.

[0111] In addition, each functional unit in various embodiments of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of a software functional unit. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, may be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods in various embodiments of the present application. The foregoing storage medium includes: various media such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disc that can store program codes.

Claims

1. An image-based halo processing method, characterized in that The method includes: Obtaining an initial dark frame image collected according to current dark frame parameters and an initial bright frame image collected according to current bright frame parameters; Adjusting the current dark frame parameters according to the brightness information of the target area in the initial dark frame image to obtain target dark frame parameters; Obtaining a target dark frame image collected according to the target dark frame parameters; Performing image fusion processing on the target dark frame image and the initial bright frame image according to the height information of the target area to obtain an anti-halo image; The step of performing image fusion processing on the target dark frame image and the initial bright frame image according to the height information of the target area to obtain an anti-halo image includes: determining the height information of the target area as the fusion height information; or determining the fusion height information according to the height information of the target area and preset fusion parameters, where the fusion height information is greater than or equal to the height information of the target area; determining a target fusion area of the target dark frame image and the initial bright frame image according to the fusion height information; and performing fusion processing on the target dark frame image and the initial bright frame image according to the target fusion area to obtain the anti-halo image.

2. The method according to claim 1, wherein The adjusting the current dark frame parameters according to the brightness information of the target area in the initial dark frame image to obtain target dark frame parameters includes: Obtaining a target brightness interval; Adjusting the current dark frame parameters according to the comparison result between the brightness value of the target area in the initial dark frame image and the left and right boundaries of the target brightness interval to obtain the target dark frame parameters.

3. The method according to claim 2, wherein The current dark frame parameters include dark frame exposure parameters, and the adjusting the current dark frame parameters according to the comparison result between the brightness value of the target area in the initial dark frame image and the left and right boundaries of the target brightness interval to obtain the target dark frame parameters includes: If the brightness value is less than the left boundary of the target brightness interval, increasing the dark frame exposure parameters to obtain the target dark frame parameters; If the brightness value is greater than the right boundary of the target brightness interval, decreasing the dark frame exposure parameters to obtain the target dark frame parameters.

4. The method according to claim 1, characterized in that, The obtaining a target dark frame image collected according to the target dark frame parameters includes: Collecting a to-be-processed dark frame image according to the target dark frame parameters; Comparing the original brightness of the to-be-processed dark frame image with a preset brightness threshold to obtain a brightness comparison result; In response to the brightness comparison result indicating that the original brightness is less than the preset brightness threshold, increasing the original brightness to obtain the target dark frame image; In response to the brightness comparison result indicating that the original brightness is greater than the preset brightness threshold, decreasing the original brightness to obtain the target dark frame image.

5. The method according to claim 1, characterized in that The obtaining a target dark frame image collected according to the target dark frame parameters includes: Collecting a to-be-processed dark frame image according to the target dark frame parameters; In response to the brightness information of the target area in the to-be-processed dark frame image being less than the left boundary of the target brightness interval, brightening the to-be-processed dark frame image using a first brightening parameter to obtain the target dark frame image; In response to the brightness information of the target area in the to-be-processed dark frame image being greater than the right boundary of the target brightness interval, perform linear mapping processing on the to-be-processed dark frame image to obtain the target dark frame image; In response to the brightness information of the target area in the to-be-processed dark frame image being within the target brightness interval, perform brightness transition processing on the to-be-processed dark frame image to obtain the target dark frame image.

6. The method according to claim 1, wherein The target area includes a traffic signal light area. Before adjusting the current dark frame parameters according to the brightness information of the target area in the initial dark frame image to obtain target dark frame parameters, the method further includes: Perform target detection processing on the initial dark frame image to obtain the traffic signal light area in the initial dark frame image; Alternatively, in response to a received area selection instruction, determine the traffic signal light area in the initial dark frame image according to the area selection instruction; Determine the average red brightness of the traffic signal light area as the brightness information.

7. The method according to claim 1, wherein The fusing the target dark frame image and the initial bright frame image according to the target fusion area to obtain the anti-halo image includes: According to the height information and the fusion height information, determine, in the target fusion area, the dark frame fusion weight of the target dark frame image corresponding to each pixel row and the bright frame fusion weight of the initial bright frame image; Fuse the target dark frame image and the initial bright frame image according to the dark frame fusion weight and the bright frame fusion weight to obtain the anti-halo image.

8. An electronic device, characterized in that, It includes a memory and a processor, and the processor is configured to execute program instructions stored in the memory to implement the method according to any one of claims 1 to 7.

9. A computer-readable storage medium having program instructions stored thereon, characterized in that, When the program instructions are executed by the processor, the method according to any one of claims 1 to 7 is implemented.

Images