Image-based halo processing method and device, and storage medium

Abstract

The invention discloses an image-based halo processing method and device and a storage medium, and the method comprises the steps: obtaining an initial dark frame image collected according to a current dark frame parameter and an initial bright frame image collected according to a current bright frame parameter, whether the brightness information of the target area in the initial dark frame image meets the brightness requirement is analyzed, and then the current dark frame parameter can be adjusted according to the brightness information of the target area to obtain the target dark frame parameter, so that the target dark frame image collected according to the target dark frame parameter meets the brightness requirement, and overexposure and underexposure of the image are avoided; and performing image fusion processing on the target dark frame image and the initial bright frame image according to the height information of the target area to obtain a halo-reducing image. According to the scheme, the halo phenomenon in the image can be effectively reduced.

Description

Technical Field

[0001] This application relates to the field of image processing technologies, and particularly to an image-based halo processing method, device, and storage medium. Background Art

[0002] In the field of image processing, a halo refers to a blurred effect that diffuses from the edges of an image after image development. This phenomenon is usually caused by reasons such as light scattering or the characteristics of an optical lens. Especially when observing bright objects in a dark scene, the halo phenomenon is very likely to occur.

[0003] For example, in a traffic application scenario, if the environment where the image acquisition device is located is relatively dark, the image acquisition device will set relatively high exposure parameters to acquire a bright image. However, when there are luminous objects such as vehicle lights and traffic signals in the environment, it may cause an obvious halo phenomenon in the image.

[0004] When the halo phenomenon is severe, it will affect the accurate analysis of the image. For example, it is impossible to distinguish information such as the direction and color of the signal light in the image. Summary of the Invention

[0005] This application provides at least an image-based halo processing method, apparatus, device, and computer-readable storage medium.

[0006] In a first aspect of this application, an image-based halo processing method is provided, including: obtaining an initial dark frame image collected according to current dark frame parameters and an initial bright frame image collected according to current bright frame parameters; adjusting the current dark frame parameters according to the brightness information of a target area in the initial dark frame image to obtain target dark frame parameters; obtaining a target dark frame image collected according to the target dark frame parameters; and performing image fusion processing on the target dark frame image and the initial bright frame image according to the height information of the target area to obtain a de-halo image.

[0007] In an embodiment, the adjusting the current dark frame parameters according to the brightness information of a target area in the initial dark frame image to obtain target dark frame parameters includes: obtaining a target brightness interval; and adjusting the current dark frame parameters according to the comparison result between the brightness value of the target area in the initial dark frame image and the left and right boundaries of the target brightness interval to obtain the target dark frame parameters.

[0008] In one embodiment, the current dark frame parameter includes a dark frame exposure parameter. Adjusting the current dark frame parameter according to the comparison result between the brightness value of the target area in the initial dark frame image and the left and right boundaries of the target brightness range to obtain the target dark frame parameter includes: if the brightness value is less than the left boundary of the target brightness range, increasing the dark frame exposure parameter to obtain the target dark frame parameter; if the brightness value is greater than the right boundary of the target brightness range, decreasing the dark frame exposure parameter to obtain the target dark frame parameter.

[0009] In one embodiment, obtaining the target dark frame image collected according to the target dark frame parameter includes: collecting a to-be-processed dark frame image according to the target dark frame parameter; comparing the original brightness of the to-be-processed dark frame image with a preset brightness threshold to obtain a brightness comparison result; in response to the brightness comparison result indicating that the original brightness is less than the preset brightness threshold, increasing the original brightness to obtain the target dark frame image; in response to the brightness comparison result indicating that the original brightness is greater than the preset brightness threshold, decreasing the original brightness to obtain the target dark frame image.

[0010] In one embodiment, obtaining the target dark frame image collected according to the target dark frame parameter includes: collecting a to-be-processed dark frame image according to the target dark frame parameter; in response to the brightness information of the target area in the to-be-processed dark frame image being less than the left boundary of the target brightness range, using a first brightening parameter to brighten the to-be-processed dark frame image to obtain the target dark frame image; in response to the brightness information of the target area in the to-be-processed dark frame image being greater than the right boundary of the target brightness range, performing a linear mapping process on the to-be-processed dark frame image to obtain the target dark frame image; in response to the brightness information of the target area in the to-be-processed dark frame image being within the target brightness range, performing a brightness transition process on the to-be-processed dark frame image to obtain the target dark frame image.

[0011] In one embodiment, the target area includes a traffic signal light area. Before adjusting the current dark frame parameter according to the brightness information of the target area in the initial dark frame image to obtain the target dark frame parameter, the method further includes: performing target detection processing on the initial dark frame image to obtain the traffic signal light area in the initial dark frame image; or, in response to a received area selection instruction, determining the traffic signal light area in the initial dark frame image according to the area selection instruction; determining the average red brightness of the traffic signal light area as the brightness information.

[0012] In one embodiment, performing image fusion processing on the target dark frame image and the initial bright frame image according to the height information of the target area to obtain a de-halo image includes: Determining the height information of the target area as the fusion height information; or determining the fusion height information according to the height information of the target area and a preset fusion parameter, where the fusion height information is greater than or equal to the height information of the target area; determining a target fusion area of the target dark frame image and the initial bright frame image according to the fusion height information; and performing fusion processing on the target dark frame image and the initial bright frame image according to the target fusion area to obtain the de-halo image.

[0013] In one embodiment, performing fusion processing on the target dark frame image and the initial bright frame image according to the target fusion area to obtain the de-halo image includes: determining, according to the height information and the fusion height information, a dark frame fusion weight of the target dark frame image corresponding to each pixel row and a bright frame fusion weight of the initial bright frame image in the target fusion area; and performing fusion processing on the target dark frame image and the initial bright frame image according to the dark frame fusion weight and the bright frame fusion weight to obtain the de-halo image.

[0014] A second aspect of the present application provides an image-based halo processing device, including: a first acquisition module, configured to acquire an initial dark frame image acquired according to a current dark frame parameter and an initial bright frame image acquired according to a current bright frame parameter; a parameter adjustment module, configured to adjust the current dark frame parameter according to the brightness information of a target area in the initial dark frame image to obtain a target dark frame parameter; a second acquisition module, configured to acquire a target dark frame image acquired according to the target dark frame parameter; and an image fusion module, configured to perform image fusion processing on the target dark frame image and the initial bright frame image according to the height information of the target area to obtain a de-halo image.

[0015] A third aspect of the present application provides an electronic device, including a memory and a processor, where the processor is configured to execute program instructions stored in the memory to implement the above-mentioned image-based halo processing method.

[0016] A fourth aspect of the present application provides a computer-readable storage medium, on which program instructions are stored, and when the program instructions are executed by a processor, the above-mentioned image-based halo processing method is implemented.

[0017] In the above solution, an initial dark frame image collected according to the current dark frame parameters and an initial bright frame image collected according to the current bright frame parameters are obtained. By analyzing whether the brightness information of the target area in the initial dark frame image reaches the brightness requirement, the current dark frame parameters can be adjusted according to the brightness information of the target area to obtain target dark frame parameters, so that the target dark frame image collected according to the target dark frame parameters meets the brightness requirement and avoids overexposure and underexposure of the image. Image fusion processing is performed on the target dark frame image and the initial bright frame image according to the height information of the target area, and then a dehalo image can be obtained, thereby effectively reducing the halo phenomenon in the image.

[0018] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and do not limit this application. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The accompanying drawings herein are incorporated into the specification and constitute a part of this specification. These drawings illustrate embodiments consistent with this application and, together with the specification, are used to explain the technical solutions of this application.

[0020] Figure 1 is a schematic flowchart of an exemplary embodiment of the image-based halo processing method of this application; Figure 2 is a schematic diagram of an exemplary traffic signal light area in the image-based halo processing method of this application; Figure 3 is a schematic diagram of an exemplary image fusion in the image-based halo processing method of this application; Figure 4 is a block diagram of an image-based halo processing device shown in an exemplary embodiment of this application; Figure 5 is a schematic structural diagram of an embodiment of an electronic device of this application; Figure 6 is a schematic structural diagram of an embodiment of a computer-readable storage medium of this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0021] The solutions of the embodiments of this application will be described in detail below with reference to the accompanying drawings of the specification.

[0022] In the following description, specific details such as specific system structures, interfaces, and technologies are presented for the purpose of illustration rather than limitation, in order to thoroughly understand this application.

[0023] In this text, the term "and / or" is merely a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. Additionally, in this text, the character " / " generally indicates that the associated objects before and after are in an "or" relationship. Furthermore, "multiple" in this text means two or more than two. Additionally, the term "at least one" in this text means any one of multiple or any combination of at least two of multiple. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set composed of A, B, and C.

[0024] In the field of image processing, halation refers to the blurred effect that spreads out from the image edge after image development. This phenomenon is usually caused by reasons such as light scattering, light reflection, or the characteristics of the optical lens. Especially when observing bright objects in a dark scene, the halation phenomenon is very likely to occur. For example, in a traffic application scenario, if the environment where the image acquisition device is located is relatively dark, the image acquisition device will set relatively high exposure parameters to acquire bright images. However, when there are luminous objects such as vehicle lights and traffic lights in the environment, it may cause an obvious halation phenomenon in the image.

[0025] It should be noted that the halation processing method of this application can be used in traffic application scenarios or other image acquisition application scenarios, and no limitation is made here. For ease of understanding, this application mainly takes the traffic application scenario as an example for illustration.

[0026] Please refer to Figure 1 , Figure 1 is a schematic flowchart of an exemplary embodiment of the image-based halation processing method of this application. Specifically, it can include the following steps: Step S110, obtain an initial dark frame image collected according to the current dark frame parameters and an initial bright frame image collected according to the current bright frame parameters.

[0027] It should be noted that in the image acquisition scenario, the image quality is often optimized by collecting bright and dark frames. For example, techniques such as Wide Dynamic Range (WDR) and High Dynamic Range (HDR) are not elaborated here.

[0028] Among them, the overall brightness of the bright frame image is greater than that of the dark frame image. The bright frame is obtained based on the bright frame acquisition parameters (the initial bright frame image is collected according to the current bright frame parameters), and the dark frame is obtained based on the dark frame acquisition parameters (the initial dark frame image is collected according to the current dark frame parameters). The acquisition parameters can include but are not limited to exposure parameters, brightening parameters, etc. By adjusting these acquisition parameters, the adjustment of the image brightness can be achieved.

[0029] It should also be noted that in the image acquisition process of this application, bright frames and dark frames can be obtained periodically or randomly, and there is no limitation here. For example, it can be to acquire a dark frame after acquiring a bright frame, or to acquire a dark frame after acquiring two bright frames, or to acquire two dark frames after acquiring two bright frames. It can also be to randomly acquire some dark frames after randomly acquiring some bright frames, etc., which will not be elaborated here. And this application also does not limit the acquisition order of the initial dark frame image and the initial bright frame image.

[0030] Thus, all the image frames acquired by the image acquisition device can be classified into bright frames and dark frames. The bright frame images can be used for tasks that require analyzing image content such as image detection, and the dark frame images can be used for image processing tasks such as image quality optimization.

[0031] Step S120: Adjust the current dark frame parameters according to the brightness information of the target area in the initial dark frame image to obtain the target dark frame parameters.

[0032] The target area is the area in the initial dark frame image that needs to be analyzed for underexposure or overexposure, usually the light source area.

[0033] Exemplarily, the target area in the initial dark frame image can be obtained, and its brightness information is compared with the obtained target brightness range, and then whether to adjust the current dark frame parameters is determined according to the comparison result to obtain the target dark frame parameters. Among them, the target brightness range is used to judge whether the target area is underexposed or overexposed according to the brightness information of the target area. The target brightness range can be preset and fixed, or preset and support dynamic adjustment, and there is no limitation here.

[0034] For example, if the brightness information of the target area is within the target brightness range [a, b], it indicates that the brightness of the target area meets the brightness requirement, and the current dark frame parameters can be determined as the target dark frame parameters. If the brightness information of the target area is outside the target brightness range [a, b], it indicates that the brightness of the target area does not meet the brightness requirement, and the current dark frame parameters need to be adjusted accordingly, and the adjusted dark frame parameters are used as the target dark frame parameters.

[0035] Step S130: Obtain the target dark frame image acquired according to the target dark frame parameters.

[0036] The above steps are referred to for illustration. If it is determined in the above steps that the current dark frame parameter does not meet the brightness requirement, after adjusting the current dark frame parameter to obtain the target dark frame parameter, the target dark frame image collected by the target dark frame parameter should meet the brightness requirement. If it is determined in the above steps that the current dark frame parameter meets the brightness requirement, the current dark frame parameter can be determined as the target dark frame parameter, and image acquisition can continue with the same parameter. In this case, the target dark frame image obtained can be a newly collected dark frame image according to the target dark frame parameter, or the initial dark frame image can be used as the target dark frame image, which is not limited here.

[0037] Step S140: Perform image fusion processing on the target dark frame image and the initial bright frame image according to the height information of the target area to obtain a de-halo image.

[0038] The above steps are referred to for illustration. The target dark frame image is an image that meets the brightness requirement. By performing image fusion processing on the target dark frame image and the initial bright frame image, the halo phenomenon in the bright frame image can be reduced to obtain a de-halo image.

[0039] It should be noted that in the specific implementation process of this application, in addition to fusing the target dark frame image and the initial bright frame image, it is also possible to collect a new bright frame image (the current bright frame image) with the current bright frame parameter (or the parameter after adjusting the current bright frame parameter) after collecting the target dark frame image, and fuse the target dark frame image and the current bright frame image to obtain a de-halo image.

[0040] It should also be noted that in the image fusion processing process of this application, the target dark frame image and the initial bright frame image can be globally fused, which means that each pixel point in the de-halo image is obtained through image fusion processing. Although the global fusion method can reduce the halo phenomenon in the image, it may also cause the overall brightness of the image to decrease, affecting subsequent processing such as image detection of the image.

[0041] Therefore, the image fusion processing process of this application can also be local fusion, that is, a part of the pixel points in the de-halo image are obtained through image fusion processing. For example, the target area of the target dark frame image and the target area of the initial bright frame image are determined as the fusion area for fusion, and the image content in the initial bright frame image can be retained in other unfused areas, so that the halo is reduced in the target area without affecting the image content in other areas. In the traffic application scenario, it is equivalent to performing image fusion processing on the area where the traffic signal is located to reduce the halo phenomenon caused by the traffic signal, but no image fusion processing is performed on the area outside the area where the traffic signal is located (such as the road area), thus avoiding the impact on traffic road detection.

[0042] Optionally, in order to make the bright and dark regions in the obtained anti-halo image natural, the target region may also be enlarged to obtain a fusion region, and then the fusion region of the target dark frame image and the fusion region of the initial bright frame image are fused.

[0043] It can be seen that in this application, by obtaining the initial dark frame image collected according to the current dark frame parameters and the initial bright frame image collected according to the current bright frame parameters, analyzing whether the brightness information of the target region in the initial dark frame image meets the brightness requirement, and then adjusting the current dark frame parameters according to the brightness information of the target region to obtain the target dark frame parameters, so that the target dark frame image collected according to the target dark frame parameters meets the brightness requirement and avoids overexposure and underexposure of the image; performing image fusion processing on the target dark frame image and the initial bright frame image according to the height information of the target region can obtain an anti-halo image, thereby effectively reducing the halo phenomenon in the image.

[0044] Based on the above embodiments, the embodiments of this application illustrate the step of adjusting the current dark frame parameters according to the brightness information of the target region in the initial dark frame image to obtain the target dark frame parameters. Specifically, the method of this embodiment includes the following steps: Obtain a target brightness interval; adjust the current dark frame parameters according to the comparison result between the brightness value of the target region in the initial dark frame image and the left and right boundaries of the target brightness interval to obtain the target dark frame parameters.

[0045] Combined with the foregoing embodiments, the target brightness region in this embodiment may refer to a pre-set brightness interval, or may be a brightness interval obtained after some adjustment processing, which is not limited here. The brightness information may include a brightness value, and there are various methods for obtaining the brightness value, which are not limited here. For example, the brightness value in the raw domain of the image may be directly obtained, or the Y value in the YUV information of the image may be used as the brightness value, etc. In this embodiment, the brightness in the raw domain of the image is mainly used as an example for illustration. The brightness value of the target region may be a statistical value determined according to the brightness values of each pixel in the target region (such as average value, weighted average value, mode, median, etc., which are not limited here).

[0046] Exemplarily, the target brightness interval [a, b] has a left boundary a and a right boundary b. Through the comparison result between the brightness value of the target region and the left and right boundaries of the target brightness interval, it can be determined whether the brightness value of the target region meets the brightness requirement. If it does not meet the brightness requirement, the current dark frame parameters need to be adjusted to obtain the target dark frame parameters, so that the brightness value of the target region in the target dark frame image collected by the image acquisition device according to the target dark frame parameters meets the brightness requirement.

[0047] Based on the above embodiments, the embodiments of the present application illustrate the step of adjusting the current dark frame parameter according to the comparison result between the brightness value of the target area in the initial dark frame image and the left and right boundaries of the target brightness interval to obtain the target dark frame parameter. Among them, the current dark frame parameter includes the dark frame exposure parameter. Specifically, the method of this embodiment includes the following steps: If the brightness value is less than the left boundary of the target brightness interval, the dark frame exposure parameter is increased to obtain the target dark frame parameter; if the brightness value is greater than the right boundary of the target brightness interval, the dark frame exposure parameter is decreased to obtain the target dark frame parameter.

[0048] Combined with the foregoing embodiments for illustration, the current dark frame parameter includes the dark frame exposure parameter, and the dark frame exposure parameter is a parameter used to control the exposure of the image acquisition device when acquiring a dark frame image. When the brightness of the target area in the dark frame image does not meet the brightness requirement, the brightness of the subsequently acquired dark frame image can be changed by adjusting the dark frame exposure parameter.

[0049] Exemplarily, if the brightness value of the target area in the initial dark frame image is less than the left boundary of the target brightness interval, it indicates that the target area is underexposed, and the dark frame exposure parameter needs to be increased to obtain the target dark frame parameter, so that the brightness value of the target area of the target dark frame image subsequently acquired according to the target dark frame parameter can be within the preset brightness interval. Similarly, if the brightness value of the target area in the initial dark frame image is greater than the right boundary of the target brightness interval, it indicates that the target area is overexposed, and the dark frame exposure parameter needs to be decreased to obtain the target dark frame parameter.

[0050] Another exemplarily, if the brightness value of the target area in the initial dark frame image is greater than or equal to the left boundary of the target brightness interval and less than or equal to the right boundary of the target brightness interval, that is, within the target brightness interval of the brightness value, it indicates that the brightness value of the target area meets the brightness requirement, and the current dark frame parameter can be determined as the target dark frame parameter without adjusting the dark frame exposure parameter.

[0051] Based on the above embodiments, the embodiments of the present application illustrate the step of obtaining the target dark frame image acquired according to the target dark frame parameter. Specifically, the method of this embodiment includes the following steps: Acquire the dark frame image to be processed according to the target dark frame parameter; compare the original brightness of the dark frame image to be processed with the preset brightness threshold to obtain a brightness comparison result; in response to the brightness comparison result indicating that the original brightness is less than the preset brightness threshold, increase the original brightness to obtain the target dark frame image; in response to the brightness comparison result indicating that the original brightness is greater than the preset brightness threshold, decrease the original brightness to obtain the target dark frame image.

[0052] In combination with the foregoing embodiments, in the present application, the brightness range of the dark frame image is restricted by setting a target brightness interval. However, the required brightness ranges of the dark frame images in different application scenarios are not the same. Therefore, in the specific implementation manner of the present application, a method is also provided to open upper-layer parameters: a preset brightness threshold (such as the target brightness that a signal light needs to reach), link the original brightness in the raw domain of the image (such as the actual brightness of the signal light), and adaptively adjust the brightness on the raw.

[0053] Exemplarily, in a specific application scenario, the preset brightness threshold can be fixed or variable. For example, it can be adjusted after receiving a setting instruction input by the user, and no limitation is made here. Additionally, the preset brightness threshold can be a numerical value or a range value, and no limitation is made here either.

[0054] Compare the original brightness (raw brightness) of the target area in the dark frame image with the preset brightness threshold to obtain a brightness comparison result. In response to the brightness comparison result indicating that the original brightness is less than the preset brightness threshold, increase the original brightness; in response to the brightness comparison result indicating that the original brightness is greater than the preset brightness threshold, decrease the original brightness. Thus, the brightness of the target area on the raw can be directly controlled by the preset brightness threshold. When the brightness of the target area on the raw needs to be brighter, the preset brightness threshold can be increased; otherwise, the preset brightness threshold can be decreased. Finally, the larger the value of the preset brightness threshold, the brighter the target area on the raw; the smaller the value of the preset brightness threshold, the darker the target area on the raw.

[0055] Based on the above embodiments, the embodiments of the present application illustrate the steps of obtaining a target dark frame image collected according to target dark frame parameters. Specifically, the method of this embodiment includes the following steps: Collect a dark frame image to be processed according to the target dark frame parameters; in response to the brightness information of the target area in the dark frame image to be processed being less than the left boundary of the target brightness interval, use a first brightening parameter to brighten the dark frame image to be processed to obtain a target dark frame image; in response to the brightness information of the target area in the dark frame image to be processed being greater than the right boundary of the target brightness interval, perform a linear mapping process on the dark frame image to be processed to obtain a target dark frame image; in response to the brightness information of the target area in the dark frame image to be processed being within the target brightness interval, perform a brightness transition process on the dark frame image to be processed to obtain a target dark frame image.

[0056] Combined with the foregoing embodiments, the exposure parameters act on the raw domain (RAW Domain). The implementation process of the foregoing embodiments mainly targets the raw data directly output by the image sensor (raw data without image processing). These raw data contain the most primitive image signal information, can truly reflect the performance of the sensor and the optical system, and avoid the deviation brought by the image signal processor (ISP) algorithm. Usually, it contains the information of three components: red (R), green (G), and blue (B).

[0057] During the image acquisition process, the ISP processing process mainly receives and processes the raw data of the photosensitive element, which is an important link in the image quality of the entire photo-taking and video recording. ISP processing is divided into Raw domain processing, RGB domain processing, and YUV domain processing. Raw domain processing includes bad pixel correction, black level correction, lens shading correction, automatic white balance, demosaicing, etc.; RGB domain processing includes Gamma curve correction, color correction matrix, color space conversion; YUV domain processing includes luminance noise reduction, color noise reduction, edge enhancement, hue and saturation control, contrast and brightness adjustment.

[0058] Therefore, in addition to adjusting the brightness of the dark frame by adjusting the exposure parameters acting on the raw domain in the foregoing embodiments, the brightness of the dark frame can also be adjusted by the brightening parameters in other domains during the ISP processing. The method for adjusting the brightness of the raw domain in the foregoing embodiments of the present application and the method for adjusting the brightness of the ISP in this embodiment can be executed alternatively or all, which is not limited here. For example, after comparing and adjusting the original brightness with the preset brightness threshold, the brightness adjustment can be made with reference to the ISP processing process in this embodiment.

[0059] Exemplarily, since the brightness of the dark frame is relatively dark and the difference from the bright frame is relatively large, if the non-brightened dark frame and bright frame are directly used for fusion processing, although the display effect of the target area is optimized, other areas may appear darker and the picture is not natural. The present application can perform further ISP brightening processing on the dark frame collected according to the target dark frame parameters to ensure that there are details in the dark area of the image and the halo phenomenon in the target area is acceptable.

[0060] Specifically, in the ISP, modules with different brightening intensities (i.e., brightening templates corresponding to different brightening parameters) can be set according to different brightness intervals to perform special ISP brightening processing on dark frames. For example, the Gamma brightening module (adjusting the brightness of the image by adjusting the Gamma curve) can be referred to. Combining the description of the foregoing embodiments, the brightness interval of the target area can be [a, b]. Map the interval values to the brightening module. If the brightness value of the target area is lower than a, a stronger brightening intensity (the first brightening parameter) is used; if the brightness value of the target area is higher than b, a linear mapping process is performed on its brightness value without brightening processing, and the original brightening parameter is maintained; if the brightness value of the target area is within the interval [a, b], a brightness transition process can be performed on the target area so that the brightness of the target area is gradually brightened. Among them, the first brightening parameter should be greater than the original brightening parameter and should also be greater than the brightening parameter used in the brightness transition process. The specific method of the brightness transition process can refer to the Gamma correction technology. By adjusting the brightness through the Gamma curve, the brightness change can be made less abrupt.

[0061] Based on the above embodiments, the embodiments of the present application will describe the steps before adjusting the current dark frame parameters according to the brightness information of the target area in the initial dark frame image to obtain the target dark frame parameters. Among them, the target area includes the traffic signal light area. Specifically, the method of this embodiment includes the following steps: Perform target detection processing on the initial dark frame image to obtain the traffic signal light area in the initial dark frame image; or, in response to the received area selection instruction, determine the traffic signal light area in the initial dark frame image according to the area selection instruction; and determine the average red brightness of the traffic signal light area as the brightness information.

[0062] Combined with the foregoing embodiments for description, an example will be given in combination with one of the specific scenarios to which the present application can be applied. The halo processing method of the present application can be applied to a traffic scene, especially a traffic scene including traffic signal lights. When collecting images in a traffic scene, it is easy to be affected by traffic signal lights, resulting in a halo phenomenon in the collected images. The target area of the present application is also the area where the traffic signal light is located.

[0063] Exemplarily, reference can be made to Figure 2 as shown in Figure 2 is a schematic diagram of an exemplary traffic signal light area in the image-based halo processing method of the present application. Before adjusting the current dark frame parameters of the initial dark frame image, it is necessary to first determine the traffic signal light area in the initial dark frame image, and then judge whether it is necessary to adjust the dark frame acquisition parameters according to the brightness information of the traffic signal light area.

[0064] Among them, the method for determining the traffic signal light area in the initial dark frame image may include, but is not limited to, performing object detection processing on the initial dark frame image according to methods such as image template matching and neural network processing to obtain the traffic signal light area in the initial dark frame image; in addition, it may also be to display the initial dark frame image through a display interface such as a web, and in response to the received area selection instruction input by the user (such as drawing an area box), determine the traffic signal light area in the initial dark frame image according to the area selection instruction. It should be noted that there may be one or more traffic signal light areas in the same image, which is not limited here.

[0065] For example, in this application, a traffic signal light area recognition module can be set. After receiving the signal light area drawn by the user on the web, the traffic signal light area recognition module will process this area and output information such as the color state of the signal light and the circumscribed rectangle area of the signal light. In the dark frame, the color state of the signal light and the circumscribed rectangle area of the signal light output by the signal light recognition module can be obtained first. Inside the circumscribed rectangle area of the signal light, the foreground points of the signal light are segmented. Since inside this rectangle area, except for the signal light, it is the lamp panel area, the brightness histogram distribution of the image is basically in the shape of two peaks. Considering that the brightness statistics need to be real-time and fully considering the time consumption of the processing process, this application selects the global threshold segmentation method to segment the foreground and background of the traffic signal light to accurately obtain the relevant information of the signal light (such as color information, brightness information, etc.).

[0066] It should be noted that this application considers that in the traffic scene, the red light of the traffic signal light is more likely to be overexposed compared to other colored lights (such as green lights, yellow lights). Therefore, this application analyzes based on the red brightness, and determines the average red brightness of the traffic signal light area as the brightness information. It is equivalent to, in the specific implementation process, obtaining the average red brightness by statistically counting the brightness of each red light on the raw data of the initial dark frame image, and then adaptively adjusting the current dark frame parameters to ensure that the brightness of the dark frame image is within a reasonable range. Assume that for the collected image with x bit data, the brightness interval is [a, b], and this interval can also be adjusted according to the value of bit (such as 8 bit, 10 bit, etc., which will not be elaborated here). At this time, the brightness of most signal lights is moderate. Inside the circumscribed rectangle frame of the signal light, the red brightness of the target area is statistically counted based on the segmented foreground points, and then the average red brightness is obtained. If the average red brightness is relatively high compared to the brightness interval, the exposure parameter is reduced; if the average red brightness is relatively low compared to the brightness interval, the exposure parameter is increased; if the average red brightness is within the brightness interval, the subsequent judgment processing is carried out by referring to the foregoing embodiments.

[0067] Based on the above embodiments, the embodiments of the present application will describe the steps of performing image fusion processing on the target dark frame image and the initial bright frame image according to the height information of the target area to obtain a dehalo image. Specifically, the method of this embodiment includes the following steps: Determine the height information of the target area as the fusion height information; or, determine the fusion height information according to the height information of the target area and a preset fusion parameter, where the fusion height information is greater than or equal to the height information of the target area; determine the target fusion area of the target dark frame image and the initial bright frame image according to the fusion height information; perform fusion processing on the target dark frame image and the initial bright frame image according to the target fusion area to obtain a dehalo image.

[0068] Exemplarily, reference can be made to Figure 3 as shown Figure 3 is an exemplary image fusion schematic diagram in the image-based halo processing method of the present application.

[0069] Combined with the foregoing embodiments, in the specific implementation process of the present application, according to actual needs, the target dark frame image and the initial bright frame image can be globally fused, or the target dark frame image and the initial bright frame image can be locally fused. Among them, global fusion takes all image areas of the target dark frame image and all image areas of the initial bright frame image as the fusion area (usually, the initial dark frame image, the target dark frame image, and the initial bright frame image, etc. are all of the same size), and local fusion takes local image areas in the target dark frame image and local image areas in the initial bright frame image as the fusion area.

[0070] Therefore, the present application can determine the fusion height information of the fusion area according to the height information of the target area. Specifically, the height information of the target area can be directly determined as the fusion height information, but there may be obvious brightness mutation phenomena in the image obtained by this fusion. Therefore, in order to make the transition area of the fusion natural, the fusion height information can also be determined according to the height information of the target area and a preset fusion parameter, and the preset fusion parameter is used to expand the fusion area.

[0071] For example, if there are multiple signal light areas in the image, first compare the height coordinates of each signal light area to obtain the maximum height value h 1 (that is, the height information of the target area). In order to make the transition of the target fusion area natural, therefore, the maximum height of the target fusion area can be n times of h 1 . Furthermore, obtain the pixel row number range of the target fusion area: the minimum row number can be 0, and the maximum row number can be , thereby determining the target fusion area.

[0072] Based on the above embodiments, the embodiments of the present application describe the step of fusing the target dark frame image and the initial bright frame image according to the target fusion region to obtain the anti-halo image. Specifically, the method of this embodiment includes the following steps: According to the height information and the fusion height information, determine the dark frame fusion weight of the target dark frame image corresponding to each pixel row and the bright frame fusion weight of the initial bright frame image in the target fusion region; fuse the target dark frame image and the initial bright frame image according to the dark frame fusion weight and the bright frame fusion weight to obtain the anti-halo image.

[0073] Combined with the foregoing embodiments for description, within the target fusion region, that is, from pixel row 0 to row of the image, a specific fusion ratio is set. For the image where all pixel rows of row 0 can use the dark frame, the rows above (until reaching the maximum row number of the image) can use the bright frame entirely. For the middle rows of the target fusion region [0, , fusion can be performed according to linear transition, and finally the fused image is obtained. Its mathematical expression can be:

[0074] Among them, refers to the target dark frame image in yuv format, refers to the initial bright frame image in yuv format, refers to the fused image in yuv format (which is also equivalent to the anti-halo image). This formula represents that when fusing the pixels from the 0th row to the th row in the fused image, according to the height information h 1 and the fusion height information , determine the dark frame fusion weight , and the bright frame fusion weight , and then perform image fusion. For the pixels from the th row to the th row, the bright frame image can be directly used. It is equivalent that the image in the target fusion region is obtained by fusing the dark frame and the bright frame, and the image outside the target fusion region is obtained from the bright frame. The anti-halo image can be obtained by splicing the fused image in the target fusion region and the bright frame image outside the target fusion region.

[0075] Furthermore, in the present application, the anti-halo intensity can also be set to control the fusion weight of the bright and dark frames. Combining the foregoing embodiments, it can be seen that the preset fusion parameter n value determines the fusion weight of the bright and dark frames. The larger the n, the larger the proportion of the dark frame and the smaller the proportion of the bright frame; the smaller the n, the smaller the proportion of the dark frame and the larger the proportion of the bright frame. Therefore, the upper layer opens the anti-halo intensity parameter and links it with the n value, which is equivalent to using the n value as the anti-halo intensity parameter. The stronger the set anti-halo intensity, the larger the n and the stronger the anti-halo effect; the weaker the set anti-halo intensity, the smaller the n and the weaker the anti-halo effect.

[0076] Exemplarily, regarding the process of determining the anti-halo intensity, it can be determined according to the received anti-halo instruction input by the user. For example, the stronger the anti-halo intensity represented by the anti-halo instruction input by the user, the larger the n value increases positively. Or it can be through technologies such as artificial intelligence, neural networks, and image processing to detect the image halo in the dark frame image and / or the bright frame image, thereby determining the halo intensity in the image, and then determining the anti-halo intensity according to its halo intensity (the halo intensity is positively correlated with the anti-halo intensity), and being able to adaptively obtain the n value during image fusion.

[0077] In summary, combining the foregoing embodiments, it can be seen that the halo processing method of the present application, based on the problem of the existence of halo in the image, proposes a method of image fusion based on bright and dark frames to achieve anti-halo processing. Among them, the brightness of the dark frame raw can be adaptively adjusted according to the brightness of the target area on the dark frame raw to ensure that the brightness of the target area on the dark frame raw is reasonable and the shape is clear. Then, specific ISP brightening processing can be performed on the dark frame to ensure that there are details in the dark area of the image and the halo effect is acceptable. Finally, according to the selected target area, the fusion range of the bright and dark frames is determined, and the bright and dark frames are fused.

[0078] In addition, upper layer linkage parameters can also be opened: preset brightness threshold (the target brightness required for the target area), ISP brightening parameter, anti-halo intensity, etc., to improve the adaptability of the present solution.

[0079] It should be further noted that the execution subject of the halo processing method based on the image can be a halo processing device based on the image. For example, the halo processing method based on the image can be executed by a terminal device, a server, or other processing devices. Among them, the terminal device can be a user equipment (UE), a computer, a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementation manners, the halo processing method based on the image can be implemented by the processor calling the computer-readable instructions stored in the memory.

[0080] Figure 4 is a block diagram of an image-based halo processing device shown in an exemplary embodiment of the present application. As Figure 4 shown, the exemplary image-based halo processing device 400 includes: a first acquisition module 410, a parameter adjustment module 420, a second acquisition module 430, and an image fusion module 440. Specifically: The first acquisition module 410 is configured to acquire an initial dark frame image acquired according to the current dark frame parameters and an initial bright frame image acquired according to the current bright frame parameters.

[0081] The parameter adjustment module 420 is configured to adjust the current dark frame parameters according to the brightness information of the target area in the initial dark frame image to obtain target dark frame parameters.

[0082] The second acquisition module 430 is configured to acquire a target dark frame image acquired according to the target dark frame parameters.

[0083] The image fusion module 440 is configured to perform image fusion processing on the target dark frame image and the initial bright frame image according to the height information of the target area to obtain a de-halo image.

[0084] In this exemplary image-based halo processing device, by acquiring the initial dark frame image acquired according to the current dark frame parameters and the initial bright frame image acquired according to the current bright frame parameters, analyzing whether the brightness information of the target area in the initial dark frame image reaches the brightness requirement, and then adjusting the current dark frame parameters according to the brightness information of the target area to obtain target dark frame parameters, so that the target dark frame image acquired according to the target dark frame parameters meets the brightness requirement and avoids overexposure and underexposure of the image; performing image fusion processing on the target dark frame image and the initial bright frame image according to the height information of the target area can obtain a de-halo image, thereby effectively reducing the halo phenomenon in the image.

[0085] It should be noted that the device provided in the above embodiment and the method provided in the above embodiment belong to the same concept. The specific manners in which each module and unit perform operations have been described in detail in the method embodiment, and will not be repeated here. In practical applications, the device provided in the above embodiment can, according to needs, allocate the above functions to different functional modules, that is, divide the internal structure of the device into different functional modules to complete all or part of the functions described above. This is not limited here.

[0086] Among them, the functions of each module can be referred to in the embodiment of the image-based halo processing method, and will not be repeated here.

[0087] Please refer to Figure 5 , Figure 5It is a schematic structural diagram of an embodiment of the electronic device of the present application. The electronic device 100 includes a memory 101 and a processor 102. The processor 102 is configured to execute program instructions stored in the memory 101 to implement the steps in any of the above-described embodiments of the image-based halo processing method. In a specific implementation scenario, the electronic device 100 may include, but is not limited to, a microcomputer, a server. In addition, the electronic device 100 may also include mobile devices such as a laptop computer, a tablet computer, etc., which are not limited herein.

[0088] Specifically, the processor 102 is configured to control itself and the memory 101 to implement the steps in any of the above-described embodiments of the image-based halo processing method. The processor 102 may also be referred to as a CPU (Central Processing Unit). The processor 102 may be an integrated circuit chip with signal processing capabilities. The processor 102 may also be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. Additionally, the processor 102 may be implemented jointly by integrated circuit chips.

[0089] In this exemplary electronic device, an initial dark frame image acquired according to the current dark frame parameters and an initial bright frame image acquired according to the current bright frame parameters are obtained. The brightness information of the target area in the initial dark frame image is analyzed to determine whether it meets the brightness requirement. Furthermore, the current dark frame parameters can be adjusted based on the brightness information of the target area to obtain target dark frame parameters, such that the target dark frame image acquired according to the target dark frame parameters meets the brightness requirement, avoiding overexposure and underexposure of the image. Image fusion processing is performed on the target dark frame image and the initial bright frame image based on the height information of the target area, and then a de-halo image can be obtained, thereby effectively reducing the halo phenomenon in the image.

[0090] Please refer to Figure 6 , Figure 6 It is a schematic structural diagram of an embodiment of the computer-readable storage medium of the present application. The computer-readable storage medium 110 stores program instructions 111 that can be run by a processor. The program instructions 111 are used to implement the steps in any of the above-described embodiments of the image-based halo processing method.

[0091] In the exemplary storage medium, by running the program instructions in the storage medium, an initial dark frame image acquired according to the current dark frame parameters and an initial bright frame image acquired according to the current bright frame parameters are obtained. The brightness information of the target area in the initial dark frame image is analyzed to determine whether it meets the brightness requirement. Furthermore, the current dark frame parameters can be adjusted according to the brightness information of the target area to obtain target dark frame parameters, so that the target dark frame image acquired according to the target dark frame parameters meets the brightness requirement, avoiding overexposure and underexposure of the image. Image fusion processing is performed on the target dark frame image and the initial bright frame image according to the height information of the target area, and then a dehalo image can be obtained, thereby effectively reducing the halo phenomenon in the image.

[0092] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the methods described in the above method embodiments. The specific implementation can refer to the description of the above method embodiments. For the sake of brevity, it will not be repeated here.

[0093] The descriptions of the above embodiments tend to emphasize the differences between the embodiments. Their similarities or similarities can be referred to each other. For the sake of brevity, they will not be repeated in this article.

[0094] In several embodiments provided in the present application, it should be understood that the disclosed methods and devices can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods. For example, units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection to each other can be through some interfaces. The indirect coupling or communication connection of the device or unit can be in electrical, mechanical or other forms.

[0095] In addition, each functional unit in the various embodiments of the present application may be integrated into one processing unit, may exist separately as individual physical units, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of a software functional unit. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it may be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, may be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the methods in the various embodiments of the present application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs that can store program codes.

Claims

1. A halo processing method based on an image, characterized in that: The method comprises: Acquire an initial dark frame image acquired according to current dark frame parameters and an initial bright frame image acquired according to current bright frame parameters; Adjusting the current dark frame parameters according to the brightness information of the target area in the initial dark frame image to obtain target dark frame parameters; Acquire a target dark frame image acquired according to the target dark frame parameters; The target dark frame image and the initial bright frame image are subjected to image fusion processing according to the height information of the target area to obtain a halo-reduced image.

2. The method according to claim 1, characterized in that The step of adjusting the current dark frame parameters according to the brightness information of the target area in the initial dark frame image to obtain the target dark frame parameters includes: Get the target brightness range; According to the comparison result between the brightness value of the target area in the initial dark frame image and the left and right boundaries of the target brightness range, the current dark frame parameter is adjusted to obtain the target dark frame parameter.

3. The method according to claim 2, characterized in that The current dark frame parameters include dark frame exposure parameters, and adjusting the current dark frame parameters according to a comparison result between a brightness value of a target area in the initial dark frame image and left and right boundaries of the target brightness interval to obtain the target dark frame parameters includes: If the brightness value is less than the left boundary of the target brightness interval, the dark frame exposure parameter is increased to obtain the target dark frame parameter; If the brightness value is greater than the right boundary of the target brightness range, the dark frame exposure parameter is reduced to obtain the target dark frame parameter.

4. The method according to claim 1, characterized in that The acquiring of the target dark frame image acquired according to the target dark frame parameters comprises: Acquire a dark frame image to be processed according to the target dark frame parameters; Comparing the original brightness of the dark frame image to be processed with a preset brightness threshold to obtain a brightness comparison result; In response to the brightness comparison result indicating that the original brightness is less than the preset brightness threshold, the original brightness is increased to obtain the target dark frame image; In response to the brightness comparison result indicating that the original brightness is greater than the preset brightness threshold, the original brightness is reduced to obtain the target dark frame image.

5. The method according to claim 1, characterized in that The acquiring of the target dark frame image acquired according to the target dark frame parameters comprises: Acquire a dark frame image to be processed according to the target dark frame parameters; In response to the brightness information of the target area in the dark frame image to be processed being smaller than the left boundary of the target brightness range, the dark frame image to be processed is brightened using a first brightening parameter to obtain the target dark frame image; In response to the brightness information of the target area in the dark frame image to be processed being greater than the right boundary of the target brightness interval, linear mapping processing is performed on the dark frame image to be processed to obtain the target dark frame image; In response to the brightness information of the target area in the dark frame image to be processed being within the target brightness range, brightness transition processing is performed on the dark frame image to be processed to obtain the target dark frame image.

6. The method according to claim 1, characterized in that The target area includes a traffic light area. Before adjusting the current dark frame parameters according to the brightness information of the target area in the initial dark frame image to obtain the target dark frame parameters, the method further includes: Performing target detection processing on the initial dark frame image to obtain a traffic light area in the initial dark frame image; or, in response to a received area selection instruction, determining a traffic light area in the initial dark frame image according to the area selection instruction; The average red brightness of the traffic light area is determined as the brightness information.

7. The method according to claim 1, characterized in that The step of performing image fusion processing on the target dark frame image and the initial bright frame image according to the height information of the target area to obtain a halo-reduced image includes: Determining the height information of the target area as fused height information; Alternatively, the fused height information is determined according to the height information of the target area and a preset fusion parameter, and the fused height information is greater than or equal to the height information of the target area; Determine a target fusion region of the target dark frame image and the initial bright frame image according to the fusion height information; The target dark frame image and the initial bright frame image are fused according to the target fusion area to obtain the halo-reduced image.

8. The method according to claim 7, characterized in that The step of fusing the target dark frame image with the initial bright frame image according to the target fusion area to obtain the halo-reduced image includes: Determine, according to the height information and the fusion height information, a dark frame fusion weight of the target dark frame image and a bright frame fusion weight of the initial bright frame image corresponding to each pixel row in the target fusion area; The target dark frame image and the initial bright frame image are fused according to the dark frame fusion weight and the bright frame fusion weight to obtain the halo-reduced image.

9. An electronic device, characterized in that: The method comprises a memory and a processor, wherein the processor is used to execute program instructions stored in the memory to implement the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having program instructions stored thereon, characterized in that: When the program instructions are executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images