Low-illumination enhancement processing method and device, equipment and storage medium

Abstract

The application is suitable for the field of image processing, and provides a low-illumination enhancement processing method, device, equipment and storage medium, comprising: acquiring a to-be-processed image and source information of the to-be-processed image; determining a first brightness enhancement coefficient according to the source information; calculating image brightness information corresponding to the to-be-processed image; determining a second brightness enhancement coefficient according to the image brightness information; determining a pixel traversal model corresponding to the to-be-processed image; determining a plurality of initial pixels corresponding to the to-be-processed image to obtain an initial pixel set; calling the pixel traversal model to traverse the initial pixel set to obtain a target pixel set corresponding to the initial pixel set; and adjusting the initial pixel set and the target pixel set according to the first brightness enhancement coefficient and the second brightness enhancement coefficient to obtain a low-illumination enhancement image. The application can improve the effect and efficiency of low-illumination enhancement.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, and in particular relates to a low-light enhancement processing method, apparatus, device and storage medium. Background Technology

[0002] Electronic devices can perform operations such as object detection, classification, recognition, scene understanding, and 3D reconstruction on captured images, enabling their application in real-world scenarios such as autonomous driving, video surveillance, and virtual / augmented reality. Image quality has a significant impact on subsequent application scenarios, and changes in lighting conditions often directly affect the quality of the acquired images. A typical example is the use of images captured in low-light environments, which often suffer from severe degradation, such as poor visibility, low contrast, and unexpected noise. Low light undoubtedly degrades the performance of most vision-based algorithms; therefore, it is necessary to perform low-light enhancement processing on images to improve their quality.

[0003] However, low-light enhancement of images is affected by many factors, and the effectiveness and efficiency of low-light enhancement cannot be guaranteed.

[0004] Application content

[0005] This application provides a low-light enhancement processing method, a low-light enhancement processing device, an electronic device, and a storage medium to solve the problem of poor effect and efficiency of low-light enhancement processing.

[0006] A first aspect of this application provides a low-light enhancement processing method, the method comprising: acquiring an image to be processed and source information of the image to be processed; determining a first brightness enhancement coefficient based on the source information; calculating image brightness information corresponding to the image to be processed; determining a second brightness enhancement coefficient based on the image brightness information; determining a pixel traversal model corresponding to the image to be processed; determining multiple initial pixels corresponding to the image to be processed to obtain an initial pixel set; calling the pixel traversal model to traverse the initial pixel set to obtain a target pixel set corresponding to the initial pixel set; adjusting the initial pixel set and the target pixel set according to the first brightness enhancement coefficient and the second brightness enhancement coefficient to obtain a low-light enhanced image.

[0007] Further, in the method provided in the embodiments of this application, the step of determining the first brightness enhancement coefficient based on the source information includes: when the source information is not video, determining the first brightness enhancement coefficient as a preset value; when the source information is video, determining the video frame rate of the video corresponding to the image to be processed and the interval difference corresponding to the preset first brightness enhancement coefficient, calculating the ratio of the interval difference to the video frame rate, and determining the first brightness enhancement coefficient corresponding to each frame of the image to be processed in the video based on the ratio.

[0008] Furthermore, in the method provided in the embodiments of this application, before calculating the image brightness information corresponding to the image to be processed, the method further includes: determining the image format of the image to be processed; when the image format is RGB data format, converting the RGB data format into YUV data format.

[0009] Further, in the method provided in the embodiments of this application, the step of calculating the image brightness information corresponding to the image to be processed includes: determining the Y-channel image corresponding to the image to be processed; determining the preset dark pixels and global image pixels corresponding to the Y-channel image; calculating the pixel ratio value between the first number of pixels corresponding to the preset dark pixels and the second number of pixels corresponding to the global image pixels; splitting the Y-channel image according to the preset ratio to obtain multiple image blocks; selecting a preset number of sampling points from the multiple image blocks, and determining a sampling interval based on the sampling points; calculating the pixel mean value corresponding to the sampling interval; and weighting the pixel ratio value and the pixel mean value to obtain the image brightness information corresponding to the image to be processed.

[0010] Further, in the method provided in the embodiments of this application, determining the pixel traversal model corresponding to the image to be processed includes: calculating the target bright channel prior value corresponding to the image to be processed; calculating the target atmospheric light intensity corresponding to the image to be processed; processing the target bright channel prior value and the target atmospheric light intensity according to a preset atmospheric light transmittance formula to obtain atmospheric light transmittance; performing gamma transformation on the atmospheric light transmittance to obtain an atmospheric light transmittance enhancement value; and generating the pixel traversal model according to the target atmospheric light intensity and the atmospheric light transmittance enhancement value.

[0011] Further, in the method provided in the embodiments of this application, the step of calculating the prior value of the target brightness channel corresponding to the image to be processed includes: determining the brightness value of each pixel in the image to be processed to obtain a first brightness value set; selecting multiple local maximum values ​​from the first brightness value set according to a first preset sliding window algorithm; sorting the multiple local maximum values ​​in descending order, and selecting the local maximum value with a first preset proportion that is sorted first; calculating the first average value of the local maximum values ​​with the first preset proportion, and using the first average value as the prior value of the target brightness channel.

[0012] Further, in the method provided in the embodiments of this application, the step of calculating the atmospheric light intensity corresponding to the image to be processed includes: determining the brightness value of each pixel of the image to be processed to obtain a second brightness value set; selecting multiple local minima from the second brightness value set according to a second preset sliding window algorithm; sorting the multiple local minima in descending order and selecting the local minima with a second preset percentage that is sorted first; calculating the second average value of the local minima with the second preset percentage and using the second average value as the target atmospheric light intensity.

[0013] A second aspect of this application also provides a low-light enhancement processing apparatus, the apparatus comprising: a source acquisition module for acquiring an image to be processed and source information of the image to be processed; a first coefficient determination module for determining a first brightness enhancement coefficient based on the source information; a brightness calculation module for calculating image brightness information corresponding to the image to be processed; a second coefficient determination module for determining a second brightness enhancement coefficient based on the image brightness information; a traversal table determination module for determining a pixel traversal model corresponding to the image to be processed; an initial pixel determination module for determining multiple initial pixels corresponding to the image to be processed to obtain an initial pixel set; a target pixel determination module for traversing the pixel traversal model to obtain a target pixel set corresponding to the initial pixel set; and a pixel adjustment module for adjusting the initial pixel set and the target pixel set according to the first brightness enhancement coefficient and the second brightness enhancement coefficient to obtain a low-light enhanced image.

[0014] A third aspect of this application also provides an electronic device, the electronic device including a processor, the processor being configured to implement the low-light enhancement processing method as described in any of the preceding embodiments when executing a computer program stored in a memory.

[0015] A fourth aspect of this application also provides a computer-readable storage medium storing a computer program that, when executed by a controller, implements the low-light enhancement processing method as described in any of the preceding embodiments.

[0016] In this application, a first brightness enhancement coefficient is determined based on the source information of the image to be processed. By setting the first brightness enhancement coefficient, when the source information is video, phenomena such as jumps between video frames can be avoided, ensuring the effect of low-light enhancement. Furthermore, this application determines the pixel traversal model corresponding to the image to be processed. The pixel traversal model performs low-light enhancement processing on the image to be processed, which can avoid the computational complexity and the need for training with real low-light paired data caused by deep learning methods for low-light enhancement, thus improving the efficiency of low-light enhancement. In addition, this application also determines a second brightness enhancement coefficient based on the image brightness information of the image to be processed, which can adaptively control the upper bound of brightness enhancement according to the current lighting conditions of the image, ensuring the effect of low-light enhancement. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a structural diagram of the low-light enhancement processing method provided in the embodiments of this application;

[0019] Figure 2 This is a schematic flowchart of the low-light enhancement processing method provided in the embodiments of this application;

[0020] Figure 3 This is a schematic diagram illustrating the process of determining the first brightness enhancement coefficient provided in an embodiment of this application;

[0021] Figure 4 This is a flowchart of the image brightness information determination process provided in an embodiment of this application;

[0022] Figure 5A This is a pixel statistics histogram provided in the embodiments of this application;

[0023] Figure 5B This is a regional fixed-point sampling map provided in the embodiments of this application;

[0024] Figure 6 This is a flowchart illustrating the determination of the pixel traversal model provided in the embodiments of this application;

[0025] Figure 7 This is a flowchart illustrating the determination of the prior value of the target bright channel provided in an embodiment of this application;

[0026] Figure 8 This is a flowchart illustrating the determination of target atmospheric light intensity provided in an embodiment of this application;

[0027] Figure 9 This is a schematic diagram of the low-light enhancement processing device provided in the embodiments of this application. Detailed Implementation

[0028] To better understand the above-mentioned objectives, features, and advantages of this application, the application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0029] Numerous specific details are set forth in the following description in order to provide a full understanding of this application. The described embodiments are only some, not all, of the embodiments of this application.

[0030] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application.

[0031] When electronic devices enhance images captured in low-light environments (also known as low-light images), the process is affected by numerous factors, making it difficult to guarantee the effectiveness and efficiency of the enhancement. For example, existing low-light image enhancement methods often employ deep learning; however, the massive computational cost and spatial complexity of neural networks lead to poor efficiency. Furthermore, deep learning methods often struggle to obtain real-world low-light images, requiring the use of artificially synthesized low-light images as training data. Using artificially synthesized low-light images results in poor model generalization to real-world low-light scenarios, compromising the effectiveness of image enhancement and ultimately leading to suboptimal results.

[0032] To address the aforementioned issues, this application provides a low-light enhancement processing method to improve the effectiveness and efficiency of low-light enhancement processing.

[0033] Combination Figure 1 This document illustrates the structural diagram of the low-light enhancement processing method provided in the embodiments of this application. (See diagram for example.) Figure 1 As shown, electronic device 3 includes a memory 31, at least one processor 32, at least one communication bus 33, and a transceiver 34. It can be understood that... Figure 1 The structure of the electronic device shown does not constitute a limitation of the embodiments of this application. It can be a bus structure or a star structure. The electronic device 3 may also include more or fewer other hardware or software than shown, or different component arrangements.

[0034] In some embodiments, electronic device 3 is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits, programmable gate arrays, digital processors, and embedded devices. Electronic device 3 may also include client devices, including, but not limited to, any electronic product that can interact with a client via a keyboard, mouse, remote control, touchpad, or voice control device, such as personal computers, tablet computers, smartphones, and digital cameras.

[0035] It should be noted that electronic device 3 is only an example. Other existing or future electronic products that are suitable for this application should also be included within the scope of protection of this application and are incorporated herein by reference.

[0036] In some embodiments, at least one communication bus 33 is configured to enable communication between the memory 31 and at least one processor 32, etc.

[0037] Although not shown, electronic device 3 may also include a power supply (such as a battery) to power various components. Preferably, the power supply can be logically connected to at least one processor 32 via a power management device, thereby enabling functions such as charging, discharging, and power consumption management. The power supply may also include one or more DC or AC power sources, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. Electronic device 3 may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be described in detail here.

[0038] Figure 2 This is a schematic flowchart of the low-light enhancement processing method provided in an embodiment of this application. Figure 2 As shown, the low-light enhancement processing method is performed by an electronic device and may include the following steps. Depending on different needs, the order of the steps in this flowchart may be changed, and some may be omitted.

[0039] S11, Obtain the image to be processed and its source information.

[0040] In at least one embodiment of this application, the image to be processed refers to an image that requires low-light enhancement processing. The source information may include both video and non-video sources. The image to be processed can be a single image or a video frame from a video. When the image to be processed is a single image, the source information refers to non-video; when the image to be processed is a video frame from a video, the source information refers to video.

[0041] S12, determine the first brightness enhancement coefficient based on the source information.

[0042] In at least one embodiment of this application, a first brightness enhancement coefficient is used to identify the brightness enhancement slope of video frames within the video; that is, the first brightness enhancement coefficient is related to the video. It can be understood that when the source information is not video, the first brightness enhancement coefficient is a preset value, which can be 1. When the source information is video, the video frame rate needs to be determined, and the first brightness enhancement coefficient is set according to the video frame rate to avoid excessive brightness changes between adjacent video frames within the same second, which could lead to flickering, frame skipping, or other phenomena.

[0043] The process for determining the first brightness enhancement coefficient can be found in the following section. Figure 3 The detailed description is in the text.

[0044] S13, calculate the image brightness information corresponding to the image to be processed.

[0045] In at least one embodiment of this application, image brightness information refers to the global illumination information of the image to be processed. In one embodiment, the image brightness information can be divided into five brightness levels from dark to bright: extremely dark, slightly dark, normal, slightly bright, and extremely bright.

[0046] In one embodiment, the image format of the image to be processed can be either RGB data format or YUV data format. The RGB data format image includes an R channel, a G channel, and a B channel, corresponding to the red, green, and blue components respectively, with the values ​​of these three components determining the image color. The YUV data format image includes a Y channel, a U channel, and a V channel, where the Y channel represents the image brightness, and the U and V channels represent the image chromaticity.

[0047] In one embodiment, before calculating the image brightness information corresponding to the image to be processed, the method further includes: determining the image format of the image to be processed; and when the image format is RGB data format, converting the RGB data format to YUV data format. The conversion of RGB data format to YUV data format is prior art and will not be elaborated here. This embodiment of the application significantly improves the efficiency of low-light brightness processing by determining the image brightness information of the Y channel image to perform low-light brightness processing on the Y channel image instead of determining the image brightness information of the RGB three-channel image to perform low-light brightness processing on the RGB three-channel image.

[0048] The process for determining image brightness information can be found in the following text. Figure 4 The detailed description is in the text.

[0049] S14, determine the second brightness enhancement coefficient based on the image brightness information.

[0050] In at least one embodiment of this application, a second brightness enhancement coefficient is used to identify the upper limit of brightness enhancement for the image to be processed. A mapping relationship exists between image brightness information and the second brightness enhancement coefficient. By querying this mapping relationship, the second brightness enhancement coefficient corresponding to the image brightness information can be obtained. For example, the numerical range of the second brightness enhancement coefficient is 0-1. When the image brightness information is extremely dark, the second brightness enhancement coefficient can be 1; when the image brightness information is slightly dark, the second brightness enhancement coefficient can be 0.8; when the image brightness information is normal, the second brightness enhancement coefficient can be 0.5; when the image brightness information is slightly bright, the second brightness enhancement coefficient can be 0.2; and when the image brightness information is extremely bright, the second brightness enhancement coefficient can be 0.

[0051] In one embodiment, the value of the second brightness enhancement coefficient can be fine-tuned according to the image conditions of the image to be processed, which may include, but are not limited to, image quality. Before determining the second brightness enhancement coefficient based on the image brightness information, the method further includes: determining the image quality of the image to be processed; when the image quality meets a preset image quality requirement, determining the second brightness enhancement coefficient to a set value; when the image quality does not meet the preset image quality requirement, determining the second brightness enhancement coefficient based on the image brightness information. The preset image quality requirement may refer to an image storing too much noise in an extremely dark state.

[0052] It is understandable that when the image quality meets the preset requirements, the image brightness information of the image to be processed is extremely dark, and the image to be processed contains too much noise. When the image quality does not meet the preset requirements, the image brightness information of the image to be processed may be extremely dark, but the number of noise points in the image to be processed is small or there is no noise; or the image brightness information of the image to be processed may not be extremely dark.

[0053] For example, when the image to be processed stores too much noise in an extremely dark state, forcibly enhancing it may cause the noise to expand, resulting in severe blockiness. In response to this situation, the second brightness enhancement coefficient is set to a value less than 0.1, that is, without expanding the noise, the brightness of the image is appropriately adjusted to obtain a better visual effect.

[0054] This application determines a second brightness enhancement coefficient based on the image brightness information of the image to be processed, and can adaptively control the upper limit of brightness enhancement according to the current illumination of the image to ensure the effect of low-light enhancement.

[0055] S15, determine the pixel traversal model corresponding to the image to be processed.

[0056] In at least one embodiment of this application, the pixel traversal model refers to a mathematical model used to perform initial low-light enhancement transformation processing on each pixel in the image to be processed. By calling the pixel traversal model to process the brightness value corresponding to each initial pixel in the image to be processed, the brightness value corresponding to the target pixel after the initial low-light enhancement transformation processing can be obtained. Then, the brightness values ​​of each target pixel are adjusted by calling the first brightness enhancement coefficient and the second brightness enhancement coefficient to obtain the final low-light enhanced photo. In one embodiment, multiple images to be processed can share a single pixel traversal model, or each image to be processed can have its own corresponding pixel traversal model; no limitation is made here.

[0057] In one embodiment, the model parameters corresponding to the pixel traversal model may include the prior value of the bright channel of the image to be processed, the atmospheric light intensity, and the atmospheric light transmittance. By calculating the prior value of the bright channel, the atmospheric light intensity, and the atmospheric light transmittance of the image to be processed, and combining the prior value of the bright channel, the atmospheric light intensity, and the atmospheric light transmittance, the pixel traversal model can be obtained.

[0058] The process for determining the pixel traversal model can be found in the following section. Figure 6 A detailed description.

[0059] In at least one embodiment of this application, a pixel traversal model (also referred to as a static pixel traversal model in this application) can be set for various low-light images to be processed, or a corresponding pixel traversal model (also referred to as a dynamic pixel traversal model in this application) can be set for each low-light image to be processed, without any limitation.

[0060] In one embodiment, when there is only one pixel traversal model, the target bright channel prior value and the target atmospheric light intensity can be determined through deep learning. Then, the target bright channel prior value and the target atmospheric light intensity are processed according to a preset atmospheric light transmittance formula to obtain the atmospheric light transmittance. The atmospheric light transmittance is then subjected to gamma transformation to obtain the atmospheric light transmittance enhancement value. The pixel traversal model is generated based on the target atmospheric light intensity and the atmospheric light transmittance enhancement value. For example, an initialized neural network model and a large amount of low-light image data are obtained. The initialized neural network model can be a convolutional neural network model. The low-light image data includes low-light images and the corresponding bright channel prior value and atmospheric light intensity. The low-light image data is used as model training data, with the low-light images as the input vector and the bright channel prior value and atmospheric light intensity as the output vector. The neural network model is trained to obtain a pixel traversal model adapted to multiple low-light images to be processed.

[0061] In other embodiments, when there are multiple pixel traversal models, a pixel traversal model corresponding to each image to be processed is calculated. For example, the prior value of the target bright channel and the target atmospheric light intensity corresponding to each image to be processed are determined. Then, the prior value of the target bright channel and the target atmospheric light intensity are processed according to a preset atmospheric light transmittance formula to obtain the atmospheric light transmittance. A gamma transformation is performed on the atmospheric light transmittance to obtain an atmospheric light transmittance enhancement value. A pixel traversal model is generated based on the target atmospheric light intensity and the atmospheric light transmittance enhancement value.

[0062] In other embodiments, determining the corresponding pixel traversal model for each video frame in a high frame rate video involves a large amount of computation, resulting in low efficiency of image enhancement processing. To address this, this application can also determine the pixel traversal model corresponding to video frames within a certain time period by establishing a long-distance dependency time period to reduce the computational load. The long-distance dependency time period can be set according to actual needs; for example, it can be 5 seconds or 10 seconds. Taking a long-distance dependency time of 5 seconds as an example, the 5-second video contains 50 video frames. The pixel traversal models corresponding to the first few frames (e.g., the first 3 frames) are obtained, namely pixel traversal model A, pixel traversal model B, and pixel traversal model C. The average value of these pixel traversal models A, B, and C is calculated (first calculating the sum of pixel traversal models A, B, and C, then calculating the quotient of the sum and the number of pixel traversal models, 3, as the average result). The resulting pixel traversal model (i.e., the average result) is used as the pixel traversal model for the subsequent 47 video frames. This application determines the pixel traversal model corresponding to the image to be processed. The pixel traversal model performs low-light enhancement processing on the image to be processed, which can avoid the computational complexity and the need for training with real low-light paired data caused by deep learning methods for low-light enhancement, thereby improving the efficiency of low-light enhancement.

[0063] S16, determine multiple initial pixels corresponding to the image to be processed, and obtain the initial pixel set.

[0064] In at least one embodiment of this application, the image to be processed contains a plurality of initial pixels, each of which contains a corresponding brightness value, and the plurality of initial pixels are combined into an initial pixel set.

[0065] S17, call the pixel traversal model to traverse the initial pixel set and obtain the target pixel set corresponding to the initial pixel set.

[0066] In at least one embodiment of this application, the brightness value corresponding to each initial pixel in the initial pixel set is input into the pixel traversal model for initial low-light enhancement processing to obtain the brightness value corresponding to the target pixel. The brightness values ​​corresponding to multiple target pixels are combined to obtain the target pixel set.

[0067] S18, adjust the initial pixel set and the target pixel set according to the first brightness enhancement coefficient and the second brightness enhancement coefficient to obtain a low-light enhanced image.

[0068] In at least one embodiment of this application, a first brightness enhancement coefficient, a second brightness enhancement coefficient, a target pixel in the target pixel set, and an initial pixel in the initial pixel set are determined, and the final low-light enhancement image is obtained based on the mathematical relationship.

[0069] In one embodiment, I(x) represents the target pixel set, and I(x) represents the initial pixel set (i.e., the image to be processed). The mathematical relationship can be: I(x) * second brightness enhancement coefficient + (1 - second brightness enhancement coefficient) * (x) * first brightness enhancement coefficient. This mathematical relationship can be adjusted according to the image enhancement effect.

[0070] The low-light enhancement processing method provided in this application determines a first brightness enhancement coefficient based on the source information of the image to be processed. By setting the first brightness enhancement coefficient, when the source information is video, phenomena such as jumps between video frames can be avoided, ensuring the effect of low-light enhancement. Furthermore, this application determines the pixel traversal model corresponding to the image to be processed, and the pixel traversal model performs low-light enhancement processing on the image to be processed. This avoids the computational complexity and the need for training with real low-light paired data caused by deep learning methods for low-light enhancement, thus improving the efficiency of low-light enhancement. In addition, this application also determines a second brightness enhancement coefficient based on the image brightness information of the image to be processed, which can adaptively control the upper limit of brightness enhancement according to the current lighting conditions of the image, ensuring the effect of low-light enhancement.

[0071] The following combination Figure 3 This application describes the process for determining the first brightness enhancement coefficient provided in its embodiments. In one embodiment, the electronic device determines the first brightness enhancement coefficient based on source information, including:

[0072] S121, when the source information is not video, determine the first brightness enhancement coefficient to be a preset value.

[0073] In one embodiment, the preset value can be set according to actual needs, for example, the preset value is 1.

[0074] S122, when the source information is video, determine the video frame rate of the video corresponding to the image to be processed and the interval difference corresponding to the preset first brightness enhancement coefficient, calculate the ratio of the interval difference to the video frame rate, and determine the first brightness enhancement coefficient corresponding to each frame of the image to be processed in the video based on the ratio.

[0075] For example, when the video frame rate is 15 frames per second, the range corresponding to the preset first brightness enhancement coefficient is determined to be [0.5, 1]. It can be understood that this range can be dynamically set according to the video frame images (i.e., the images to be processed), and is not limited here. To avoid flickering, frame skipping, or other phenomena between video frames, the range difference corresponding to the first brightness enhancement coefficient is calculated to be 0.5, and the ratio of the range difference to the video frame rate is calculated to be 0.5 / 15 = 0.033. At this time, there are 15 images to be processed each second. For the first image to be processed in each second, the first brightness enhancement coefficient is 0.5; for the second image to be processed, the first brightness enhancement coefficient is 0.5 + 0.033 = 0.533; for the third image to be processed, the first brightness enhancement coefficient is 0.533 + 0.033 = 0.566; and so on, until the first brightness enhancement coefficient corresponding to the fifteenth image to be processed is determined.

[0076] This application embodiment controls the brightness enhancement slope between video frames by setting a first brightness enhancement coefficient, which can avoid phenomena such as jumps between video frames when the source information is video, and ensure the effect of low-light enhancement.

[0077] The following combination Figure 4 This application describes the image brightness information determination process provided in its embodiments. In one embodiment, the electronic device calculates the image brightness information corresponding to the image to be processed, including:

[0078] S133, determine the Y channel image corresponding to the image to be processed.

[0079] In one embodiment, the image brightness information corresponding to the image to be processed can be determined by performing pixel statistics on the Y channel image corresponding to the image to be processed in a histogram manner. For example, the brightness values ​​of the pixels of the image to be processed are set in the range [0, 255], wherein the closer the pixel value is to 255, the higher the brightness, and the closer the pixel value is to 0, the lower the brightness.

[0080] S134, determine the preset dark area pixels and global image pixels corresponding to the Y channel image.

[0081] In one embodiment, preset dark pixels and global image pixels are determined in the image to be processed. Preset dark pixels refer to pixels within a specific range of brightness values ​​that are set in advance. For example, preset dark pixels may refer to pixels within the brightness value range of [0, 45]. Global image pixels refer to all pixels in the entire image to be processed. For example, global image pixels may refer to pixels within the brightness value range of [0, 255].

[0082] S135, calculate the pixel ratio between the number of first pixels corresponding to preset dark area pixels and the number of second pixels corresponding to global image pixels.

[0083] In one embodiment, the brightness level of the image to be processed, i.e., the image brightness information, can be determined by statistically analyzing the pixel ratio of preset dark pixels in the global image pixels using histogram pixel statistics. In one embodiment, the number of first pixels and the number of second pixels in the global image pixels are calculated for pixels in the brightness value range of [0, 45] (also called preset dark pixels), and the ratio of the number of first pixels to the number of second pixels is calculated as the pixel ratio value.

[0084] Please see Figure 5A In the histogram, the horizontal axis represents the brightness value of a pixel in a given image to be processed, ranging from 0 to 255. The vertical axis represents the number of pixels in the image to be processed for each brightness value. By consulting the histogram, the proportion of preset dark pixels in the global image can be clearly seen.

[0085] S136, split the Y channel image according to a preset ratio to obtain multiple image blocks.

[0086] In one embodiment, the preset ratio is a splitting ratio set according to the pixel size of the image to be processed. The Y-channel image is split according to the preset ratio to obtain multiple image blocks of the same shape and area.

[0087] S137, Select a preset number of sampling points from multiple image blocks, and determine the sampling interval based on the sampling points.

[0088] Considering that using histograms for pixel statistics to determine image brightness information can be affected by the color interference of large objects—for example, the actual overall brightness of the image to be processed might be normal, but if the image contains a black can of cola, the histogram method is easily influenced by the black cola, leading to an underestimation of the overall brightness of the image and compromising the accuracy of the brightness determination—this application, in addition to histogram pixel statistics, also incorporates region-based fixed-point sampling to assist in determining the image brightness information of the image to be processed.

[0089] In one embodiment, the preset number is a value determined based on the pixel size of the image to be processed; the larger the pixel size, the larger the preset number; the smaller the pixel size, the smaller the preset number. In one embodiment, the correspondence between the pixel size of the image to be processed and the preset number can be predetermined. By querying this correspondence, the number of corresponding sampling points of the image to be processed can be obtained. For example, the preset number can be 6. The region formed by the image block (containing the sampling points) surrounded by the 6 sampling points is defined as the sampling interval.

[0090] Please see Figure 5BThe image to be processed is divided into 5*7 equal-sized image blocks in the figure. The circled image blocks in the figure are the selected sampling points, and there are 6 sampling points. The region formed by the image blocks (including the sampling points) surrounded by the 6 sampling points is called the sampling interval. In the figure, the sampling interval is a region composed of 3*5 image blocks.

[0091] S138, calculate the average pixel value corresponding to the sampling interval.

[0092] In one embodiment, firstly, the sum of the brightness values ​​of all pixels within the sampling interval is calculated, for example, the brightness values ​​are set in the interval [0, 255]; then, the number of pixels within the sampling interval is determined; finally, the ratio of the sum of the brightness values ​​to the number of pixels is calculated as the pixel mean value corresponding to the sampling interval.

[0093] S139, weighted processing of pixel ratio values ​​and pixel mean values ​​to obtain the image brightness information corresponding to the image to be processed.

[0094] In one embodiment, the pixel ratio value calculated using histogram statistics and the pixel mean value calculated using region fixed-point sampling are weighted to obtain the image brightness information of the image to be processed. Weighting coefficients are assigned to the pixel ratio value and the pixel mean value. These weighting coefficients can be set according to actual needs, and the sum of the weighting coefficients is 1. For example, the weighting coefficient for the pixel ratio value is 0.5, and the weighting coefficient for the pixel mean value is 0.5. The weighted result of this calculation is used as the image brightness information corresponding to the image to be processed.

[0095] This application determines the image brightness information of the image to be processed by combining histogram pixel statistics with regional fixed-point sampling. This avoids the problem of color interference from some large objects, improves the accuracy of image brightness information determination, and thus ensures the effect of low-light enhancement processing.

[0096] The following combination Figure 6 This application describes the pixel traversal model determination process provided in its embodiments. In one embodiment, the electronic device determines the pixel traversal model corresponding to the image to be processed, including:

[0097] S151, calculate the prior value of the target brightness channel corresponding to the image to be processed.

[0098] In one embodiment, the brightness value of the dark channel image corresponding to a normal image is close to 0, while the brightness value of the dark channel image corresponding to a foggy image affected by fog will be significantly higher than 0. Correspondingly, the brightness value of the inverse dark channel (also known as the bright channel) of a low-light image is close to the maximum value of the color range, which is referred to as the bright channel prior value in this application.

[0099] The process for determining the prior value of the target brightness channel can be found in the following text. Figure 8 A detailed description.

[0100] S152, calculate the target atmospheric light intensity corresponding to the image to be processed.

[0101] The procedure for determining the target atmospheric light intensity can be found in the following text. Figure 9 A detailed description.

[0102] S153, based on the preset atmospheric light transmittance formula, process the prior value of the target bright channel and the target atmospheric light intensity to obtain the atmospheric light transmittance.

[0103] In one embodiment, atmospheric light transmittance can be calculated using the following formula 1:

[0104]

[0105] Where t(x) represents atmospheric light transmittance, ω is a pre-set darkening parameter in the interval [0,1], and J bright A is used to represent the prior value of the target bright channel. bright Used to represent the atmospheric light intensity of a target. This is used to represent the set of maximum brightness values ​​corresponding to the same position in the RGB three channels, or the set of maximum brightness values ​​obtained by traversing the Y channel using a sliding window algorithm. The sliding window algorithm is preset and is not restricted here.

[0106] S154, perform gamma transformation on atmospheric light transmittance to obtain the atmospheric light transmittance enhancement value.

[0107] In one embodiment, the atmospheric light transmittance can be gamma-transformed using a pre-set gamma function to obtain an enhanced value for atmospheric light transmittance.

[0108] S155, a pixel traversal model is generated based on the target atmospheric light intensity and atmospheric light transmittance enhancement value.

[0109] In one embodiment, the pixel traversal model can be calculated using the following formula 2:

[0110]

[0111] Where J(x) represents the pixel traversal model, I(x) represents the image to be processed, and t0 is a pre-set value to avoid a denominator of 0.

[0112] The following combination Figure 7 The brightness channel prior value determination process provided in this application embodiment is defined as follows: In one embodiment, the electronic device calculates the target brightness channel prior value corresponding to the image to be processed, including:

[0113] S1511, determine the brightness value of each pixel in the image to be processed, and obtain the first brightness value set.

[0114] In one embodiment, when the image to be processed is in RGB data format, the maximum value of the pixel values ​​corresponding to the same position in the RGB three channels is determined, and a first luminance value set is generated from the maximum value of the pixel values ​​at each position in the image to be processed. When the image to be processed is in YUV data format, the first luminance value set is generated from the pixel values ​​at each position in the Y channel image.

[0115] S1512, Select multiple local maximum values ​​from the first brightness value set according to the first preset sliding window algorithm.

[0116] For example, the overall brightness of the image to be processed is normal, but because the image contains a glass of cola, which is black, the black cola can easily affect the overall brightness judgment of the image to be processed. Therefore, this application uses a first preset sliding window algorithm to select multiple local maxima from a first brightness set, which can filter out the influence of extremely black objects (in this embodiment, cola) on the overall brightness of the image to be processed. The first preset sliding window algorithm is existing technology, and the window size can be set according to actual needs, and is not limited here.

[0117] S1513, sort the multiple local maximum values ​​in descending order, and select the first local maximum value with the first preset percentage that is sorted first.

[0118] In one embodiment, the first preset percentage can be set according to actual needs, for example, the first preset percentage is 0.1%.

[0119] S1514, calculate the first average value of the local maximum value of the first preset proportion, and use the first average value as the prior value of the target bright channel.

[0120] In one embodiment, taking the number of local maximum values ​​of the first preset proportion as three as an example, namely local maximum value A, local maximum value B and local maximum value C, the first average value is (A+B+C) / 3, and the first average value is used as the prior value of the target bright channel.

[0121] The following combination Figure 8 This application describes the process for determining the target atmospheric light intensity provided in its embodiments. The atmospheric light assumption has poor generalization because it often assumes the brightest part of an image is atmospheric light. However, even in low-light images, the brightest pixel may not be atmospheric light, and the influence of pure white objects must also be considered. In one embodiment, the electronic device calculates the atmospheric light intensity corresponding to the image to be processed, including:

[0122] S1521, determine the brightness value of each pixel in the image to be processed to obtain the second brightness value set.

[0123] In one embodiment, when the image to be processed is in RGB data format, the minimum value of the pixel values ​​corresponding to the same position in the RGB three channels is determined, and a second luminance value set is generated from the minimum value of the pixel values ​​at each position in the image to be processed. When the image to be processed is in YUV data format, the second luminance value set is generated from the pixel values ​​at each position in the Y channel image.

[0124] S1522, Select multiple local minimum values ​​from the second brightness value set according to the second preset sliding window algorithm.

[0125] For example, the overall brightness of the image to be processed is normal, but because the image contains a piece of tissue paper, which is pure white, the pure white tissue paper can easily affect the overall brightness judgment of the image to be processed. Therefore, this application uses a second preset sliding window algorithm to select multiple local minima from a second brightness value set, which can filter out the influence of extremely white objects (tissue paper in this embodiment) on the overall brightness of the image to be processed. The second preset sliding window algorithm is prior art and is not intended to limit it further.

[0126] S1523, sort the multiple local minimums in descending order, and select the local minimum with the second preset percentage that is ranked first.

[0127] In one embodiment, the second preset percentage can be set according to actual needs, for example, the second preset percentage is 0.1%.

[0128] S1524, calculate the second average value of the local minimum of the second preset proportion, and use the second average value as the target atmospheric light intensity.

[0129] In one embodiment, taking the number of local minimums of the second preset proportion as three as an example, namely local minimum D, local minimum E and local minimum F, the second average value is (D+E+F) / 3, and the second average value is used as the target atmospheric light intensity.

[0130] Please see Figure 9 , Figure 9 This is a schematic diagram of the low-light enhancement processing device provided in an embodiment of this application. In some embodiments, the low-light enhancement processing device 20 may include multiple functional modules composed of computer program segments. The computer programs of each program segment in the low-light enhancement processing device 20 may be stored in the memory of an electronic device and executed by at least one controller to perform (see details). Figure 2 (Description) Low-light enhancement processing function.

[0131] In this embodiment, the low-light enhancement processing device 20 can be divided into multiple functional modules according to its functions. These functional modules may include: a source acquisition module 201, a first coefficient determination module 202, a brightness calculation module 203, a second coefficient determination module 204, a traversal table determination module 205, an initial pixel determination module 206, a target pixel determination module 207, and a pixel adjustment module 208. The term "module" in this application refers to a series of computer program segments that can be executed by at least one controller and perform a fixed function, and which are stored in memory. In this embodiment, the functions of each module will be detailed in subsequent embodiments.

[0132] The source acquisition module 201 can be used to acquire the image to be processed and the source information of the image to be processed.

[0133] The first coefficient determination module 202 can be used to determine the first brightness enhancement coefficient based on the source information.

[0134] The brightness calculation module 203 can be used to calculate the image brightness information corresponding to the image to be processed.

[0135] The second coefficient determination module 204 can be used to determine the second brightness enhancement coefficient based on the image brightness information.

[0136] The traversal table determination module 205 can be used to determine the pixel traversal model corresponding to the image to be processed.

[0137] The initial pixel determination module 206 can be used to determine multiple initial pixels corresponding to the image to be processed, and obtain an initial pixel set.

[0138] The target pixel determination module 207 can be used to call the pixel traversal model to traverse the initial pixel set and obtain the target pixel set corresponding to the initial pixel set.

[0139] The pixel adjustment module 208 can be used to adjust the initial pixel set and the target pixel set according to the first brightness enhancement coefficient and the second brightness enhancement coefficient to obtain a low-light enhanced image.

[0140] The images to be processed are shown in the following embodiments. In this application, the functional modules of the low-light enhancement processing device 20 and the low-light enhancement processing methods of the above embodiments belong to the same application concept. The specific implementation of each module of the low-light enhancement processing device 20 corresponds to each step of the low-light enhancement processing method in the above embodiments, and will not be described again in this application.

[0141] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may exist in actual implementation. The integrated units implemented in the form of software functional modules described above can be stored in a computer-readable storage medium. The software functional modules stored in a storage medium include several instructions to cause a computer device (which may be a personal computer, computer equipment, or network device, etc.) or processor to execute portions of the methods of the various embodiments of this application.

[0142] In some embodiments, the memory 31 stores a computer program that, when executed by at least one processor 32, implements all or part of the steps in a low-light enhancement processing method. The memory 31 includes read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), one-time programmable read-only memory (OTPROM), electrically-erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, disk storage, magnetic tape storage, or any other computer-readable medium capable of carrying or storing data.

[0143] Furthermore, a computer-readable storage medium may primarily include a stored program area and a stored data area, wherein the stored program area may store the operating system, an application program required for at least one function, etc.; and the stored data area may store created data, etc.

[0144] In some embodiments, at least one processor 32 is the control unit of the electronic device 3, connecting various components of the electronic device 3 via various interfaces and lines. It executes programs or modules stored in the memory 31 and calls data stored in the memory 31 to perform various functions and process data. For example, when at least one processor 32 executes a computer program stored in the memory, it implements all or part of the steps of the low-light enhancement processing method in the embodiments of this application; or it implements all or part of the functions of the low-light enhancement processing device. At least one processor 32 may be composed of integrated circuits, such as a single-packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips.

[0145] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0146] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.

[0147] It will be apparent to those skilled in the art that this application is not limited to the details of the exemplary embodiments described above, and that it can be implemented in other specific forms without departing from the spirit or essential characteristics of this application. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of this application is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within this application. No reference numerals in the claims should be construed as limiting the scope of the claims. Furthermore, it is clear that the word "comprising" does not exclude other elements or, and the singular does not exclude the plural. Multiple elements or devices recited in the specification may also be implemented by a single element or device through software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any particular order.

[0148] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application and are not intended to limit it. Although this application has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of this application without departing from the spirit and scope of the technical solutions of this application.

Claims

1. A low-light enhancement processing method, characterized in that, The method includes: Obtain the image to be processed and its source information; Determining a first brightness enhancement coefficient based on the source information includes: when the source information is not video, determining the first brightness enhancement coefficient as a preset value; when the source information is video, determining the video frame rate of the video corresponding to the image to be processed and the interval difference between the preset interval corresponding to the first brightness enhancement coefficient, calculating the ratio of the interval difference to the video frame rate, and determining the first brightness enhancement coefficient corresponding to each frame of the image to be processed in the video based on the ratio; wherein, the interval corresponding to the first brightness enhancement coefficient is the value range of the first brightness enhancement coefficient corresponding to the video frame rate; Calculate the image brightness information corresponding to the image to be processed; A second brightness enhancement coefficient is determined based on the image brightness information; Determining the pixel traversal model corresponding to the image to be processed includes: calculating the target bright channel prior value corresponding to the image to be processed; calculating the target atmospheric light intensity corresponding to the image to be processed; processing the target bright channel prior value and the target atmospheric light intensity according to a preset atmospheric light transmittance formula to obtain atmospheric light transmittance; performing a gamma transformation on the atmospheric light transmittance to obtain an atmospheric light transmittance enhancement value; and generating the pixel traversal model based on the target atmospheric light intensity and the atmospheric light transmittance enhancement value. Determine multiple initial pixels corresponding to the image to be processed to obtain an initial pixel set; The pixel traversal model is invoked to traverse the initial pixel set to obtain the target pixel set corresponding to the initial pixel set; Adjusting the initial pixel set and the target pixel set according to the first brightness enhancement coefficient and the second brightness enhancement coefficient to obtain a low-light enhanced image includes: determining a mathematical relationship between the first brightness enhancement coefficient, the second brightness enhancement coefficient, the target pixel set, and the initial pixel set; and determining the low-light enhanced image based on the mathematical relationship, wherein the mathematical relationship includes: , wherein For the target pixel set, the This is the initial pixel set.

2. The method as described in claim 1, characterized in that, Before calculating the image brightness information corresponding to the image to be processed, the method further includes: Determine the image format of the image to be processed; When the image format is RGB data format, the RGB data format is converted to YUV data format.

3. The method as described in claim 2, characterized in that, The calculation of the image brightness information corresponding to the image to be processed includes: Determine the Y channel image corresponding to the image to be processed; Determine the preset dark area pixels and global image pixels corresponding to the Y channel image; Calculate the pixel ratio between the number of first pixels corresponding to the preset dark area pixels and the number of second pixels corresponding to the global image pixels; The Y-channel image is split according to a preset ratio to obtain multiple image blocks; A preset number of sampling points are selected from the plurality of image blocks, and a sampling interval is determined based on the sampling points; Calculate the mean pixel value corresponding to the sampling interval; The pixel ratio value and the pixel mean value are weighted to obtain the image brightness information corresponding to the image to be processed.

4. The method as described in claim 1, characterized in that, The calculation of the prior value of the target brightness channel corresponding to the image to be processed includes: The brightness value of each pixel in the image to be processed is determined to obtain a first set of brightness values; Multiple local maximum values ​​are selected from the first set of brightness values ​​according to the first preset sliding window algorithm; The multiple local maximum values ​​are sorted in descending order, and the local maximum value with the first preset percentage is selected. Calculate the first average value of the local maximum value of the first preset proportion, and use the first average value as the prior value of the target bright channel.

5. The method as described in claim 4, characterized in that, The calculation of the target atmospheric light intensity corresponding to the image to be processed includes: The brightness value of each pixel in the image to be processed is determined to obtain a second brightness value set; According to the second preset sliding window algorithm, multiple local minimum values ​​are selected from the second brightness value set; The local minimum values ​​are sorted in descending order, and the local minimum value with the second preset percentage that appears first in the sort is selected. Calculate the second average value of the local minimum of the second preset proportion, and use the second average value as the target atmospheric light intensity.

6. A low-light enhancement processing apparatus for performing the low-light enhancement processing method as described in any one of claims 1 to 5, characterized in that, The device includes: The source acquisition module is used to acquire the image to be processed and the source information of the image to be processed; The first coefficient determination module is used to determine the first brightness enhancement coefficient based on the source information; A brightness calculation module is used to calculate the image brightness information corresponding to the image to be processed; The second coefficient determination module is used to determine the second brightness enhancement coefficient based on the image brightness information; The traversal table determination module is used to determine the pixel traversal model corresponding to the image to be processed; An initial pixel determination module is used to determine multiple initial pixels corresponding to the image to be processed, thereby obtaining an initial pixel set; The target pixel determination module is used to traverse the pixel traversal model to obtain the target pixel set corresponding to the initial pixel set. The pixel adjustment module is used to adjust the initial pixel set and the target pixel set according to the first brightness enhancement coefficient and the second brightness enhancement coefficient to obtain a low-light enhanced image.

7. An electronic device, characterized in that, The electronic device includes a processor and a memory, wherein the processor is configured to implement the low-light enhancement processing method as described in any one of claims 1 to 5 when executing a computer program stored in the memory.

8. A computer-readable storage medium storing a computer program thereon, characterized in that, When the computer program is executed by the controller, it implements the low-light enhancement processing method as described in any one of claims 1 to 5.

Images