Video dark-light enhancement method, apparatus and device, and storage medium and product

Abstract

Provided in the embodiments of the present application are a video dark-light enhancement method, apparatus and device, and a storage medium and a product. In the technical solution provided in the embodiments of the present application, on the basis of mean brightness information, image dark part information and image bright part information of an image to be processed, dark-light strength information of said image is determined; filtering processing is performed on the dark-light strength information, so as to obtain filtered dark-light strength information; and dark-light enhancement processing is performed on said image on the basis of the filtered dark-light strength information. Therefore, a user does not need to manually adjust a dark-light enhancement intensity, and adaptive dark-light enhancement is automatically carried out on the basis of brightness information of an image to be processed in a video call, thereby effectively improving the dark-light enhancement effect in the video call.

Description

Video dark light enhancement method, device, equipment, storage medium and product

[0001] This application claims priority to the Chinese patent application filed with the China Patent Office on September 12, 2023, with application number 202311178460.9, the entire contents of which are incorporated by reference into this application. Technical Field

[0002] The embodiments of the present application relate to the field of video processing technology, and in particular to a method, device, equipment, storage medium and product for enhancing low-light conditions in a video. Background Art

[0003] In a real-time video call scenario, when the user is in an environment with poor lighting conditions or a dark environment, the video call may have poor picture clarity and low face recognition, seriously affecting the user's video call experience.

[0004] To improve call quality in low-light environments, low-light enhancement technology is commonly used to enhance the brightness of camera-captured images. Traditional low-light enhancement typically uses histogram equalization to increase the overall brightness of the image. However, this low-light enhancement solution lacks adaptability and controllability, resulting in poor performance during video calls.

[0005] Summary of the Invention

[0006] The embodiments of the present application provide a method, apparatus, device, storage medium, and product for enhancing low-light video, so as to solve the technical problem in the related art that low-light enhancement has a poor effect in video calls, and effectively improve the low-light enhancement effect in video calls.

[0007] In a first aspect, an embodiment of the present application provides a method for enhancing low-light conditions in a video, comprising:

[0008] Determine the average brightness information, image dark part information and image bright part information of the image to be processed;

[0009] Determining the dark light level information of the image to be processed according to the average brightness information, the dark part information of the image, and the bright part information of the image;

[0010] performing filtering processing on the dim light level information to obtain filtered dim light level information;

[0011] Perform dark light enhancement processing on the image to be processed according to the filtered dark light level information.

[0012] In a second aspect, an embodiment of the present application provides a video dark light enhancement device, comprising:…

[0013] In a third aspect, an embodiment of the present application provides a video low-light enhancement device, comprising: a memory and one or more processors;

[0014] The memory is used to store one or more programs;

[0015] When the one or more programs are executed by the one or more processors, the one or more processors implement the video low-light enhancement method as described in the first aspect.

[0016] In a fourth aspect, an embodiment of the present application provides a non-volatile storage medium storing computer-executable instructions, which, when executed by a computer processor, are used to perform the video low-light enhancement method as described in the first aspect.

[0017] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor of the device reads and executes the computer program from the computer-readable storage medium, so that the device performs the video low-light enhancement method as described in the first aspect.

[0018] The embodiment of the present application determines the dark light level information of the image to be processed based on the average brightness information, dark part information and bright part information of the image to be processed, and filters the dark light level information to obtain filtered dark light level information. Then, dark light enhancement processing is performed on the image to be processed according to the filtered dark light level information. There is no need for the user to manually adjust the dark light enhancement intensity. The dark light enhancement is automatically and adaptively performed according to the brightness information of the image to be processed in the video call, thereby effectively improving the dark light enhancement effect in the video call. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] FIG1 is a flow chart of a method for enhancing low-light conditions in a video according to an embodiment of the present application;

[0020] FIG2 is a flow chart of another method for enhancing low-light conditions in a video provided by an embodiment of the present application;

[0021] FIG3 is a schematic diagram of a process for determining dark image information and bright image information provided by an embodiment of the present application;

[0022] FIG4 is a schematic diagram of a flow chart of determining an authority coefficient according to an embodiment of the present application;

[0023] FIG5 is a schematic diagram of a dark light enhancement process for an image to be processed provided by an embodiment of the present application;

[0024] FIG6 is a schematic structural diagram of a video dark light enhancement device provided in an embodiment of the present application;

[0025] FIG7 is a schematic structural diagram of a video dark light enhancement device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical solutions and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below in conjunction with the accompanying drawings. It is understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. It should also be noted that, for ease of description, only some, but not all, of the contents related to the present application are shown in the accompanying drawings. Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flow charts. Although the flow charts describe each operation (or step) as a sequential process, many of the operations therein can be implemented in parallel, concurrently or simultaneously. In addition, the order of the operations can be rearranged. The above process can be terminated when its operation is completed, but it can also have additional steps not included in the accompanying drawings. The above process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0027] The video dark light enhancement method provided in the present application can be applied to video call scenarios, for example, to perform dark light enhancement on the image to be processed collected during the video call, aiming to determine the filtered dark light level information based on the brightness information of the image to be processed in the video call, and perform dark light enhancement processing on the image to be processed based on the filtered dark light level information, thereby improving the dark light enhancement effect in the video call. In existing dark light enhancement schemes, histogram equalization is generally used to stretch the overall brightness of the image, but this dark light enhancement method has poor adaptability and controllability, and the dark light enhancement effect is poor. If deep learning is used for dark light enhancement, the computational complexity is large, and it is not suitable for video call scenarios. Based on this, a video dark light enhancement method according to an embodiment of the present application is provided to solve the technical problems of poor adaptability and controllability, poor dark light enhancement effect, and large computational complexity of existing dark light enhancement schemes.

[0028] Figure 1 shows a flowchart of a video dark light enhancement method provided in an embodiment of the present application. The video dark light enhancement method provided in an embodiment of the present application can be executed by a video dark light enhancement device, which can be implemented by hardware and / or software and integrated into a video dark light enhancement device (such as a mobile phone, tablet, etc.).

[0029] The following description is based on an example of a method for enhancing video dark light by using a video dark light enhancement device. Referring to FIG1 , the method for enhancing video dark light includes:

[0030] S110: Determine average brightness information, image dark portion information, and image bright portion information of the image to be processed.

[0031] For example, during a video call, the image to be processed is captured by the video dark light enhancement device through a built-in or external camera device, and the average brightness information, dark part information and bright part information of the image to be processed are determined.

[0032] The average brightness information provided by this solution can be used to indicate the average brightness of the image to be processed, the dark part information of the image can be used to indicate the information related to brightness and area corresponding to the lower brightness part of the image to be processed (such as the average value of the dark area of ​​the dark area and the proportion of the dark area), and the bright part information of the image can be used to indicate the information related to area corresponding to the higher brightness part (such as the part other than the dark area) of the image to be processed (such as the proportion of the bright area of ​​the bright area). The dark area can be understood as the area formed by the pixels in the image to be processed whose brightness is lower than the set brightness threshold, and the bright area can be understood as the area formed by the pixels in the image to be processed whose brightness reaches the set brightness threshold.

[0033] In one embodiment, the dark light enhancement method provided by the present solution may be to determine the average brightness information, image dark information, and image bright information of the image to be processed based on the color information of each pixel in the image to be processed. For example, the brightness information of each pixel is determined based on the color information of the image to be processed, and the average brightness information of the image to be processed and the dark area and bright area in the image to be processed are determined based on the brightness information, and the image dark information corresponding to the dark area and the image bright information corresponding to the bright area are determined. For example, when the image to be processed is an image in RGB format, the brightness information of each pixel is determined based on the color values ​​of the three RGB color channels of the image to be processed, and the average brightness information of the image to be processed and the dark area and bright area in the image to be processed are determined based on the brightness information of these pixels, and the image dark information corresponding to the dark area and the image bright information corresponding to the bright area are determined.

[0034] In one possible embodiment, the dark light enhancement method provided by the present solution may be to determine the average brightness information, image dark part information, and image bright part information of the image to be processed based on the brightness information of each pixel point in the image to be processed on the brightness channel. Based on this, the video dark light enhancement method provided by the present solution, before determining the average brightness information, image dark part information, and image bright part information of the image to be processed, further includes: converting the color space of the image to be processed to a set color space (a color space including a brightness channel). Correspondingly, when determining the average brightness information, image dark part information, and image bright part information of the image to be processed, the average brightness information, image dark part information, and image bright part information of the image to be processed may be determined based on the brightness information of each pixel point in the image to be processed on the brightness channel.

[0035] For example, the image to be processed in RGB format is converted into the image to be processed in YUV format (where Y represents the brightness channel and UV represents the chrominance channel) or HSV format (where V represents the brightness channel and HS represents the chrominance channel). The average brightness information, dark part information and bright part information of the image to be processed can be determined based on the brightness information of each pixel in the brightness channel (for example, the V channel in the YUV format).

[0036] It needs to be explained that this scheme evaluates the degree of darkness of the scene rather than color-related information. This scheme can effectively reduce the data processing amount of the video dark light enhancement method (the calculation amount is reduced by two-thirds compared to the image calculation amount in RGB format) by converting the color space of the image to be processed to the set color space and performing relevant data processing of the video dark light enhancement method on the brightness information of the brightness channel, thereby improving the efficiency of dark light enhancement while ensuring the dark light enhancement effect.

[0037] S120: Determine the dark light level information of the image to be processed according to the average brightness information, the dark part information of the image, and the bright part information of the image.

[0038] Exemplarily, the darkness level information of the image to be processed is determined based on the above-determined average brightness information, image dark area information, and image bright area information. The darkness level information can be used to indicate the darkness level of the image to be processed. A higher darkness level corresponds to a higher intensity of darkness enhancement processing during the dark light enhancement process. Optionally, the darkness level information of the image to be processed can be mapped to an interval of [0, 1], where as the darkness level information changes from 0 to 1, the corresponding darkness level increases.

[0039] In one embodiment, the image dark area information provided by this solution includes the average dark area value and the dark area ratio of the dark area in the image to be processed, and the image bright area information includes the bright area ratio of the bright area in the image to be processed. The dark light level information is inversely proportional to the average brightness information, the average dark area value, and the bright area ratio, and is directly proportional to the dark area ratio.

[0040] S130: Filtering the dark light level information to obtain filtered dark light level information.

[0041] Exemplarily, after determining the dark light level information of the image to be processed based on the average brightness information, the dark portion information of the image, and the bright portion information of the image, the dark light level information is further filtered to obtain the filtered dark light level information. Alternatively, the dark light level information may be filtered in the time domain using a filtering method such as time domain filtering or Kalman filtering to obtain the filtered dark light level information.

[0042] It needs to be explained that after determining the dark light level information of the image to be processed based on the average brightness information, the dark part information of the image and the bright part information of the image, if the dark light level information is directly used to perform dark light enhancement on the image to be processed, it is easy for the dark light enhancement effect to oscillate in a cycle. For example, after dark light enhancement is performed on the image to be processed based on the dark light level information, the brightness of the dark light enhanced image displayed on the screen increases, causing the face to become brighter. At this time, the brightness of the face area in the collected image to be processed is greatly improved, and the dark light intensity reflected by the corresponding determined dark light level information is greatly reduced (for example, when dark light is not detected). At this time, the dark light enhancement intensity of the image to be processed is greatly reduced or no dark light enhancement is performed. At this time, the brightness of the face area in the next collected image to be processed is greatly reduced, and the dark light intensity reflected by the corresponding determined dark light level information is greatly increased. According to this cycle, the dark light enhancement intensity repeatedly jumps, resulting in obvious jitter in the dark light enhancement effect, and the dark light enhancement effect is poor. This solution obtains filtered dark light level information by filtering the dark light level information, effectively reducing the jitter of the dark light enhancement effect and improving the dark light enhancement effect.

[0043] S140: Performing dark light enhancement processing on the image to be processed according to the filtered dark light level information.

[0044] Exemplarily, after filtering the dark light level information to obtain filtered dark light level information, dark light enhancement processing is performed on the image to be processed according to the filtered dark light level information. The higher the dark light level reflected by the filtered dark light level information, the higher the intensity of the dark light enhancement processing on the image to be processed.

[0045] In one embodiment, the filtered dark light level information can be mapped to the interval [0, 1], wherein when the filtered dark light level information changes from 0 to 1, the corresponding dark light enhancement intensity becomes increasingly higher. When the filtered dark light level information is 0, no dark light enhancement is performed, and when the filtered dark light level information is 1, the highest intensity dark light enhancement processing is performed. In one embodiment, performing dark light enhancement processing on the image to be processed based on the filtered dark light level information is performing dark light enhancement processing on the image to be processed before color space conversion based on the filtered dark light level information, for example, performing dark light enhancement processing on the image to be processed in RGB format.

[0046] In the above, by determining the dark light level information of the image to be processed based on the average brightness information, dark part information and bright part information of the image to be processed, and filtering the dark light level information to obtain filtered dark light level information, the image to be processed is subjected to dark light enhancement processing according to the filtered dark light level information. There is no need for the user to manually adjust the dark light enhancement intensity, and the dark light enhancement is automatically and adaptively performed according to the brightness information of the image to be processed in the video call, thereby effectively improving the dark light enhancement effect in the video call.

[0047] Based on the above embodiment, FIG2 shows a flowchart of another method for enhancing low-light video provided by an embodiment of the present application. The method for enhancing low-light video is a specific embodiment of the above method for enhancing low-light video. Referring to FIG2 , the method for enhancing low-light video includes:

[0048] S210: Determine the picture brightness information of the image to be processed in the brightness channel, and determine the average brightness information of the image to be processed according to the picture brightness information.

[0049] Exemplarily, an image to be processed is obtained, and the color space of the image to be processed is converted to a set color space including a brightness channel, and the picture brightness information of the image to be processed in the brightness channel (that is, the brightness value of each pixel point of the image to be processed in the brightness channel) is determined. For example, the image to be processed is converted into an image in YUV format, and the value of each pixel point of the image to be processed in the Y channel is the picture brightness information of the corresponding pixel point.

[0050] In one embodiment, after determining the image brightness information of each pixel in the image to be processed in the brightness channel, average brightness information of the image to be processed is determined based on the image brightness information. The average brightness information can be the ratio of the sum of the image brightness information corresponding to each pixel in the image to be processed to the number of pixels in the image to be processed.

[0051] S220: Determine the dark area and the bright area in the image to be processed according to the picture brightness information, and determine the dark part information of the dark area and the bright part information of the bright area.

[0052] Exemplarily, the dark area and the bright area in the image to be processed are determined based on the picture brightness information of each pixel in the image to be processed. For example, for each pixel, the picture brightness information is compared with a set brightness threshold. If the picture brightness information is less than the set brightness threshold, the pixel is classified into the dark area. If the picture brightness information reaches the set brightness threshold, the pixel is classified into the bright area. In one embodiment, the set brightness threshold can be set to 60. When the picture brightness information of the pixel is less than 60, the pixel is classified into the dark area. When the picture brightness information of the pixel is ≥60, the pixel is classified into the bright area.

[0053] After determining the dark area in the image to be processed, the image dark information corresponding to the dark area and the image bright information corresponding to the bright area are determined. In one embodiment, the image dark information corresponding to the dark area provided by this solution includes the dark area average value and the dark area ratio of the dark area, and the image bright information corresponding to the bright area includes the bright area ratio of the bright area. This solution accurately reflects the picture brightness of the image to be processed, as well as the color information and ratio information of the bright and dark areas, by determining the average brightness information, the image dark information of the dark area, and the image bright information of the bright area based on the picture brightness information of the image to be processed in the brightness channel, thereby improving the evaluation accuracy of the dark light level information of the image to be processed and improving the dark light enhancement effect of the video.

[0054] In one possible embodiment, as shown in a flowchart of determining dark image information and bright image information provided in FIG3 , the dark-light enhancement method provided in this solution includes, when determining dark image information of dark areas and bright image information of bright areas:

[0055] S221: Determine a dark area average value of the dark area according to the image brightness information of the dark area and the number of dark pixels in the dark area.

[0056] S222: Determine a dark area ratio of the dark area according to the number of dark pixels in the dark area and the number of image pixels of the image to be processed.

[0057] S223: Determine a bright area ratio of the bright area according to the number of bright pixels in the bright area and the number of image pixels of the image to be processed.

[0058] Exemplarily, the dark area average value of the dark area is calculated based on the image brightness information of each pixel in the dark area of ​​the image to be processed and the number of dark pixels contained in the dark area. For example, the ratio of the cumulative sum of the image brightness information of each pixel in the dark area to the number of dark pixels contained in the dark area is used as the dark area average value of the dark area.

[0059] Optionally, the average value of the dark area provided by this solution can be determined according to the following dark area average value calculation formula:

[0060] Among them, dark_pixel_mean is the average value of the dark area, dark_pixel_valu is the image brightness information of the pixels in the dark area, and dark_pixel_num is the number of dark pixels in the dark area.

[0061] The dark area ratio of the dark area is calculated based on the number of dark pixels in the dark area and the number of image pixels of the image to be processed. For example, the ratio of the number of dark pixels in the dark area to the number of image pixels of the image to be processed can be used as the dark area ratio of the dark area. Optionally, the dark area ratio provided by this solution can be determined according to the following dark area ratio determination formula:

[0062] Among them, dark_pixel_ratio is the ratio of dark areas, dark_pixel_num is the number of dark pixels in the dark area, and image_pixel_num is the number of image pixels in the image to be processed.

[0063] The bright area ratio of the bright area is calculated based on the number of bright pixels in the bright area and the number of image pixels in the image to be processed. For example, the ratio of the number of bright pixels in the bright area to the number of image pixels in the image to be processed can be used as the bright area ratio of the bright area. Optionally, the bright area ratio provided in this solution can be determined using the following bright area ratio determination formula:

[0064] Where bright_pixel_ratio is the ratio of bright areas to bright areas, bright_pixel_num is the number of bright pixels in the bright areas, and image_pixel_num is the number of image pixels in the image to be processed. This solution accurately determines the average dark area value, the ratio of dark areas to bright areas, and the ratio of bright areas to bright areas based on the brightness information of the dark areas, the number of dark pixels in the dark areas, the number of image pixels in the image to be processed, and the number of bright pixels in the bright areas. This effectively improves the accuracy of the dark light level assessment information of the processed image and enhances the dark light enhancement effect of the video.

[0065] S230: Determine the average brightness weight coefficient, dark pixel average value authority coefficient, dark pixel proportion authority coefficient, average brightness dark pixel average value authority coefficient and bright pixel proportion weight coefficient of the image to be processed based on the average brightness information, image dark part information and image bright part information.

[0066] Exemplarily, based on the above-determined average brightness information, image dark information (including the dark area average value and the dark area proportion) and image bright information (including the bright area proportion of the bright area), the average brightness weight coefficient, dark pixel average value authority coefficient, dark pixel proportion authority coefficient, average brightness dark pixel average value authority coefficient and bright pixel proportion weight coefficient of the image to be processed are calculated.

[0067] Among them, the higher the overall brightness of the image to be processed, the smaller the corresponding average brightness weight coefficient; the greater the average brightness of the dark light pixels in the dark area, the smaller the corresponding dark pixel average value authority coefficient; the larger the proportion of the dark area, the larger the corresponding dark pixel proportion authority coefficient; the smaller the average brightness of the dark light pixels is lower than the average brightness of the image to be processed, the larger the corresponding average brightness dark pixel average value authority coefficient; the larger the proportion of the bright area in the bright area, the larger the corresponding bright pixel proportion weight coefficient.

[0068] In one possible embodiment, as shown in a flowchart of authority coefficient determination provided in FIG4 , the video dark light enhancement method provided by the present solution determines the average brightness weight coefficient, the dark pixel average authority coefficient, the dark pixel ratio authority coefficient, the average brightness dark pixel average authority coefficient, and the bright pixel ratio weight coefficient of the image to be processed based on the average brightness information, the image dark part information, and the image bright part information, including:

[0069] S231: performing smooth transition processing on the average brightness information within a set first brightness range according to a set smoothing function to obtain an average brightness weight coefficient of the image to be processed.

[0070] S232: performing smooth transition processing on the average value of the dark area in the dark area of ​​the image to be processed within a set second brightness range according to a smoothing function to obtain an authority coefficient of the average value of dark pixels in the image to be processed.

[0071] S233: performing a smooth transition process on the authority coefficient of the average value of dark pixels within a set first weight coefficient range according to a smoothing function to obtain the authority coefficient of the proportion of dark pixels of the image to be processed.

[0072] S234: performing smooth transition processing on the difference between the average brightness information and the average value of the dark area of ​​the dark area in the image to be processed within a set third brightness range according to a smoothing function to obtain an average brightness dark area pixel average value authority coefficient of the image to be processed.

[0073] S235: performing a smooth transition process on the bright area ratio of the bright area in the image to be processed within a set second weight coefficient range according to a smoothing function to obtain a bright area pixel ratio weight coefficient of the image to be processed.

[0074] The smoothing function provided by this solution can be a polynomial interpolation function (such as a Hermite polynomial interpolation function), which maps the weight coefficient to the range of [0, 1] for smooth transition. For example, the smoothing function can be expressed as:

[0075] Where x is the parameter that needs to be smoothly transitioned. At the same time, assuming that a smooth transition needs to be performed within the set range [edge_left, edge_right], we can set:

[0076] Among them, edge_left is the first extreme value of the set range, edge_right is the second extreme value of the set range, and x new is the value after smooth transition within the set range [edge_left, edge_right], x old is the value before smooth transition. new Substituting this into the smoothing function will normalize the smoothed value to the range [0, 1]. Correspondingly, the smoothing function smoothstep can be used to smooth the parameters within a given range as follows: smoothstep(x, edge_left, edge_right). When the first extreme value is less than the second, the corresponding parameter smoothly transitions from 0 to 1; when the second extreme value is less than the first, the corresponding parameter smoothly transitions from 1 to 0.

[0077] Exemplarily, the average brightness information is smoothed in the set first brightness range according to the smoothing function set above to obtain the average brightness weight coefficient of the image to be processed. Optionally, the first extreme value and the second extreme value corresponding to the first brightness range can be 70 and 30. Optionally, the average brightness weight coefficient provided by this solution can be determined by the following average brightness weight coefficient determination formula: image_mean_factor=0.1+0.9*smoothstep(image_mean,70,30)

[0078] Among them, image_mean_factor is the average brightness weight coefficient, image_mean is the average brightness information. The larger the average brightness information, the smaller the average brightness weight coefficient, that is, the brighter the overall image, the lower the average brightness weight coefficient.

[0079] In one embodiment, a smooth transition process is performed on the dark area average value of the dark area in the image to be processed within a set second brightness range according to a smoothing function to obtain a dark pixel average value authority coefficient of the image to be processed. Optionally, the first extreme value and the second extreme value corresponding to the second brightness range may be 50 and 30. Optionally, the dark pixel average value authority coefficient provided by this solution may be determined by the following dark pixel average value authority coefficient determination formula: dark_pixel_mean_factor=smoothstep(dark_pixel_mean,50,30)

[0080] Where dark_pixel_mean_factor is the dark pixel mean value authority factor of the image to be processed, and dark_pixel_mean is the dark area mean value of the dark area. The larger the dark area mean value of the dark area, the lower the corresponding dark pixel mean value authority factor.

[0081] In one embodiment, the dark pixel mean value authority coefficient is smoothed in a set first weight coefficient range according to a smoothing function to obtain the dark pixel ratio authority coefficient of the image to be processed. Optionally, the first extreme value and the second extreme value corresponding to the first weight coefficient range can be 0.3 and 0.5. Optionally, the dark pixel ratio authority coefficient provided by this solution can be determined by the following dark pixel ratio authority coefficient determination formula: dark_pixel_ratio_factor=smoothstep(dark_pixel_mean_factor,0.3,0.5)

[0082] Where dark_pixel_ratio_factor is the dark pixel ratio factor of the image to be processed, and dark_pixel_mean_factor is the dark pixel mean factor. A larger dark pixel mean factor corresponds to a larger dark pixel ratio factor.

[0083] In one embodiment, a smooth transition process is performed on the difference between the average brightness information and the average value of the dark area of ​​the dark area in the dark area of ​​the image to be processed within a set third brightness range according to a smoothing function to obtain the average brightness dark pixel average value authority coefficient of the image to be processed. Optionally, the first extreme value and the second extreme value corresponding to the third brightness range can be 20 and 40. Optionally, the average brightness dark pixel average value authority coefficient provided by this solution can be determined by the following average brightness dark pixel average value authority coefficient determination formula: mean_diff_factor=1.0+0.5*smoothstep(image_mean-dark_pixel_mean,20,40)

[0084] Where mean_diff_factor is the weight factor for the average brightness of the image being processed, image_mean is the average brightness information, and dark_pixel_mean is the average value of the dark area. The greater the difference between the average brightness information and the average value of the dark area, the larger the corresponding average brightness dark pixel weight factor. This formula can achieve a weight factor amplification effect, with the maximum weight factor amplified to 1.5 times the original value.

[0085] In one embodiment, a smooth transition process is performed on the bright area ratio of the bright area in the image to be processed within a set second weight coefficient range according to a smoothing function to obtain a bright pixel ratio weight coefficient of the image to be processed. Optionally, the first extreme value and the second extreme value corresponding to the second weight coefficient range may be 0.2 and 0.1. Optionally, the bright pixel ratio weight coefficient provided by this solution may be determined by the following bright pixel ratio weight coefficient determination formula: bright_pixel_ratio_factor = smoothstep(bright_pixel_ratio, 0.2, 0.1)

[0086] Where bright_pixel_ratio_factor is the weight coefficient for the proportion of bright pixels in the image to be processed, and bright_pixel_ratio is the proportion of bright areas in the bright area of ​​the image to be processed. The larger the proportion of bright areas, the smaller the corresponding bright pixel ratio weight coefficient.

[0087] This solution accurately determines the average brightness weight coefficient, dark pixel average value authority coefficient, dark pixel ratio authority coefficient, average brightness dark pixel average value authority coefficient and bright pixel ratio weight coefficient of the image to be processed based on the set smoothing function and average brightness information, image dark information and image bright information, thereby improving the estimation accuracy of the dark light level information of the image to be processed and effectively improving the dark light enhancement effect.

[0088] S240: Determine the dark light level information of the image to be processed according to the average brightness weight coefficient, the dark pixel average value authority coefficient, the dark pixel ratio authority coefficient, the average brightness dark pixel average value authority coefficient and the bright pixel ratio weight coefficient.

[0089] Exemplarily, the dark light level information of the current image to be processed is calculated based on the above-determined average brightness weight coefficient, dark pixel average authority coefficient, dark pixel ratio authority coefficient, average brightness dark pixel average authority coefficient and bright pixel ratio weight coefficient. In one embodiment, the product of the average brightness weight coefficient, dark pixel average authority coefficient, dark pixel ratio authority coefficient, average brightness dark pixel average authority coefficient and bright pixel ratio weight coefficient can be used as the dark light level information of the image to be processed. Optionally, the dark light level information provided by this solution can be determined according to the following dark light level information determination formula: dark_strength = image_mean_factor*dark_pixel_mean_factor*dark_pixel_ratio_factor *mean_diff_factor*bright_pixel_ratio_factor

[0090] Among them, dark_strength is the darkness information, image_mean_factor is the average brightness weight coefficient, dark_pixel_mean_factor is the dark pixel mean value authority coefficient, dark_pixel_ratio_factor is the dark pixel ratio authority coefficient, mean_diff_factor is the average brightness dark pixel mean value authority coefficient, and bright_pixel_ratio_factor is the bright pixel ratio weight coefficient. This solution accurately determines the darkness information of the image to be processed by determining the average brightness weight coefficient, dark pixel mean value authority coefficient, dark pixel ratio authority coefficient, average brightness dark pixel mean value authority coefficient, and bright pixel ratio weight coefficient of the image to be processed based on the average brightness information, dark information, and bright information of the image to be processed, effectively improving the dark light enhancement effect.

[0091] In one possible embodiment, the video dark light enhancement method provided by the present solution can truncate the dark light level information within a set intensity range (for example, [0, 1]) after determining the dark light level information of the image to be processed, and can determine the dark light enhancement intensity of the image to be processed based on the dark light level information to ensure the dark light enhancement effect.

[0092] S250: Filter the dark light level information to obtain filtered dark light level information.

[0093] In one embodiment, the video dark light enhancement method provided by the present solution can filter the dark light level information to obtain filtered dark light level information by performing time domain filtering on the dark light level information according to an adaptive smoothing factor to obtain filtered dark light level information, wherein the adaptive smoothing factor is determined according to the set minimum cutoff frequency, constant cutoff frequency, and image acquisition period.

[0094] For example, based on the adaptive smoothing factor and the filtered dark light level information determined by performing dark light enhancement processing on the previous frame of the image to be processed, the dark light level information corresponding to the current image to be processed is subjected to time domain filtering processing to obtain the filtered dark light level information. For example, the filtered dark light level information can be expressed as:

[0095] in, is the filtered dark light level information corresponding to the current frame to be processed, is the filtered dark light level information corresponding to the previous frame to be processed, and α is the adaptive smoothing factor. The adaptive smoothing factor can be determined based on the following adaptive smoothing factor determination formula: T e =T i -T i-1

[0096] in, is the absolute value of the speed (i.e., the rate of change of the dark light level information of the current frame relative to the filtered dark light level information of the previous frame), is the absolute value of the filtered speed (i.e., the rate of change of the filtered dark light level information of the current frame relative to the filtered dark light level information of the previous frame), is the minimum cutoff frequency, β is the set speed coefficient, optional, β=0.007, f c is the frequency parameter determined according to the minimum cutoff frequency, the set speed coefficient and the speed, τ is the first period parameter determined according to the frequency parameter, T e According to the current frame timestamp T i and the previous frame timestamp T i-1 The corresponding time difference, that is, the image acquisition period, optional, T e =0.04. Among them, the speed It can be determined based on the following formula:

[0097] in, For a constant cutoff frequency, optional, τ d is the second period parameter determined according to the constant cutoff frequency, α d is the velocity filter coefficient determined according to the second cycle parameter and the image acquisition cycle, is the speed corresponding to the current image to be processed (i.e., the rate of change of the dark light level information of the current frame relative to the filtered dark light level information of the previous frame), is the speed corresponding to the previous frame of the image to be processed (i.e., the rate of change of the filtered dark light level information of the current frame relative to the filtered dark light level information of the previous frame), X i is the dark light level information corresponding to the current image to be processed, is the filtered dark light level information corresponding to the previous frame of the image to be processed. This solution performs time-domain filtering on the dark light level information based on an adaptive smoothing factor. Compared to filtering the dark light level information with a fixed smoothing factor, this solution can more accurately and quickly eliminate cyclic oscillations in the dark light enhancement effect under extreme conditions, effectively reducing jitter in the dark light enhancement effect and improving the dark light enhancement effect.

[0098] S260: Perform dark light enhancement processing on the image to be processed according to the filtered dark light level information.

[0099] In one possible embodiment, as shown in FIG5 , a schematic diagram of a dark light enhancement process for an image to be processed is provided. The dark light enhancement method of the video provided in this solution includes the following steps when performing dark light enhancement processing on the image to be processed based on filtered dark light level information:

[0100] S261: Determine a mapping color corresponding to the initial color of each image pixel in the image to be processed by displaying a lookup table filter.

[0101] S262: Determine a target color of each image pixel after performing dark light enhancement processing according to the filtered dark light level information, the initial color of each image pixel, and the mapped color.

[0102] This solution uses a display lookup table filter (LUT filter) to perform dark-light enhancement on the image being processed. For example, a pre-defined display lookup table filter determines the mapped color corresponding to the initial (RGB) color of each pixel in the image being processed. The display lookup table corresponding to the display lookup table filter records the dark-light-enhanced mapped colors corresponding to different color values. The mapped colors have a higher brightness than the initial colors.

[0103] In one possible embodiment, a corresponding mapping color can be set for each color value of each RGB channel in the display lookup table, and the mapping color corresponding to each initial color can be directly determined based on the display lookup table filter, wherein the value range of each RGB channel is [0, 255]. Therefore, to complete the mapping of all colors, it is necessary to set corresponding mapping colors for 256x256x256 initial colors in the texture map of the display lookup table. Optionally, 64x64x64 (512x512) colors can be used to define the mapping intervals, and the intermediate color mapping can be calculated by bilinear interpolation. Then, corresponding mapping colors can be set for the 512x512 initial colors in the texture map of the display lookup table. While ensuring the dark light enhancement effect, the memory usage of the display lookup table filter and the calculation overhead of the target color are effectively reduced.

[0104] In one embodiment, dark light enhancement processing is performed on the image to be processed based on the filtered dark light level information determined above, the initial color of each image pixel, and the mapped color, to obtain the target color of each image pixel after the dark light enhancement processing, thereby achieving dark light enhancement processing for the image to be processed. In one embodiment, a weighted summation processing can be performed on the initial color and the mapped color of each image pixel based on the filtered dark light level information to achieve dark light enhancement processing for each image pixel in the image to be processed. Optionally, the target color of each image pixel after the dark light enhancement processing can be determined based on the following target color determination formula: RGB final=(1-dark_strength)*RGB old +dark_strength*RGB new

[0105] Among them, RGB final is the target color, dark_strength is the filtered dark light information, RGB old is the initial color, RGB new This solution uses a display lookup table filter to determine the mapped color corresponding to the initial color of each image pixel in the processed image. Based on the filtered dark light level information, the initial color of each image pixel, and the mapped color, it more quickly and accurately determines the target color of each image pixel after dark light enhancement processing. This allows dark light enhancement processing of the processed image to be performed, effectively improving the dark light enhancement processing efficiency while maintaining the dark light enhancement effect.

[0106] In the above, by determining the dark light level information of the image to be processed based on its average brightness information, dark area information, and bright area information, and filtering the dark light level information to obtain filtered dark light level information, the image to be processed is then subjected to dark light enhancement processing based on the filtered dark light level information. This eliminates the need for the user to manually adjust the dark light enhancement intensity, and automatically and adaptively performs dark light enhancement based on the brightness information of the image to be processed during a video call, effectively improving the dark light enhancement effect during video calls. Furthermore, by determining the average brightness information, dark area information for dark areas, and bright area information for bright areas based on the brightness information of the image to be processed in the brightness channel, the image brightness of the image to be processed, as well as the color information and proportion of bright and dark areas, is accurately reflected. This improves the accuracy of the dark light level assessment of the image to be processed and enhances the dark light enhancement effect during video calls.

[0107] FIG6 is a schematic diagram of the structure of a video low-light enhancement device provided by an embodiment of the present application. Referring to FIG6 , the video low-light enhancement device includes an information determination module 61 , a low-light analysis module 62 , a filtering processing module 63 and a low-light enhancement module 64 .

[0108] Among them, the information determination module 61 is configured to determine the average brightness information, image dark part information and image bright part information of the image to be processed; the dark light analysis module 62 is configured to determine the dark light level information of the image to be processed based on the average brightness information, image dark part information and image bright part information; the filtering processing module 63 is configured to filter the dark light level information to obtain filtered dark light level information; the dark light enhancement module 64 is configured to perform dark light enhancement processing on the image to be processed based on the filtered dark light level information.

[0109] In the above, by determining the dark light level information of the image to be processed based on the average brightness information, dark part information and bright part information of the image to be processed, and filtering the dark light level information to obtain filtered dark light level information, the image to be processed is subjected to dark light enhancement processing according to the filtered dark light level information. There is no need for the user to manually adjust the dark light enhancement intensity, and the dark light enhancement is automatically and adaptively performed according to the brightness information of the image to be processed in the video call, thereby effectively improving the dark light enhancement effect in the video call.

[0110] In a possible embodiment, when determining the average brightness information, image dark part information, and image bright part information of the image to be processed, the information determination module 61 is configured as follows:

[0111] Determine the brightness information of the image to be processed in the brightness channel;

[0112] Determine the average brightness information of the image to be processed according to the picture brightness information;

[0113] Determine the dark area and the bright area in the image to be processed according to the brightness information of the image;

[0114] The dark information of the image in the dark area and the bright information of the image in the bright area are determined.

[0115] In a possible embodiment, when determining the dark portion information of the dark area and the bright portion information of the bright area, the information determining module 61 is configured as follows:

[0116] Determining a dark area average value of the dark area according to the image brightness information of the dark area and the number of dark pixels in the dark area;

[0117] determining a dark area ratio of the dark area according to the number of dark pixels in the dark area and the number of image pixels of the image to be processed;

[0118] The bright area ratio of the bright area is determined according to the number of bright pixels in the bright area and the number of image pixels of the image to be processed.

[0119] In a possible embodiment, when the dark light analysis module 62 determines the dark light level information of the image to be processed based on the average brightness information, the dark part information of the image, and the bright part information of the image, it is configured as follows:

[0120] Determine the average brightness weight coefficient, dark pixel average value authority coefficient, dark pixel ratio authority coefficient, average brightness dark pixel average value authority coefficient and bright pixel ratio weight coefficient of the image to be processed according to the average brightness information, image dark part information and image bright part information;

[0121] The dark light level information of the image to be processed is determined according to the average brightness weight coefficient, the dark pixel average value authority coefficient, the dark pixel proportion authority coefficient, the average brightness dark pixel average value authority coefficient and the bright pixel proportion weight coefficient.

[0122] In one possible embodiment, when the dark light analysis module 62 determines the average brightness weight coefficient, the dark pixel average value authority coefficient, the dark pixel ratio authority coefficient, the average brightness dark pixel average value authority coefficient, and the bright pixel ratio weight coefficient of the image to be processed based on the average brightness information, the image dark part information, and the image bright part information, the configuration is as follows:

[0123] Performing a smooth transition process on the average brightness information within a set first brightness range according to a set smoothing function to obtain an average brightness weight coefficient of the image to be processed;

[0124] Performing smooth transition processing on the average value of the dark area of ​​the dark area in the image to be processed within a set second brightness range according to a smoothing function to obtain an authority coefficient of the average value of the dark area pixels in the image to be processed;

[0125] Performing a smooth transition process on the authority coefficient of the average value of dark pixels within a set first weight coefficient range according to a smoothing function to obtain the authority coefficient of the proportion of dark pixels in the image to be processed;

[0126] Performing a smooth transition process on the difference between the average brightness information and the average value of the dark area of ​​the dark area in the image to be processed within a set third brightness range according to a smoothing function to obtain an authority coefficient for the average brightness dark area pixel average value of the image to be processed;

[0127] A smooth transition process is performed on the bright area ratio of the bright area in the image to be processed within a set second weight coefficient range according to the smoothing function to obtain a bright area pixel ratio weight coefficient of the image to be processed.

[0128] In a possible embodiment, when filtering the dim light level information to obtain the filtered dim light level information, the filtering processing module 63 is configured as follows:

[0129] The dark light level information is subjected to time domain filtering according to an adaptive smoothing factor to obtain filtered dark light level information, wherein the adaptive smoothing factor is determined according to a set minimum cutoff frequency, a constant cutoff frequency, and an image acquisition period.

[0130] In a possible embodiment, when performing dark light enhancement processing on the image to be processed according to the filtered dark light level information, the dark light enhancement module 64 is configured as follows:

[0131] Determining the mapped color corresponding to the initial color of each image pixel in the image to be processed by displaying a lookup table filter;

[0132] The target color of each image pixel after dark light enhancement processing is determined according to the filtered dark light level information, the initial color of each image pixel and the mapped color.

[0133] It is worth noting that in the embodiment of the above-mentioned video low-light enhancement device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the protection scope of the embodiments of this application.

[0134] The embodiment of the present application also provides a video dark light enhancement device, which can integrate the video dark light enhancement apparatus provided by the embodiment of the present application. Figure 7 is a structural diagram of a video dark light enhancement device provided by the embodiment of the present application. Referring to Figure 7, the video dark light enhancement device includes: an input device 73, an output device 74, a memory 72 and one or more processors 71; the memory 72 is used to store one or more programs; when one or more programs are executed by one or more processors 71, the one or more processors 71 implement the video dark light enhancement method provided by the above embodiment. The above-mentioned video dark light enhancement apparatus, device and computer can be used to execute the video dark light enhancement method provided by any of the above embodiments, and have corresponding functions and beneficial effects.

[0135] The embodiments of the present application also provide a non-volatile storage medium that stores computer-executable instructions, which, when executed by a computer processor, are used to execute the video low-light enhancement method provided in the above-mentioned embodiments. Of course, the non-volatile storage medium that stores computer-executable instructions provided in the embodiments of the present application, whose computer-executable instructions are not limited to the video low-light enhancement method provided above, can also execute the related operations in the video low-light enhancement method provided in any embodiment of the present application. The video low-light enhancement apparatus, device, and storage medium provided in the above-mentioned embodiments can execute the video low-light enhancement method provided in any embodiment of the present application. For technical details not fully described in the above-mentioned embodiments, please refer to the video low-light enhancement method provided in any embodiment of the present application.

[0136] Based on the above embodiments, the embodiments of the present application also provide a computer program product. The essence of the technical solution of the present application or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product. The computer program product is stored in a storage medium and includes a number of instructions for enabling a computer device, a mobile terminal or the processor therein to execute all or part of the steps of the video dark light enhancement method provided in each embodiment of the present application.

Claims

1. A method for enhancing low light in a video, wherein: include: Determine the average brightness information, dark part information and bright part information of the image to be processed; Determine the dark light degree information of the image to be processed according to the average brightness information, the dark part information of the image and the bright part information of the image; Performing filtering on the dim light level information to obtain filtered dim light level information; Perform dark light enhancement processing on the image to be processed according to the filtered dark light level information.

2. The method for enhancing dark light in a video according to claim 1, wherein: The determining of average brightness information, image dark part information, and image bright part information of the image to be processed includes: Determine the picture brightness information of the image to be processed in the brightness channel; Determine the average brightness information of the image to be processed according to the picture brightness information; Determine the dark area and the bright area in the image to be processed according to the picture brightness information; The dark portion information of the dark portion image and the bright portion information of the bright portion image are determined.

3. The video dark light enhancement method according to claim 2, wherein: The determining of the dark portion information of the dark portion image and the bright portion information of the bright portion image includes: Determine a dark area average value of the dark area according to the picture brightness information of the dark area and the number of dark pixels in the dark area; Determining a dark area ratio of the dark area according to the number of dark area pixels in the dark area and the number of image pixels of the image to be processed; The bright area ratio of the bright area is determined according to the number of bright area pixels in the bright area and the number of image pixels in the image to be processed.

4. The video dark light enhancement method according to claim 1, wherein: The determining the dark light level information of the image to be processed according to the average brightness information, the dark part information of the image and the bright part information of the image includes: Determine the average brightness weight coefficient, the dark pixel average value authority coefficient, the dark pixel proportion authority coefficient, the average brightness dark pixel average value authority coefficient and the bright pixel proportion weight coefficient of the image to be processed according to the average brightness information, the image dark part information and the image bright part information; The dark light degree information of the image to be processed is determined according to the average brightness weight coefficient, the dark pixel average value authority coefficient, the dark pixel proportion authority coefficient, the average brightness dark pixel average value authority coefficient and the bright pixel proportion weight coefficient.

5. The method for enhancing dark light in a video according to claim 4, wherein: The method of determining the average brightness weight coefficient, the dark pixel average value authority coefficient, the dark pixel proportion authority coefficient, the average brightness dark pixel average value authority coefficient and the bright pixel proportion weight coefficient of the image to be processed according to the average brightness information, the image dark part information and the image bright part information includes: Performing smooth transition processing on the average brightness information in a set first brightness range according to a set smoothing function to obtain an average brightness weight coefficient of the image to be processed; Performing smooth transition processing on the average value of the dark area in the dark area of ​​the image to be processed within a set second brightness range according to the smoothing function to obtain an authority coefficient of the average value of the dark area pixels of the image to be processed; According to the smoothing function, a smooth transition process is performed on the authority coefficient of the average value of dark pixels within a set first weight coefficient range to obtain an authority coefficient of the proportion of dark pixels in the image to be processed; According to the smoothing function, a difference between the average brightness information and the average value of the dark area in the dark area of ​​the image to be processed is subjected to smooth transition processing within a set third brightness range to obtain an average brightness dark area pixel average value authority coefficient of the image to be processed; According to the smoothing function, a smooth transition processing is performed on the bright area ratio of the bright area in the image to be processed within a set second weight coefficient range to obtain a bright pixel ratio weight coefficient of the image to be processed.

6. The video dark light enhancement method according to claim 1, wherein: The filtering the dim light level information to obtain filtered dim light level information includes: The dark light level information is subjected to time domain filtering according to an adaptive smoothing factor to obtain filtered dark light level information, wherein the adaptive smoothing factor is determined according to a set minimum cutoff frequency, a constant cutoff frequency, and an image acquisition period.

7. The video dark light enhancement method according to claim 1, wherein: The performing dark light enhancement processing on the image to be processed according to the filtered dark light level information comprises: Determining a mapping color corresponding to the initial color of each image pixel in the image to be processed by displaying a lookup table filter; According to the filtered dark light level information, the initial color of each image pixel and the mapped color, a target color of each image pixel after dark light enhancement processing is determined.

8. A video dark light enhancement device, wherein: It includes an information determination module, a dark light analysis module, a filtering processing module and a dark light enhancement module, wherein: The information determination module is configured to determine average brightness information, image dark part information, and image bright part information of the image to be processed; The dark light analysis module is configured to determine the dark light degree information of the image to be processed according to the average brightness information, the dark part information of the image and the bright part information of the image; The filtering processing module is configured to filter the dim light level information to obtain filtered dim light level information; The dark light enhancement module is configured to perform dark light enhancement processing on the image to be processed according to the filtered dark light level information.

9. A video dark light enhancement device, wherein: include: memory and one or more processors; The memory is used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the video low-light enhancement method as described in any one of claims 1-7.

10. A non-volatile storage medium storing computer executable instructions, wherein: The computer executable instructions are used to execute the video low-light enhancement method as described in any one of claims 1-7 when executed by a computer processor.

11. A computer program product comprising a computer program, wherein: When the computer program is executed by a processor, the video low-light enhancement method according to any one of claims 1 to 7 is implemented.

Images