Image processing method and apparatus, device, and storage medium

Abstract

The present application relates to the technical field of image processing, and provides an image processing method and apparatus, a device, and a storage medium. The method comprises: when the illuminance of a target environment is less than an illuminance threshold and a dynamic range is greater than a dynamic range threshold, on the basis of brightness information of a live picture of the target environment, determining an environment type corresponding to the target environment, wherein the environment type is used for representing an illuminance level of the target environment and the proportion of a high-brightness region, and the brightness of the high-brightness region is greater than a preset brightness; and then on the basis of the environment type, performing, in a targeted manner, brightness adjustment on an initial image acquired for the target environment, so as to obtain a target image.

Description

Image processing methods, apparatus, devices and storage media

[0001] Cross-reference of related applications

[0002] This application claims priority to Chinese Patent Application No. 2024119280690, filed on December 25, 2024, entitled “Image Processing Method, Apparatus, Device and Storage Medium”, which is incorporated herein by reference in its entirety. Technical Field

[0003] This application relates to the field of image processing technology, and in particular to an image processing method, apparatus, device, and storage medium. Background Technology

[0004] In low-light environments, such as dusk, and in high dynamic range scenes, the camera may struggle to capture sufficient detail in low light conditions, resulting in poor image quality in dark areas of the captured image.

[0005] To address the issue of poor image quality in dark areas, the initial image can be stretched dynamically to obtain a processed image. This processed image is then smoothed to remove noise. The smoothed image is then magnified to highlight the noise. Finally, the magnified image is smoothed again to remove noise. The processed image is then fused with the initial image to obtain an image with better quality in dark areas.

[0006] However, the above method of processing noise before fusing images will lose details of the original image, making it unsuitable for environments with extremely low illumination, such as at night, and high dynamic range scenes, resulting in poor image quality in dark areas of the image. Summary of the Invention

[0007] This application provides an image processing method, apparatus, device, and storage medium to solve the problem of poor image quality in dark areas of images in scenes with extremely low ambient light and high dynamic range in the prior art, thereby improving the image quality in dark areas.

[0008] This application provides an image processing method, including:

[0009] When the illuminance of the target environment is less than the illuminance threshold and the dynamic range is greater than the dynamic range threshold, the environment type corresponding to the target environment is determined based on the brightness information of the real-time image of the target environment; wherein, the environment type is used to characterize the illuminance level and the proportion of high brightness areas of the target environment, and the brightness of the high brightness areas is greater than the preset brightness.

[0010] The initial image acquired for the target environment is processed based on the environment type to obtain the target image.

[0011] According to an image processing method provided in this application, determining the environment type corresponding to the target environment based on the brightness information of the real-time image of the target environment includes:

[0012] The live feed is divided into multiple screen blocks;

[0013] For each of the aforementioned image blocks, determine the brightness difference between the RAW brightness and the YUV brightness of that image block;

[0014] Based on the RAW brightness, YUV brightness, and corresponding brightness difference of each image block, the environment type corresponding to the target environment is determined.

[0015] According to an image processing method provided in this application, determining the environment type corresponding to the target environment based on the RAW brightness, YUV brightness, and corresponding brightness difference of each image block includes:

[0016] Based on the RAW brightness of each of the multiple frame blocks, the first RAW brightness corresponding to the multiple frame blocks, the second RAW brightness corresponding to at least one frame block whose RAW brightness is greater than or equal to the RAW brightness threshold, and the third RAW brightness corresponding to at least one frame block whose RAW brightness is less than the RAW brightness threshold are determined.

[0017] Based on the brightness difference corresponding to each of the picture blocks, a first brightness difference corresponding to multiple brightness differences, a second brightness difference corresponding to at least one picture block whose brightness difference is less than the brightness difference threshold, and a third brightness difference corresponding to at least one picture block whose brightness difference is greater than or equal to the brightness difference threshold are determined.

[0018] The first YUV brightness difference is determined based on the YUV brightness of each frame block whose brightness difference is greater than or equal to the brightness difference threshold.

[0019] Based on the first RAW brightness, the second RAW brightness, the third RAW brightness, the first brightness difference, the second brightness difference, the third brightness difference, and the first YUV brightness difference, the environment type corresponding to the target environment is determined.

[0020] According to an image processing method provided in this application, determining the environment type corresponding to the target environment based on the first RAW brightness, the second RAW brightness, the third RAW brightness, the first brightness difference, the second brightness difference, the third brightness difference, and the first YUV brightness difference includes:

[0021] The second YUV brightness difference is determined based on the YUV brightness of each frame block whose brightness difference is less than the brightness difference threshold.

[0022] When the absolute value of the difference between the first brightness difference and the second brightness difference is less than the absolute value of the difference between the first brightness difference and the third brightness difference, the absolute value of the difference between the first RAW brightness and the second RAW brightness is less than the absolute value of the difference between the first RAW brightness and the third RAW brightness, and the absolute value of the difference between the first YUV brightness difference and the second YUV brightness difference is within a preset YUV brightness difference range, the environment type is determined to be a first environment type; wherein, the first environment type is the illuminance level of the target environment is the first illuminance level, and the proportion of the high brightness area is greater than a preset proportion.

[0023] If the absolute value of the difference between the first brightness difference and the second brightness difference is greater than or equal to the absolute value of the difference between the first brightness difference and the third brightness difference, the absolute value of the difference between the first RAW brightness and the second RAW brightness is greater than or equal to the absolute value of the difference between the first RAW brightness and the third RAW brightness, and the first YUV brightness difference is greater than the YUV brightness difference threshold, then the environment type is determined to be the second environment type; wherein, the second environment type is the illuminance level of the target environment is the second illuminance level, and the proportion of the high brightness area is less than or equal to the preset proportion, and the second illuminance level is lower than the first illuminance level.

[0024] According to an image processing method provided in this application, the step of performing brightness processing on an initial image acquired for the target environment based on the environment type includes:

[0025] When the environment type is the first environment type and the gain of the initial image is within a preset gain range, the brightness enhancement parameter of the dark area in the initial image is reduced until the YUV brightness of the reduced image is less than or equal to the first YUV brightness threshold.

[0026] Wherein, the brightness difference between the RAW brightness and YUV brightness of the image block included in the dark area is greater than or equal to the brightness difference threshold, and the brightness enhancement parameter is used to characterize the enhancement parameter of the initial image in the brightness information dimension.

[0027] According to an image processing method provided in this application, reducing the brightness enhancement parameter of the dark areas in the initial image includes:

[0028] Based on the YUV brightness of the image blocks included in the dark area, the average YUV brightness corresponding to the dark area is determined, and based on the RAW brightness of the image blocks included in the dark area, the average RAW brightness corresponding to the dark area is determined.

[0029] The brightness reduction step size is determined based on the difference between the average YUV brightness and the average RAW brightness.

[0030] Based on the brightness reduction step size, the brightness enhancement parameter of the dark areas in the initial image is reduced.

[0031] According to an image processing method provided in this application, the step of performing brightness processing on an initial image acquired for the target environment based on the environment type includes:

[0032] When the environment type is the second environment type and the gain of the initial image is greater than or equal to the gain threshold, the brightness enhancement parameter of the dark area in the initial image is reduced until the YUV brightness of the reduced image is less than or equal to the second YUV brightness threshold.

[0033] Wherein, the second YUV brightness threshold is less than the first YUV brightness threshold.

[0034] This application also provides an image processing apparatus, comprising:

[0035] The determining unit is configured to determine the environment type corresponding to the target environment based on the brightness information of the real-time image of the target environment when the illuminance of the target environment is less than the illuminance threshold and the dynamic range is greater than the dynamic range threshold; wherein, the environment type is used to characterize the illuminance level and the proportion of high brightness areas of the target environment, and the brightness of the high brightness areas is greater than a preset brightness.

[0036] The processing unit is configured to perform brightness processing on the initial image acquired for the target environment based on the environment type to obtain the target image.

[0037] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the image processing method as described above.

[0038] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image processing method as described above.

[0039] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the image processing method as described above.

[0040] The image processing method, apparatus, device, and storage medium provided in this application relate to the field of image processing technology. When the illuminance of the target environment is less than an illuminance threshold and the dynamic range is greater than a dynamic range threshold, the environment type corresponding to the target environment is determined based on the brightness information of the real-time image of the target environment. The environment type characterizes the illuminance level and the proportion of high-brightness areas in the target environment, with the brightness of the high-brightness areas exceeding a preset brightness. Then, based on the environment type, brightness processing is performed on the initial image acquired for the target environment in a targeted manner. Attached Figure Description

[0041] Figure 1 is a schematic flowchart of an image processing method provided in an embodiment of this application.

[0042] Figure 2 is a schematic flowchart of a method for determining the environment type of a target environment based on the brightness information of a real-time image of the target environment, according to an embodiment of this application.

[0043] Figure 3 is a schematic diagram of the structure of an image processing device provided in an embodiment of this application.

[0044] Figure 4 is a schematic diagram of the physical structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0045] The technical solutions of this application will now be described with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0046] The technical solutions provided in this application can be adapted to scenarios such as target monitoring and visual navigation in environments with extremely low illumination and high dynamic range. In these scenarios, in order to better analyze and identify information in images, it is usually necessary to perform brightness processing on dark areas in the initial acquired image to improve the image quality of dark areas in the acquired image.

[0047] Given that current methods of processing noise before fusing images lose details of the initial image, making it unsuitable for scenes with extremely low illumination and high dynamic range, this application provides an image processing method to improve the image quality of dark areas in such scenes. In cases of extremely low illumination and high dynamic range, the method first determines the environment type of the target environment based on the brightness information of the real-time image. The environment type characterizes the illumination level and the proportion of high-brightness areas in the target environment, with the brightness of these high-brightness areas exceeding a preset brightness. Then, based on the environment type, targeted brightness processing is applied to the initial image acquired for the target environment. This effectively solves the problem of poor image quality in dark areas in existing technologies for scenes with extremely low illumination and high dynamic range, thereby improving the image quality of dark areas.

[0048] It is understood that the subject executing this method can be an electronic device such as a camera, a specially designed image processing device, a computer or server, or an image processing device installed in the electronic device. The image processing device can be implemented by software, hardware or a combination of both, and can be configured according to actual needs.

[0049] It should be noted that the image processing method provided in this application embodiment can also be extended to scenarios of contrast processing and motion blur processing. The brightness-related parameters can be replaced with contrast parameters or motion blur parameters as needed. Here, this application embodiment will not elaborate further.

[0050] The image processing method provided in this application will now be described in detail through the following embodiments. It is understood that these embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0051] Figure 1 is a flowchart illustrating an image processing method provided in an embodiment of this application. For example, as shown in Figure 1, the image processing method may include:

[0052] S101. When the illuminance of the target environment is less than the illuminance threshold and the dynamic range is greater than the dynamic range threshold, the environment type corresponding to the target environment is determined based on the brightness information of the real-time image of the target environment. The environment type is used to characterize the illuminance level and the proportion of high-brightness areas of the target environment. The brightness of the high-brightness areas is greater than the preset brightness.

[0053] The values ​​of the illuminance threshold and the dynamic range threshold can be set according to actual needs. For example, the illuminance threshold can be 50 lux or 51 lux, etc., depending on actual needs.

[0054] For example, with a maximum brightness of 255, the preset brightness value can be 100, 101, etc., which can be set according to actual needs.

[0055] For example, the brightness information of the live footage can include RAW brightness and YUV brightness, which can be set according to actual needs. RAW brightness is usually the raw data obtained directly from the sensor of the camera device, without any processing; while YUV brightness is the brightness information after color space conversion.

[0056] It is understood that in the embodiments of this application, the illuminance level and the proportion of high-brightness areas of the target environment represented by different environment types are different.

[0057] For example, in the embodiments of this application, when determining the environment type corresponding to the target environment based on the brightness information of the live screen of the target environment, the live screen of the target environment can be treated as a whole, and the environment type corresponding to the target environment can be determined based on its brightness information; or the live screen of the target environment can be divided into multiple screen blocks, and the environment type corresponding to the target environment can be determined based on the brightness information of each screen block, etc., which can be set according to actual needs.

[0058] After determining the environment type corresponding to the target environment based on the brightness information of the live image of the target environment, the initial image of the target environment can be subjected to brightness processing in a targeted manner based on the environment type, i.e., the following S102 is executed.

[0059] S102. Based on the environment type, perform brightness processing on the initial image acquired for the target environment to obtain the target image.

[0060] As can be seen, in this embodiment, when the illuminance of the target environment is less than the illuminance threshold and the dynamic range is greater than the dynamic range threshold, the environment type corresponding to the target environment is determined based on the brightness information of the real-time image of the target environment. The environment type is used to characterize the illuminance level and the proportion of high-brightness areas of the target environment, and the brightness of the high-brightness areas is greater than the preset brightness. Based on the environment type, the initial image acquired for the target environment is subjected to targeted brightness processing. This can effectively solve the problem of poor image quality in dark areas of the image in the prior art in scenes with extremely low ambient illuminance and high dynamic range, thereby improving the image quality of dark areas.

[0061] Based on the embodiment shown in Figure 1 above, for example, in S101 above, the implementation of determining the environment type corresponding to the target environment based on the brightness information of the real-time image of the target environment can be referred to the embodiment shown in Figure 2 below.

[0062] Figure 2 is a flowchart illustrating a method for determining the environment type of a target environment based on the brightness information of a real-time image of the target environment, according to an embodiment of this application. For example, as shown in Figure 2, the method may include:

[0063] S201. Divide the live video into multiple video blocks.

[0064] For example, in this embodiment of the application, the live video can be divided into m×n video blocks, where each video block has its own RAW luminance and YUV luminance. For example, the RAW luminance of the m×n video blocks can be denoted as Rm×n. m1 R m2 , ..., R mn The YUV brightness of m×n image blocks can be denoted as Y m1 Y m2 , ..., Ymn Typically, the luminance value of RAW is less than that of YUV.

[0065] Generally, for any given frame block, such as the m×n-th frame block, its RAW brightness can be understood as the average RAW brightness of all pixels in the m×n-th frame block; similarly, its YUV brightness can be understood as the average YUV brightness of all pixels in the m×n-th frame block.

[0066] S202. For each frame block, determine the brightness difference between the RAW brightness and the YUV brightness of the frame block.

[0067] For example, when calculating the brightness difference between the RAW and YUV brightness of each frame block, taking any frame block, such as the m×n-th frame block, as an example, the brightness difference between the RAW and YUV brightness of the m×n-th frame block can be seen in Formula 2 below. ΔL mn =Y mn -R mn Formula 2

[0068] Where, ΔL mn This represents the brightness difference between the RAW brightness and the YUV brightness of the m×n-th frame block, where Y... mn R represents the YUV brightness of the m×n-th frame block. mn This represents the RAW brightness of the m×n-th frame block.

[0069] S203. Based on the RAW brightness, YUV brightness and corresponding brightness difference of each frame block, determine the environment type corresponding to the target environment.

[0070] For example, in the embodiments of this application, when determining the environment type corresponding to the target environment based on the RAW brightness, YUV brightness, and corresponding brightness difference of each image block, it may include:

[0071] Based on the RAW brightness of each frame block, the first RAW brightness corresponding to multiple frame blocks, the second RAW brightness corresponding to at least one frame block whose RAW brightness is greater than or equal to a RAW brightness threshold, and the third RAW brightness corresponding to at least one frame block whose RAW brightness is less than a RAW brightness threshold are determined. Based on the brightness difference corresponding to each frame block, the first brightness difference corresponding to multiple brightness differences, the second brightness difference corresponding to at least one frame block whose brightness difference is less than a brightness difference threshold, and the third brightness difference corresponding to at least one frame block whose brightness difference is greater than or equal to a brightness difference threshold are determined. Based on the YUV brightness corresponding to each frame block whose brightness difference is greater than or equal to a brightness difference threshold, the first YUV brightness difference is determined. Finally, based on the first RAW brightness, the second RAW brightness, the third RAW brightness, the first brightness difference, the second brightness difference, the third brightness difference, and the first YUV brightness difference, the environment type corresponding to the target environment is determined.

[0072] For example, in the embodiments of this application, the RAW luminance threshold can be denoted as Rt, and the luminance difference threshold can be denoted as ΔL. th The values ​​of the RAW brightness threshold and brightness difference threshold can be set according to actual needs. For example, the value of the RAW brightness threshold Rt can be set according to the sensor characteristics of the camera device, and the value of the brightness difference threshold ΔL... th The setting can be based on the RAW brightness of the live stream footage. Generally, the brighter the live stream footage, the higher the RAW brightness threshold ΔL should be. th The value can be set to be larger.

[0073] For example, when determining the first RAW brightness corresponding to multiple frame blocks based on the RAW brightness of each frame block, one can calculate the corresponding average RAW brightness based on the RAW brightness of each frame block and determine this average RAW brightness as the first RAW brightness; alternatively, one can select a portion of frame blocks based on the RAW brightness of each frame block, calculate the average RAW brightness corresponding to that portion of frame blocks, and determine this average RAW brightness as the first RAW brightness, etc., depending on actual needs. For example, the first RAW brightness corresponding to multiple frame blocks can be denoted as R.

[0074] When determining the second RAW brightness corresponding to at least one frame block whose RAW brightness is greater than or equal to the RAW brightness threshold Rt among multiple frame blocks based on the RAW brightness of each frame block, the frame blocks whose RAW brightness is greater than or equal to the RAW brightness threshold can be determined from the multiple frame blocks based on the RAW brightness of each frame block; and the corresponding second RAW brightness can be determined based on the RAW brightness of the frame blocks whose RAW brightness is greater than or equal to the RAW brightness threshold.

[0075] For example, when determining the corresponding second RAW brightness based on the RAW brightness of frame blocks whose RAW brightness is greater than or equal to the RAW brightness threshold, the average RAW brightness can be calculated based on the RAW brightness of each frame block whose RAW brightness is greater than or equal to the RAW brightness threshold, and this average RAW brightness can be determined as the second RAW brightness; alternatively, the average RAW brightness can be calculated based on the RAW brightness of a subset of frame blocks whose RAW brightness is greater than or equal to the RAW brightness threshold, and this average RAW brightness can be determined as the second RAW brightness, etc., depending on actual needs. For example, the second RAW brightness can be denoted as RB.

[0076] When determining the third RAW brightness corresponding to at least one frame block whose RAW brightness is less than the RAW brightness threshold based on the RAW brightness of each frame block, the corresponding frame block whose RAW brightness is less than the RAW brightness threshold can be determined from multiple frame blocks based on the RAW brightness of each frame block; and the corresponding third RAW brightness can be determined based on the RAW brightness of the frame block whose RAW brightness is less than the RAW brightness threshold.

[0077] For example, when determining the corresponding third RAW brightness based on the RAW brightness of frame blocks whose RAW brightness is less than the RAW brightness threshold, the average RAW brightness can be calculated based on the RAW brightness of each frame block whose RAW brightness is less than the RAW brightness threshold, and this average RAW brightness can be determined as the third RAW brightness; alternatively, the average RAW brightness can be calculated based on the RAW brightness of a subset of frame blocks whose RAW brightness is less than the RAW brightness threshold, and this average RAW brightness can be determined as the third RAW brightness, etc., depending on actual needs. For example, the third RAW brightness can be denoted as RD.

[0078] For example, when determining the first brightness difference corresponding to multiple brightness differences based on the brightness difference corresponding to each frame block, one can calculate the average brightness difference of all brightness differences based on the brightness difference corresponding to each frame block, as shown in Formula 3 below, and determine this average brightness difference as the first brightness difference; alternatively, one can calculate the average brightness difference of a portion of all brightness differences based on the brightness difference corresponding to each frame block, and determine this average brightness difference as the first brightness difference, etc., depending on actual needs. For example, the first brightness difference can be denoted as ΔL. Amn .

[0079] Where i represents the i-th frame block in an m×n frame block, ΔL i This represents the brightness difference between the RAW brightness and the YUV brightness of the i-th frame block.

[0080] When determining the second brightness difference corresponding to at least one frame block whose brightness difference is less than a brightness difference threshold and the third brightness difference corresponding to at least one frame block whose brightness difference is greater than or equal to the brightness difference threshold based on the brightness difference corresponding to each frame block, it is possible to first determine, from multiple frame blocks, at least one frame block whose brightness difference is less than the brightness difference threshold and at least one frame block whose brightness difference is greater than or equal to the brightness difference threshold. For example, frame blocks whose brightness difference is less than the brightness difference threshold can be marked as 0, and frame blocks whose brightness difference is greater than or equal to the brightness difference threshold can be marked as 1. Among them, the frame block marked as 0 can be understood as the frame block in the bright area of ​​the live image, and the frame block marked as 1 can be understood as the frame block in the dark area of ​​the live image.

[0081] Separately count the image blocks marked as 0 and the image blocks marked as 1. Assuming there are C image blocks marked as 0, then there are m × nC image blocks marked as 1. When determining the second brightness difference corresponding to at least one image block whose brightness difference is less than the brightness difference threshold, the average brightness difference corresponding to the C image blocks marked as 0 can be calculated based on the brightness differences corresponding to each of the C image blocks marked as 0, as shown in Formula 4 below. The calculated average brightness difference is then determined as the second brightness difference.

[0082] Where, ΔL A0 This represents the second brightness difference, and C represents the number of image blocks marked as 0, which is the number of image blocks whose brightness difference is less than the brightness difference threshold.

[0083] When determining the third brightness difference corresponding to at least one screen block whose brightness difference is greater than or equal to the brightness difference threshold, the average brightness difference corresponding to the m×nC screen blocks can be calculated based on the brightness differences corresponding to each of the m×nC screen blocks marked as 1, as shown in Formula 5 below, and the calculated average brightness difference is determined as the third brightness difference.

[0084] Where, ΔL A1 The third brightness difference is represented by m×nC, which represents the number of image blocks marked as 1, i.e., the number of image blocks whose brightness difference is greater than or equal to the brightness difference threshold.

[0085] For example, when determining the first YUV brightness difference based on the YUV brightness of each frame block whose brightness difference is greater than or equal to the brightness difference threshold, the average YUV brightness of the m×nC frame blocks can be calculated based on the YUV brightness of each of the m×nC frame blocks marked as 1, as shown in Formula 6 below, and the calculated average YUV brightness is determined as the first YUV brightness difference.

[0086] Among them, Y A1This represents the first YUV brightness difference, Y i This represents the YUV brightness of the i-th frame block out of m×nC frame blocks.

[0087] Based on the above description, after determining the first RAW brightness R, the second RAW brightness RB, the third RAW brightness RD, and the first brightness difference ΔL respectively... Amn Second brightness difference ΔL A0 The third brightness difference ΔL A1 The difference in brightness between the first YUV and Y A1 Then, the environment type corresponding to the target environment can be further determined.

[0088] For example, based on the first RAW luminance R, the second RAW luminance RB, the third RAW luminance RD, and the first luminance difference ΔL Amn Second brightness difference ΔL A0 The third brightness difference ΔL A1 The difference in brightness between the first YUV and Y A1 When determining the environment type corresponding to the target environment, the first brightness difference ΔL can be used. Amn Difference from the second brightness ΔL A0 The absolute value of the difference is less than the first brightness difference ΔL Amn Difference in brightness ΔL from the third brightness A1 The absolute value of the difference, the absolute value of the difference between the first RAW brightness R and the second RAW brightness RB is less than the absolute value of the difference between the first RAW brightness R and the third RAW brightness RD, and the first YUV brightness difference Y A1 The difference in brightness between the second YUV and Y A0 The absolute value of the difference is within the preset YUV brightness difference range, i.e., |ΔL Amn -ΔL A0 |<|ΔL Amn -ΔL A1 |、|R-RB|<|R-RD|、and|Y A1 -Y A0 If the exposure is within the preset YUV brightness difference range, it indicates that the target scene's exposure is primarily focused on brighter areas, with some darker areas present. Furthermore, the difference in YUV brightness between the darker and brighter areas is relatively small. Therefore, the target environment can be classified as the first environment type. The first environment type corresponds to the target environment's illuminance level being the first illuminance level, and the proportion of high-brightness areas exceeding the preset proportion. The preset proportion can be set according to actual needs.

[0089] Among them, the second YUV brightness difference Y A0 It is determined based on the YUV brightness of each frame block whose brightness difference is less than the brightness difference threshold. For example, see Formula 7 below.

[0090] Among them, Y A0 This represents the second YUV brightness difference, Y i This represents the YUV brightness of the i-th frame block out of C frame blocks, i.e., the YUV brightness of the frame block whose brightness difference is less than the brightness difference threshold.

[0091] First brightness difference ΔL Amn Difference from the second brightness ΔL A0 The absolute value of the difference is greater than or equal to the first brightness difference ΔL Amn Difference in brightness ΔL from the third brightness A1 The absolute value of the difference between the first RAW brightness R and the second RAW brightness RB is greater than or equal to the absolute value of the difference between the first RAW brightness R and the third RAW brightness RD, and the first YUV brightness Y A1 The difference is greater than the YUV brightness difference threshold, i.e., |ΔL Amn -ΔL A0 |≥|ΔL Amn -ΔL A1 |、|R-RB|≥|R-RD|, and Y A1 If the difference between the YUV brightness and the target environment is greater than the threshold, it indicates that the exposure of the target scene is mainly dark areas, and there are large areas of dark areas. The YUV brightness of the dark areas is also relatively high. Therefore, the environment type corresponding to the target environment can be determined as the second environment type. The second environment type is the target environment with an illuminance level of the second illuminance level, and the proportion of high brightness areas is less than or equal to the preset proportion. The second illuminance level is lower than the first illuminance level.

[0092] For example, in this embodiment of the application, in order to more accurately determine the second environment type, the number of screen blocks with a brightness difference less than the brightness difference threshold in the live screen can be used for judgment. If the number of screen blocks with a brightness difference less than the brightness difference threshold is relatively small, for example, less than the number of screen blocks, then the environment type corresponding to the target environment can be determined as the second environment type.

[0093] Based on the above description, after determining the environment type corresponding to the target environment based on the RAW brightness, YUV brightness and corresponding brightness difference of each image block, the initial image acquired for the target environment can be processed in a targeted manner based on the environment type. This can effectively solve the problem of poor image quality in dark areas of the image in the existing technology in scenes with extremely low ambient light and high dynamic range, thereby improving the image quality in dark areas.

[0094] Based on any of the above embodiments, for example, in S102 above, when performing brightness processing on the initial image acquired for the target environment based on the environment type, at least two of the following possible scenarios may be included.

[0095] In one possible scenario, the environment type is the first environment type, and the gain of the initial image is within a preset gain range. The value of the preset gain range can be set according to actual needs.

[0096] When the environment type is the first environment type and the gain of the initial image is within the preset gain range, the brightness enhancement parameter of the dark areas in the initial image can be reduced until the YUV brightness of the reduced image is less than or equal to the first YUV brightness threshold. The value of the first YUV brightness threshold can be set according to actual needs.

[0097] Among them, the brightness difference between the RAW brightness and YUV brightness of the image blocks included in the dark area is greater than or equal to the brightness difference threshold. The brightness enhancement parameter is used to characterize the enhancement parameters of the initial image in the brightness information dimension.

[0098] For example, in the embodiments of this application, when reducing the brightness enhancement parameter of the dark area in the initial image, the brightness enhancement parameter of the dark area in the initial image can be reduced by a different brightness reduction value each time it is reduced, until the YUV brightness of the reduced image is less than or equal to the first YUV brightness threshold; or a brightness reduction step size can be predetermined, and the brightness enhancement parameter of the dark area in the initial image can be reduced based on the brightness reduction step size each time it is reduced, until the YUV brightness of the reduced image is less than or equal to the first YUV brightness threshold.

[0099] For example, in this embodiment of the application, when reducing the brightness enhancement parameter of the dark area in the initial image based on the brightness reduction step size, the average YUV brightness corresponding to the dark area can be determined based on the YUV brightness of the image blocks included in the dark area, and the average RAW brightness corresponding to the dark area can be determined based on the RAW brightness of the image blocks included in the dark area; then, the brightness reduction step size is determined based on the difference between the average YUV brightness and the average RAW brightness corresponding to the dark area. For example, see Formula 8 below. In this way, after determining the brightness reduction step size, the brightness enhancement parameter of the dark area in the initial image can be reduced based on the determined brightness reduction step size, thereby improving the image quality of the dark area. S=(YR)×γ Formula 8

[0100] Where S represents the brightness reduction step size, Y represents the average YUV brightness corresponding to the dark area, R represents the average RAW brightness corresponding to the dark area, and γ represents the brightness adjustment coefficient, the value of which can be set according to actual needs.

[0101] In another possible scenario, the environment type is either the first or the second environment type, and the gain of the initial image is greater than or equal to a gain threshold. The value of the gain threshold can be set according to actual needs. Typically, this gain threshold is greater than the upper limit of the aforementioned preset gain range.

[0102] If the environment type is the second environment type and the gain of the initial image is greater than or equal to the gain threshold, it indicates that the image quality is relatively poor, especially the image quality of the dark areas. In this case, the dark areas are often no longer important and can be ignored. Therefore, the brightness enhancement parameter of the dark areas in the initial image can be reduced until the YUV brightness of the reduced image is less than or equal to the second YUV brightness threshold.

[0103] The second YUV brightness threshold is less than the first YUV brightness threshold.

[0104] For example, in the embodiments of this application, when reducing the brightness enhancement parameter of the dark area in the initial image, the implementation is similar to the implementation of reducing the brightness enhancement parameter of the dark area in the initial image in the possible scenarios described above. Please refer to the relevant description of reducing the brightness enhancement parameter of the dark area in the initial image above. Here, the embodiments of this application will not repeat the description.

[0105] Understandably, in the two possible scenarios mentioned above, in order to improve the image quality of dark areas, in addition to reducing the brightness enhancement parameters of dark areas in the initial image, one can also reduce the sharpening intensity and noise reduction intensity of dark areas, etc., which can be set according to actual needs.

[0106] The image processing apparatus provided in this application is described below. The image processing apparatus described below can be referred to in correspondence with the image processing method described above.

[0107] Figure 3 is a schematic diagram of an image processing apparatus provided in an embodiment of this application. For example, as shown in Figure 3, the image processing apparatus 30 may include:

[0108] The determining unit 301 is configured to determine the environment type corresponding to the target environment based on the brightness information of the real-time image of the target environment when the illuminance of the target environment is less than the illuminance threshold and the dynamic range is greater than the dynamic range threshold; wherein, the environment type is used to characterize the illuminance level and the proportion of high brightness area of ​​the target environment, and the brightness of the high brightness area is greater than the preset brightness.

[0109] The processing unit 302 is configured to perform brightness processing on the initial image acquired for the target environment based on the environment type to obtain the target image.

[0110] For example, in this embodiment of the application, the determining unit 301 is configured to determine the environment type corresponding to the target environment based on the brightness information of the real-time image of the target environment, including:

[0111] The live feed is divided into multiple screen blocks;

[0112] For each of the aforementioned image blocks, determine the brightness difference between the RAW brightness and the YUV brightness of that image block;

[0113] Based on the RAW brightness, YUV brightness, and corresponding brightness difference of each image block, the environment type corresponding to the target environment is determined.

[0114] For example, in this embodiment of the application, the determining unit 301 is configured to determine the environment type corresponding to the target environment based on the RAW brightness, YUV brightness and the corresponding brightness difference of each of the image blocks, including:

[0115] Based on the RAW brightness of each of the multiple frame blocks, the first RAW brightness corresponding to the multiple frame blocks, the second RAW brightness corresponding to at least one frame block whose RAW brightness is greater than or equal to the RAW brightness threshold, and the third RAW brightness corresponding to at least one frame block whose RAW brightness is less than the RAW brightness threshold are determined.

[0116] Based on the brightness difference corresponding to each of the picture blocks, a first brightness difference corresponding to multiple brightness differences, a second brightness difference corresponding to at least one picture block whose brightness difference is less than the brightness difference threshold, and a third brightness difference corresponding to at least one picture block whose brightness difference is greater than or equal to the brightness difference threshold are determined.

[0117] The first YUV brightness difference is determined based on the YUV brightness of each frame block whose brightness difference is greater than or equal to the brightness difference threshold.

[0118] Based on the first RAW brightness, the second RAW brightness, the third RAW brightness, the first brightness difference, the second brightness difference, the third brightness difference, and the first YUV brightness difference, the environment type corresponding to the target environment is determined.

[0119] For example, in this embodiment of the application, the determining unit 301 is configured to determine the environment type corresponding to the target environment based on the first RAW brightness, the second RAW brightness, the third RAW brightness, the first brightness difference, the second brightness difference, the third brightness difference, and the first YUV brightness difference, including:

[0120] The second YUV brightness difference is determined based on the YUV brightness of each frame block whose brightness difference is less than the brightness difference threshold.

[0121] When the absolute value of the difference between the first brightness difference and the second brightness difference is less than the absolute value of the difference between the first brightness difference and the third brightness difference, the absolute value of the difference between the first RAW brightness and the second RAW brightness is less than the absolute value of the difference between the first RAW brightness and the third RAW brightness, and the absolute value of the difference between the first YUV brightness difference and the second YUV brightness difference is within a preset YUV brightness difference range, the environment type is determined to be a first environment type; wherein, the first environment type is the illuminance level of the target environment is the first illuminance level, and the proportion of high brightness areas is greater than a preset proportion.

[0122] If the absolute value of the difference between the first brightness difference and the second brightness difference is greater than or equal to the absolute value of the difference between the first brightness difference and the third brightness difference, the absolute value of the difference between the first RAW brightness and the second RAW brightness is greater than or equal to the absolute value of the difference between the first RAW brightness and the third RAW brightness, and the first YUV brightness difference is greater than the YUV brightness difference threshold, then the environment type is determined to be the second environment type; wherein, the second environment type is the illuminance level of the target environment is the second illuminance level, and the proportion of high brightness areas is less than or equal to the preset proportion, and the second illuminance level is lower than the first illuminance level.

[0123] For example, in an embodiment of this application, the processing unit 302 is configured to perform brightness processing on an initial image acquired for the target environment based on the environment type, including:

[0124] When the environment type is the first environment type and the gain of the initial image is within a preset gain range, the brightness enhancement parameter of the dark area in the initial image is reduced until the YUV brightness of the reduced image is less than or equal to the first YUV brightness threshold.

[0125] Wherein, the brightness difference between the RAW brightness and YUV brightness of the image block included in the dark area is greater than or equal to the brightness difference threshold, and the brightness enhancement parameter is used to characterize the enhancement parameter of the initial image in the brightness information dimension.

[0126] For example, in an embodiment of this application, the processing unit 302 is configured to reduce the brightness enhancement parameter of the dark areas in the initial image, including:

[0127] Based on the YUV brightness of the image blocks included in the dark area, the average YUV brightness corresponding to the dark area is determined, and based on the RAW brightness of the image blocks included in the dark area, the average RAW brightness corresponding to the dark area is determined.

[0128] The brightness reduction step size is determined based on the difference between the average YUV brightness and the average RAW brightness.

[0129] Based on the brightness reduction step size, the brightness enhancement parameter of the dark areas in the initial image is reduced.

[0130] For example, in an embodiment of this application, the processing unit 302 is configured to perform brightness processing on an initial image acquired for the target environment based on the environment type, including:

[0131] When the environment type is the second environment type and the gain of the initial image is greater than or equal to the gain threshold, the brightness enhancement parameter of the dark area in the initial image is reduced until the YUV brightness of the reduced image is less than or equal to the second YUV brightness threshold.

[0132] Wherein, the second YUV brightness threshold is less than the first YUV brightness threshold.

[0133] The image processing apparatus 30 provided in this application embodiment can execute the technical solution of the image processing method in any of the above embodiments. Its implementation principle and beneficial effects are similar to those of the image processing method. Please refer to the implementation principle and beneficial effects of the image processing method. It will not be repeated here.

[0134] Figure 4 is a schematic diagram of the physical structure of an electronic device provided in an embodiment of this application. As shown in Figure 4, the electronic device may include: a processor 410, a communication interface 420, a memory 430, and a communication bus 440. The processor 410, communication interface 420, and memory 430 communicate with each other via the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute an image processing method. This method includes: determining the environment type corresponding to the target environment based on the brightness information of the real-time image of the target environment when the illuminance of the target environment is less than an illuminance threshold and the dynamic range is greater than a dynamic range threshold; wherein the environment type is used to characterize the illuminance level and the proportion of high-brightness areas of the target environment, and the brightness of the high-brightness areas is greater than a preset brightness; and performing brightness processing on an initial image acquired for the target environment based on the environment type to obtain a target image.

[0135] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of this application. The aforementioned storage medium includes: USB flash drive, portable hard drive, read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk, etc., any medium capable of storing program code.

[0136] On the other hand, this application also provides a computer program product, which includes a computer program that can be stored on a computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the image processing method provided by each of the above methods. The method includes: when the illuminance of the target environment is less than an illuminance threshold and the dynamic range is greater than a dynamic range threshold, determining the environment type corresponding to the target environment based on the brightness information of the real-time image of the target environment; wherein the environment type is used to characterize the illuminance level and the proportion of high-brightness areas of the target environment, and the brightness of the high-brightness areas is greater than a preset brightness; and performing brightness processing on an initial image acquired for the target environment based on the environment type to obtain a target image.

[0137] In another aspect, this application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, is implemented to perform the image processing method provided by each of the above methods. The method includes: determining an environment type corresponding to the target environment based on brightness information of a real-time image of the target environment when the illuminance of the target environment is less than an illuminance threshold and the dynamic range is greater than a dynamic range threshold; wherein the environment type characterizes the illuminance level and the proportion of high-brightness areas of the target environment, and the brightness of the high-brightness areas is greater than a preset brightness; and performing brightness processing on an initial image acquired for the target environment based on the environment type to obtain a target image.

[0138] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0139] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0140] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in each of the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of each embodiment of this application.

Claims

1. An image processing method, comprising: When the illuminance of the target environment is less than the illuminance threshold and the dynamic range is greater than the dynamic range threshold, the environment type corresponding to the target environment is determined based on the brightness information of the real-time image of the target environment; wherein, the environment type is used to characterize the illuminance level and the proportion of high brightness areas of the target environment, and the brightness of the high brightness areas is greater than the preset brightness. The initial image acquired for the target environment is processed based on the environment type to obtain the target image.

2. The image processing method according to claim 1, wherein, The determination of the environment type corresponding to the target environment based on the brightness information of the real-time image of the target environment includes: The live feed is divided into multiple screen blocks; For each of the aforementioned image blocks, determine the brightness difference between the RAW brightness and the YUV brightness of that image block; Based on the RAW brightness, YUV brightness, and corresponding brightness difference of each image block, the environment type corresponding to the target environment is determined.

3. The image processing method according to claim 2, wherein, The determination of the environment type corresponding to the target environment based on the RAW brightness, YUV brightness, and corresponding brightness difference of each image block includes: Based on the RAW brightness of each of the multiple frame blocks, the first RAW brightness corresponding to the multiple frame blocks, the second RAW brightness corresponding to at least one frame block whose RAW brightness is greater than or equal to the RAW brightness threshold, and the third RAW brightness corresponding to at least one frame block whose RAW brightness is less than the RAW brightness threshold are determined. Based on the brightness difference corresponding to each of the picture blocks, a first brightness difference corresponding to multiple brightness differences, a second brightness difference corresponding to at least one picture block whose brightness difference is less than the brightness difference threshold, and a third brightness difference corresponding to at least one picture block whose brightness difference is greater than or equal to the brightness difference threshold are determined. The first YUV brightness difference is determined based on the YUV brightness of each frame block whose brightness difference is greater than or equal to the brightness difference threshold. Based on the first RAW brightness, the second RAW brightness, the third RAW brightness, the first brightness difference, the second brightness difference, the third brightness difference, and the first YUV brightness difference, the environment type corresponding to the target environment is determined.

4. The image processing method according to claim 3, wherein, The step of determining the environment type corresponding to the target environment based on the first RAW brightness, the second RAW brightness, the third RAW brightness, the first brightness difference, the second brightness difference, the third brightness difference, and the first YUV brightness difference includes: The second YUV brightness difference is determined based on the YUV brightness of each frame block whose brightness difference is less than the brightness difference threshold. When the absolute value of the difference between the first brightness difference and the second brightness difference is less than the absolute value of the difference between the first brightness difference and the third brightness difference, the absolute value of the difference between the first RAW brightness and the second RAW brightness is less than the absolute value of the difference between the first RAW brightness and the third RAW brightness, and the absolute value of the difference between the first YUV brightness difference and the second YUV brightness difference is within a preset YUV brightness difference range, the environment type is determined to be a first environment type; wherein, the first environment type is the illuminance level of the target environment is the first illuminance level, and the proportion of the high brightness area is greater than a preset proportion. If the absolute value of the difference between the first brightness difference and the second brightness difference is greater than or equal to the absolute value of the difference between the first brightness difference and the third brightness difference, the absolute value of the difference between the first RAW brightness and the second RAW brightness is greater than or equal to the absolute value of the difference between the first RAW brightness and the third RAW brightness, and the first YUV brightness difference is greater than the YUV brightness difference threshold, then the environment type is determined to be the second environment type; wherein, the second environment type is the illuminance level of the target environment is the second illuminance level, and the proportion of the high brightness area is less than or equal to the preset proportion, and the second illuminance level is lower than the first illuminance level.

5. The image processing method according to any one of claims 1-4, wherein, The brightness processing of the initial image acquired for the target environment based on the environment type includes: When the environment type is the first environment type and the gain of the initial image is within a preset gain range, the brightness enhancement parameter of the dark area in the initial image is reduced until the YUV brightness of the reduced image is less than or equal to the first YUV brightness threshold. Wherein, the brightness difference between the RAW brightness and YUV brightness of the image block included in the dark area is greater than or equal to the brightness difference threshold, and the brightness enhancement parameter is used to characterize the enhancement parameter of the initial image in the brightness information dimension.

6. The image processing method according to claim 5, wherein, The parameter for reducing the brightness enhancement of dark areas in the initial image includes: Based on the YUV brightness of the image blocks included in the dark area, the average YUV brightness corresponding to the dark area is determined, and based on the RAW brightness of the image blocks included in the dark area, the average RAW brightness corresponding to the dark area is determined. The brightness reduction step size is determined based on the difference between the average YUV brightness and the average RAW brightness. Based on the brightness reduction step size, the brightness enhancement parameter of the dark areas in the initial image is reduced.

7. The image processing method according to claim 5, wherein, The brightness processing of the initial image acquired for the target environment based on the environment type includes: When the environment type is the second environment type and the gain of the initial image is greater than or equal to the gain threshold, the brightness enhancement parameter of the dark area in the initial image is reduced until the YUV brightness of the reduced image is less than or equal to the second YUV brightness threshold. Wherein, the second YUV brightness threshold is less than the first YUV brightness threshold.

8. An image processing apparatus, comprising: The determining unit is configured to determine the environment type corresponding to the target environment based on the brightness information of the real-time image of the target environment when the illuminance of the target environment is less than the illuminance threshold and the dynamic range is greater than the dynamic range threshold; wherein, the environment type is used to characterize the illuminance level and the proportion of high brightness areas of the target environment, and the brightness of the high brightness areas is greater than a preset brightness. The processing unit is configured to perform brightness processing on the initial image acquired for the target environment based on the environment type to obtain the target image.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, wherein the processor executes the computer program to implement the image processing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, the computer program, when executed by a processor, implementing the image processing method as described in any one of claims 1 to 7.

Images