Image processing method, apparatus, device, and storage medium

Abstract

This application provides an image processing method, apparatus, device, and storage medium, relating to the field of image processing technology. The method includes: when the illuminance of a target environment is less than an illuminance threshold and the dynamic range is greater than a dynamic range threshold, determining the environment type corresponding to the target environment based on the brightness information of the real-time image of the target environment; wherein the environment type is used to characterize the illuminance level and the proportion of high-brightness areas in the target environment, and the brightness of the high-brightness areas is greater than a preset brightness; then, based on the environment type, performing targeted brightness processing on an initial image acquired for the target environment to obtain a target image. Using the technical solution provided in this application can solve the problem of poor image quality in dark areas of images in scenarios with extremely low ambient illuminance and high dynamic range, thereby improving the image quality in dark areas.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image processing method, apparatus, device, and storage medium. Background Technology

[0002] In low-light environments, such as dusk, and in high dynamic range scenes, the camera may struggle to capture sufficient detail in low light conditions, resulting in poor image quality in dark areas of the captured image.

[0003] To address the issue of poor image quality in dark areas, the initial image can be stretched dynamically to obtain a processed image. This processed image is then smoothed to remove noise. The smoothed image is then magnified to highlight the noise. Finally, the magnified image is smoothed again to remove noise. The processed image is then fused with the initial image to obtain an image with better quality in dark areas.

[0004] However, the above method of processing noise before fusing images will lose details of the original image, making it unsuitable for environments with extremely low illumination, such as at night, and high dynamic range scenes, resulting in poor image quality in dark areas of the image. Summary of the Invention

[0005] This application provides an image processing method, apparatus, device, and storage medium to solve the problem of poor image quality in dark areas of images in scenes with extremely low ambient light and high dynamic range in the prior art, thereby improving the image quality in dark areas.

[0006] This application provides an image processing method, including: When the illuminance of the target environment is less than the illuminance threshold and the dynamic range is greater than the dynamic range threshold, the environment type corresponding to the target environment is determined based on the brightness information of the real-time image of the target environment; wherein, the environment type is used to characterize the illuminance level and the proportion of high brightness areas of the target environment, and the brightness of the high brightness areas is greater than the preset brightness. The initial image acquired for the target environment is processed based on the environment type to obtain the target image.

[0007] According to an image processing method provided in this application, determining the environment type corresponding to the target environment based on the brightness information of the real-time image of the target environment includes: The live feed is divided into multiple screen blocks; For each of the aforementioned image blocks, determine the brightness difference between the RAW brightness and the YUV brightness of the image block; Based on the RAW brightness, YUV brightness, and corresponding brightness difference of each image block, the environment type corresponding to the target environment is determined.

[0008] According to an image processing method provided in this application, determining the environment type corresponding to the target environment based on the RAW brightness, YUV brightness, and corresponding brightness difference of each image block includes: Based on the RAW brightness of each of the multiple screen blocks, the first RAW brightness corresponding to the multiple screen blocks, the second RAW brightness corresponding to at least one screen block whose RAW brightness is greater than or equal to the RAW brightness threshold, and the third RAW brightness corresponding to at least one screen block whose RAW brightness is less than the RAW brightness threshold are determined. Based on the brightness difference corresponding to each of the aforementioned picture blocks, a first brightness difference corresponding to multiple brightness differences, a second brightness difference corresponding to at least one picture block whose brightness difference is less than a brightness difference threshold, and a third brightness difference corresponding to at least one picture block whose brightness difference is greater than or equal to the brightness difference threshold are determined. The first YUV brightness difference is determined based on the YUV brightness of each frame block whose brightness difference is greater than or equal to the brightness difference threshold. Based on the first RAW brightness, the second RAW brightness, the third RAW brightness, the first brightness difference, the second brightness difference, the third brightness difference, and the first YUV brightness difference, the environment type corresponding to the target environment is determined.

[0009] According to an image processing method provided in this application, determining the environment type corresponding to the target environment based on the first RAW brightness, the second RAW brightness, the third RAW brightness, the first brightness difference, the second brightness difference, the third brightness difference, and the first YUV brightness difference includes: The second YUV brightness difference is determined based on the YUV brightness of each frame block whose brightness difference is less than the brightness difference threshold. When the absolute value of the difference between the first brightness difference and the second brightness difference is less than the absolute value of the difference between the first brightness difference and the third brightness difference, the absolute value of the difference between the first RAW brightness and the second RAW brightness is less than the absolute value of the difference between the first RAW brightness and the third RAW brightness, and the absolute value of the difference between the first YUV brightness difference and the second YUV brightness difference is within a preset YUV brightness difference range, the environment type is determined to be a first environment type; wherein, the first environment type is the illuminance level of the target environment is the first illuminance level, and the proportion of the high brightness area is greater than a preset proportion. If the absolute value of the difference between the first brightness difference and the second brightness difference is greater than or equal to the absolute value of the difference between the first brightness difference and the third brightness difference, the absolute value of the difference between the first RAW brightness and the second RAW brightness is greater than or equal to the absolute value of the difference between the first RAW brightness and the third RAW brightness, and the first YUV brightness difference is greater than the YUV brightness difference threshold, then the environment type is determined to be the second environment type; wherein, the second environment type is the illuminance level of the target environment is the second illuminance level, and the proportion of the high brightness area is less than or equal to the preset proportion, and the second illuminance level is lower than the first illuminance level.

[0010] According to an image processing method provided in this application, the step of performing brightness processing on an initial image acquired for the target environment based on the environment type includes: When the environment type is the first environment type and the gain of the initial image is within a preset gain range, the brightness enhancement parameter of the dark area in the initial image is reduced until the YUV brightness of the reduced image is less than or equal to the first YUV brightness threshold. Wherein, the brightness difference between the RAW brightness and YUV brightness of the image block included in the dark area is greater than or equal to the brightness difference threshold, and the brightness enhancement parameter is used to characterize the enhancement parameter of the initial image in the brightness information dimension.

[0011] According to an image processing method provided in this application, reducing the brightness enhancement parameter of the dark areas in the initial image includes: Based on the YUV brightness of the image blocks included in the dark area, the average YUV brightness corresponding to the dark area is determined, and based on the RAW brightness of the image blocks included in the dark area, the average RAW brightness corresponding to the dark area is determined. The brightness reduction step size is determined based on the difference between the average YUV brightness and the average RAW brightness. Based on the brightness reduction step size, the brightness enhancement parameter of the dark areas in the initial image is reduced.

[0012] According to an image processing method provided in this application, the step of performing brightness processing on an initial image acquired for the target environment based on the environment type includes: When the environment type is the second environment type and the gain of the initial image is greater than or equal to the gain threshold, the brightness enhancement parameter of the dark area in the initial image is reduced until the YUV brightness of the reduced image is less than or equal to the second YUV brightness threshold. Wherein, the second YUV brightness threshold is less than the first YUV brightness threshold.

[0013] This application also provides an image processing apparatus, comprising: The determining unit is used to determine the environment type corresponding to the target environment based on the brightness information of the real-time image of the target environment when the illuminance of the target environment is less than the illuminance threshold and the dynamic range is greater than the dynamic range threshold; wherein, the environment type is used to characterize the illuminance level and the proportion of high brightness area of ​​the target environment, and the brightness of the high brightness area is greater than the preset brightness. The processing unit is used to perform brightness processing on the initial image acquired for the target environment based on the environment type to obtain the target image.

[0014] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the image processing method as described above.

[0015] This application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image processing method as described above.

[0016] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the image processing method as described above.

[0017] The image processing method, apparatus, device, and storage medium provided in this application relate to the field of image processing technology. When the illuminance of the target environment is less than an illuminance threshold and the dynamic range is greater than a dynamic range threshold, the method determines the environment type corresponding to the target environment based on the brightness information of the real-time image of the target environment. The environment type characterizes the illuminance level and the proportion of high-brightness areas in the target environment, with the brightness of the high-brightness areas exceeding a preset brightness. Based on the environment type, the initial image acquired for the target environment is then subjected to targeted brightness processing. This effectively solves the problem of poor image quality in dark areas of images in scenarios with extremely low ambient illuminance and high dynamic range, thereby improving the image quality of dark areas. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating an image processing method provided in an embodiment of this application.

[0020] Figure 2This is a flowchart illustrating a method for determining the environment type of a target environment based on the brightness information of a real-time image of the target environment, as provided in an embodiment of this application.

[0021] Figure 3 This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application.

[0022] Figure 4 This is a schematic diagram of the physical structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0024] In the embodiments of this application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone, where A and B can be singular or plural. In the textual description of this application, the character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0025] The technical solutions provided in this application can be adapted to scenarios such as target monitoring and visual navigation in environments with extremely low illumination and high dynamic range. In these scenarios, in order to better analyze and identify information in images, it is usually necessary to perform brightness processing on dark areas in the initial acquired image to improve the image quality of dark areas in the acquired image.

[0026] Given that current methods of processing noise before fusing images lose details of the initial image, making it unsuitable for scenes with extremely low illumination and high dynamic range, this application provides an image processing method to improve the image quality of dark areas in such scenes. In cases of extremely low illumination and high dynamic range, the method first determines the environment type of the target environment based on the brightness information of the real-time image. The environment type characterizes the illumination level and the proportion of high-brightness areas in the target environment, with the brightness of these high-brightness areas exceeding a preset brightness. Then, based on the environment type, targeted brightness processing is applied to the initial image acquired for the target environment. This effectively solves the problem of poor image quality in dark areas in existing technologies for scenes with extremely low illumination and high dynamic range, thereby improving the image quality of dark areas.

[0027] It is understood that the subject executing this method can be an electronic device such as a camera, a specially designed image processing device, a computer or server, or an image processing device installed in the electronic device. The image processing device can be implemented by software, hardware or a combination of both, and can be set according to actual needs.

[0028] It should be noted that the image processing method provided in this application embodiment can also be extended to scenarios of contrast processing and motion blur processing. Specifically, the brightness-related parameters can be replaced with contrast parameters or motion blur parameters according to actual needs. Here, this application embodiment will not elaborate further.

[0029] The image processing method provided in this application will be described in detail below through several specific embodiments. It is understood that these specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0030] Figure 1 This is a flowchart illustrating an image processing method provided in an embodiment of this application. For example, please refer to... Figure 1 As shown, the image processing method may include: S101. When the illuminance of the target environment is less than the illuminance threshold and the dynamic range is greater than the dynamic range threshold, the environment type corresponding to the target environment is determined based on the brightness information of the real-time image of the target environment. The environment type is used to characterize the illuminance level and the proportion of high-brightness areas of the target environment. The brightness of the high-brightness areas is greater than the preset brightness.

[0031] The values ​​of the illuminance threshold and the dynamic range threshold can be set according to actual needs. For example, the illuminance threshold can be 50 lux or 51 lux, etc., and can be set according to actual needs.

[0032] For example, with a maximum brightness of 255, the preset brightness value can be 100, 101, etc., and can be set according to actual needs.

[0033] For example, the brightness information of the live footage can include RAW brightness and YUV brightness, which can be set according to actual needs. RAW brightness is usually the raw data obtained directly from the camera device's sensor, without any processing; while YUV brightness is the brightness information after color space conversion.

[0034] It is understood that in the embodiments of this application, the illuminance level and the proportion of high-brightness areas of the target environment represented by different environment types are different.

[0035] For example, in the embodiments of this application, when determining the environment type corresponding to the target environment based on the brightness information of the live image of the target environment, the live image of the target environment can be treated as a whole, and the environment type corresponding to the target environment can be determined based on its brightness information; or the live image of the target environment can be divided into multiple image blocks, and the environment type corresponding to the target environment can be determined based on the brightness information of each image block, etc. The specific settings can be made according to actual needs.

[0036] After determining the environment type corresponding to the target environment based on the brightness information of the live image of the target environment, the initial image of the target environment can be subjected to brightness processing in a targeted manner based on the environment type, i.e., the following S102 is executed.

[0037] S102. Based on the environment type, perform brightness processing on the initial image acquired for the target environment to obtain the target image.

[0038] As can be seen, in this embodiment, when the illuminance of the target environment is less than the illuminance threshold and the dynamic range is greater than the dynamic range threshold, the environment type corresponding to the target environment is determined based on the brightness information of the real-time image of the target environment. The environment type is used to characterize the illuminance level and the proportion of high-brightness areas of the target environment, and the brightness of the high-brightness areas is greater than the preset brightness. Based on the environment type, the initial image acquired for the target environment is subjected to targeted brightness processing. This can effectively solve the problem of poor image quality in dark areas of the image in the prior art in scenes with extremely low ambient illuminance and high dynamic range, thereby improving the image quality of dark areas.

[0039] Based on the above Figure 1 In the illustrated embodiment, for example, the specific implementation of determining the environment type corresponding to the target environment based on the brightness information of the real-time image of the target environment in S101 described above can be found in the following... Figure 2 The example shown.

[0040] Figure 2 This application provides a flowchart illustrating a method for determining the environment type of a target environment based on the brightness information of a real-time image of that environment. For example, please refer to [link to flowchart illustration]. Figure 2 As shown, the method may include: S201. Divide the live video into multiple video blocks.

[0041] For example, in an embodiment of this application, the live video can be divided into... Each frame is a pixel, and each pixel has its own RAW luminance and YUV luminance. For example, The raw brightness of each image block can be denoted as: , , ..., , The YUV brightness of each image block can be denoted as... , , ..., Typically, the luminance value of RAW is less than that of YUV.

[0042] Typically, for any given frame, such as the first... The RAW brightness of the first image block can be understood as the brightness of the second image block. The average RAW brightness of all pixels in a given frame block; similarly, its YUV brightness can be understood as the average RAW brightness of the first frame block. The average YUV brightness of all pixels in a frame block.

[0043] S202. For each frame block, determine the brightness difference between the RAW brightness and the YUV brightness of the frame block.

[0044] For example, when calculating the brightness difference between the RAW brightness and YUV brightness of each frame block, take any frame block, such as the first one... Taking the first screen block as an example, the first... The brightness difference between the RAW brightness and YUV brightness of each frame block can be seen in Formula 2 below.

[0045] Formula 2 in, Indicates the first The brightness difference between the RAW brightness and YUV brightness of each frame block. Indicates the first The YUV brightness of each pixel block Indicates the first RAW brightness of each frame block.

[0046] S203. Based on the RAW brightness, YUV brightness and corresponding brightness difference of each image block, determine the environment type corresponding to the target environment.

[0047] For example, in the embodiments of this application, when determining the environment type corresponding to the target environment based on the RAW brightness, YUV brightness and the corresponding brightness difference of each frame block, it may include: Based on the RAW brightness of each frame block, the first RAW brightness corresponding to multiple frame blocks, the second RAW brightness corresponding to at least one frame block whose RAW brightness is greater than or equal to a RAW brightness threshold, and the third RAW brightness corresponding to at least one frame block whose RAW brightness is less than a RAW brightness threshold are determined. Based on the brightness difference corresponding to each frame block, the first brightness difference corresponding to multiple brightness differences, the second brightness difference corresponding to at least one frame block whose brightness difference is less than a brightness difference threshold, and the third brightness difference corresponding to at least one frame block whose brightness difference is greater than or equal to a brightness difference threshold are determined. Based on the YUV brightness corresponding to each frame block whose brightness difference is greater than or equal to a brightness difference threshold, the first YUV brightness difference is determined. Finally, based on the first RAW brightness, the second RAW brightness, the third RAW brightness, the first brightness difference, the second brightness difference, the third brightness difference, and the first YUV brightness difference, the environment type corresponding to the target environment is determined.

[0048] For example, in the embodiments of this application, the RAW luminance threshold can be denoted as Rt, and the luminance difference threshold can be denoted as... The values ​​of the RAW brightness threshold and brightness difference threshold can be set according to actual needs. For example, the value of the RAW brightness threshold Rt can be set according to the characteristics of the camera device's sensor, and the brightness difference threshold... The brightness can be set based on the RAW brightness of the live stream footage. Generally, the brighter the live stream footage, the higher the RAW brightness threshold should be. The value can be set to be larger.

[0049] For example, when determining the first RAW brightness corresponding to multiple frame blocks based on the RAW brightness of each frame block, one can calculate the corresponding average RAW brightness based on the RAW brightness of each frame block and determine this average RAW brightness as the first RAW brightness; alternatively, one can select a portion of the frame blocks based on their RAW brightness, calculate the average RAW brightness corresponding to that portion of the frame blocks, and determine this average RAW brightness as the first RAW brightness, etc. The specific settings can be configured according to actual needs. For example, the first RAW brightness corresponding to multiple frame blocks can be denoted as R.

[0050] When determining the second RAW brightness corresponding to at least one frame block whose RAW brightness is greater than or equal to the RAW brightness threshold Rt based on the RAW brightness of each frame block, the frame block whose RAW brightness is greater than or equal to the RAW brightness threshold can be determined from the multiple frame blocks based on the RAW brightness of each frame block; and the corresponding second RAW brightness can be determined based on the RAW brightness of the frame block whose RAW brightness is greater than or equal to the RAW brightness threshold.

[0051] For example, when determining the corresponding second RAW brightness based on the RAW brightness of frame blocks whose RAW brightness is greater than or equal to the RAW brightness threshold, the average RAW brightness can be calculated based on the RAW brightness of each frame block whose RAW brightness is greater than or equal to the RAW brightness threshold, and this average RAW brightness can be determined as the second RAW brightness; alternatively, the average RAW brightness can be calculated based on the RAW brightness of a subset of frame blocks whose RAW brightness is greater than or equal to the RAW brightness threshold, and this average RAW brightness can be determined as the second RAW brightness, etc. The specific settings can be configured according to actual needs. For example, the second RAW brightness can be denoted as RB.

[0052] When determining the third RAW brightness corresponding to at least one frame block whose RAW brightness is less than the RAW brightness threshold based on the RAW brightness of each frame block, the corresponding frame block whose RAW brightness is less than the RAW brightness threshold can be determined from multiple frame blocks based on the RAW brightness of each frame block; and the corresponding third RAW brightness can be determined based on the RAW brightness of the frame block whose RAW brightness is less than the RAW brightness threshold.

[0053] For example, when determining the corresponding third RAW brightness based on the RAW brightness of frame blocks whose RAW brightness is less than the RAW brightness threshold, one can calculate the corresponding average RAW brightness based on the RAW brightness of each frame block whose RAW brightness is less than the RAW brightness threshold, and determine this average RAW brightness as the third RAW brightness; alternatively, one can calculate the average RAW brightness corresponding to a subset of frame blocks whose RAW brightness is less than the RAW brightness threshold, and determine this average RAW brightness as the third RAW brightness, etc., depending on actual needs. For example, the third RAW brightness can be denoted as RD.

[0054] For example, when determining the first brightness difference based on the brightness differences corresponding to each screen block, one can calculate the average brightness difference of all brightness differences based on the brightness differences corresponding to each screen block, as shown in Formula 3 below, and determine this average brightness difference as the first brightness difference; alternatively, one can calculate the average brightness difference of a portion of all brightness differences based on the brightness differences corresponding to each screen block, and determine this average brightness difference as the first brightness difference, etc. The specific setting can be determined according to actual needs. For example, the first brightness difference can be denoted as... .

[0055] Formula 3 in, express The first in the first image block Each image block Indicates the first The brightness difference between the RAW brightness and YUV brightness of each frame block.

[0056] When determining the second brightness difference corresponding to at least one frame block whose brightness difference is less than a brightness difference threshold and the third brightness difference corresponding to at least one frame block whose brightness difference is greater than or equal to the brightness difference threshold based on the brightness differences corresponding to each frame block, it is possible to first determine, from multiple frame blocks, at least one frame block whose brightness difference is less than the brightness difference threshold and at least one frame block whose brightness difference is greater than or equal to the brightness difference threshold. For example, frame blocks whose brightness difference is less than the brightness difference threshold can be marked as 0, and frame blocks whose brightness difference is greater than or equal to the brightness difference threshold can be marked as 1. Among them, the frame block marked as 0 can be understood as the frame block in the bright area of ​​the live image, and the frame block marked as 1 can be understood as the frame block in the dark area of ​​the live image.

[0057] Count the number of screen blocks marked as 0 and the number of screen blocks marked as 1 respectively. Assume the number of screen blocks marked as 0 is... If there are [number], then the number of image blocks marked as 1 is [number]. n- When determining the second brightness difference corresponding to at least one image block whose brightness difference is less than a brightness difference threshold, it can be based on a value marked as 0. Calculate the brightness difference corresponding to each image block. The average brightness difference corresponding to each screen block can be found in Formula 4 below, and the calculated average brightness difference is determined as the second brightness difference.

[0058] Formula 4 in, Indicates the second brightness difference. This indicates the number of image blocks marked as 0, which is the number of image blocks with a brightness difference less than the brightness difference threshold.

[0059] When determining the third brightness difference corresponding to at least one frame block whose brightness difference is greater than or equal to a brightness difference threshold, it can be based on the value marked as 1. n- Calculate the brightness difference corresponding to each image block. n- The average brightness difference corresponding to each screen block can be found in Formula 5 below, and the calculated average brightness difference is determined as the third brightness difference.

[0060] Formula 5 in, Indicates the third brightness difference. n- This indicates the number of image blocks marked as 1, which is the number of image blocks with a brightness difference greater than or equal to the brightness difference threshold.

[0061] For example, when determining the first YUV brightness difference based on the YUV brightness of each frame block whose brightness difference is greater than or equal to a brightness difference threshold, it can be based on the value marked as 1. n- Calculate the YUV brightness corresponding to each image block. n- The average YUV brightness corresponding to each frame block can be found in Formula 6 below, and the calculated average YUV brightness is determined as the first YUV brightness difference.

[0062] Formula 6 in, Indicates the first YUV brightness difference. express n- In the first frame, the... The YUV brightness of each image block.

[0063] Based on the above description, after determining the first RAW brightness R, the second RAW brightness RB, the third RAW brightness RD, and the first brightness difference... Second brightness difference Third brightness difference Difference in brightness between the first YUV and the second YUV Then, the environment type corresponding to the target environment can be further determined.

[0064] For example, based on the first RAW luminance R, the second RAW luminance RB, the third RAW luminance RD, and the first luminance difference Second brightness difference Third brightness difference Difference in brightness between the first YUV and the second YUV When determining the environment type corresponding to the target environment, the first brightness difference can be used. Difference from the second brightness The absolute value of the difference is less than the first brightness difference. Difference from the third brightness The absolute value of the difference, the absolute value of the difference between the first RAW brightness R and the second RAW brightness RB is less than the absolute value of the difference between the first RAW brightness R and the third RAW brightness RD, and the first YUV brightness difference Difference in brightness with the second YUV The absolute value of the difference is within the preset YUV brightness difference range, that is... , ,and If the exposure is within the preset YUV brightness difference range, it indicates that the target scene's exposure is primarily focused on brighter areas, with some darker areas present. Furthermore, the difference in YUV brightness between the darker and brighter areas is relatively small. Therefore, the target environment can be classified as the first environment type. The first environment type is defined as having the first illuminance level, and the proportion of high-brightness areas is greater than the preset proportion. The preset proportion can be set according to actual needs.

[0065] Among them, the second YUV brightness difference It is determined based on the YUV brightness of each frame block whose brightness difference is less than the brightness difference threshold. For example, see Formula 7 below.

[0066] Formula 7 in, This indicates the brightness difference between the second and third YUV wavelengths. express In the first frame, the... The YUV brightness of a single frame block, i.e., the YUV brightness of a frame block whose brightness difference is less than the brightness difference threshold.

[0067] First brightness difference Difference from the second brightness The absolute value of the difference is greater than or equal to the first brightness difference. Difference from the third brightness The absolute value of the difference between the first RAW brightness R and the second RAW brightness RB is greater than or equal to the absolute value of the difference between the first RAW brightness R and the third RAW brightness RD, and the first YUV brightness... The difference is greater than the YUV brightness difference threshold, i.e. , ， and If the difference between the YUV brightness and the target environment is greater than the threshold, it indicates that the exposure of the target scene is mainly dark areas, and there are large areas of dark areas. The YUV brightness of the dark areas is also relatively high. Therefore, the environment type corresponding to the target environment can be determined as the second environment type. The second environment type is the target environment with an illuminance level of the second illuminance level, and the proportion of high brightness areas is less than or equal to the preset proportion. The second illuminance level is lower than the first illuminance level.

[0068] For example, in this embodiment of the application, in order to more accurately determine the second environment type, the number of screen blocks with a brightness difference less than the brightness difference threshold in the real-time image can be further combined for judgment. If the number of screen blocks with a brightness difference less than the brightness difference threshold is relatively small, for example, less than the number of screen blocks, then the environment type corresponding to the target environment can be determined as the second environment type.

[0069] Based on the above description, after determining the environment type corresponding to the target environment based on the RAW brightness, YUV brightness and corresponding brightness difference of each image block, the initial image acquired for the target environment can be processed in a targeted manner based on the environment type. This can effectively solve the problem of poor image quality in dark areas of the image in the existing technology in scenes with extremely low ambient light and high dynamic range, thereby improving the image quality in dark areas.

[0070] Based on any of the above embodiments, for example, in S102 above, when performing brightness processing on the initial image acquired for the target environment based on the environment type, at least two of the following possible scenarios may be included.

[0071] In one possible scenario, the environment type is the first environment type, and the gain of the initial image is within a preset gain range. The value of the preset gain range can be set according to actual needs.

[0072] When the environment type is the first environment type and the gain of the initial image is within the preset gain range, the brightness enhancement parameter of the dark areas in the initial image can be reduced until the YUV brightness of the reduced image is less than or equal to the first YUV brightness threshold. The value of the first YUV brightness threshold can be set according to actual needs.

[0073] Among them, the brightness difference between the RAW brightness and YUV brightness of the image blocks included in the dark area is greater than or equal to the brightness difference threshold. The brightness enhancement parameter is used to characterize the enhancement parameters of the initial image in the brightness information dimension.

[0074] For example, in the embodiments of this application, when reducing the brightness enhancement parameter of the dark area in the initial image, the brightness enhancement parameter of the dark area in the initial image can be reduced by a different brightness reduction value each time it is reduced, until the YUV brightness of the reduced image is less than or equal to the first YUV brightness threshold; or a brightness reduction step size can be predetermined, and the brightness enhancement parameter of the dark area in the initial image can be reduced based on the brightness reduction step size each time it is reduced, until the YUV brightness of the reduced image is less than or equal to the first YUV brightness threshold.

[0075] For example, in this embodiment of the application, when reducing the brightness enhancement parameter of the dark area in the initial image based on the brightness reduction step size, the average YUV brightness corresponding to the dark area can be determined based on the YUV brightness of the image blocks included in the dark area, and the average RAW brightness corresponding to the dark area can be determined based on the RAW brightness of the image blocks included in the dark area; then, the brightness reduction step size is determined based on the difference between the average YUV brightness and the average RAW brightness corresponding to the dark area. For example, see Formula 8 below. In this way, after determining the brightness reduction step size, the brightness enhancement parameter of the dark area in the initial image can be reduced based on the determined brightness reduction step size, thereby improving the image quality of the dark area.

[0076] Formula 8 in, This indicates the step size for decreasing brightness, where Y represents the average YUV brightness of the dark area, and R represents the average RAW brightness of the dark area. This represents the brightness adjustment factor, and its value can be set according to actual needs.

[0077] In another possible scenario, the environment type is either the first or the second environment type, and the gain of the initial image is greater than or equal to a gain threshold. The value of the gain threshold can be set according to actual needs. Typically, this gain threshold is greater than the upper limit of the aforementioned preset gain range.

[0078] If the environment type is the second environment type and the gain of the initial image is greater than or equal to the gain threshold, it indicates that the image quality is relatively poor, especially the image quality of the dark areas. In this case, the dark areas are often no longer important and can be ignored. Therefore, the brightness enhancement parameter of the dark areas in the initial image can be reduced until the YUV brightness of the reduced image is less than or equal to the second YUV brightness threshold.

[0079] The second YUV luminance threshold is less than the first YUV luminance threshold.

[0080] For example, in the embodiments of this application, when reducing the brightness enhancement parameter of the dark area in the initial image, the specific implementation is similar to the specific implementation of reducing the brightness enhancement parameter of the dark area in the initial image in the above possible scenarios. Please refer to the relevant description of reducing the brightness enhancement parameter of the dark area in the initial image above. Here, the embodiments of this application will not repeat it.

[0081] Understandably, in the two possible scenarios mentioned above, in order to improve the image quality of dark areas, in addition to reducing the brightness enhancement parameters of dark areas in the initial image, one can also reduce the sharpening intensity and noise reduction intensity of dark areas, etc., which can be set according to actual needs.

[0082] The image processing apparatus provided in this application is described below. The image processing apparatus described below can be referred to in correspondence with the image processing method described above.

[0083] Figure 3 This is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this application. For example, please refer to [link to relevant documentation]. Figure 3 As shown, the image processing device 30 may include: The determining unit 301 is used to determine the environment type corresponding to the target environment based on the brightness information of the real-time image of the target environment when the illuminance of the target environment is less than the illuminance threshold and the dynamic range is greater than the dynamic range threshold; wherein, the environment type is used to characterize the illuminance level and the proportion of high brightness area of ​​the target environment, and the brightness of the high brightness area is greater than the preset brightness. The processing unit 302 is used to perform brightness processing on the initial image acquired for the target environment based on the environment type to obtain the target image.

[0084] For example, in this embodiment of the application, the determining unit 301 is used to determine the environment type corresponding to the target environment based on the brightness information of the real-time image of the target environment, including: The live feed is divided into multiple screen blocks; For each of the aforementioned image blocks, determine the brightness difference between the RAW brightness and the YUV brightness of the image block; Based on the RAW brightness, YUV brightness, and corresponding brightness difference of each image block, the environment type corresponding to the target environment is determined.

[0085] For example, in an embodiment of this application, the determining unit 301 is used to determine the environment type corresponding to the target environment based on the RAW brightness, YUV brightness, and corresponding brightness difference of each of the image blocks, including: Based on the RAW brightness of each of the multiple screen blocks, the first RAW brightness corresponding to the multiple screen blocks, the second RAW brightness corresponding to at least one screen block whose RAW brightness is greater than or equal to the RAW brightness threshold, and the third RAW brightness corresponding to at least one screen block whose RAW brightness is less than the RAW brightness threshold are determined. Based on the brightness difference corresponding to each of the aforementioned picture blocks, a first brightness difference corresponding to multiple brightness differences, a second brightness difference corresponding to at least one picture block whose brightness difference is less than a brightness difference threshold, and a third brightness difference corresponding to at least one picture block whose brightness difference is greater than or equal to the brightness difference threshold are determined. The first YUV brightness difference is determined based on the YUV brightness of each frame block whose brightness difference is greater than or equal to the brightness difference threshold. Based on the first RAW brightness, the second RAW brightness, the third RAW brightness, the first brightness difference, the second brightness difference, the third brightness difference, and the first YUV brightness difference, the environment type corresponding to the target environment is determined.

[0086] For example, in this embodiment of the application, the determining unit 301 is used to determine the environment type corresponding to the target environment based on the first RAW brightness, the second RAW brightness, the third RAW brightness, the first brightness difference, the second brightness difference, the third brightness difference, and the first YUV brightness difference, including: The second YUV brightness difference is determined based on the YUV brightness of each frame block whose brightness difference is less than the brightness difference threshold. When the absolute value of the difference between the first brightness difference and the second brightness difference is less than the absolute value of the difference between the first brightness difference and the third brightness difference, the absolute value of the difference between the first RAW brightness and the second RAW brightness is less than the absolute value of the difference between the first RAW brightness and the third RAW brightness, and the absolute value of the difference between the first YUV brightness difference and the second YUV brightness difference is within a preset YUV brightness difference range, the environment type is determined to be a first environment type; wherein, the first environment type is the illuminance level of the target environment is the first illuminance level, and the proportion of high brightness areas is greater than a preset proportion. If the absolute value of the difference between the first brightness difference and the second brightness difference is greater than or equal to the absolute value of the difference between the first brightness difference and the third brightness difference, the absolute value of the difference between the first RAW brightness and the second RAW brightness is greater than or equal to the absolute value of the difference between the first RAW brightness and the third RAW brightness, and the first YUV brightness difference is greater than the YUV brightness difference threshold, then the environment type is determined to be the second environment type; wherein, the second environment type is the illuminance level of the target environment is the second illuminance level, and the proportion of high brightness areas is less than or equal to the preset proportion, and the second illuminance level is lower than the first illuminance level.

[0087] For example, in an embodiment of this application, the processing unit 302 is used to perform brightness processing on an initial image acquired for the target environment based on the environment type, including: When the environment type is the first environment type and the gain of the initial image is within a preset gain range, the brightness enhancement parameter of the dark area in the initial image is reduced until the YUV brightness of the reduced image is less than or equal to the first YUV brightness threshold. Wherein, the brightness difference between the RAW brightness and YUV brightness of the image block included in the dark area is greater than or equal to the brightness difference threshold, and the brightness enhancement parameter is used to characterize the enhancement parameter of the initial image in the brightness information dimension.

[0088] For example, in an embodiment of this application, the processing unit 302 is used to reduce the brightness enhancement parameter of the dark areas in the initial image, including: Based on the YUV brightness of the image blocks included in the dark area, the average YUV brightness corresponding to the dark area is determined, and based on the RAW brightness of the image blocks included in the dark area, the average RAW brightness corresponding to the dark area is determined. The brightness reduction step size is determined based on the difference between the average YUV brightness and the average RAW brightness. Based on the brightness reduction step size, the brightness enhancement parameter of the dark areas in the initial image is reduced.

[0089] For example, in an embodiment of this application, the processing unit 302 is used to perform brightness processing on an initial image acquired for the target environment based on the environment type, including: When the environment type is the second environment type and the gain of the initial image is greater than or equal to the gain threshold, the brightness enhancement parameter of the dark area in the initial image is reduced until the YUV brightness of the reduced image is less than or equal to the second YUV brightness threshold. Wherein, the second YUV brightness threshold is less than the first YUV brightness threshold.

[0090] The image processing apparatus 30 provided in this application embodiment can execute the technical solution of the image processing method in any of the above embodiments. Its implementation principle and beneficial effects are similar to those of the image processing method. Please refer to the implementation principle and beneficial effects of the image processing method. It will not be repeated here.

[0091] Figure 4 This is a schematic diagram of the physical structure of an electronic device provided in an embodiment of this application, such as... Figure 4 As shown, the electronic device may include a processor 410, a communications interface 420, a memory 430, and a communication bus 440, wherein the processor 410, communications interface 420, and memory 430 communicate with each other via the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute an image processing method. This method includes: determining the environment type corresponding to the target environment based on the brightness information of the real-time image of the target environment when the illuminance of the target environment is less than an illuminance threshold and the dynamic range is greater than a dynamic range threshold; wherein the environment type characterizes the illuminance level and the proportion of high-brightness areas of the target environment, and the brightness of the high-brightness areas is greater than a preset brightness; and performing brightness processing on an initial image acquired for the target environment based on the environment type to obtain a target image.

[0092] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0093] On the other hand, this application also provides a computer program product, which includes a computer program that can be stored on a computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the image processing methods provided by the above methods. The method includes: when the illuminance of the target environment is less than an illuminance threshold and the dynamic range is greater than a dynamic range threshold, determining the environment type corresponding to the target environment based on the brightness information of the real-time image of the target environment; wherein the environment type is used to characterize the illuminance level and the proportion of high-brightness areas of the target environment, and the brightness of the high-brightness areas is greater than a preset brightness; and performing brightness processing on an initial image acquired for the target environment based on the environment type to obtain a target image.

[0094] In another aspect, this application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the image processing methods provided by the above methods. The method includes: determining an environment type corresponding to the target environment based on brightness information of a real-time image of the target environment when the illuminance of the target environment is less than an illuminance threshold and the dynamic range is greater than a dynamic range threshold; wherein the environment type characterizes the illuminance level and the proportion of high-brightness areas of the target environment, and the brightness of the high-brightness areas is greater than a preset brightness; and performing brightness processing on an initial image acquired for the target environment based on the environment type to obtain a target image.

[0095] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0096] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0097] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. An image processing method, characterized in that, include: When the illuminance of the target environment is less than the illuminance threshold and the dynamic range is greater than the dynamic range threshold, the environment type corresponding to the target environment is determined based on the brightness information of the real-time image of the target environment; wherein, the environment type is used to characterize the illuminance level and the proportion of high brightness areas of the target environment, and the brightness of the high brightness areas is greater than the preset brightness. The initial image acquired for the target environment is processed based on the environment type to obtain the target image.

2. The image processing method according to claim 1, characterized in that, The determination of the environment type corresponding to the target environment based on the brightness information of the real-time image of the target environment includes: The live feed is divided into multiple screen blocks; For each of the aforementioned image blocks, determine the brightness difference between the RAW brightness and the YUV brightness of the image block; Based on the RAW brightness, YUV brightness, and corresponding brightness difference of each image block, the environment type corresponding to the target environment is determined.

3. The image processing method according to claim 2, characterized in that, The determination of the environment type corresponding to the target environment based on the RAW brightness, YUV brightness, and corresponding brightness difference of each of the image blocks includes: Based on the RAW brightness of each of the multiple screen blocks, the first RAW brightness corresponding to the multiple screen blocks, the second RAW brightness corresponding to at least one screen block whose RAW brightness is greater than or equal to the RAW brightness threshold, and the third RAW brightness corresponding to at least one screen block whose RAW brightness is less than the RAW brightness threshold are determined. Based on the brightness difference corresponding to each of the aforementioned picture blocks, a first brightness difference corresponding to multiple brightness differences, a second brightness difference corresponding to at least one picture block whose brightness difference is less than a brightness difference threshold, and a third brightness difference corresponding to at least one picture block whose brightness difference is greater than or equal to the brightness difference threshold are determined. The first YUV brightness difference is determined based on the YUV brightness of each frame block whose brightness difference is greater than or equal to the brightness difference threshold. Based on the first RAW brightness, the second RAW brightness, the third RAW brightness, the first brightness difference, the second brightness difference, the third brightness difference, and the first YUV brightness difference, the environment type corresponding to the target environment is determined.

4. The image processing method according to claim 3, characterized in that, The step of determining the environment type corresponding to the target environment based on the first RAW brightness, the second RAW brightness, the third RAW brightness, the first brightness difference, the second brightness difference, the third brightness difference, and the first YUV brightness difference includes: The second YUV brightness difference is determined based on the YUV brightness of each frame block whose brightness difference is less than the brightness difference threshold. When the absolute value of the difference between the first brightness difference and the second brightness difference is less than the absolute value of the difference between the first brightness difference and the third brightness difference, the absolute value of the difference between the first RAW brightness and the second RAW brightness is less than the absolute value of the difference between the first RAW brightness and the third RAW brightness, and the absolute value of the difference between the first YUV brightness difference and the second YUV brightness difference is within a preset YUV brightness difference range, the environment type is determined to be a first environment type; wherein, the first environment type is the illuminance level of the target environment is the first illuminance level, and the proportion of the high brightness area is greater than a preset proportion. If the absolute value of the difference between the first brightness difference and the second brightness difference is greater than or equal to the absolute value of the difference between the first brightness difference and the third brightness difference, the absolute value of the difference between the first RAW brightness and the second RAW brightness is greater than or equal to the absolute value of the difference between the first RAW brightness and the third RAW brightness, and the first YUV brightness difference is greater than the YUV brightness difference threshold, then the environment type is determined to be the second environment type; wherein, the second environment type is the illuminance level of the target environment is the second illuminance level, and the proportion of the high brightness area is less than or equal to the preset proportion, and the second illuminance level is lower than the first illuminance level.

5. The image processing method according to any one of claims 1-4, characterized in that, The brightness processing of the initial image acquired for the target environment based on the environment type includes: When the environment type is the first environment type and the gain of the initial image is within a preset gain range, the brightness enhancement parameter of the dark area in the initial image is reduced until the YUV brightness of the reduced image is less than or equal to the first YUV brightness threshold. Wherein, the brightness difference between the RAW brightness and YUV brightness of the image block included in the dark area is greater than or equal to the brightness difference threshold, and the brightness enhancement parameter is used to characterize the enhancement parameter of the initial image in the brightness information dimension.

6. The image processing method according to claim 5, characterized in that, The parameter for reducing the brightness enhancement of dark areas in the initial image includes: Based on the YUV brightness of the image blocks included in the dark area, the average YUV brightness corresponding to the dark area is determined, and based on the RAW brightness of the image blocks included in the dark area, the average RAW brightness corresponding to the dark area is determined. The brightness reduction step size is determined based on the difference between the average YUV brightness and the average RAW brightness. Based on the brightness reduction step size, the brightness enhancement parameter of the dark areas in the initial image is reduced.

7. The image processing method according to claim 5, characterized in that, The brightness processing of the initial image acquired for the target environment based on the environment type includes: When the environment type is the second environment type and the gain of the initial image is greater than or equal to the gain threshold, the brightness enhancement parameter of the dark area in the initial image is reduced until the YUV brightness of the reduced image is less than or equal to the second YUV brightness threshold. Wherein, the second YUV brightness threshold is less than the first YUV brightness threshold.

8. An image processing apparatus, characterized in that, include: The determining unit is used to determine the environment type corresponding to the target environment based on the brightness information of the real-time image of the target environment when the illuminance of the target environment is less than the illuminance threshold and the dynamic range is greater than the dynamic range threshold; wherein, the environment type is used to characterize the illuminance level and the proportion of high brightness area of ​​the target environment, and the brightness of the high brightness area is greater than the preset brightness. The processing unit is used to perform brightness processing on the initial image acquired for the target environment based on the environment type to obtain the target image.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the image processing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the image processing method as described in any one of claims 1 to 7.

Images