Image processing method and device, computer readable storage medium, and terminal equipment

Abstract

The application provides an image processing method and device, a computer readable storage medium and a terminal device, the image processing method comprising: obtaining an image to be processed; calculating a statistical value of low-frequency information of the image to be processed in a luminance domain; determining a corresponding mapping curve according to a scene to which the statistical value belongs, the mapping curve representing a mapping relationship between an original luminance value and an adjusted luminance value of each pixel in the luminance domain, different scenes corresponding to different mapping curves; determining an adjusted luminance value of each pixel in the image to be processed according to the mapping curve; and mapping a gain of the adjusted luminance value of each pixel to each color channel of the corresponding pixel to obtain an adjusted image. The application provides an image processing scheme capable of improving the luminance of a low-light area of an image and avoiding overexposure of a highlight area.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image processing method and apparatus, a computer-readable storage medium, and a terminal device. Background Technology

[0002] The ultimate goal of image display is to reproduce images that closely resemble natural scenes as observed by the human eye. Current image capture devices can acquire images with high brightness, deep bit width, and wide color gamut, containing a wealth of information. However, due to hardware processing limitations, these images cannot be displayed on ordinary monitors. Therefore, it is necessary to reduce the bit width of high-bit-width data for display. However, linear compression causes a severe loss of dynamic range in the displayed image, resulting in significant distortion and an inability to capture the full range of illumination in a natural scene. When an image has only a limited dynamic range, it appears either too bright or too dark, with a significant loss of detail.

[0003] Current methods for improving the dynamic range of images involve increasing the overall brightness of a generally dark image to enhance the brightness of low-brightness areas.

[0004] However, while existing technologies improve the overall brightness of images, they can also cause overexposure in bright areas and significantly increase noise in low-brightness areas. Summary of the Invention

[0005] This application provides a communication method and apparatus, and provides an image processing scheme that can improve the brightness of low-brightness areas of an image and avoid overexposure of high-brightness areas.

[0006] To achieve the above objectives, this application provides the following technical solution:

[0007] In a first aspect, an image processing method is provided, comprising: acquiring an image to be processed; calculating statistical values ​​of low-frequency information of the image to be processed in the luminance domain; determining a corresponding mapping curve based on the scene to which the statistical values ​​belong, wherein the mapping curve represents the mapping relationship between the original luminance value and the adjusted luminance value of each pixel in the luminance domain, and different mapping curves correspond to different scenes; determining the adjusted luminance value of each pixel in the image to be processed based on the mapping curve; and mapping the gain of the adjusted luminance value of each pixel to each color channel of the corresponding pixel to obtain the adjusted image.

[0008] Optionally, in the mapping curve, when the original brightness value is lower than a first preset threshold, the adjusted brightness value corresponding to the original brightness value is higher than the original brightness value; when the original brightness value is higher than the first preset threshold, the adjusted brightness value corresponding to the original brightness value is lower than the original brightness value.

[0009] Optionally, in the mapping curve, when the original brightness value is lower than the second preset threshold, the original brightness value and its corresponding adjusted brightness value have a linear relationship and the adjusted brightness value is greater than the original brightness value, and the second preset threshold is less than the first preset threshold.

[0010] Optionally, determining the adjusted brightness value of each pixel in the image to be processed based on the mapping curve includes: when the image to be processed is the current video frame, determining the target brightness value of each pixel based on the original brightness value of each pixel in the current video frame and the mapping curve; calculating the brightness difference between the current video frame and multiple video frames located before the current video frame; determining the adjusted brightness value of each pixel based on the brightness difference and the target brightness value, wherein the difference between the adjusted brightness value and the target brightness value is less than a preset threshold.

[0011] Optionally, determining the corresponding mapping curve based on the scenario to which the statistical value belongs includes: inputting the statistical value into a mapping model to obtain the mapping curve corresponding to the scenario to which the statistical value belongs, wherein the mapping model is pre-trained.

[0012] Optionally, mapping the gain of the adjusted brightness value of each pixel to the corresponding color channel includes: calculating the adjustment gain under each original brightness value, wherein the adjustment gain is the ratio of the adjusted brightness value to the original brightness value; and mapping the adjustment gain of each pixel to the corresponding color channel to obtain the adjusted image.

[0013] Optionally, the image to be processed is a color image, and the calculation of the statistical values ​​of the low-frequency information of the image to be processed in the brightness domain includes: performing bilateral filtering on the image to be processed to obtain the low-frequency image and high-frequency image of the image to be processed in the brightness domain; and performing non-uniformly distributed histogram statistics on the low-frequency image to obtain the statistical values.

[0014] Optionally, before mapping the gain of the adjusted brightness value of each pixel to the corresponding pixel in each color channel, the method further includes: adding the high-frequency information of each pixel in the high-frequency image to the adjusted brightness value of each pixel for conversion in each color channel.

[0015] Secondly, this application also discloses an image processing apparatus, comprising: an acquisition module for acquiring an image to be processed; a statistics module for calculating statistical values ​​of low-frequency information of the image to be processed in the luminance domain; a mapping curve determination module for determining a corresponding mapping curve based on the scene to which the statistical values ​​belong, wherein the mapping curve represents the mapping relationship between the original luminance value and the adjusted luminance value of each pixel in the luminance domain, and different scenes correspond to different mapping curves; a luminance adjustment module for determining the adjusted luminance value of each pixel in the image to be processed based on the mapping curve; and a conversion module for mapping the gain of the adjusted luminance value of each pixel to each color channel of the corresponding pixel to obtain the adjusted image.

[0016] Thirdly, a computer-readable storage medium is provided having a computer program stored thereon, the computer program being executed by a processor to perform the method provided in the first aspect.

[0017] Fourthly, a terminal device is provided, including a memory and a processor, wherein the memory stores a computer program executable on the processor, and the processor executes the computer program to perform a method provided in the first aspect.

[0018] Fifthly, a communication device is provided, including a memory and a processor, wherein the memory stores a computer program executable on the processor, characterized in that the processor executes the computer program to perform a method provided in the first aspect.

[0019] Eighthly, a computer program product is provided, on which a computer program is stored, the computer program being executed by a processor to perform the method provided in the first aspect.

[0020] Compared with the prior art, the technical solution of this application embodiment has the following beneficial effects:

[0021] In this technical solution, for the image to be processed, a mapping curve can be determined based on the statistical values ​​of its low-frequency information in the luminance domain, and the luminance values ​​of each pixel in the image can be adjusted according to the mapping curve. Since the low-frequency information of the image in the luminance domain usually does not contain noise, luminance adjustment based on the low-frequency information in the luminance domain can suppress noise amplification while preserving the image's detail information.

[0022] Furthermore, in the mapping curve, when the original brightness value is lower than a first preset threshold, the adjusted brightness value corresponding to the original brightness value is higher than the original brightness value; when the original brightness value is higher than the first preset threshold, the adjusted brightness value corresponding to the original brightness value is lower than the original brightness value. The technical solution of this invention, through the above mapping curve, can achieve brightness enhancement in low-brightness areas and brightness suppression in high-brightness areas of the image, thereby ensuring a significant improvement in the overall dynamic range of the image while ensuring that the overall image noise does not increase significantly.

[0023] Furthermore, when the image to be processed is the current video frame, the target brightness value of each pixel is determined based on the original brightness value of each pixel in the current video frame and the mapping curve; the brightness difference between the current video frame and multiple video frames preceding the current video frame is calculated; and the adjusted brightness value of each pixel is determined based on the brightness difference and the target brightness value, wherein the adjusted brightness value is less than or equal to the target brightness value. This technical solution considers the brightness difference between the current video frame and multiple video frames preceding it when performing tone mapping on each video frame, resulting in a smooth transition of the mapping curve. This minimizes the brightness difference between adjacent video frames, avoids inter-frame flickering, and improves the user's video viewing experience. Attached Figure Description

[0024] Figure 1 This is a flowchart of an image processing method provided in an embodiment of this application;

[0025] Figure 2 This is a schematic diagram of a mapping curve provided in an embodiment of this application;

[0026] Figure 3 This is a schematic diagram of an image processing application scenario provided in an embodiment of this application;

[0027] Figure 4 This is a schematic diagram of the structure of an image processing device provided in an embodiment of this application. Detailed Implementation

[0028] As described in the background section, while existing technologies can improve the overall brightness of an image, they can also cause overexposure in bright areas and a significant increase in noise in low-brightness areas.

[0029] Furthermore, video possesses unique temporal characteristics and spatial content relationships. Simply applying the tone mapping method for each frame directly to video tone mapping will cause significant image quality problems, such as flickering between frames. Existing methods for improving image dynamic range mostly focus on tone mapping of single frames without considering temporal transformations between frames, which easily leads to inter-frame flickering.

[0030] In this technical solution, for the image to be processed, a mapping curve can be determined based on the statistical values ​​of its low-frequency information in the luminance domain, and the luminance values ​​of each pixel in the image can be adjusted according to the mapping curve. Since the low-frequency information of the image in the luminance domain usually does not contain noise, luminance adjustment based on the low-frequency information in the luminance domain can suppress noise amplification while preserving the image's detail information.

[0031] Furthermore, in the mapping curve, when the original brightness value is lower than a first preset threshold, the adjusted brightness value corresponding to the original brightness value is higher than the original brightness value; when the original brightness value is higher than the first preset threshold, the adjusted brightness value corresponding to the original brightness value is lower than the original brightness value. The technical solution of this invention, through the above mapping curve, can achieve brightness enhancement in low-brightness areas and brightness suppression in high-brightness areas of the image, thereby ensuring a significant improvement in the overall dynamic range of the image while ensuring that the overall image noise does not increase significantly.

[0032] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0033] This application provides an image processing method that can improve the brightness of low-brightness areas of an image and avoid overexposure of high-brightness areas.

[0034] See Figure 1 The image processing method provided in this application includes:

[0035] Step 101: Obtain the image to be processed.

[0036] Step 102: Calculate the statistical values ​​of low-frequency information of the image to be processed in the brightness domain.

[0037] Step 103: Determine the corresponding mapping curve based on the scene to which the statistical value belongs. The mapping curve represents the mapping relationship between the original brightness value and the adjusted brightness value of each pixel in the brightness domain. Different scenes correspond to different mapping curves.

[0038] Step 104: Determine the adjusted brightness value of each pixel in the image to be processed based on the mapping curve.

[0039] Step 105: Map the gain of the adjusted brightness value of each pixel to the corresponding color channel of the pixel to obtain the adjusted image.

[0040] It should be noted that the sequence number of each step in this embodiment does not represent a limitation on the execution order of each step.

[0041] It is understood that, in specific implementations, the image processing method can be implemented using software programs, which run within a processor integrated into the chip or chip module. The method can also be implemented using a combination of software and hardware; this application does not impose any limitations on this approach.

[0042] In this embodiment of the invention, the image to be processed can be a picture, a photograph, or a video frame.

[0043] In the specific implementation of step 102, the low-frequency information of the image to be processed in the luminance domain is statistically analyzed, that is, the luminance values ​​of each pixel in the low-frequency image of the image to be processed in the luminance domain are statistically analyzed to obtain statistical values. These statistical values ​​can reflect the overall luminance of the image to be processed. Specifically, the statistical values ​​may include the luminance values ​​of low-brightness areas in the image, the luminance values ​​of high-brightness areas in the image, the logarithmic mean, and / or the arithmetic mean.

[0044] It should be noted that the low-brightness area in the image can be the area with a pixel value lower than the first preset threshold, and the bright area can be the area with a pixel value higher than the second preset threshold, wherein the second preset threshold is greater than or equal to the first preset threshold.

[0045] Specifically, a non-uniformly distributed histogram is performed on the low-frequency image to obtain the histogram data corresponding to the low-frequency image. The number of pixels (M1) in the low-brightness region is obtained according to the proportion of pixels in the entire image. The brightness value corresponding to the accumulation of M1 pixels from left to right in the histogram is the brightness value of the low-brightness region (also called the low brightness value). The number of pixels (M2) in the high-brightness region is obtained according to the proportion of pixels in the entire image. The brightness value corresponding to the accumulation of M2 pixels from right to left in the histogram is the brightness value of the high-brightness region (also called the high brightness value).

[0046] Specifically, the low-frequency information (also called low-frequency components) in the brightness domain of an image refers to the brightness values ​​of each pixel in areas of the image where brightness changes smoothly. The high-frequency information (also called high-frequency components) refers to the brightness values ​​of each pixel in areas of the image where brightness changes drastically, typically at the edges (or contours) of the image. High-frequency information in an image often contains noise. Therefore, this application performs statistical analysis on the low-frequency information to avoid the influence of noise.

[0047] In one specific embodiment, when the image to be processed is a color image, such as an RGB image, in order to obtain the low-frequency information of the image in the luminance domain, it is necessary to convert the image to be processed into a luminance map. For example, the maximum value of the R, G, and B channels corresponding to each pixel of the high-bit RGB image is taken as the luminance value of the current pixel, thereby obtaining the luminance map corresponding to the current image. Then, bilateral filtering is performed on the luminance map to obtain the low-frequency and high-frequency images of the image to be processed in the luminance domain.

[0048] More specifically, it could be a bilateral filtering process performed on the Luma map corresponding to the image to be processed.

[0049] In the specific implementation of step 103, a mapping curve can be determined based on the scene to which the statistical value of the low-frequency information of the image to be processed belongs in the brightness domain. This mapping curve is used to adjust the brightness value of each pixel in the low-frequency image of the image to be processed in the brightness domain.

[0050] Specifically, please refer to Figure 2 , Figure 2 A mapping curve is shown, which represents the mapping relationship between the original brightness value and the adjusted brightness value of each pixel in the brightness domain. Figure 2 The horizontal axis of the coordinate system in which the mapping curve is located represents the original low-frequency brightness value of the pixel, and the vertical axis represents the adjusted low-frequency brightness value of the pixel (i.e., the adjusted brightness value).

[0051] In one specific implementation, statistical values ​​can be input into a mapping model to obtain a mapping curve corresponding to the scenario to which the statistical values ​​belong. The mapping model is pre-trained.

[0052] Specifically, the mapping model can be trained using pre-labeled training data, which may include the scene to which the statistical value belongs and its corresponding mapping curve. Specifically, the scene to which the statistical value belongs can be determined based on the presence of a face in the image and the dynamic range level. For example, the scene to which the statistical value belongs may include: face present, no face present, low dynamic range level I, low dynamic range level II, high dynamic range level I, high dynamic range level II, high dynamic range level III, etc. After inputting the scene to which the statistical value belongs into the mapping model, the mapping model can output the corresponding mapping curve. The mapping curve corresponding to different scenes has different brightness adjustment ranges for pixels.

[0053] Then, in step 104, the adjusted brightness value of each pixel in the image to be processed is determined according to the mapping curve. Specifically, for each pixel, its original brightness value is searched on the mapping curve, and the adjusted brightness value corresponding to the original brightness value found on the mapping curve is the adjusted brightness value of that pixel.

[0054] Since the pixel brightness adjustment in step 104 is performed in the luminance domain, step 105 also requires mapping the gain of the adjusted brightness value of each pixel to the corresponding color channels of the pixel to obtain the adjusted image. The adjusted image is a color image, such as an RGB image, where the color channels of the pixels are the R channel, G channel, and B channel, respectively.

[0055] The embodiments of this application can compress image brightness while preserving image details and color information. By using dynamic adaptive tone mapping technology, high dynamic range images and video data are mapped into images and videos that are compatible with low bit width devices, enabling high dynamic range images or videos to be better compatible with low bit width display devices.

[0056] In a non-limiting embodiment of this application, in the mapping curve, when the original brightness value is lower than a first preset threshold, the adjusted brightness value corresponding to the original brightness value is higher than the original brightness value; when the original brightness value is higher than the first preset threshold, the adjusted brightness value corresponding to the original brightness value is lower than the original brightness value.

[0057] Continue to refer to Figure 2 For the mapping curve S, when the original brightness value is lower than the first preset threshold T1, the adjusted brightness value corresponding to the original brightness value on the mapping curve S is higher than the original brightness value; correspondingly, when the original brightness value is higher than the first preset threshold T1, the adjusted brightness value corresponding to the original brightness value is lower than the original brightness value. Therefore, adjusting the brightness value of each pixel using the curve achieves the effect of increasing the brightness value of each pixel in the low-brightness area and suppressing the brightness value of each pixel in the high-brightness area.

[0058] Furthermore, in the mapping curve, when the original brightness value is lower than the second preset threshold, the original brightness value and its corresponding adjusted brightness value have a linear relationship and the adjusted brightness value is greater than or equal to the original brightness value, and the second preset threshold is less than the first preset threshold.

[0059] Continue to refer to Figure 2 When the original brightness value of a pixel is lower than the second preset threshold Ymin, the brightness of the low-brightness area can be linearly increased according to the set low-brightness control parameters. That is, the adjusted brightness value is higher than or equal to the original brightness value, and the two are linearly related, thereby controlling the gain of the low-brightness area. This setting is to avoid excessive increase in noise caused by excessive brightness increase of pixels with low brightness.

[0060] Furthermore, when determining the mapping curve in step 103, the mapping curve can be determined first based on the mapping model, and then adjusted to obtain the final mapping curve. During the adjustment of the mapping curve, the proportion of each brightness range in the entire image can also be considered to ensure that the image contrast is not excessively reduced. Specifically, histogram data can be analyzed, the histogram can be analyzed for brightness range proportions, and the histograms can be accumulated to obtain a histogram equalization curve. Weighting this equalization curve with the mapping curve yields the final mapping curve, thus ensuring an improvement in the overall dynamic range of the image while reducing image contrast loss.

[0061] Specifically, the image to be processed is partitioned according to brightness, for example, divided into 128 equal intervals, i.e., 0 to 127. With pixel brightness data in 10-bit format, the 10-bit brightness value is first right-shifted by 3 bits to obtain the corresponding 7-bit value, i.e., the value from 0 to 127, and the corresponding interval is then located. The histogram is then divided into 10 histogram intervals according to the brightness range, and the proportion of pixels in each brightness interval relative to the total number of pixels in the image is calculated. This proportion is part of a statistical value. This proportion, along with the ambient light brightness (BV) from Auto Exposure (AE), is input into a mapping model to obtain the corresponding mapping curve. The mapping model is trained on a large amount of data previously.

[0062] Histogram data is accumulated across the entire image to be processed. Specifically, the mapping value corresponding to the first histogram interval on the mapping curve is the original histogram value, and the mapping value corresponding to the second histogram interval on the mapping curve is the mapping value obtained by accumulating the first and second histogram intervals. This process is repeated to obtain the corresponding histogram equalization mapping curve.

[0063] The histogram equalization mapping curve is weighted and combined with the previously obtained global brightening mapping curve. Specifically, multiple control point positions (such as a first preset threshold and a second preset threshold) are set across the entire brightness range to control the weighting of the histogram equalization mapping curve and the global brightening mapping curve in different brightness ranges, ultimately obtaining the ideal mapping curve (e.g., ...). Figure 2 (The mapping curve shown).

[0064] In a non-limiting embodiment of this application, a different scheme is needed when mapping each frame in a video compared to single-frame image mapping. Video tone mapping maps the tone of consecutive frames. The biggest difference between video tone mapping and single-frame tone mapping is that the temporal relationship between frames needs to be considered; otherwise, considering only the tone mapping of a single frame will cause inter-frame flickering.

[0065] In this embodiment of the invention, when the image to be processed is the current video frame, the target brightness value of each pixel is determined based on the original brightness value and mapping curve of each pixel in the current video frame; the brightness difference between the current video frame and multiple video frames located before the current video frame is calculated; and the adjusted brightness value of each pixel is determined based on the brightness difference and the target brightness value, so that the adjusted brightness value can progressively approach the target brightness value.

[0066] In practice, when the brightness difference between the current video frame and several previous video frames reaches a preset threshold, to avoid inter-frame flicker, the actual brightness adjustment value needs to progressively approach the target brightness value when determining the pixel's adjusted brightness value. That is, when the brightness difference reaches the preset threshold, the adjusted brightness value progressively approaches the target brightness value. When the brightness difference is below the preset threshold, it means the brightness difference between the current video frame and several previous video frames is small; in this case, the adjusted brightness value can be equal to the target brightness value.

[0067] This application embodiment limits the degree of brightness adjustment of the current frame and gradually changes to the target brightness in multiple frames. This ensures that the change in the mapping curve between the current frame and the previous frame is not too large, so that the human eye can see the video more naturally and improve the viewing experience.

[0068] In a non-limiting embodiment of this application, high-frequency information of each pixel in the high-frequency image is added to the adjusted brightness value of each pixel to map the adjusted brightness value gain of each pixel onto each color channel of the current pixel to obtain the adjusted image.

[0069] This application, after adjusting the brightness of each pixel in the low-frequency image of the image to be processed (i.e., performing tone mapping), needs to add back the detailed information of the high-frequency image to the adjusted low-frequency image. This process preserves the texture details of the image and avoids excessive noise amplification.

[0070] Furthermore, after adding back the high-frequency information, the added luminance image needs to be combined with the chrominance image to convert it into a color image. For example, the added luminance image is combined with the chrominance image to convert it into an RGB image.

[0071] In a non-limiting embodiment of this application, in the specific implementation of step 105, when the gain of adjusting the brightness value of each pixel is mapped to the corresponding color channel of the pixel, in order to avoid abnormal saturation of the mapped image, the color channels can also be adjusted.

[0072] Specifically, the adjustment gain is calculated for each original brightness value, where the adjustment gain is the ratio of the adjusted brightness value to the original brightness value. The adjustment gain of each pixel is mapped to the corresponding color channel of that pixel, and the adjustment gain of each pixel is used to weight each color channel of that pixel to obtain the adjusted image.

[0073] Taking an RGB image as an example, this application calculates the corresponding adjustment gain for each brightness value in the brightness map. Specifically, the adjustment gain can be the ratio of the adjusted brightness value to the original brightness value. This value is then mapped to the R, G, and B channels, thereby ensuring that the gain factors of the R, G, and B values ​​corresponding to the same brightness are consistent.

[0074] In one specific embodiment, after obtaining the adjusted image, the image can also undergo certain noise reduction processing, distortion correction processing, saturation adjustment processing, and color gamut conversion processing, and finally be displayed on the display interface.

[0075] In a specific application scenario of this application, the image to be processed is an RGB image, which will be discussed below. Figure 3 The image processing procedure is described in detail.

[0076] Light information is projected onto the photosensitive area of ​​a photosensitive element, such as a charge-coupled device (CCD) or complementary metal-oxide-semiconductor (CMOS), through the lens of an image capture device. The photosensitive element converts the light signal into an electrical signal through photoelectric conversion, thereby obtaining a high-bit Bayer format raw image. The raw image is then preprocessed, including noise reduction, dead pixel correction, and color interpolation. Finally, the preprocessed high-bit Bayer format image is interpolated using a demosaicing method to obtain a high-bit RGB domain image.

[0077] The maximum value of the R, G, and B values ​​corresponding to each pixel in the high-bit RGB image is used as the luminance value of the current pixel, thus obtaining the luminance map (e.g., luma map) and chrominance map corresponding to the current image. A bilateral filter is applied to the luminance map to obtain a low-frequency image (i.e., the LumaBase image) and a high-frequency image (i.e., the LumaDetail image). Then, a non-uniformly distributed histogram is performed on the low-frequency image to obtain various parameters reflecting image information. These parameters are input to the image processing control unit, which adaptively obtains the image mapping curve.

[0078] When performing image processing on video, the image control processing unit needs to combine the parameters of the previous frame with the mapping curve of the current frame in order to limit the degree of transformation of the current frame.

[0079] After performing bilateral filtering on the Luma map corresponding to the high-bit RGB input image and tone mapping on the obtained low-frequency image, the high-frequency image detail information obtained from the bilateral filtering of the Luma map is added back to the mapped image. After adding the high-frequency information, the mapped image with the added high-frequency information is combined with the chromaticity map and transformed to the RGB color gamut. During the transformation to the RGB color gamut, the corresponding luminance gain can be calculated for each luminance value in the luminance map, and this value is mapped to the R, G, and B channels, so that the gain factors of the R, G, and B values ​​corresponding to the same luminance are consistent.

[0080] For more specific implementations of the embodiments of this application, please refer to the foregoing embodiments, which will not be repeated here.

[0081] Please refer to Figure 4 , Figure 4 An image processing apparatus 40 is shown, which may include:

[0082] Acquisition module 401 is used to acquire the image to be processed;

[0083] The statistics module 402 is used to calculate the statistical values ​​of the low-frequency information of the image to be processed in the brightness domain;

[0084] The mapping curve determination module 403 is used to determine the corresponding mapping curve according to the scene to which the statistical value belongs. The mapping curve represents the mapping relationship between the original brightness value and the adjusted brightness value of each pixel in the brightness domain. Different scenes correspond to different mapping curves.

[0085] Brightness adjustment module 404 is used to determine the adjusted brightness value of each pixel in the image to be processed according to the mapping curve;

[0086] The conversion module 405 is used to map the gain of the adjusted brightness value of each pixel to the corresponding color channel of the pixel to obtain the adjusted image.

[0087] In this embodiment, for the image to be processed, a mapping curve can be determined based on the statistical values ​​of its low-frequency information in the luminance domain, and the luminance values ​​of each pixel in the image can be adjusted according to the mapping curve. Since the low-frequency information of an image in the luminance domain usually does not contain noise, luminance adjustment based on the low-frequency information in the luminance domain can suppress noise amplification while preserving the image's detail information.

[0088] This application can adaptively improve the dynamic range of an image, preserving the texture details of low-brightness areas while suppressing overexposure in high-brightness areas, and also achieving a good denoising effect to a certain extent.

[0089] In specific implementations, the aforementioned image processing device 40 may correspond to a chip with image processing function in a terminal device, such as a system-on-a-chip (SOC), a baseband chip, etc.; or to a chip module in a terminal device that includes a chip with image processing function; or to a chip module with a chip with data processing function; or to a terminal device.

[0090] In a non-limiting embodiment of this application, the brightness adjustment module 404 may include:

[0091] The target brightness value determination unit is used to determine the target brightness value of each pixel based on the original brightness value and the mapping curve of each pixel in the current video frame when the image to be processed is the current video frame.

[0092] The brightness difference calculation unit is used to calculate the brightness difference between the current video frame and multiple video frames preceding the current video frame.

[0093] The adjustment unit determines the adjusted brightness value for each pixel based on the brightness difference and the target brightness value, ensuring that the difference between the adjusted brightness value and the target brightness value is less than a preset threshold. In other words, the adjusted brightness value progressively approaches the target brightness value.

[0094] The technical solution of this application takes into account the brightness difference between the current video frame and the previous video frames when performing tone mapping on each video frame, thereby making the brightness difference between adjacent video frames smaller, avoiding the problem of inter-frame flickering, and improving the user's video viewing experience.

[0095] In a non-limiting embodiment of this application, the mapping curve determination module 403 can input the statistical value into the mapping model to obtain the mapping curve corresponding to the scene to which the statistical value belongs. The mapping model is pre-trained. In a non-limiting embodiment of this application, the conversion module 405 may include:

[0096] An adjustment gain calculation unit is used to calculate the adjustment gain for each original brightness value, wherein the adjustment gain is the ratio of the adjusted brightness value to the original brightness value;

[0097] The weighted adjustment unit is used to map the gain of the adjusted brightness value of each pixel to each color channel of the corresponding pixel, and to weight each color channel of the pixel using the adjustment gain corresponding to each pixel to obtain the adjusted image.

[0098] In a non-limiting embodiment of this application, the statistics module 402 may include:

[0099] A filtering unit is used to perform bilateral filtering on the image to be processed to obtain the low-frequency image and high-frequency image of the image to be processed in the brightness domain;

[0100] The statistical unit is used to perform non-uniformly distributed histogram statistics on the low-frequency image to obtain the statistical value.

[0101] Furthermore, the statistics module 402 adds the high-frequency information of each pixel in the high-frequency image to the adjusted brightness value of each pixel, so as to map the gain of the adjusted brightness value of each pixel to each color channel of the corresponding pixel, thereby obtaining the adjusted image.

[0102] Regarding the modules / units included in the various devices and products described in the above embodiments, they can be software modules / units, hardware modules / units, or a combination of both. For example, for devices and products applied to or integrated into a chip, all modules / units can be implemented using hardware methods such as circuits, or at least some modules / units can be implemented using software programs running on a processor integrated within the chip, while the remaining (if any) modules / units can be implemented using hardware methods such as circuits. For devices and products applied to or integrated into a chip module, all modules / units can be implemented using hardware methods such as circuits. Different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or different components of the chip module, or at least some modules / units can be implemented using hardware methods such as circuits. The implementation is achieved through a software program that runs on a processor integrated within the chip module. The remaining modules / units (if any) can be implemented using hardware methods such as circuits. For various devices and products applied to or integrated into terminal equipment, each of their modules / units can be implemented using hardware methods such as circuits. Different modules / units can be located in the same component (e.g., chip, circuit module, etc.) or different components within the terminal equipment. Alternatively, at least some modules / units can be implemented using a software program that runs on a processor integrated within the terminal equipment, while the remaining modules / units (if any) can be implemented using hardware methods such as circuits.

[0103] This application also discloses a storage medium, which is a computer-readable storage medium storing a computer program. When the computer program is executed, it can perform the steps of the aforementioned image processing method. The storage medium may include read-only memory (ROM), random access memory (RAM), a magnetic disk, or an optical disk, etc. The storage medium may also include non-volatile memory or non-transitory memory, etc.

[0104] It should be understood that the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article indicates that the preceding and following related objects have an "or" relationship.

[0105] In the embodiments of this application, "multiple" refers to two or more.

[0106] The descriptions of "first," "second," etc., appearing in the embodiments of this application are for illustrative purposes and to distinguish the objects being described. They have no order and do not indicate any special limitation on the number of devices in the embodiments of this application, nor do they constitute any limitation on the embodiments of this application.

[0107] In this application embodiment, "connection" refers to various connection methods such as direct connection or indirect connection to realize communication between devices. This application embodiment does not limit this in any way.

[0108] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, as a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer program are loaded or executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means.

[0109] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0110] In the several embodiments provided in this application, it should be understood that the disclosed methods, apparatuses, and systems can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for example, the division of units is merely a logical functional division, and other division methods may exist in actual implementation; for example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0111] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0112] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can be physically included separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0113] The integrated unit implemented as a software functional unit described above can be stored in a computer-readable storage medium. This software functional unit, stored in a storage medium, includes several instructions to cause a computer device (which may be a personal computer, a server, or a network device, etc.) to execute some steps of the methods described in the various embodiments of this application.

[0114] While this application discloses the above information, it is not limited thereto. Any person skilled in the art can make various modifications and alterations without departing from the spirit and scope of this application; therefore, the scope of protection of this application shall be determined by the scope defined in the claims.

Claims

1. An image processing method, characterized in that, include: Obtain the image to be processed; Calculate the statistical values ​​of low-frequency information in the brightness domain of the image to be processed; The corresponding mapping curve is determined based on the scene to which the statistical value belongs. The mapping curve represents the mapping relationship between the original brightness value and the adjusted brightness value of each pixel in the brightness domain. Different scenes correspond to different mapping curves. The step of determining the corresponding mapping curve based on the scene to which the statistical value belongs includes: inputting the statistical value into a mapping model to determine the mapping curve, and obtaining the mapping curve corresponding to the mapping model; performing histogram data accumulation on the image to be processed to obtain the mapping curve corresponding to histogram equalization; and performing weighted processing on the mapping region corresponding to the mapping model and the mapping curve corresponding to histogram equalization to obtain the mapping curve corresponding to the scene to which the statistical value belongs. Determining the adjusted brightness value of each pixel in the image to be processed according to the mapping curve includes: when the image to be processed is the current video frame, determining the target brightness value of each pixel according to the original brightness value of each pixel in the current video frame and the mapping curve; calculating the brightness difference between the current video frame and multiple video frames located before the current video frame; determining the adjusted brightness value of each pixel according to the brightness difference and the target brightness value, wherein the difference between the adjusted brightness value and the target brightness value is less than a preset threshold; The gain of each pixel's adjusted brightness value is mapped to the corresponding color channel of that pixel to obtain the adjusted image.

2. The image processing method according to claim 1, characterized in that, In the mapping curve, when the original brightness value is lower than the first preset threshold, the adjusted brightness value corresponding to the original brightness value is higher than the original brightness value; when the original brightness value is higher than the first preset threshold, the adjusted brightness value corresponding to the original brightness value is lower than the original brightness value.

3. The image processing method according to claim 2, characterized in that, In the mapping curve, when the original brightness value is lower than the second preset threshold, the original brightness value and its corresponding adjusted brightness value have a linear relationship and the adjusted brightness value is greater than the original brightness value. The second preset threshold is less than the first preset threshold.

4. The image processing method according to claim 1, characterized in that, The step of mapping the gain of each pixel's adjusted brightness value to each color channel of the corresponding pixel includes: Calculate the adjustment gain for each original brightness value, where the adjustment gain is the ratio of the adjusted brightness value to the original brightness value; The adjustment gain of each pixel is mapped to the corresponding color channel of the pixel to obtain the adjusted image.

5. The image processing method according to claim 1, characterized in that, The image to be processed is a color image, and the statistical values ​​for calculating the low-frequency information of the image to be processed in the brightness domain include: Bilateral filtering is performed on the image to be processed to obtain the low-frequency image and high-frequency image of the image to be processed in the brightness domain; The low-frequency image is subjected to non-uniformly distributed histogram statistics to obtain the statistical values.

6. The image processing method according to claim 5, characterized in that, The step of mapping the gain of each pixel's adjusted brightness value to the corresponding pixel before each color channel also includes: The high-frequency information of each pixel in the high-frequency image is added to the adjusted brightness value of each pixel to convert it into the color channel of each pixel.

7. An image processing apparatus, characterized in that, include: The acquisition module is used to acquire the image to be processed; The statistics module is used to calculate the statistical values ​​of low-frequency information of the image to be processed in the brightness domain; The mapping curve determination module is used to determine the corresponding mapping curve based on the scene to which the statistical value belongs. The mapping curve represents the mapping relationship between the original brightness value and the adjusted brightness value of each pixel in the brightness domain. Different scenes correspond to different mapping curves. The step of determining the corresponding mapping curve based on the scene to which the statistical value belongs includes: inputting the statistical value into a mapping model to determine the mapping curve, and obtaining the mapping curve corresponding to the mapping model; performing histogram data accumulation on the image to be processed to obtain the mapping curve corresponding to histogram equalization; and performing weighted processing on the mapping region corresponding to the mapping model and the mapping curve corresponding to histogram equalization to obtain the mapping curve corresponding to the scene to which the statistical value belongs. A brightness adjustment module is used to determine the adjusted brightness value of each pixel in the image to be processed according to the mapping curve, including: when the image to be processed is the current video frame, determining the target brightness value of each pixel according to the original brightness value of each pixel in the current video frame and the mapping curve; calculating the brightness difference between the current video frame and multiple video frames located before the current video frame; determining the adjusted brightness value of each pixel according to the brightness difference and the target brightness value, wherein the difference between the adjusted brightness value and the target brightness value is less than a preset threshold; The conversion module maps the gain of the adjusted brightness value of each pixel to the corresponding color channel of the pixel to obtain the adjusted image.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program, when run by a processor, performs the steps of the image processing method according to any one of claims 1 to 6.

9. A terminal device, comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, characterized in that, When the processor runs the computer program, it performs the steps of the image processing method according to any one of claims 1 to 6.

Images