Image processing method, device, electronic device and storage medium

Abstract

The present application relates to an image processing method, device, electronic device, and storage medium. The image processing method includes: converting an original image into a light domain and extracting a brightness channel to obtain a first brightness value, where the original image is an HDR image; performing a PQ transformation on the first brightness value to obtain a second brightness value in the PQ domain; the calculation formula of the PQ transformation is #imgabs0#L HDR,PQ is the second brightness value, L light1 is the first brightness value, and γ1 is the PQ transformation constant; according to the equation of the global mapping curve, the second brightness value is converted to a third brightness value to obtain a first intermediate image; the first intermediate image is converted from the PQ domain to the optical domain to obtain a second intermediate image; according to the color coordinates of the second intermediate image and the color conversion matrix, the original color domain is converted to the target color domain to obtain a target image; the target image is an SDR image. The technical solution of this application can provide high visual quality SDR image output for HDR images with different brightness conditions, and is hardware-friendly and computationally low.

Description

[0001] This application is a divisional application of the Chinese patent application submitted to the State Intellectual Property Office on October 23, 2023, with application number 202311377991.0 and invention name "Image processing method, device, electronic device and storage medium". Technical Field

[0002] The present invention relates to the field of image processing technology, and in particular to an image processing method, device, electronic equipment and storage medium. Background Art

[0003] High dynamic range (HDR) technology has emerged as a new revolution in digital media and has recently been adopted by the industry as a new standard for capturing, transmitting, and displaying video content. However, since most existing commercial displays cannot display true HDR image content, backward compatibility with these legacy displays is a very important issue. Over the years, the industry has proposed several tone mapping operators (TMOs) to convert HDR image formats to standard dynamic range (SDR) image formats. With the recent development of SDR displays, the need for video TMOs has become critical.

[0004] Most advanced TMOs in the related art are suitable for specific types of input content, either considering normal (average brightness) content in HDR format images or focusing on dark and / or bright content. These have certain limitations and cannot provide high-quality visual quality output for different brightness levels, nor can they maintain the artistic impression of the original HDR image content. In addition, current algorithms are not hardware-friendly and computationally intensive. Therefore, how to overcome the above-mentioned technical shortcomings of converting HDR format images to SDR format images is a technical problem that needs to be solved. Summary of the Invention

[0005] The purpose of this application is to provide an image processing method, device, electronic device and storage medium, which can convert the entire HDR image into an SDR image, provide high-quality visual quality output for HDR images with different brightness conditions, and be hardware-friendly and computationally small.

[0006] According to a first aspect of an embodiment of the present application, there is provided an image processing method, comprising:

[0007] Converting the original image into a light domain and extracting a brightness channel for each pixel to obtain a first brightness value, wherein the original image is a high dynamic range (HDR) image;

[0008] The first brightness value is subjected to a perceptual quantization PQ transformation to obtain a corresponding second brightness value in the PQ domain; wherein the calculation formula of the PQ transformation is: Among them, LHDR,PQ is the second brightness value, L light1 is the first brightness value, γ1 is the PQ transformation constant;

[0009] converting the second brightness value into a third brightness value according to a specified global mapping curve equation to obtain a first intermediate image; wherein the third brightness value is a brightness value of a brightness channel of the first intermediate image;

[0010] Converting the first intermediate image from the PQ domain to the optical domain to obtain a second intermediate image;

[0011] According to the color coordinates of the second intermediate image and the color conversion matrix, the original color gamut is converted to the target color gamut to obtain a target image; wherein the original color gamut is the color gamut of the original image, the target color gamut is the color gamut of the target image, and the target image is a standard dynamic range (SDR) image.

[0012] In one embodiment, the γ1 is 4.

[0013] In one embodiment, converting the first intermediate image from the PQ domain to the optical domain to obtain a second intermediate image includes:

[0014] The first intermediate image is converted from the PQ domain to the optical domain using an inverse PQ transform formula to obtain the second intermediate image, wherein the inverse PQ transform formula is: L SDR,PQ is the third brightness value, L light2 is the fourth luminance value of the luminance channel of the pixel of the second intermediate image, and γ2 is the inverse PQ transformation constant.

[0015] In one embodiment, the γ2 is 0.25.

[0016] In one embodiment, after converting the second luminance value into a third luminance value according to the equation of the specified global mapping curve to obtain a first intermediate image and before converting the first intermediate image from the PQ domain to the optical domain to obtain a second intermediate image, the method further includes:

[0017] For each color channel, perform color adjustment on the first intermediate image according to a specified color adjustment equation, keeping the color hue unchanged and the brightness substantially unchanged, to obtain a third intermediate image;

[0018] After obtaining the third intermediate image, the third intermediate image is converted from the PQ domain to the optical domain to obtain the second intermediate image.

[0019] In one embodiment, the color adjustment equation is

[0020] Among them, C SDR,PQ is the color saturation of any color channel of the third intermediate image in the PQ domain, C HDR,PQ is the color saturation of any color channel of the first intermediate image in the PQ domain, L HDR,PQ is the second brightness value, L SDR,PQ is the third brightness value, α and β are two constants responsible for controlling the hue and color saturation of the color.

[0021] In one embodiment, the global mapping curve is a piecewise linear curve, including two endpoints, two dividing points, and three linear curve segments, the three linear curve segments forming a continuous curve, the two endpoints and the two dividing points dividing the original image into a dark area, a normal area, and a bright area; the three linear curve segments correspond one-to-one to the dark area, the normal area, and the bright area;

[0022] The converting the second brightness value into a third brightness value according to a specified global mapping curve equation to obtain a first intermediate image includes:

[0023] Determining the two endpoints and the two dividing points according to the equation of the global mapping curve;

[0024] Dividing the original image into a dark area, a normal area, and a bright area according to the two endpoints and the two dividing points;

[0025] For each region of the original image, the second brightness value is converted into a corresponding third brightness value according to an equation of a global mapping curve to obtain the first intermediate image.

[0026] In one embodiment, the equation of the global mapping curve is:

[0027]

[0028] Among them, L HDR,PQ is the second brightness value, L SDR,PQ is the third brightness value, s1, s2 and s3 are the slopes of the three linear curves, a1, a2 and a3 are the intercepts of the three linear curves, x1 and x2 are the horizontal coordinates of the two dividing points, HDR min,PQ is the minimum value of the second brightness value, HDR max,PQ is the maximum value of the second brightness value.

[0029] In one embodiment, x1 is calculated using the following formula:

[0030] x1=P1×(HDR max,PQ -HDR min,PQ )+HDR min,PQ ;

[0031] x2 is calculated by the following formula:

[0032] x2=HDR max,PQ -P2×(HDR max,PQ -HDR min,PQ );

[0033] Among them, x1 is the horizontal coordinate of the dividing point between the dark area and the normal area, x2 is the horizontal coordinate of the dividing point between the normal area and the bright area, and P1 and P2 are two constants.

[0034] In one embodiment, P1 is 0.15 and P2 is 0.4.

[0035] In one embodiment, s1, s2, s3, a1, a2, and a3 are calculated as follows:

[0036] Determine the values ​​of y1 and y2 corresponding to x1 and x2 in the SDR domain;

[0037] Obtaining the coordinates of the two endpoints;

[0038] According to x1, x2, y1, y2 and the coordinates of the two endpoints, the values ​​of s1, s2, s3, a1, a2 and a3 are calculated.

[0039] In one embodiment, the values ​​of y1 and y2 are calculated using the following formula:

[0040] y1=P3×(SDR max,PQ -SDR min,PQ )+SDR min,PQ ;

[0041] y2=P4×(SDR max,PQ -SDR min,PQ )+SDR min,PQ ;

[0042] Among them, P3 and P4 are two constants, SDR min,PQ is the minimum luminance value of the target SDR display in the PQ domain, SDR max,PQ is the maximum luminance value of the target SDR display in the PQ domain.

[0043] In one embodiment, P3 is 0.1369 and P4 is 0.7518.

[0044] In one embodiment, the global mapping curve is a gamma curve, and the equation of the gamma curve is as follows:

[0045]

[0046] Among them, LSDR,PQ is the third brightness value, L HDR,PQ is the second brightness value, c1 and c2 are constants;

[0047] The calculation formula for c2 is as follows:

[0048]

[0049] The calculation formula of c1 is as follows:

[0050]

[0051] Among them, HDR min,PQ is the minimum value of the second brightness value, HDR max,PQ is the maximum value of the second brightness value, SDR min,PQ is the minimum luminance value of the target SDR display in the PQ domain, SDR max,PQ is the maximum luminance value of the target SDR display in the PQ domain.

[0052] According to a second aspect of an embodiment of the present application, there is provided an image processing apparatus, including:

[0053] a first acquisition module configured to convert an original image into a light domain and extract a brightness channel for each pixel to obtain a first brightness value, wherein the original image is a high dynamic range (HDR) image;

[0054] The second acquisition module is configured to perform a perceptual quantization PQ transformation on the first brightness value to obtain a corresponding second brightness value in the PQ domain; wherein the calculation formula of the PQ transformation used is: Among them, L HDR,PQ is the second brightness value, L light1 is the first brightness value, γ1 is the PQ transformation constant;

[0055] a first conversion module configured to convert the second brightness value into a third brightness value according to a specified global mapping curve equation to obtain a first intermediate image; the third brightness value is a brightness value of a brightness channel of the first intermediate image;

[0056] a second conversion module, configured to convert the first intermediate image from the PQ domain to an optical domain to obtain a second intermediate image;

[0057] The third conversion module is configured to convert from the original color gamut to the target color gamut according to the color coordinates and the color conversion matrix of the second intermediate image to obtain a target image; wherein the original color gamut is the color gamut of the original image, the target color gamut is the color gamut of the target image, and the target image is a standard dynamic range (SDR) image.

[0058] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising a memory and a processor, wherein the memory is used to store a computer program executable by the processor; and the processor is used to execute the computer program in the memory to implement the above method.

[0059] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which a computer program is stored, characterized in that when the executable computer program in the storage medium is executed by a processor, the above method can be implemented.

[0060] Compared with the prior art, the beneficial effect of the present application is that: since the original image (HDR image) is converted to the light domain, and the brightness channel is extracted for each pixel to obtain a first brightness value, and then the first brightness value is subjected to perceptual quantization (PQ) conversion to obtain a corresponding second brightness value in the PQ domain, that is, the HDR image is converted to the perceptual domain, thus helping to maintain the overall visual impression of the frame. Moreover, the calculation formula of the PQ conversion adopted is It is hardware-friendly and can be easily implemented with limited resources. It has a small amount of computation and can reduce computing costs.

[0061] Then, according to the equation of the specified global mapping curve, the second brightness value of each pixel is converted to a corresponding third brightness value to obtain a first intermediate image, wherein the third brightness value is the brightness value of the brightness channel of the first intermediate image. On the one hand, since a global mapping curve is used rather than a local mapping curve, the brightness of each pixel of the entire original image can be converted, that is, the brightness conversion is performed on all areas of different brightness levels in the HDR image, rather than only performing local brightness conversion on areas of a certain type of specific brightness level. Therefore, the entire HDR image can be converted into an SDR image, providing high-quality visual quality output for HDR images of different brightness conditions. On the other hand, since the brightness of the entire original image is adjusted as a whole using the global mapping curve, rapid changes in brightness that may cause flicker can be avoided, thereby avoiding flicker artifacts.

[0062] Then, the first intermediate image is converted from the PQ domain to the optical domain to obtain the second intermediate image. According to the color coordinates of the second intermediate image and the color conversion matrix, it is converted from the original color domain to the target color domain to obtain the target image (SDR image).

[0063] To sum up, the technical solution provided by this application can convert the entire HDR image into an SDR image, providing high-quality visual quality output for HDR images with different brightness conditions. Moreover, it is hardware-friendly and can be easily implemented with limited resources, with small computational complexity, which can reduce computing costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 The figure is a flowchart of an image processing method according to an exemplary embodiment.

[0065] Figure 2 is a schematic diagram showing a gamma curve according to an exemplary embodiment.

[0066] Figure 3 is a flowchart of an image processing method according to another exemplary embodiment.

[0067] Figure 4 is a schematic diagram of a piecewise linear curve according to an exemplary embodiment.

[0068] Figure 5 is a flowchart of an image processing method according to another exemplary embodiment.

[0069] Figure 6 is a block diagram of an image processing apparatus according to an exemplary embodiment.

[0070] Figure 7 is a block diagram of an image processing apparatus according to another exemplary embodiment.

[0071] Figure 8 It is a block diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0072] Unless otherwise defined, the technical or scientific terms used in this specification and claims shall have the ordinary meanings understood by persons having ordinary skills in the technical field to which the invention belongs. Specific embodiments of the present invention will be described below in conjunction with the accompanying drawings. It should be noted that in the specific description of these embodiments, for the sake of clarity and clarity, this specification cannot provide a detailed description of all features of the actual embodiments. Without departing from the spirit and scope of the present invention, those skilled in the art may modify and replace the embodiments of the present invention, and the resulting embodiments are also within the scope of protection of the present invention.

[0073] In the related art, applying traditional TMO directly to HDR video is not an effective solution due to the generation of visual artifacts such as flickering, brightness, and color inconsistencies. On the other hand, TMO for generating high-quality SDR content is complex and requires manual intervention to set various parameters within the implementation method, making it unsuitable for real-time, hardware-friendly implementation. Another challenge with existing methods is the inability to preserve the artistic impression of the original HDR content. Maintaining the overall visual impression of the input HDR content is crucial when converting to SDR format.

[0074] To address the above technical issues, this application proposes an image processing method, device, electronic device, and storage medium that can convert an entire HDR image into an SDR image, providing high-quality visual output for HDR images with different brightness conditions. It can also avoid issues such as flickering artifacts, brightness and color inconsistencies, etc. generated during the image format conversion process. Furthermore, the method is hardware-friendly and can be easily implemented with limited resources, with low computational effort, thus reducing computational costs.

[0075] Figure 1 This is a flowchart of an image processing method according to an exemplary embodiment. The image processing method is performed in the RGB color space and can be applied to electronic devices with image processing functions such as image processors, display chips, and displays. Figure 1 , the image processing method may include the following steps:

[0076] Step 101 : converting an original image into a light domain, and extracting a brightness channel for each pixel to obtain a first brightness value, wherein the original image is a high dynamic range (HDR) image.

[0077] In this step, the input original image (HDR image) is converted from the HDR domain to the optical domain, that is, the HDR format encoding value of the original image is converted into grayscale values. Then, the brightness channel is extracted for each pixel to obtain the first brightness value.

[0078] In this embodiment, the color gamut of the original image, that is, the original color gamut, is the BT.2020 color gamut. According to the BT.2020 standard, the formula for extracting the brightness channel from the red, green, and blue color channels is as follows:

[0079] L light1 =0.2627R+0.6780G+0.0593B;

[0080] Among them, R is the red channel, G is the green channel, B is the blue channel, L light1 is the first brightness value.

[0081] In related technologies, there are two HDR formats: HLG10 or HDR10. The difference between these two formats lies in their different transfer functions. The HDR10 format uses the PQ function, while the HLG10 format uses the Hybrid Log-Gamma (HLG) function.

[0082] In this embodiment, when the format of the original image is HDR10, the original image is converted into the optical domain using an inverse PQ transformation function; when the format of the original image is HLG10, the original image is converted into the optical domain using an inverse HLG transformation function.

[0083] Step 102: Perform a perceptual quantization (PQ) transformation on the first brightness value to obtain a corresponding second brightness value in the PQ domain. The calculation formula of the PQ transformation is: Among them, L HDR,PQ is the second brightness value, L light1 is the first brightness value, and γ1 is the PQ transformation constant.

[0084] The PQ (Perceptual Quantizer) domain is a method of encoding and compressing video signals. The PQ domain belongs to the perceptual domain and is a perceptually uniform color space designed to more closely match the way the human visual system perceives color. The PQ domain uses a logarithmic transfer function, which allows for more efficient use of the available dynamic range. This makes it ideal for displaying high dynamic range (HDR) video content.

[0085] Perceptual Quantization (PQ) aims to optimize the light intensity distribution associated with the HVS properties, converting physically linear values ​​into perceptually linear values. Perceptual linearity means that the human eye sees any intensity change at any brightness level in the same way (i.e., two consecutive brightness levels are just below the threshold that the HVS just notices). Perceptual linearity is fundamental to the implementation because it allows us to identify what the human eye can see and map that information to a higher dynamic range in the best way. Therefore, we first transfer the first luminance value of the luminance channel of the input HDR image to the PQ domain, which allows the luminance values ​​of the HDR image and the luminance values ​​of the SDR image to be in the same domain during the mapping process. This allows us to work in the perceptual domain, where adjacent luminance values ​​correspond to differences that are perceptible to the human eye.

[0086] In this embodiment, the first luminance value is subjected to a PQ transformation to obtain a corresponding second luminance value in the PQ domain.

[0087] In this embodiment, a perceptual quantizer may be used to perform a PQ transform on the first luminance value. A perceptual quantizer is a transfer function that maps the linear light intensity of a display device into a nonlinear signal to better match the characteristics of the human visual system (HVS). The perceptual quantizer optimizes the distribution of light intensity to most efficiently utilize the available dynamic range and provide the best reproduction of the original image.

[0088] In related technologies, the calculation formula for PQ transformation is:

[0089]

[0090] Among them, L light1 is the first brightness value, L HDR,PQ is the second brightness value. c1, c2, c3, m1, and m2 are constants that can be set to 0.8359, 18.8515, 18.6875, 0.1593, and 78.8437, respectively.

[0091] However, the calculation formula of the above-mentioned PQ transform has a high computational cost when implemented in hardware and requires a large amount of resources to implement.

[0092] In order to overcome the above challenges, in this embodiment, the calculation formula of the PQ transform used is:

[0093]

[0094] Among them, L HDR,PQ is the second brightness value, L light1 is the first brightness value, and γ1 is the PQ transformation constant.

[0095] In this embodiment, the calculation formula of the PQ transform is simple, has few parameters, does not require manual intervention to set various parameters, has a small amount of calculation, and has a relatively low calculation cost when implemented in hardware. It is hardware-friendly, can be easily implemented with limited resources, and can be extended to real-time applications.

[0096] In this embodiment, γ1 is 4. According to extensive simulation experiments, setting γ1 to 4 achieves high visual quality results in both subjective and objective evaluations.

[0097] Step 103 : Convert the second brightness value into a third brightness value according to the specified global mapping curve equation to obtain a first intermediate image; the third brightness value is the brightness value of the brightness channel of the first intermediate image.

[0098] Flicker is a common problem that can occur during tone mapping, especially when using a local tone mapping algorithm. This is because the local algorithm independently adjusts the brightness of a single pixel or a small group of pixels, which can cause rapid changes in brightness across the entire image. These brightness changes can be particularly noticeable in high-contrast areas and can be distracting and visually jarring. To avoid this problem, the global mapping algorithm in this application (the equation for the global mapping curve) uses a different approach. Instead of adjusting the brightness of a single pixel, it takes the average of the brightness values ​​for the entire image. Because the brightness of the entire image is adjusted as a whole, this approach avoids rapid changes in brightness that can cause flicker.

[0099] In this embodiment, the global mapping curve can be determined according to the curve identifier input by the user. For example, if the curve identifier is 00, the specified global mapping curve is a gamma curve, and if the curve identifier is 01, the specified global mapping curve is a piecewise linear curve.

[0100] In this embodiment, the global mapping curve is a gamma curve, and the equation of the gamma curve is as follows:

[0101]

[0102] Among them, L SDR,PQ is the third brightness value, L HDR,PQ is the second brightness value, c1 and c2 are constants.

[0103] In this embodiment, c2 can be calculated using the following formula:

[0104]

[0105] Then, calculate c1 using the following formula:

[0106]

[0107] Among them, HDR min,PQ is the minimum value of the second brightness value, HDR max,PQ is the maximum value of the second brightness value, SDR min,PQ is the minimum luminance value of the target SDR display in the PQ domain, SDR max,PQ is the maximum luminance value of the target SDR display in the PQ domain.

[0108] In this embodiment, the gamma curve C1 is as follows: Figure 2 shown.

[0109] In this embodiment, the global mapping curve uses a gamma curve, which has a simple mathematical form, low computational complexity, and can generate high-quality SDR images. Furthermore, the generated SDR image has more details in brighter areas than in dark areas. If you need to generate an SDR image with more details in brighter areas than in dark areas, using a gamma curve is recommended.

[0110] In this embodiment, the mapping curve is constructed by taking into account the maximum and minimum luminance values ​​of the target SDR display. This enables the image processing method to generate content that looks pleasing on all types of SDR displays. Therefore, the algorithm provided in this application is display-adaptive and can generate SDR image content that matches the capabilities of any target SDR display. Therefore, the TMO proposed in this article is display-adaptive and can generate SDR content that matches the capabilities of any target display.

[0111] Step 104 : Convert the first intermediate image from the PQ domain to the optical domain to obtain a second intermediate image.

[0112] In this embodiment, the first intermediate image needs to be converted from the PQ domain to the optical domain to display the image content.

[0113] In related art, the following inverse PQ transformation formula is used to convert the first intermediate image from the PQ domain to the optical domain:

[0114]

[0115] Among them, L light2 is the fourth brightness value of the brightness channel of the pixel in the second intermediate image.

[0116] However, the calculation formula of the above-mentioned inverse PQ transform is very expensive to implement in hardware and requires a lot of resources to implement. In order to overcome the above challenges, in this embodiment, the calculation formula of the inverse PQ transform is:

[0117]

[0118] Among them, L SDR,PQ is the third brightness value, L light2 is the fourth luminance value of the luminance channel of the pixel of the second intermediate image, and γ2 is the inverse PQ transformation constant.

[0119] In this embodiment, the calculation formula of the inverse PQ transform is simple, has few parameters, and has a small amount of calculation. The calculation cost of implementing it in hardware is relatively low, it is hardware-friendly, and can be easily implemented with limited resources.

[0120] In this embodiment, γ2 is 0.25. According to extensive simulation experiments, setting γ2 to 0.25 achieves high visual quality results in both subjective and objective evaluations.

[0121] Step 105, converting from the original color gamut to the target color gamut according to the color coordinates of the second intermediate image and the color conversion matrix to obtain a target image; wherein the original color gamut is the color gamut of the original image, the target color gamut is the color gamut of the target image, and the target image is a standard dynamic range (SDR) image.

[0122] In this embodiment, the color gamut of the target image, i.e., the target color gamut, is BT.709. Since BT.709 is specifically designed for SDR images, converting the image to BT.709 can ensure that the generated content displays correct colors.

[0123] Converting an image from the BT.2020 color gamut to the BT.709 color gamut involves converting the color coordinates of a given image or video from one color space to another.

[0124] In one embodiment, a color gamut conversion can be performed using a 3*3 color conversion matrix that maps colors in the source color space (BT.2020) to colors in the target color space (BT.709). The process typically begins by converting the image or video from the RGB color space to the native color representation of the color space. The color conversion matrix is ​​then applied to the color coordinates of the image to convert them to the new color space. The matrix for converting from the BT.2020 color gamut to the BT.709 color gamut can be expressed as:

[0125] R709=T11×R2020+T12×G2020+T13×B2020

[0126] G709=T21×R2020+T22×G2020+T23×B2020

[0127] B709=T31×R2020+T32×G2020+T33×B2020

[0128] Where R, G, and B represent the red, green, and blue channels, respectively. R709, G709, and B709 are the color coordinates of the target image in the BT.709 color gamut, and R2020, G2020, and B2020 are the color coordinates of the second intermediate image in the BT.2020 color gamut. T11, T12, T13, T21, T22, T23, T31, T32, and T33 are conversion coefficients.

[0129] It should be noted that this step is quite complex and requires very high precision. This color conversion matrix is ​​only an approximation. If you want to perform accurate conversion, you need to use a three-dimensional color lookup table (3D-CLUT).

[0130] In this embodiment, the original image (HDR image) is converted to the optical domain, and the brightness channel is extracted for each pixel to obtain a first brightness value. Then, the first brightness value is subjected to perceptual quantization (PQ) conversion to obtain a corresponding second brightness value in the PQ domain, that is, the HDR image is converted to the perceptual domain. Therefore, it helps to maintain the overall visual impression of the frame. Moreover, the calculation formula of the PQ conversion is: It is hardware-friendly and can be easily implemented with limited resources. It has a small amount of computation and can reduce computing costs.

[0131] Then, according to the equation of the specified global mapping curve, the second brightness value of each pixel is converted to a corresponding third brightness value to obtain a first intermediate image, wherein the third brightness value is the brightness value of the brightness channel of the first intermediate image. On the one hand, since a global mapping curve is used rather than a local mapping curve, the brightness of each pixel of the entire original image can be converted, that is, the brightness conversion is performed on all areas of different brightness levels in the HDR image, rather than only performing local brightness conversion on areas of a certain type of specific brightness level. Therefore, the entire HDR image can be converted into an SDR image, providing high-quality visual quality output for HDR images of different brightness conditions. On the other hand, since the brightness of the entire original image is adjusted as a whole using the global mapping curve, rapid changes in brightness that may cause flicker can be avoided, thereby avoiding flicker artifacts.

[0132] Then, the first intermediate image is converted from the PQ domain to the optical domain to obtain the second intermediate image. According to the color coordinates of the second intermediate image and the color conversion matrix, it is converted from the original color domain to the target color domain to obtain the target image (SDR image).

[0133] In summary, the technical solution provided by this application can convert the entire HDR image into an SDR image, providing high-quality visual quality output for HDR images with different brightness conditions. Moreover, it is hardware-friendly and can be easily implemented with limited resources, with small computational complexity, which can reduce computing costs.

[0134] Figure 3 This is a flowchart illustrating an image processing method according to another exemplary embodiment. Unlike the above embodiment, in this embodiment, after converting the second luminance value to a third luminance value according to a specified global mapping curve equation to obtain a first intermediate image, and before converting the first intermediate image from the PQ domain to the optical domain to obtain a second intermediate image, color adjustment is performed on the first intermediate image according to a specified color adjustment equation for each color channel to maintain the color hue and brightness substantially unchanged. See [see ]. Figure 3 , the image processing method may include the following steps:

[0135] Step 301 : converting the original image into a light domain, and extracting a brightness channel for each pixel to obtain a first brightness value, wherein the original image is a high dynamic range (HDR) image.

[0136] This step is similar to the above-mentioned step 101 and will not be described again here.

[0137] Step 302: Perform a perceptual quantization (PQ) transformation on the first luminance value to obtain a corresponding second luminance value in the PQ domain. The PQ transformation formula used is: Among them, L HDR,PQ is the second brightness value, L light1 is the first brightness value, and γ1 is the PQ transformation constant.

[0138] This step is similar to the above-mentioned step 102 and will not be described again here.

[0139] Step 303 : Convert the second brightness value into a third brightness value according to the specified global mapping curve equation to obtain a first intermediate image; the third brightness value is the brightness value of the brightness channel of the first intermediate image.

[0140] In this embodiment, the global mapping curve is a piecewise linear curve. Figure 4As shown, the global mapping curve C2 includes two endpoints D1 and D2, two dividing points D3 and D4, and three linear curves C21, C22, and C23. The three linear curves C21, C22, and C23 form a continuous curve. The two endpoints D1 and D2 and the two dividing points D3 and D4 divide the original image into a dark area, a normal area, and a bright area. The three linear curves correspond one-to-one to the dark area, the normal area, and the bright area. Specifically, linear curve C21 corresponds to the dark area, linear curve C22 corresponds to the normal area, and linear curve C23 corresponds to the bright area. x1 and x2 are the horizontal coordinates of the two dividing points D3 and D4, y1 and y2 are the vertical coordinates of the two dividing points D3 and D4, and s1, s2, and s3 are the slopes of the three linear curves C21, C22, and C23, respectively. In other embodiments, the number of segments in the global mapping curve can be expanded to more than three.

[0141] In this embodiment, the equation of the global mapping curve is as follows:

[0142]

[0143] Among them, L HDR,PQ is the second brightness value, L SDR,PQ is the third brightness value, s1, s2 and s3 are the slopes of the three linear curves, a1, a2 and a3 are the intercepts of the three linear curves, x1 and x2 are the horizontal coordinates of the two dividing points, HDR min,PQ is the minimum value of the second brightness value, HDR max,PQ is the maximum value of the second brightness value.

[0144] In this embodiment, the aforementioned piecewise linear curve is used as the equation of the designated global mapping curve to convert the second luminance value into a third luminance value, thereby obtaining a first intermediate image. The specific method for performing global mapping is as follows: First, based on the equation C2 of the global mapping curve, two endpoints D1 and D2 and two demarcation points D3 and D4 are determined. Then, based on the two endpoints D1 and D2 and the two demarcation points D3 and D4, the original image is divided into a dark region, a normal region, and a bright region. Next, for each region of the original image, the second luminance value is converted into a corresponding third luminance value based on the equation of the global mapping curve. For the dark region of the original image, the second luminance value is converted into a corresponding third luminance value based on the linear curve C21; for the normal region of the original image, the second luminance value is converted into a corresponding third luminance value based on the linear curve C22; and for the bright region of the original image, the second luminance value is converted into a corresponding third luminance value based on the linear curve C23. This helps convert each region of the HDR image into its corresponding region in the SDR domain while maintaining the overall artistic impression. Using a piecewise linear curve can make the output image more balanced, thereby achieving the best balance between the overall brightness and contrast of the image.

[0145] In this embodiment, the method for generating the piecewise linear curve is as follows:

[0146] (1) Determine the segmentation method.

[0147] In this step, first, the values ​​of x1 and x2 are calculated to segment the image, and the original image is divided into a dark area, a normal area, and a bright area.

[0148] x1 is calculated by the following formula:

[0149] x1=P1×(HDR max,PQ -HDR min,PQ )+HDR min,PQ ;

[0150] x2 is calculated by the following formula:

[0151] x2=HDR max,PQ -P2×(HDR max,PQ -HDR min,PQ );

[0152] Among them, x1 is the horizontal coordinate of the dividing point between the dark area and the normal area, x2 is the horizontal coordinate of the dividing point between the normal area and the bright area, and P1 and P2 are two constants.

[0153] In this embodiment, P1 is 0.15 and P2 is 0.4. According to extensive research, setting P1 and P2 to 0.15 and 0.4 respectively can achieve good results in segmenting the HDR image into three brightness regions.

[0154] After calculating the values ​​of x1 and x2, determine the corresponding values ​​y1 and y2 of x1 and x2 in the SDR domain. The values ​​of y1 and y2 can be calculated using the following formula:

[0155] y1=P3×(SDR max,PQ -SDR min,PQ )+SDR min,PQ ;

[0156] y2=P4×(SDR max,PQ -SDR min,PQ )+SDR min,PQ ;

[0157] Among them, P3 and P4 are two constants, SDR min,PQ is the minimum luminance value of the target SDR display in the PQ domain, SDR max,PQ is the maximum luminance value of the target SDR display in the PQ domain.

[0158] In this embodiment, P3 is 0.1369 and P4 is 0.7518. According to extensive simulations, setting P3 and P4 to 0.1369 and 0.7518 respectively will produce an SDR image with good visual quality.

[0159] Next, the coordinates of the two endpoints D1 and D2 are obtained.

[0160] Next, the values ​​of s1, s2, s3, a1, a2, and a3 are calculated based on the coordinates of x1, x2, y1, y2, and the two endpoints D1 and D2.

[0161] (2) Generate piecewise linear curve

[0162] In this embodiment, the above-mentioned piecewise linear curve is generated according to the values ​​of s1, s2, s3, a1, a2 and a3, that is, the equation of the global mapping curve C2 is obtained.

[0163] In this embodiment, the proposed TMO works in the perceptual domain to take into account the sensitivity of the human eye to changes in brightness in different regions of the scene (i.e., dark, normal, and bright). This helps the TMO process each region differently based on how the eye perceives them.

[0164] To preserve the overall impression of the original HDR image content, the proposed TMO uses a novel segmentation method to divide the HDR image into dark, normal, and bright regions. This not only preserves the overall brightness of each region but also enables the TMO to process all types of HDR images, a problem that most existing TMOs cannot solve.

[0165] Step 304 : For each color channel, perform color adjustment on the first intermediate image according to a specified color adjustment equation, keeping the color hue unchanged and the brightness substantially unchanged, to obtain a third intermediate image.

[0166] A key consideration in designing an efficient TMO is maintaining color accuracy between the input HDR image and the generated SDR image. This is a challenge because color shifts can occur when attempting to compress brightness levels from HDR to SDR. Therefore, an effective color adjustment scheme is crucial when designing a TMO. The goal of a color adjustment method is to maintain color hue during the transformation while making changes in color brightness negligible.

[0167] In this embodiment, to ensure the color accuracy of the generated image, the color adjustment equation used is as follows:

[0168]

[0169] Among them, C SDR,PQ is the color saturation of any color channel (red, green, or blue) of the third intermediate image in the PQ domain, C HDR,PQ is the color saturation of any color channel of the first intermediate image in the PQ domain, L HDR,PQ is the second brightness value, L SDR,PQ is the third brightness value, α and β are two constants responsible for controlling the hue and color saturation of the color.

[0170] In this embodiment, since color adjustment is performed according to a specified color adjustment equation, the color hue is kept unchanged and the brightness is kept basically unchanged, so that the color accuracy between the input HDR image and the output SDR image can be guaranteed.

[0171] Additionally, based on extensive simulations, the above color adjustment equation preserves the hue of colors while having a negligible impact on the brightness when mapping an HDR image to an SDR image.

[0172] In this embodiment, by performing color adjustment in the perceptual domain, the image processing method preserves the color hue of the HDR image and generates the color of the SDR image that is very close to the color of the HDR image.

[0173] Step 305: Convert the third intermediate image from the PQ domain to the optical domain to obtain a second intermediate image.

[0174] This step is similar to the above-mentioned step 104 and will not be described again here.

[0175] Step 306: Convert the original color gamut to the target color gamut based on the color coordinates of the second intermediate image and the color conversion matrix to obtain a target image; wherein the original color gamut is the color gamut of the original image, the target color gamut is the color gamut of the target image, and the target image is a standard dynamic range (SDR) image.

[0176] This step is similar to the above-mentioned step 105 and will not be described again here.

[0177] In this embodiment, after converting the second luminance value to a third luminance value according to a specified global mapping curve equation to obtain a first intermediate image, and before converting the first intermediate image from the PQ domain to the optical domain to obtain a second intermediate image, color adjustment is performed on each color channel of the first intermediate image according to a specified color adjustment equation to maintain the color hue and brightness substantially unchanged. By performing color adjustment in the perceptual domain, the image processing method preserves the color hue of the HDR image and generates an SDR image with colors that closely resemble those of the HDR image, while simultaneously making changes in color brightness negligible during the color adjustment process.

[0178] Figure 5 5 is a flowchart of an image processing method according to another exemplary embodiment, and is also a block diagram of the principle of the TMO proposed in this article. The main part of the method is the part surrounded by the dotted box 51. Figure 5 , the image processing method may include the following steps:

[0179] (1) Extract the brightness channel

[0180] This step is the same as step 101 and will not be repeated here.

[0181] (2) PQ transform

[0182] This step is the same as step 102 and will not be repeated here.

[0183] (3) Select the mapping curve

[0184] In this embodiment, the mapping curve is a global mapping curve, which includes a gamma curve and a piecewise linear curve. The electronic device can select a mapping curve based on a curve identifier input by the user. For example, if the user inputs a curve identifier of 00, the global mapping curve is designated as a gamma curve; if the user inputs a curve identifier of 01, the global mapping curve is designated as a piecewise linear curve.

[0185] (4) When the user inputs the curve identifier of the gamma curve, the gamma curve is generated and globally mapped.

[0186] In this embodiment, when the curve identifier input by the user is a curve identifier of a gamma curve, a gamma curve is generated and global mapping is performed.

[0187] (5) When the user inputs the curve identifier of the piecewise linear curve, determine the curve segmentation method

[0188] In this embodiment, when the curve identifier input by the user is a curve identifier of a piecewise linear curve, a curve segmentation method is determined.

[0189] This step is the same as the content of determining the curve segmentation method in the above step 303, and will not be repeated here.

[0190] (6) Generate piecewise linear curves and perform global mapping

[0191] In this embodiment, after the curve segmentation method is determined, a piecewise linear curve is generated and global mapping is performed.

[0192] This step is the same as the global mapping method in step 303 above, and will not be described again here.

[0193] (7) Color adjustment

[0194] This step is the same as step 304 and will not be repeated here.

[0195] (8) Reverse PQ conversion

[0196] This step is the same as step 305 and will not be repeated here.

[0197] (9) Inverse color gamut conversion

[0198] This step is the same as step 306 and will not be repeated here.

[0199] In this embodiment, an HDR image can be converted into a standard dynamic range (SDR) image to achieve high visual quality at all brightness levels. Moreover, it is hardware-friendly and can be implemented in hardware with limited resources.

[0200] Figure 6 FIG. 1 is a block diagram of an image processing apparatus according to an exemplary embodiment. Figure 6 As shown, in this embodiment, the image processing device includes:

[0201] a first acquisition module 61 configured to convert the original image into a light domain and extract a brightness channel for each pixel to obtain a first brightness value, wherein the original image is a high dynamic range (HDR) image;

[0202] The second acquisition module 62 is configured to perform a perceptual quantization PQ transformation on the first brightness value to obtain a corresponding second brightness value in the PQ domain; wherein the calculation formula of the PQ transformation used is: Among them, L HDR,PQ is the second brightness value, Llight1 is the first brightness value, γ1 is the PQ transformation constant;

[0203] A first conversion module 63 is configured to convert the second brightness value into a third brightness value according to a specified global mapping curve equation to obtain a first intermediate image; the third brightness value is a brightness value of a brightness channel of the first intermediate image;

[0204] A second conversion module 64 is configured to convert the first intermediate image from the PQ domain to the optical domain to obtain a second intermediate image;

[0205] The third conversion module 65 is configured to convert from the original color gamut to the target color gamut according to the color coordinates and the color conversion matrix of the second intermediate image to obtain a target image; wherein the original color gamut is the color gamut of the original image, the target color gamut is the color gamut of the target image, and the target image is a standard dynamic range SDR image.

[0206] In one embodiment, Figure 7 As shown, the image processing device further includes:

[0207] A color adjustment module 66 is configured to perform color adjustment on the first intermediate image according to a specified color adjustment equation for each color channel, keeping the color hue unchanged and the brightness substantially unchanged, to obtain a third intermediate image;

[0208] The second conversion module 64 is further configured to convert the third intermediate image from the PQ domain to the optical domain to obtain a second intermediate image.

[0209] An embodiment of the present application further proposes an electronic device comprising a processor and a memory; the memory is used to store a computer program executable by the processor; and the processor is used to execute the computer program in the memory to implement the image processing method of any of the above embodiments.

[0210] The embodiments of the present application further provide a computer-readable storage medium, which can implement the image processing method of any of the above embodiments when the executable computer program in the storage medium is executed by a processor.

[0211] Regarding the apparatus in the above embodiment, the specific manner in which the processor performs operations has been described in detail in the embodiment of the method, and will not be elaborated on here.

[0212] Figure 8 8 is a block diagram of an electronic device according to an exemplary embodiment. For example, the electronic device 800 can be provided as a server. Figure 8The device 800 includes a processing component 822, which further includes one or more processors, and a memory resource represented by a memory 832 for storing instructions, such as applications, that can be executed by the processing component 822. The application stored in the memory 832 can include one or more modules, each corresponding to a set of instructions. In addition, the processing component 822 is configured to execute the instructions to perform the above-mentioned method for image processing.

[0213] The device 800 may also include a power supply component 826 configured to perform power management of the device 800, a wired or wireless network interface 850 configured to connect the device 800 to a network, and an input / output (I / O) interface 858. The device 800 may operate based on an operating system stored in the memory 832, such as Windows Server™, MacOS X™, Unix™, Linux™, FreeBSD™, or the like.

[0214] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 832 including instructions, which can be executed by the processing component 822 of the device 800 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0215] In the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance. The term "plurality" refers to two or more, unless otherwise clearly defined.

[0216] The above description of the embodiments is intended to facilitate understanding and application of the present application by those skilled in the art. It will be apparent that those skilled in the art can readily make various modifications to these embodiments and apply the general principles described herein to other embodiments without expending any creative effort. Therefore, the present application is not limited to the embodiments described herein, and improvements and modifications made by those skilled in the art based on the disclosure of this application without departing from the scope and spirit of this application are within the scope of this application.

Claims

1. An image processing method, characterized in that: include: Converting the original image into a light domain and extracting a brightness channel for each pixel to obtain a first brightness value, wherein the original image is a high dynamic range (HDR) image; The first brightness value is subjected to a perceptual quantization PQ transformation to obtain a corresponding second brightness value in the PQ domain; wherein the calculation formula of the PQ transformation is: Among them, L HDR,PQ is the second brightness value, L light1 is the first brightness value, γ1 is the PQ transformation constant; converting the second brightness value into a third brightness value according to a specified global mapping curve equation to obtain a first intermediate image; the third brightness value is a brightness value of a brightness channel of the first intermediate image; wherein the global mapping curve is a three-segment linear continuous curve; The first intermediate image is converted from the PQ domain to the optical domain using an inverse PQ transform formula to obtain a second intermediate image. The inverse PQ transform formula is: L SDR,PQ is the third brightness value, L light2 is the fourth luminance value of the luminance channel of the pixel of the second intermediate image, γ2 is an inverse PQ transformation constant, and γ2 is 0.25; According to the color coordinates of the second intermediate image and the color conversion matrix, the original color gamut is converted to the target color gamut to obtain a target image; wherein the original color gamut is the color gamut of the original image, the target color gamut is the color gamut of the target image, and the target image is a standard dynamic range (SDR) image.

2. An image processing device, characterized in that include: a first acquisition module configured to convert an original image into a light domain and extract a brightness channel for each pixel to obtain a first brightness value, wherein the original image is a high dynamic range (HDR) image; The second acquisition module is configured to perform a perceptual quantization PQ transformation on the first brightness value to obtain a corresponding second brightness value in the PQ domain; wherein the calculation formula of the PQ transformation used is: Among them, L HDR,PQ is the second brightness value, L light1 is the first brightness value, γ1 is the PQ transformation constant; a first conversion module configured to convert the second brightness value into a third brightness value according to a specified global mapping curve equation to obtain a first intermediate image; the third brightness value is a brightness value of a brightness channel of the first intermediate image; wherein the global mapping curve is a three-segment linear continuous curve; The second conversion module is configured to convert the first intermediate image from the PQ domain to the optical domain using an inverse PQ conversion formula to obtain a second intermediate image. The inverse PQ conversion formula is: L SDR,PQ is the third brightness value, L light2 is the fourth luminance value of the luminance channel of the pixel of the second intermediate image, γ2 is an inverse PQ transformation constant, and γ2 is 0.25; The third conversion module is configured to convert from the original color gamut to the target color gamut according to the color coordinates and the color conversion matrix of the second intermediate image to obtain a target image; wherein the original color gamut is the color gamut of the original image, the target color gamut is the color gamut of the target image, and the target image is a standard dynamic range (SDR) image.

3. An electronic device, characterized in that: The system comprises a memory and a processor, wherein the memory is used to store a computer program executable by the processor; and the processor is used to execute the computer program in the memory to implement the method according to claim 1.

4. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the executable computer program in the storage medium is executed by a processor, the method according to claim 1 can be implemented.

Images