Image processing method and device, electronic equipment and storage medium

Abstract

This application relates to an image processing method, apparatus, electronic device, and storage medium. The image processing method includes: converting an original image to the optical domain and extracting a luminance channel to obtain a first luminance value; wherein the original image is an HDR image; performing a PQ transform on the first luminance value to obtain a second luminance value in the PQ domain; the calculation formula for the PQ transform is L... HDR,PQ L is the second brightness value. light1 Let γ1 be the first luminance value and γ1 be the PQ transform constant. Based on the equation of the global mapping curve, the second luminance value is converted to a third luminance value to obtain a first intermediate image. The first intermediate image is then transformed from the PQ domain to the optical domain to obtain a second intermediate image. Based on the color coordinates and color transformation matrix of the second intermediate image, the color space is transformed from the original color space to the target color space to obtain the target image. The target image is an SDR image. The technical solution of this application can provide high-quality SDR image output for HDR images under different luminance conditions, and is hardware-friendly with low computational cost.

Description

[0001] This application is a divisional application of Chinese Patent Application No. 202311377991.0, filed with the Chinese Patent Office on October 23, 2023, entitled "Image Processing Method, Apparatus, Electronic Device and Storage Medium". Technical Field

[0002] This invention relates to the field of image processing technology, and more specifically to an image processing method, apparatus, electronic device, and storage medium. Background Technology

[0003] High Dynamic Range (HDR) technology has emerged as a new revolution in digital media and has recently been adopted by the industry as a new standard for capturing, transmitting, and displaying video content. However, since most existing commercial displays cannot display true HDR image content, backward compatibility between HDR image content and these traditional displays is a very important topic. Over the years, the industry has proposed several tone mapping operators (TMOs) to convert HDR image formats to standard Dynamic Range (SDR) image formats. With the latest developments in SDR displays, the need for video TMOs has become crucial.

[0004] In related technologies, most advanced TMOs are suitable for specific types of input content, either considering normal (average brightness) content in HDR format images or focusing on dark and / or bright content. This has limitations, failing to provide high-quality visual output for different brightness levels and unable to maintain the artistic impression of the original HDR image content. Furthermore, current algorithms are not hardware-friendly and computationally intensive. Therefore, overcoming these technical shortcomings in converting HDR format images to SDR format images is a technical problem that needs to be solved. Summary of the Invention

[0005] The purpose of this application is to provide an image processing method, apparatus, electronic device, and storage medium that can convert an entire HDR image into an SDR image, provide high-quality visual output for HDR images under different brightness conditions, and is hardware-friendly with low computational requirements.

[0006] According to a first aspect of the embodiments of this application, an image processing method is provided, comprising:

[0007] The original image is converted to the optical domain, and the luminance channel is extracted for each pixel to obtain the first luminance value, wherein the original image is a high dynamic range (HDR) image;

[0008] The first luminance value is subjected to perceptual quantization (PQ) transform to obtain the corresponding second luminance value in the PQ domain; wherein the calculation formula for the PQ transform is as follows: Among them, LHDR,PQ L is the second brightness value. light1 γ1 is the first brightness value, and γ1 is the PQ transformation constant;

[0009] According to the equation of the specified global mapping curve, the second brightness value is converted into a third brightness value to obtain a first intermediate image; the third brightness value is the brightness value of the brightness channel of the first intermediate image.

[0010] The first intermediate image is converted from the PQ domain to the optical domain to obtain the second intermediate image;

[0011] Based on the color coordinates and color transformation matrix of the second intermediate image, the original color gamut is transformed to the target color gamut to obtain the target image; wherein, the original color gamut is the color gamut of the original image, the target color gamut is the color gamut of the target image, and the target image is a standard dynamic range (SDR) image.

[0012] In one implementation, γ1 is 4.

[0013] In one embodiment, converting the first intermediate image from the PQ domain to the optical domain to obtain a second intermediate image includes:

[0014] The first intermediate image is transformed from the PQ domain to the optical domain using the inverse PQ transform formula to obtain the second intermediate image, wherein the inverse PQ transform formula is: L SDR,PQ L is the third brightness value. light2 γ is the fourth luminance value of the luminance channel of the pixel in the second intermediate image, and γ2 is the inverse PQ transform constant.

[0015] In one embodiment, γ2 is 0.25.

[0016] In one implementation, after converting the second luminance value to a third luminance value according to the equation of the specified global mapping curve to obtain a first intermediate image, and before converting the first intermediate image from the PQ domain to the optical domain to obtain a second intermediate image, the method further includes:

[0017] For each color channel, the first intermediate image is color-adjusted according to the specified color adjustment equation, while keeping the hue and brightness of the colors essentially unchanged, to obtain the third intermediate image;

[0018] After obtaining the third intermediate image, the third intermediate image is converted from the PQ domain to the optical domain to obtain the second intermediate image.

[0019] In one embodiment, the color adjustment equation is:

[0020] Among them, C SDR,PQ C represents the color saturation of any color channel of the third intermediate image in the PQ domain. HDR,PQ L represents the color saturation of any color channel of the first intermediate image in the PQ domain. HDR,PQ L is the second brightness value. SDR,PQ The third brightness value is α, and β are two constants that control the hue and saturation of the color.

[0021] In one implementation, the global mapping curve is a piecewise linear curve, including two endpoints, two boundary points, and three linear curve segments. The three linear curve segments form a continuous curve. The two endpoints and the two boundary points divide the original image into dark areas, normal areas, and bright areas. The three linear curve segments correspond one-to-one with the dark areas, normal areas, and bright areas.

[0022] The step of converting the second brightness value into a third brightness value according to the equation of the specified global mapping curve to obtain the first intermediate image includes:

[0023] Based on the equation of the global mapping curve, determine the two endpoints and the two boundary points;

[0024] Based on the two endpoints and the two dividing points, the original image is divided into a dark area, a normal area, and a bright area;

[0025] For each region of the original image, the second brightness value is converted into the corresponding third brightness value according to the equation of the global mapping curve to obtain the first intermediate image.

[0026] In one implementation, the equation of the global mapping curve is:

[0027]

[0028] Among them, L HDR,PQ L is the second brightness value. SDR,PQ The third brightness value is given, where s1, s2, and s3 are the slopes of the three linear curves, a1, a2, and a3 are the intercepts of the three linear curves, and x1 and x2 are the x-coordinates of the two dividing points. HDR min,PQ The minimum value of the second brightness value, HDR max,PQ This is the maximum value of the second brightness value.

[0029] In one implementation, x1 is calculated using the following formula:

[0030] x1=P1×(HDR max,PQ -HDR min,PQ )+HDR min,PQ ;

[0031] x2 is calculated using the following formula:

[0032] x2 = HDR max,PQ -P2×(HDR max,PQ -HDR min,PQ );

[0033] Where x1 is the x-coordinate of the boundary between the dark area and the normal area, x2 is the x-coordinate of the boundary between the normal area and the bright area, and P1 and P2 are two constants.

[0034] In one implementation, P1 is 0.15 and P2 is 0.4.

[0035] In one implementation, s1, s2, s3, a1, a2, and a3 are calculated as follows:

[0036] Determine the values ​​of y1 and y2 corresponding to x1 and x2 in the SDR domain;

[0037] Obtain the coordinates of the two endpoints;

[0038] Based on x1, x2, y1, y2 and the coordinates of the two endpoints, the values ​​of s1, s2, s3, a1, a2 and a3 are calculated.

[0039] In one implementation, the values ​​of y1 and y2 are calculated using the following formula:

[0040] y1=P3×(SDR max,PQ -SDR min,PQ )+SDR min,PQ ;

[0041] y2=P4×(SDR max,PQ -SDR min,PQ )+SDR min,PQ ;

[0042] Where P3 and P4 are two constants, SDR min,PQ The minimum brightness value of the target SDR display in the PQ domain, SDR max,PQ This represents the maximum brightness value of the target SDR display in the PQ domain.

[0043] In one implementation, P3 is 0.1369 and P4 is 0.7518.

[0044] In one implementation, the global mapping curve is a gamma curve, and the equation of the gamma curve is as follows:

[0045]

[0046] Among them, LSDR,PQ L is the third brightness value. HDR,PQ Here, c1 and c2 are constants, representing the second brightness value.

[0047] The formula for calculating c2 is as follows:

[0048]

[0049] The formula for calculating c1 is as follows:

[0050]

[0051] Among them, HDR min,PQ The minimum value of the second brightness value, HDR max,PQ The maximum value of the second brightness value, SDR min,PQ The minimum brightness value of the target SDR display in the PQ domain, SDR max,PQ This represents the maximum brightness value of the target SDR display in the PQ domain.

[0052] According to a second aspect of the embodiments of this application, an image processing apparatus is provided, comprising:

[0053] The first acquisition module is configured to convert the original image to the optical domain and extract the luminance channel for each pixel to obtain a first luminance value, wherein the original image is a high dynamic range (HDR) image;

[0054] The second acquisition module is configured to perform a perceptual quantization (PQ) transform on the first brightness value to obtain a second brightness value corresponding to the PQ domain; wherein the calculation formula for the PQ transform is as follows: Among them, L HDR,PQ L is the second brightness value. light1 γ1 is the first brightness value, and γ1 is the PQ transformation constant;

[0055] The first conversion module is configured to convert the second brightness value into a third brightness value according to the equation of a specified global mapping curve to obtain a first intermediate image; the third brightness value is the brightness value of the brightness channel of the first intermediate image.

[0056] The second conversion module is configured to convert the first intermediate image from the PQ domain to the optical domain to obtain the second intermediate image;

[0057] The third conversion module is configured to convert from the original color gamut to the target color gamut based on the color coordinates and color conversion matrix of the second intermediate image to obtain the target image; wherein the original color gamut is the color gamut of the original image, the target color gamut is the color gamut of the target image, and the target image is a standard dynamic range (SDR) image.

[0058] According to a third aspect of the embodiments of this application, an electronic device is provided, including a memory and a processor, wherein the memory is used to store a computer program executable by the processor; and the processor is used to execute the computer program in the memory to implement the method described above.

[0059] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided, on which a computer program is stored, characterized in that, when the executable computer program in the storage medium is executed by a processor, the above-described method can be implemented.

[0060] Compared with existing technologies, the advantages of this application are as follows: By converting the original image (HDR image) to the optical domain and extracting the luminance channel for each pixel to obtain a first luminance value, and then performing perceptual quantization (PQ) transformation on the first luminance value to obtain the corresponding second luminance value in the PQ domain, i.e., converting the HDR image to the perceptual domain, it helps to maintain the overall visual impression of the frame. Furthermore, the calculation formula for the PQ transformation used is... It is hardware-friendly, can be easily implemented with limited resources, requires little computation, and can reduce computing costs.

[0061] Then, according to the equation of the specified global mapping curve, the second luminance value of each pixel is converted into the corresponding third luminance value to obtain the first intermediate image, where the third luminance value is the luminance value of the luminance channel of the first intermediate image. On the one hand, since a global mapping curve is used instead of a local mapping curve, the luminance of each pixel in the entire original image can be converted, that is, the luminance of all regions with different luminance levels in the HDR image is converted, rather than only the luminance of a certain type of specific luminance level is converted locally. Therefore, it is possible to convert the entire HDR image into an SDR image, providing high-quality visual output for HDR images under different luminance conditions. On the other hand, since the luminance of the entire original image is adjusted as a whole using a global mapping curve, rapid changes in luminance that may cause flicker can be avoided, thereby avoiding flicker artifacts.

[0062] Then, the first intermediate image is converted from the PQ domain to the optical domain to obtain the second intermediate image. Based on the color coordinates of the second intermediate image and the color conversion matrix, the image is converted from the original color domain to the target color domain to obtain the target image (SDR image).

[0063] In summary, the technical solution provided in this application can convert an entire HDR image into an SDR image, providing high-quality visual output for HDR images under different brightness conditions. Moreover, it is hardware-friendly, can be easily implemented with limited resources, and requires little computation, thus reducing computational costs. Attached Figure Description

[0064] Figure 1 This is a flowchart illustrating an image processing method according to an exemplary embodiment.

[0065] Figure 2 This is a schematic diagram of a gamma curve according to an exemplary embodiment.

[0066] Figure 3 This is a flowchart illustrating an image processing method according to another exemplary embodiment.

[0067] Figure 4 This is a schematic diagram illustrating a piecewise linear curve according to an exemplary embodiment.

[0068] Figure 5 This is a flowchart illustrating an image processing method according to another exemplary embodiment.

[0069] Figure 6 This is a block diagram illustrating an image processing apparatus according to an exemplary embodiment.

[0070] Figure 7 This is a block diagram illustrating an image processing apparatus according to another exemplary embodiment.

[0071] Figure 8 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Detailed Implementation

[0072] Unless otherwise defined, the technical or scientific terms used in this specification and claims shall have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. Specific embodiments of the invention will be described below with reference to the accompanying drawings. It should be noted that, in order to provide a concise description, this specification cannot provide a detailed description of all features of the actual embodiments. Without departing from the spirit and scope of the invention, those skilled in the art can make modifications and substitutions to the embodiments of the invention, and the resulting embodiments are also within the protection scope of the invention.

[0073] In related technologies, directly applying traditional TMO to HDR video is not an effective solution because it introduces visual artifacts such as flickering, brightness inconsistencies, and color variations. On the other hand, generating high-quality SDR content with TMO is complex, requiring manual intervention to set various parameters within the implementation method, making it unsuitable for real-time, hardware-friendly implementations. Another challenge with existing methods is the inability to preserve the artistic impression of the original HDR content. Maintaining the overall visual impression of the input HDR content is crucial when converting to SDR format.

[0074] To address the aforementioned technical problems, this application proposes an image processing method, apparatus, electronic device, and storage medium that can convert an entire HDR image into an SDR image, providing high-quality visual output for HDR images under different brightness conditions. It also avoids issues such as flickering artifacts and inconsistencies in brightness and color during image format conversion. Furthermore, it is hardware-friendly, can be easily implemented with limited resources, requires minimal computation, and reduces computational costs.

[0075] Figure 1 This is a flowchart illustrating an image processing method according to an exemplary embodiment. This image processing method is performed in the RGB color space and can be applied to electronic devices with image processing capabilities, such as image processors, display chips, and displays. Please see [link to relevant documentation]. Figure 1 The image processing method may include the following steps:

[0076] Step 101: Convert the original image to the optical domain and extract the luminance channel for each pixel to obtain the first luminance value. The original image is a high dynamic range (HDR) image.

[0077] In this step, the input raw image (HDR image) is converted from the HDR domain to the optical domain, that is, the HDR format encoded values ​​of the raw image are converted into grayscale values. Then, the luminance channel is extracted for each pixel to obtain the first luminance value.

[0078] In this embodiment, the color gamut of the original image, i.e., the original color gamut, is the BT.2020 color gamut. According to the BT.2020 standard, the formula for extracting the luminance channel from the red, green, and blue color channels is as follows:

[0079] L light1 =0.2627R + 0.6780G + 0.0593B;

[0080] Where R is the red channel, G is the green channel, B is the blue channel, and L is the green channel. light1 This is the first brightness value.

[0081] In related technologies, the HDR format includes two formats: HLG10 or HDR10. The difference between these two formats lies in their transfer functions. The HDR10 format uses the PQ function, while the HLG10 format uses the Hybrid Log-Gamma (HLG) function.

[0082] In this embodiment, when the original image is in HDR10 format, the inverse PQ transform function is used to convert the original image to the optical domain; when the original image is in HLG10 format, the inverse HLG transform function is used to convert the original image to the optical domain.

[0083] Step 102: Perform perceptual quantization (PQ) transformation on the first luminance value to obtain the corresponding second luminance value in the PQ domain; wherein, the calculation formula for the PQ transformation is as follows: Among them, L HDR,PQ L is the second brightness value. light1 γ is the first brightness value, and γ1 is the PQ transformation constant.

[0084] The PQ (Perceptual Quantizer) domain is a method of encoding and compressing video signals. Belonging to the perceptual domain, the PQ domain is a perceptually uniform color space designed to more closely match how the human visual system perceives color. Using a logarithmic transfer function, the PQ domain can more efficiently utilize the available dynamic range. This makes it ideal for displaying high dynamic range (HDR) video content.

[0085] Perceptual quantization (PQ) aims to optimize the light intensity distribution associated with HVS properties, converting physically linear values ​​into perceptually linear values. Perceptual linearity means that the human eye perceives any intensity change at any brightness level in the same way (i.e., two consecutive brightness levels just below the threshold that the HVS has just noticed). Perceptual linearity is fundamental to the implementation because it allows for the identification of what the human eye can see and the optimal mapping of that information to a higher dynamic range. Therefore, the first brightness value of the brightness channel of the input HDR image is first transferred to the PQ domain, ensuring that the brightness values ​​of the HDR image and the SDR image are in the same domain during the mapping process. This allows us to work in the perceptual domain, where adjacent brightness values ​​correspond to differences perceptible to the human eye.

[0086] In this embodiment, the first luminance value is subjected to PQ transformation to obtain the corresponding second luminance value in the PQ domain.

[0087] In this embodiment, a perceptual quantizer can be used to perform a PQ transformation on the first luminance value. A perceptual quantizer is a transfer function that maps the linear light intensity of a display device to a non-linear signal, thereby better matching the characteristics of the human visual system (HVS). The function of the perceptual quantizer is to optimize the distribution of light intensity in order to most effectively utilize the available dynamic range and provide the best reproduction of the original image.

[0088] In related technologies, the formula for calculating the PQ transform is:

[0089]

[0090] Among them, L light1 This is the first brightness value, L. HDR,PQ This is the second brightness value. c1, c2, c3, m1 and m2 are constants, which can be set to 0.8359, 18.8515, 18.6875, 0.1593 and 78.8437 respectively.

[0091] However, the computational cost of implementing the above PQ transformation formula in hardware is very high, and it requires a lot of resources to implement.

[0092] To overcome the above challenges, in this embodiment, the calculation formula for the PQ transform is as follows:

[0093]

[0094] Among them, L HDR,PQ L is the second brightness value. light1 γ is the first brightness value, and γ1 is the PQ transformation constant.

[0095] In this embodiment, the PQ transform has a simple formula with few parameters, requires no manual intervention to set various parameters, has a small computational load, and has a low computational cost in hardware implementation. It is hardware-friendly, can be easily implemented with limited resources, and can be extended to real-time applications.

[0096] In this embodiment, γ1 is 4. Based on extensive simulation experiments, setting γ1 to 4 achieves high visual quality results in both subjective and objective evaluations.

[0097] Step 103: Based on the equation of the specified global mapping curve, convert the second brightness value into a third brightness value to obtain the first intermediate image; the third brightness value is the brightness value of the brightness channel of the first intermediate image.

[0098] Flickering is a common problem that can occur during tone mapping, especially when using local tone mapping algorithms. This is because local algorithms independently adjust the brightness of individual pixels or a small group of pixels, causing rapid changes in the overall brightness of the image. These brightness changes can be particularly noticeable in high-contrast areas and can be distracting and visually jarring. To avoid this problem, the global mapping algorithm (the equation for the global mapping curve) in this application uses a different approach. Instead of adjusting the brightness of individual pixels, it takes the average of the brightness values ​​across the entire image. Because the brightness of the entire image is adjusted as a whole, this method avoids rapid brightness changes that can lead to flickering.

[0099] In this embodiment, the global mapping curve can be determined based on the curve identifier input by the user. For example, if the curve identifier is 00, the specified global mapping curve is a gamma curve; if the curve identifier is 01, the specified global mapping curve is a piecewise linear curve.

[0100] In this embodiment, the global mapping curve is a gamma curve, and the equation of the gamma curve is as follows:

[0101]

[0102] Among them, L SDR,PQ The third brightness value, L HDR,PQ c1 and c2 are constants, representing the second brightness value.

[0103] In this embodiment, c2 can be calculated using the following formula:

[0104]

[0105] Then, calculate c1 using the following formula:

[0106]

[0107] Among them, HDR min,PQ The minimum value of the second brightness value, HDR max,PQ The maximum value of the second brightness value, SDR min,PQ The minimum brightness value of the target SDR display in the PQ domain, SDR max,PQ This represents the maximum brightness value of the target SDR display in the PQ domain.

[0108] In this embodiment, the gamma curve C1 used is as follows: Figure 2 As shown.

[0109] In this embodiment, a gamma curve is used for the global mapping curve. This curve has a simple mathematical form, low computational complexity, and can generate high-quality SDR images. Furthermore, the generated SDR image has more detail in brighter areas than in darker areas. If we need the generated SDR image to have more detail in brighter areas than in darker areas, then using a gamma curve is recommended.

[0110] In this embodiment, the mapping curve is constructed by considering the maximum and minimum brightness values ​​of the target SDR display. This allows the image processing method to generate content that looks pleasing on all types of SDR displays. Therefore, the algorithm provided in this application is display-adaptive and can generate SDR image content that matches the capabilities of any target SDR display. Thus, the TMO proposed herein is display-adaptive and can generate SDR content that matches the capabilities of any target display.

[0111] Step 104: Convert the first intermediate image from the PQ domain to the optical domain to obtain the second intermediate image.

[0112] In this embodiment, the first intermediate image needs to be converted from the PQ domain to the optical domain to display the image content.

[0113] In related technologies, the first intermediate image is transformed from the PQ domain to the optical domain using the following inverse PQ transform calculation formula:

[0114]

[0115] Among them, L light2 This is the fourth luminance value of the luminance channel of the pixel in the second intermediate image.

[0116] However, the computational cost of implementing the inverse PQ transform formula described above is very high in hardware and requires a large amount of resources. To overcome these challenges, in this embodiment, the inverse PQ transform formula is as follows:

[0117]

[0118] Among them, L SDR,PQ The third brightness value, L light2 γ is the fourth luminance value of the luminance channel of the pixel in the second intermediate image, and γ2 is the inverse PQ transform constant.

[0119] In this embodiment, the inverse PQ transform has a simple formula, few parameters, and low computational cost. It is relatively low-cost to implement in hardware, making it hardware-friendly and easy to implement with limited resources.

[0120] In this embodiment, γ2 is 0.25. Extensive simulation experiments have shown that setting γ2 to 0.25 achieves high visual quality results in both subjective and objective assessments.

[0121] Step 105: Based on the color coordinates and color transformation matrix of the second intermediate image, convert from the original color gamut to the target color gamut to obtain the target image; wherein, the original color gamut is the color gamut of the original image, the target color gamut is the color gamut of the target image, and the target image is a standard dynamic range (SDR) image.

[0122] In this embodiment, the color gamut of the target image, i.e., the target color gamut, is BT.709. Because BT.709 is specifically designed for SDR images, converting the image to BT.709 ensures that the generated content displays the correct colors.

[0123] Converting an image from the BT.2020 color gamut to the BT.709 color gamut involves converting the color coordinates of a given image or video from one color space to another.

[0124] In one embodiment, a 3x3 color conversion matrix can be used to perform gamut conversion, mapping colors in the source color space (BT.2020) to colors in the target color space (BT.709). This process typically begins by converting an image or video from the RGB color space to its native color representation. The color conversion matrix is ​​then applied to the image's color coordinates, transforming them to the new color space. The matrix used for converting from the BT.2020 color gamut to the BT.709 color gamut can be represented as follows:

[0125] R709=T11×R2020+T12×G2020+T13×B2020

[0126] G709=T21×R2020+T22×G2020+T23×B2020

[0127] B709=T31×R2020+T32×G2020+T33×B2020

[0128] Where R, G, and B represent the red, green, and blue channels, respectively. R709, G709, and B709 are the color coordinates of the target image in the BT.709 color gamut, and R2020, G2020, and B2020 are the color coordinates of the second intermediate image in the BT.2020 color gamut. T11, T12, T13, T21, T22, T23, T31, T32, and T33 are conversion coefficients.

[0129] It should be noted that this step is quite complex and requires high precision. This color conversion matrix is ​​only an approximation. For accurate conversion, a three-dimensional color lookup table (3D-CLUT) is required.

[0130] In this embodiment, by converting the original image (HDR image) to the optical domain and extracting the luminance channel for each pixel to obtain a first luminance value, and then performing perceptual quantization (PQ) transformation on the first luminance value to obtain the corresponding second luminance value in the PQ domain, i.e., converting the HDR image to the perceptual domain, it helps to maintain the overall visual impression of the frame. Furthermore, the calculation formula for the PQ transformation used is as follows: It is hardware-friendly, can be easily implemented with limited resources, requires little computation, and can reduce computing costs.

[0131] Then, according to the equation of the specified global mapping curve, the second luminance value of each pixel is converted into the corresponding third luminance value to obtain the first intermediate image, where the third luminance value is the luminance value of the luminance channel of the first intermediate image. On the one hand, since a global mapping curve is used instead of a local mapping curve, the luminance of each pixel in the entire original image can be converted, that is, the luminance of all regions with different luminance levels in the HDR image is converted, rather than only the luminance of a certain type of specific luminance level is converted locally. Therefore, it is possible to convert the entire HDR image into an SDR image, providing high-quality visual output for HDR images under different luminance conditions. On the other hand, since the luminance of the entire original image is adjusted as a whole using a global mapping curve, rapid changes in luminance that may cause flicker can be avoided, thereby avoiding flicker artifacts.

[0132] Then, the first intermediate image is converted from the PQ domain to the optical domain to obtain the second intermediate image. Based on the color coordinates of the second intermediate image and the color conversion matrix, the image is converted from the original color domain to the target color domain to obtain the target image (SDR image).

[0133] In summary, the technical solution provided in this application can convert an entire HDR image into an SDR image, providing high-quality visual output for HDR images under different brightness conditions. Moreover, it is hardware-friendly, can be easily implemented with limited resources, and requires little computation, thus reducing computational costs.

[0134] Figure 3 This is a flowchart illustrating an image processing method according to another exemplary embodiment. Unlike the previous embodiment, in this embodiment, after converting the second luminance value to a third luminance value according to the equation of a specified global mapping curve to obtain a first intermediate image, and before converting the first intermediate image from the PQ domain to the optical domain to obtain a second intermediate image, color adjustment is performed on the first intermediate image for each color channel according to a specified color adjustment equation to maintain the hue and luminance of the colors substantially unchanged. Please refer to... Figure 3 The image processing method may include the following steps:

[0135] Step 301: Convert the original image to the optical domain and extract the luminance channel for each pixel to obtain the first luminance value, wherein the original image is a high dynamic range (HDR) image.

[0136] This step is similar to step 101 above, and will not be repeated here.

[0137] Step 302: Perform perceptual quantization (PQ) transformation on the first luminance value to obtain the corresponding second luminance value in the PQ domain; wherein, the calculation formula for the PQ transformation is as follows: Among them, L HDR,PQ L is the second brightness value. light1 γ is the first brightness value, and γ1 is the PQ transformation constant.

[0138] This step is similar to step 102 above, and will not be repeated here.

[0139] Step 303: Based on the equation of the specified global mapping curve, convert the second brightness value into a third brightness value to obtain the first intermediate image; the third brightness value is the brightness value of the brightness channel of the first intermediate image.

[0140] In this embodiment, the global mapping curve is a piecewise linear curve. For example... Figure 4As shown, the global mapping curve C2 includes two endpoints D1 and D2, two boundary points D3 and D4, and three linear curves C21, C22, and C23. These three linear curves form a continuous curve. The two endpoints D1 and D2 and the two boundary points D3 and D4 divide the original image into dark areas, normal areas, and bright areas. Each of the three linear curves corresponds one-to-one with a dark area, a normal area, and a bright area. Specifically, linear curve C21 corresponds to the dark area, linear curve C22 corresponds to the normal area, and linear curve C23 corresponds to the bright area. x1 and x2 are the abscissas of the two boundary points D3 and D4, y1 and y2 are the ordinates of the two boundary points D3 and D4, and s1, s2, and s3 are the slopes of the three linear curves C21, C22, and C23, respectively. In other embodiments, the number of segments in the global mapping curve can be extended to more than three.

[0141] In this embodiment, the equation of the global mapping curve is as follows:

[0142]

[0143] Among them, L HDR,PQ L is the second brightness value. SDR,PQ The third brightness value is represented by s1, s2, and s3, which are the slopes of the three linear curves, respectively. a1, a2, and a3 are the intercepts of the three linear curves, respectively. x1 and x2 are the x-coordinates of the two dividing points. HDR min,PQ The minimum value of the second brightness value, HDR max,PQ This is the maximum value of the second brightness value.

[0144] In this embodiment, the piecewise linear curve described above is used as the equation for the specified global mapping curve to convert the second brightness value into a third brightness value, resulting in a first intermediate image. The specific method for global mapping is as follows: First, based on equation C2 of the global mapping curve, two endpoints D1 and D2 and two boundary points D3 and D4 are determined. Then, based on the two endpoints D1 and D2 and the two boundary points D3 and D4, the original image is divided into dark areas, normal areas, and bright areas. Next, for each area of ​​the original image, the second brightness value is converted into the corresponding third brightness value according to the equation of the global mapping curve. Specifically, for the dark areas of the original image, the second brightness value is converted into the corresponding third brightness value according to linear curve C21; for the normal areas of the original image, the second brightness value is converted into the corresponding third brightness value according to linear curve C22; and for the bright areas of the original image, the second brightness value is converted into the corresponding third brightness value according to linear curve C23. This helps to convert each region of the HDR image into its corresponding region in the SDR domain, while also helping to maintain the overall artistic impression. Using piecewise linear curves can make the output image more balanced, thus achieving the best balance between the overall brightness and contrast of the image.

[0145] In this embodiment, the method for generating piecewise linear curves is as follows:

[0146] (1) Determine the segmentation method.

[0147] In this step, firstly, the values ​​of x1 and x2 are calculated to segment the image, dividing the original image into dark areas, normal areas, and bright areas.

[0148] x1 is calculated using the following formula:

[0149] x1=P1×(HDR max,PQ -HDR min,PQ )+HDR min,PQ ;

[0150] x2 is calculated using the following formula:

[0151] x2 = HDR max,PQ -P2×(HDR max,PQ -HDR min,PQ );

[0152] Where x1 is the x-coordinate of the boundary between the dark area and the normal area, x2 is the x-coordinate of the boundary between the normal area and the bright area, and P1 and P2 are two constants.

[0153] In this embodiment, P1 is 0.15 and P2 is 0.4. Extensive research indicates that setting P1 and P2 to 0.15 and 0.4 respectively achieves good results in segmenting the HDR image into three brightness regions.

[0154] After calculating the values ​​of x1 and x2, determine their corresponding values ​​y1 and y2 in the SDR domain. The values ​​of y1 and y2 can be calculated using the following formulas:

[0155] y1=P3×(SDR max,PQ -SDR min,PQ )+SDR min,PQ ;

[0156] y2=P4×(SDR max,PQ -SDR min,PQ )+SDR min,PQ ;

[0157] Where P3 and P4 are two constants, SDR min,PQ The minimum brightness value of the target SDR display in the PQ domain, SDR max,PQ This represents the maximum brightness value of the target SDR display in the PQ domain.

[0158] In this embodiment, P3 is 0.1369 and P4 is 0.7518. Extensive simulations show that setting P3 and P4 to 0.1369 and 0.7518, respectively, produces SDR images with good visual quality.

[0159] Next, obtain the coordinates of the two endpoints D1 and D2.

[0160] Next, based on the coordinates of x1, x2, y1, y2 and the two endpoints D1, D2, the values ​​of s1, s2, s3, a1, a2 and a3 are calculated.

[0161] (2) Generate piecewise linear curves

[0162] In this embodiment, the piecewise linear curve described above is generated based on the values ​​of s1, s2, s3, a1, a2, and a3, thus obtaining the equation of the global mapping curve C2.

[0163] In this embodiment, the proposed TMO operates in the perception domain to take into account the human eye's sensitivity to brightness variations in different areas of a scene (i.e., dark, normal, and bright). This allows the TMO to process each area differently based on how the eye perceives them.

[0164] To preserve the overall impression of the original HDR image content, the TMO proposed in this paper uses a novel segmentation method to divide HDR images into dark, normal, and bright regions. This not only enables the preservation of the overall brightness of each region but also allows the TMO to process all types of HDR images, a problem that most existing TMOs cannot solve.

[0165] Step 304: For each color channel, adjust the color of the first intermediate image according to the specified color adjustment equation, keeping the hue and brightness of the colors unchanged, to obtain the third intermediate image.

[0166] A key consideration in designing an efficient TMO is maintaining color accuracy between the input HDR image and the generated SDR image. This is challenging because attempting to compress brightness levels from HDR to SDR can lead to color shifts. Therefore, an effective color adjustment scheme is crucial when designing a TMO. The goal of the color adjustment method is to preserve the hue of the colors during the transformation process while making changes in color brightness negligible.

[0167] In this embodiment, to ensure the accuracy of the colors in the generated image, the color adjustment equation used is as follows:

[0168]

[0169] Among them, C SDR,PQ C represents the color saturation of any color channel (red, green, or blue) of the third intermediate image in the PQ domain. HDR,PQ L represents the color saturation of any color channel of the first intermediate image in the PQ domain. HDR,PQ L is the second brightness value. SDR,PQ The third brightness value is α, and β are two constants that control the hue and saturation of the color.

[0170] In this embodiment, since color adjustment is performed according to a specified color adjustment equation, the hue and brightness of the colors remain unchanged, thus ensuring the color accuracy between the input HDR image and the output SDR image.

[0171] Furthermore, based on extensive simulations, the aforementioned color adjustment equation preserves the hue of the colors, while having a negligible impact on brightness when mapping HDR images to SDR images.

[0172] In this embodiment, by performing color adjustment in the perceptual domain, the image processing method preserves the hue of the HDR image and generates the color of the SDR image that is very close to the color of the HDR image.

[0173] Step 305: Convert the third intermediate image from the PQ domain to the optical domain to obtain the second intermediate image.

[0174] This step is similar to step 104 above, and will not be repeated here.

[0175] Step 306: Based on the color coordinates and color transformation matrix of the second intermediate image, the color gamut is transformed from the original color gamut to the target color gamut to obtain the target image; wherein, the original color gamut is the color gamut of the original image, the target color gamut is the color gamut of the target image, and the target image is a standard dynamic range (SDR) image.

[0176] This step is similar to step 105 above, and will not be repeated here.

[0177] In this embodiment, after converting the second luminance value to a third luminance value according to the equation of the specified global mapping curve to obtain the first intermediate image, and before converting the first intermediate image from the PQ domain to the optical domain to obtain the second intermediate image, color adjustment is performed on the first intermediate image for each color channel according to the specified color adjustment equation to maintain the hue and luminance of the colors. By performing color adjustment in the perceptual domain, this image processing method preserves the hue of the HDR image and generates the colors of the SDR image that are very close to those of the HDR image, while making the change in color luminance negligible during the color adjustment process.

[0178] Figure 5 This is a flowchart illustrating an image processing method according to another exemplary embodiment, and also a block diagram of the TMO principle proposed herein. The main part of the method is the portion enclosed by the dashed box 51. Please refer to... Figure 5 The image processing method may include the following steps:

[0179] (1) Extract the brightness channel

[0180] This step is the same as step 101, and will not be repeated here.

[0181] (2) PQ transformation

[0182] This step is the same as step 102, and will not be repeated here.

[0183] (3) Select the mapping curve

[0184] In this embodiment, the mapping curve is a global mapping curve, which includes a gamma curve and a piecewise linear curve. The electronic device can select the mapping curve based on the curve identifier input by the user. For example, if the curve identifier input by the user is 00, the specified global mapping curve is a gamma curve; if the curve identifier input by the user is 01, the specified global mapping curve is a piecewise linear curve.

[0185] (4) When the user inputs the curve identifier of the gamma curve, the gamma curve is generated and globally mapped.

[0186] In this embodiment, when the curve identifier input by the user is the curve identifier of the gamma curve, a gamma curve is generated and globally mapped.

[0187] (5) When the user inputs the curve identifier of the piecewise linear curve, determine the curve segmentation method.

[0188] In this embodiment, when the curve identifier input by the user is the curve identifier of a piecewise linear curve, the curve segmentation method is determined.

[0189] This step is the same as the step 303 above in determining the curve segmentation method, and will not be repeated here.

[0190] (6) Generate piecewise linear curves and perform global mapping.

[0191] In this embodiment, after determining the curve segmentation method, a piecewise linear curve is generated and a global mapping is performed.

[0192] This step is the same as the global mapping method in step 303 above, and will not be repeated here.

[0193] (7) Color Adjustment

[0194] This step is the same as step 304, and will not be repeated here.

[0195] (8) Reverse PQ conversion

[0196] This step is the same as step 305, and will not be repeated here.

[0197] (9) Inverse color gamut conversion

[0198] This step is the same as step 306, and will not be repeated here.

[0199] In this embodiment, HDR images can be converted to standard dynamic range (SDR) images to achieve high visual quality across all brightness levels. Furthermore, it is hardware-friendly and can be implemented on resource-constrained hardware.

[0200] Figure 6 This is a block diagram of an image processing apparatus according to an exemplary embodiment. Figure 6 As shown, in this embodiment, the image processing apparatus includes:

[0201] The first acquisition module 61 is configured to convert the original image to the light domain and extract the brightness channel for each pixel to obtain a first brightness value, wherein the original image is a high dynamic range (HDR) image.

[0202] The second acquisition module 62 is configured to perform a perceptual quantization (PQ) transformation on the first brightness value to obtain a second brightness value corresponding to the PQ domain; wherein the calculation formula for the PQ transformation is as follows: Among them, L HDR,PQ L is the second brightness value.light1 γ1 is the first brightness value, and γ1 is the PQ transformation constant;

[0203] The first conversion module 63 is configured to convert the second brightness value into a third brightness value according to the equation of a specified global mapping curve to obtain a first intermediate image; the third brightness value is the brightness value of the brightness channel of the first intermediate image.

[0204] The second conversion module 64 is configured to convert the first intermediate image from the PQ domain to the optical domain to obtain the second intermediate image;

[0205] The third conversion module 65 is configured to convert from the original color gamut to the target color gamut based on the color coordinates of the second intermediate image and the color conversion matrix to obtain the target image; wherein, the original color gamut is the color gamut of the original image, the target color gamut is the color gamut of the target image, and the target image is a standard dynamic range (SDR) image.

[0206] In one embodiment, such as Figure 7 As shown, the image processing apparatus further includes:

[0207] The color adjustment module 66 is configured to adjust the color of the first intermediate image for each color channel according to a specified color adjustment equation, while keeping the hue and brightness of the colors essentially unchanged, to obtain the third intermediate image.

[0208] The second conversion module 64 is also configured to convert the third intermediate image from the PQ domain to the optical domain to obtain the second intermediate image.

[0209] Embodiments of this application also provide an electronic device, including a processor and a memory; the memory is used to store a computer program executable by the processor; the processor is used to execute the computer program in the memory to implement the image processing method of any of the above embodiments.

[0210] Embodiments of this application also propose a computer-readable storage medium that, when an executable computer program in the storage medium is executed by a processor, can implement the image processing method of any of the above embodiments.

[0211] Regarding the apparatus in the above embodiments, the specific manner in which the processor performs the operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0212] Figure 8 This is a block diagram illustrating an electronic device according to an exemplary embodiment. For example, electronic device 800 may be provided as a server. (Refer to...) Figure 8The device 800 includes a processing component 822, which further includes one or more processors, and memory resources represented by memory 832 for storing instructions, such as application programs, that can be executed by the processing component 822. The application programs stored in memory 832 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 822 is configured to execute instructions to perform the aforementioned image processing method.

[0213] Device 800 may also include a power supply component 826 configured to perform power management of device 800, a wired or wireless network interface 850 configured to connect device 800 to a network, and an input / output (I / O) interface 858. Device 800 can operate on an operating system stored in memory 832, such as Windows Server™, MacOS X™, Unix™, Linux™, FreeBSD™, or similar.

[0214] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 832 including instructions, which can be executed by a processing component 822 of the device 800 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0215] In this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance. The term "multiple" refers to two or more unless otherwise expressly defined.

[0216] The above description of the embodiments is intended to enable those skilled in the art to understand and apply this application. It will be apparent to those skilled in the art that various modifications can be easily made to these embodiments, and the general principles described herein can be applied to other embodiments without creative effort. Therefore, this application is not limited to the embodiments described herein, and any improvements and modifications made by those skilled in the art based on the disclosure of this application without departing from the scope and spirit of this application are within the scope of this application.

Claims

1. An image processing method, characterized by, The method comprises the following steps: Converting an original image to a light domain and extracting a luminance channel for each pixel to obtain a first luminance value, wherein the original image is a high dynamic range (HDR) image; The first luminance value is subjected to perceptual quantization (PQ) transform to obtain the corresponding second luminance value in the PQ domain; wherein the calculation formula for the PQ transform is as follows: Among them, L HDR,PQ L is the second brightness value. light1 γ1 is the first brightness value, and γ1 is the PQ transformation constant; Converting the second luminance value to a third luminance value according to an equation of a specified global mapping curve to obtain a first intermediate image; the third luminance value is a luminance value of a luminance channel of the first intermediate image; the converting the second luminance value to the third luminance value according to the equation of the specified global mapping curve to obtain the first intermediate image comprises: determining two end points D1 and D2 and two demarcation points D3 and D4 according to the equation of the global mapping curve C2, dividing the original image into a dark region, a normal region and a bright region according to the two end points D1 and D2 and the two demarcation points D3 and D4, and converting the second luminance value to a corresponding third luminance value according to the equation of the global mapping curve for each region of the original image; wherein the second luminance value is converted to a corresponding third luminance value according to a linear curve C21 for the dark region of the original image, the second luminance value is converted to a corresponding third luminance value according to a linear curve C22 for the normal region of the original image, and the second luminance value is converted to a corresponding third luminance value according to a linear curve C23 for the bright region of the original image; For each color channel, color adjustment is performed on the first intermediate image according to a specified color adjustment equation, keeping the color hue unchanged and the brightness substantially unchanged, to obtain a third intermediate image; the color adjustment equation is wherein C SDR,PQ is the color saturation of any color channel of the third intermediate image in the PQ domain, C HDR,PQ is the color saturation of any color channel of the first intermediate image in the PQ domain, L HDR,PQ is the second luminance value, L SDR,PQ is the third luminance value, and α and β are two constants responsible for controlling the color hue and color saturation. Converting the third intermediate image from the PQ domain to the light domain to obtain a second intermediate image; Converting from an original color gamut to a target color gamut according to color coordinates of the second intermediate image and a color conversion matrix to obtain a target image; wherein the original color gamut is a color gamut of the original image, the target color gamut is a color gamut of the target image, and the target image is a standard dynamic range (SDR) image.

2. An image processing apparatus characterized by comprising: The method comprises the following steps: A first acquisition module is configured to convert an original image to a light domain and extract a luminance channel for each pixel to obtain a first luminance value, wherein the original image is a high dynamic range (HDR) image; The second acquisition module is configured to perform a perceptual quantization PQ transformation on the first luminance value to obtain a corresponding second luminance value in a PQ domain; wherein a calculation formula of the PQ transformation is wherein L HDR,PQ is the second luminance value, L light1 is the first luminance value, and γ1 is a PQ transformation constant; A first conversion module is configured to convert the second luminance value to a third luminance value according to an equation of a specified global mapping curve to obtain a first intermediate image; the third luminance value is a luminance value of a luminance channel of the first intermediate image; the converting the second luminance value to the third luminance value according to the equation of the specified global mapping curve to obtain the first intermediate image comprises: determining two end points D1 and D2 and two demarcation points D3 and D4 according to the equation of the global mapping curve C2, dividing the original image into a dark region, a normal region and a bright region according to the two end points D1 and D2 and the two demarcation points D3 and D4, and converting the second luminance value to a corresponding third luminance value according to the equation of the global mapping curve for each region of the original image; wherein the second luminance value is converted to a corresponding third luminance value according to a linear curve C21 for the dark region of the original image, the second luminance value is converted to a corresponding third luminance value according to a linear curve C22 for the normal region of the original image, and the second luminance value is converted to a corresponding third luminance value according to a linear curve C23 for the bright region of the original image; a color adjustment module configured to, for each color channel, perform color adjustment on the first intermediate image according to a specified color adjustment equation, keeping the hue of the color unchanged and the brightness substantially unchanged, to obtain a third intermediate image; the color adjustment equation is wherein C SDR,PQ is the color saturation of any color channel of the third intermediate image in the PQ domain, C HDR,PQ is the color saturation of any color channel of the first intermediate image in the PQ domain, L HDR,PQ is the second luminance value, L SDR,PQ is the third luminance value, and a and b are two constants responsible for controlling the hue and color saturation of the color. The second conversion module is further configured to convert the third intermediate image from a PQ domain to a light domain to obtain a second intermediate image; The third conversion module is configured to convert from an original color gamut to a target color gamut according to color coordinates of the second intermediate image and a color conversion matrix to obtain a target image; wherein the original color gamut is a color gamut of the original image, the target color gamut is a color gamut of the target image, and the target image is a standard dynamic range (SDR) image.

3. An electronic device, comprising: A computer readable storage medium, storing a computer program executable by a processor, wherein the computer program is configured to implement the method according to claim 1 when executed by the processor.

4. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program stored in the storage medium is executable by the processor, and the computer program is configured to implement the method according to claim 1 when executed by the processor.

Images