Image processing method and apparatus

Abstract

Provided in the embodiments of the present application are an image processing method and apparatus. The method comprises: in an HDR domain, performing first compression on an image on the basis of a first model that conforms to visual perception, wherein the first compression comprises first luminance compression and first saturation compression; and in an SDR domain, on the basis of a second model that conforms to visual perception, performing second compression on the image which has been subjected to the first compression, wherein the second compression comprises second luminance compression and second saturation compression.

Description

Image processing method and apparatus Technical Field

[0001] This application relates to the field of display technology, and in particular to an image processing method and apparatus. Background Technology

[0002] In recent years, high dynamic range (HDR, PQ brightness range up to 10,000 nits, HLG brightness range 1,000 nits) and wide color gamut standards (such as Rec. 2020) have become increasingly popular in the field of display technology. However, wide color gamut standards and high dynamic range place high demands on the reproduction capabilities of display devices, such as display panels.

[0003] Most existing display panels have a color gamut (color space) that differs significantly from wide color gamut standards (such as Rec. 2020). Furthermore, most existing display panels only support Standard Dynamic Range (SDR), making it difficult to fully reproduce high-brightness standards and thus failing to meet HDR requirements. Therefore, color processing and restoration (reconstruction) methods are needed to process the input HDR and wide color gamut standard display data into display data that the display panel can reproduce.

[0004] In existing color processing and restoration (reconstruction) methods, firstly, in the HDR domain, EETF tone mapping is performed on the three RGB channels of the input HDR display data to compress the brightness range of the HDR display data into the SDR brightness range, that is, into the SDR domain. In the SDR domain, color gamut mapping is first performed to map the information of the Rec.2020 HDR color space to the display device's color space. Then, clipping is performed to remove values ​​that are out of range, and finally, the SDR display data is output.

[0005] It should be noted that the above introduction to the technical background is only for the purpose of providing a clear and complete explanation of the technical solutions of this application and facilitating understanding by those skilled in the art. It should not be assumed that the above technical solutions are known to those skilled in the art simply because they have been described in this background section.

[0006] Summary of the Invention

[0007] When existing methods are used in color processing and reproduction, most colors are beyond the reproduction capabilities of the display panel. Therefore, the range of color values ​​is limited; for example, values ​​less than 0 are forcibly set to 0, and values ​​greater than 1 are forcibly set to 1. This results in excessive loss of color gradation and hue shift. Excessive loss of color gradation prevents subtle color changes in the signal source (the input high dynamic range display data) from being displayed, thus failing to reproduce the details in the signal source. Hue shifts, on the other hand, cause the user's visual perception to change the color type, such as from blue to purple, or from red to yellow. This affects the reproduction of image content, leading to a discrepancy between the displayed effect and the user's visual perception, thus degrading the user experience.

[0008] To address one or more of the aforementioned problems, embodiments of this application provide an image processing method and apparatus. This method enables the reproduction of image content with a wide color gamut and high dynamic range in a manner consistent with visual perception, utilizing display devices with limited color reproduction capabilities. It reduces excessive loss of color gradation and hue shift, thereby improving display quality and user experience.

[0009] According to a first aspect of the embodiments of this application, an image processing method is provided, the method comprising: in the HDR domain, performing a first compression on an image based on a first model conforming to visual perception, the first compression including a first brightness compression and a first saturation (chroma) compression; and in the SDR domain, performing a second compression on the image after the first compression based on a second model conforming to visual perception, the second compression including a second brightness compression and a second saturation compression.

[0010] In some embodiments, in the first compression, the first brightness compression is performed first, and then the first saturation compression is performed according to the degree of the first brightness compression; and / or, in the second compression, the second brightness compression is performed first, and then the second saturation compression is performed according to the degree of the second brightness compression.

[0011] In some embodiments, during the first brightness compression, the degree of first brightness compression is greater for the brighter portions of the image.

[0012] In the first saturation compression, the greater the degree of the first brightness compression, the greater the degree of the first saturation compression, and / or, the smaller the brightness of the signal source color space boundary, the greater the degree of the first saturation compression, and / or, the smaller the saturation ratio between the target color space boundary and the signal source color space boundary, the greater the degree of the first saturation compression.

[0013] In some embodiments, the degree of the second brightness compression is determined based on at least one of saturation, the relationship between the target brightness and the signal source brightness, the brightness of the target color space boundary, and the relationship between the target color space boundary and the signal source color space boundary.

[0014] In some embodiments, in the second brightness compression, the degree of second brightness compression is greater for the portion of the image with higher saturation; and / or, in the second brightness compression, the greater the ratio of the source brightness to the target brightness, the greater the degree of second brightness compression; and / or, in the second brightness compression, the smaller the brightness of the target color space boundary, the greater the degree of second brightness compression; and / or, in the second brightness compression, the smaller the saturation ratio of the target color space boundary to the source color space boundary, the greater the degree of second brightness compression.

[0015] In some embodiments, after the second brightness compression, the degree of the second saturation compression is determined by the degree of the second brightness compression: the greater the degree of the second brightness compression, the greater the degree of the second saturation compression.

[0016] In some embodiments, the method further includes: when there is a portion in the image after the second compression that extends beyond the boundary of the target color space, performing a third compression on the image after the second compression based on a second model, the third compression including a third saturation compression.

[0017] In some embodiments, the greater the extent of the third saturation compression, the more portions of the image that extend beyond the target color space boundary exist in the second compressed image.

[0018] In some embodiments, the first saturation compression employs an approximately elliptic curve compression method. In some embodiments, the first model includes the ICT PLC color space or the IPT color space, and / or the second model includes the OKLab color space or the IPT color space.

[0019] According to a second aspect of the embodiments of this application, an image processing apparatus is provided, the apparatus comprising: a first compression unit that performs a first compression on an image in the HDR domain based on a first model conforming to visual perception, the first compression including a first brightness compression and a first saturation compression; and a second compression unit that performs a second compression on the image after the first compression in the SDR domain based on a second model conforming to visual perception, the second compression including a second brightness compression and a second saturation compression.

[0020] According to a third aspect of the present application, a display device is provided, the device comprising: a memory storing a computer program; and a processor executing the computer program to implement the method described in the first aspect of the present application.

[0021] According to a fourth aspect of the embodiments of this application, a computer-readable storage medium is provided, on which a computer program is stored, wherein the computer program, when executed by a processor, implements the method described in the first aspect of the embodiments of this application.

[0022] According to a fifth aspect of the embodiments of this application, a computer program product is provided, including a computer program that, when executed by a processor, implements the method described in the first aspect of the embodiments of this application.

[0023] One of the beneficial effects of the embodiments of this application is that:

[0024] In the HDR domain, the image is subjected to first compression, including first brightness compression and first saturation compression, based on a first model that conforms to visual perception. In the SDR domain, the image after the first compression is subjected to second compression, including second brightness compression and second saturation compression, based on a second model that conforms to visual perception. In this way, since both the HDR domain compression and SDR domain compression are based on the model of visual perception, brightness compression and saturation compression are performed. This allows display devices with limited reproduction capabilities to reproduce image content with a wide color gamut and high dynamic range in a way that conforms to visual perception while reducing excessive loss of color gradation and hue shift. This improves the consistency of visual perception and enhances the user experience.

[0025] Furthermore, in the first compression, a first luminance compression is performed first, followed by a first saturation compression based on the degree of luminance compression. This supplements visual recognizability. Since if saturation and hue remain unchanged after luminance compression, the compressed color will have an overly dense luminance distribution, leading to reduced visual recognizability. To prevent the user's visual perception from changing the color type due to hue changes, i.e., maintaining hue, the problem of reduced visual recognizability caused by luminance compression is compensated by reducing saturation, thereby improving the consistency of visual perception.

[0026] Furthermore, in the first brightness compression, the higher the brightness of the image, the greater the degree of brightness compression. This is because a large amount of image content information is still in low-brightness areas. Therefore, low-brightness content below a certain level is not compressed or compressed to a very low degree. For high-brightness image content, which is usually bright spots formed by light or metallic reflections—content whose visual brightness perception is not very precise—the higher the brightness, the greater the compression. Through this compression, a large range of signal source brightness—for example, a PQ brightness range of 10,000 nits in film and television, or a HLG brightness range of 1,000 nits in broadcasting—is compressed into a limited, relatively low target brightness range. This satisfies the needs of display devices with different brightness levels, as well as the needs of signal conversion devices with standard brightness (e.g., 100 nits, but not limited to 100 nits target brightness). Visually, it preserves the visually sensitive low-brightness information in the input HDR signal while also restoring high-brightness information as much as possible through compression.

[0027] Furthermore, in the first saturation compression, the greater the degree of first luminance compression, the greater the degree of first saturation compression, and / or, the smaller the luminance of the signal source color space boundary, the greater the degree of first saturation compression, and / or, the smaller the saturation ratio between the target color space boundary and the signal source color space boundary, the greater the degree of first saturation compression.

[0028] Furthermore, in the second compression, similarly, a second luminance compression is performed first, followed by a second saturation compression based on the degree of luminance compression. The greater the degree of luminance compression, the greater the degree of saturation compression. This also helps to improve visual recognizability and further enhance the user experience.

[0029] Furthermore, the degree of the second brightness compression is determined based on at least one of saturation, the relationship between the target brightness and the signal source brightness, the brightness of the target color space boundary, and the relationship between the target color space boundary and the signal source color space boundary. For example, the higher the saturation after the first compression, the greater the degree of the second brightness compression; the greater the ratio of the signal source brightness to the target brightness after the first compression in the image, the greater the degree of the second brightness compression; the smaller the brightness of the target color space boundary, the greater the degree of the second brightness compression; and the smaller the ratio of the saturation of the target color space boundary to the signal source color space boundary, the greater the degree of the second brightness compression, thus determining the degree of brightness compression for different hues.

[0030] Thus, since the three-dimensional form of the color space after the first compression is usually quite different from the target color space, especially since the brightness of highly saturated colors is usually much higher than the brightness that can be achieved under the same hue and saturation in the target color space, if a direct mapping conversion is performed to the target color space, it will cause a large number of highly saturated colors to overflow. Therefore, by compressing the brightness of highly saturated colors, the greater the color saturation, the greater the degree of brightness compression, so as to minimize the overflow of color values ​​after color gamut mapping in subsequent steps. The greater the ratio of the source brightness to the target brightness, the greater the degree of the second brightness compression.

[0031] Furthermore, considering that the boundary brightness of different target color spaces varies, the degree of brightness compression is determined by the relationship between the target color space boundary and the source color space boundary—that is, the smaller the saturation ratio between the target color space boundary and the source color space boundary, the greater the degree of brightness compression. This allows the target color space to better accommodate the compressed source color space, which originally had a high dynamic range and wide color gamut.

[0032] Furthermore, the degree of compression in the second saturation compression is determined by the degree of second luminance compression; the greater the degree of second luminance compression, the greater the degree of second saturation compression.

[0033] Furthermore, when the image after second compression contains portions exceeding the target color space boundary, a third compression is applied based on the second model. This third compression includes third saturation compression. For example, the more saturation exceeds the target color space boundary, the greater the saturation compression. This allows for further saturation correction based on the target color space boundary, thereby improving visual perception consistency and enhancing the user experience. Additionally, brightness compression can be further performed under certain conditions, but the primary focus is on saturation compression.

[0034] Furthermore, the first saturation compression adopts an approximate elliptic curve compression method, so that the degree of change of the initial compression point approaches 0, which allows the compression change to be continuous; and by adjusting the parameters, the elliptic curve can take on different shapes, thereby making the degree of compression controllable.

[0035] Furthermore, the first model used in the first compression includes the ICTP color space or the IPT color space. In this way, the EETF hue mapping of luminance is performed using the ICTP color space or the IPT color space that conforms to visual perception. Thus, apart from the luminance being compressed, the hue of the chromaticity information is not affected, further improving the consistency of visual perception.

[0036] Furthermore, the second model used in the second compression includes the OKLab color space or the IPT color space. By utilizing the OKLab color space or the IPT color space that conforms to visual perception, the consistency of hue is perceived, and the change in tone (hue) is not perceived. Moreover, the calculation based on the OKLab color space or the IPT color space is simple and fast, which can improve the processing speed and reduce the requirements on processor hardware.

[0037] Specific embodiments of this application are disclosed in detail with reference to the following description and accompanying drawings, indicating how the principles of this application can be adopted. It should be understood that the embodiments of this application are not limited in scope. Within the spirit and scope of the appended claims, embodiments of this application include many changes, modifications, and equivalents.

[0038] Features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, combined with features in other embodiments, or substituted for features in other embodiments.

[0039] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, whole, step, or component, but does not exclude the presence or addition of one or more other features, wholes, steps, or components. Attached Figure Description

[0040] The accompanying drawings, which form part of the specification, are used to provide a further understanding of the embodiments of this application and illustrate the implementation methods of this application, together with the textual description, to explain the principles of this application. Obviously, the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any creative effort. In the drawings:

[0041] Figure 1 is a schematic diagram of an image processing method according to an embodiment of this application;

[0042] Figure 2 is a schematic diagram of a method for executing step 101 according to an embodiment of this application;

[0043] Figure 3 is a schematic diagram of adjusting the degree of saturation compression using a function according to an embodiment of this application;

[0044] Figure 4 is a schematic diagram of a method for executing step 102 according to an embodiment of this application;

[0045] Figure 5 is a flowchart of one embodiment of the image processing method of this application;

[0046] Figure 6 is a schematic diagram of an image processing apparatus according to an embodiment of this application;

[0047] Figure 7 is a schematic diagram of the first compression section according to an embodiment of this application;

[0048] Figure 8 is a schematic diagram of the second compression section according to an embodiment of this application;

[0049] Figure 9 is a schematic block diagram of the system configuration of a display device according to an embodiment of this application. Detailed Implementation

[0050] Referring to the accompanying drawings, the foregoing and other features of this application will become apparent from the following description. Specific embodiments of this application are specifically disclosed in the description and drawings, illustrating partial implementations in which the principles of this application may be employed. It should be understood that this application is not limited to the described embodiments; rather, it includes all modifications, variations, and equivalents falling within the scope of the appended claims.

[0051] Example 1

[0052] This application provides an image processing method.

[0053] Figure 1 is a schematic diagram of an image processing method according to an embodiment of this application. As shown in Figure 1, the method includes:

[0054] Step 101: In the HDR domain, perform a first compression on the image based on a first model conforming to visual perception. The first compression includes a first luminance compression and a first saturation compression; and

[0055] Step 102: In the SDR domain, the image after the first compression is subjected to a second compression based on a second model that conforms to visual perception. The second compression includes a second brightness compression and a second saturation compression.

[0056] In this way, since both HDR and SDR compression use models that conform to visual perception, and both brightness and saturation compression are performed, it is possible to display a wide color gamut and high dynamic range image content with limited reproduction capabilities while reducing excessive loss of color gradation and hue shift, thus improving the consistency of visual perception and enhancing the user experience.

[0057] In the embodiments of this application, terms such as "first" and "second" are used to distinguish the descriptive methods without imposing any other restrictions on the features themselves.

[0058] In the embodiments of this application, the input image, i.e. the image to be processed, can be an HDR image, while the output image, i.e. the processed image, can be an SDR image, thereby enabling the display of picture content with a wide color gamut and high dynamic range using display devices with limited reproduction capabilities.

[0059] In addition, the input and output images can be still images or one or more frames from a video.

[0060] In the embodiments of this application, saturation can also be referred to as chroma.

[0061] In step 101, the input image is first compressed in the HDR domain based on a first model that conforms to visual perception. The input image is, for example, an HDR image.

[0062] In some embodiments, the first model can be various models that conform to visual perception, such as various color spaces that conform well to human eye perception, such as the ICTCP color space or the IPT color space. In this way, using the ICTCP color space or the IPT color space that conforms to visual perception for EETF hue mapping of brightness, the hue of the color information is not affected except for the compression of brightness, which further improves the consistency of visual perception.

[0063] In some embodiments, in step 101, a first luminance compression is performed first, and then a first saturation compression is performed according to the degree of the first luminance compression.

[0064] Figure 2 is a schematic diagram of a method for executing step 101 according to an embodiment of this application. As shown in Figure 2, the method includes:

[0065] Step 1011: In the HDR domain, perform first luminance compression based on a first model that conforms to visual perception;

[0066] Step 1012: In the HDR domain, perform first saturation compression on the image after first luminance compression based on a first model that conforms to visual perception.

[0067] This can supplement visual recognizability. Since if saturation and hue remain unchanged after brightness compression, the compressed color will lack visual recognizability due to the excessively dense distribution of brightness. In order to prevent users from perceiving a change in color type due to hue changes, i.e., to keep the hue unchanged, the problem of reduced recognizability caused by brightness compression is compensated by reducing saturation, thereby further improving the consistency of visual perception.

[0068] In some embodiments, in step 1011, luminance EETF hue mapping is performed using an ICTP color space or IPT color space that conforms to visual perception. Thus, apart from luminance compression, the hue of the chromaticity information remains unaffected, improving the consistency of visual perception.

[0069] In some embodiments, the higher the brightness of a portion of the image, the greater the degree of brightness compression. This way, considering that a large amount of image content information is still in low-brightness areas, low-brightness content below a certain level is not compressed or compressed to a very low degree, while high-brightness image content is compressed more extensively, thus ensuring uniform brightness of the image content and preventing localized overly dark areas.

[0070] In some embodiments, in step 1012, the first saturation compression may employ an approximate elliptic curve compression method, for example, a 1 / 4 approximate elliptic compression method.

[0071] Figure 3 is a schematic diagram of adjusting the degree of saturation compression using a function according to an embodiment of this application.

[0072] As shown in Figure 3, the function in Figure 3 adjusts the degree of saturation compression through a variable n. In addition, the range of n in Figure 3 is not limited to the ranges listed in the figure.

[0073] In this way, the degree of change in the initial compression point approaches 0, which allows the compression change to be continuous; and by adjusting the parameters, the elliptic curve can take on different shapes, thereby making the degree of compression controllable.

[0074] The compression in the HDR domain was performed in step 101. For example, EETF maintains the consistency of chroma by compressing brightness, but the volume of the compressed color space still exceeds the color space of the display device, especially for high brightness and high saturation colors. Therefore, further compression is performed in step 102 to make the boundary shape of the compressed color space closer to the boundary of the display device's color space.

[0075] In step 102, the image after the first compression is subjected to second brightness compression and second saturation compression in the SDR domain based on a second model that conforms to visual perception.

[0076] In some embodiments, the second model can be various models that conform to visual perception, such as various color spaces that have good conformity to human eye perception, such as the OKLab color space or the IPT color space. Utilizing the OKLab color space or the IPT color space that conforms to visual perception ensures consistency in hue, preventing the perception of variations in tone (hue). Furthermore, calculations based on the OKLab color space or the IPT color space are simple and fast, which can improve processing speed and reduce the requirements on processor hardware.

[0077] In some embodiments, in step 102, similar to step 101, a second brightness compression is performed first, followed by a second saturation compression based on the degree of the second brightness compression. This also helps to improve visual recognizability and further enhance the user experience.

[0078] Figure 4 is a schematic diagram of a method for executing step 102 according to an embodiment of this application. As shown in Figure 4, the method includes:

[0079] Step 1021: In the SDR domain, perform a second luminance compression based on a second model that conforms to visual perception;

[0080] Step 1022: In the SDR domain, perform second saturation compression on the image after the second brightness compression based on the second model that conforms to visual perception.

[0081] In step 1021, the higher the saturation of a portion of the image, the greater the degree of brightness compression; and / or, the greater the ratio of the source brightness to the target brightness, the greater the degree of brightness compression; and / or, the smaller the brightness of the target color space boundary, the greater the degree of brightness compression; and / or, based on the relationship between the target color space boundary and the source color space boundary, i.e., the smaller the saturation ratio between the target color space boundary and the source color space boundary, the greater the degree of brightness compression, thus determining the degree of brightness compression for different hues. In this way, since the three-dimensional form of the color space after the first compression is usually quite different from the target color space, especially since the brightness of highly saturated colors is usually much higher than the brightness achievable under the same hue and saturation in the target color space, directly mapping to the target color space would result in a large amount of color value overflow. Therefore, by compressing the brightness of highly saturated colors, a large amount of color value overflow can be avoided.

[0082] Furthermore, considering that the brightness of different target color space boundaries will vary, the degree of brightness compression is determined by considering that the smaller the brightness of the target color space boundary, the greater the degree of second brightness compression; and / or, the smaller the saturation ratio between the target color space boundary and the signal source color space boundary, the greater the degree of second brightness compression.

[0083] This will further improve the consistency of visual perception and enhance the user experience.

[0084] After the second luminance compression, the degree of second saturation compression is determined by the degree of second luminance compression; that is, the greater the degree of second luminance compression, the greater the degree of second saturation compression.

[0085] In some embodiments, when there are portions in the second compressed image that extend beyond the target color space boundary, a third compression is performed on the second compressed image based on the second model, the third compression including a third saturation compression.

[0086] For example, based on the portion of the color gamut that extends beyond the target color space boundary, the greater the excess, the greater the saturation compression. This allows for further saturation correction based on the target color space boundary, thereby improving visual perception consistency and enhancing the user experience. Additionally, under certain conditions, brightness compression can be further applied, but the primary focus is on saturation compression.

[0087] The main steps of the image processing method according to the embodiments of this application have been described above. Specific implementation methods are described below.

[0088] Figure 5 is a flowchart of one embodiment of the image processing method of this application.

[0089] As shown in Figure 5, the processing flow includes:

[0090] First, compress the HDR input image in the HDR domain:

[0091] First, perform EETF tone mapping from RGB to ICtCp in the ICtCp color space, which is the aforementioned first luminance compression;

[0092] Second, saturation (chroma) compression is performed in the ICtCp color space, that is, the aforementioned first saturation compression is performed;

[0093] After compression in the HDR domain, the compressed image is extended to the SDR domain in the RGB space, and then compression in the SDR domain is performed:

[0094] First, luminance compression is performed in the OKLab color space, which is the aforementioned second luminance compression.

[0095] Second, saturation (chroma) compression is performed in the OKLab color space, that is, the aforementioned second saturation compression is performed;

[0096] Third, perform gamut mapping in the XYZ space;

[0097] Fourth, the portions exceeding the target color space boundary are processed in the OKLab color space. For example, if the image after the aforementioned processing contains portions exceeding the target color space boundary, further saturation compression is performed in the OKLab color space, i.e., the aforementioned third saturation compression. Additionally, under certain conditions, brightness compression can also be further performed, but primarily saturation compression is applied.

[0098] Fifth, perform OETF conversion in the RGB space to obtain the SDR output image. For example, OETF here includes, but is not limited to: Gamma 1.0, SDR OETF (sRGB, Rec.709, etc.), or Gamma 1.8-2.6, or the GSDF curve specified by the DICOM standard.

[0099] The specific implementation of the main processing steps in the above process can be found in the above description, and the remaining steps can be found in relevant technologies, which will not be repeated here.

[0100] As can be seen from the above embodiments, in the HDR domain, the image is subjected to first compression, including first brightness compression and first saturation compression, based on a first model that conforms to visual perception. In the SDR domain, the image after the first compression is subjected to second compression, including second brightness compression and second saturation compression, based on a second model that conforms to visual perception. In this way, since both the compression in the HDR domain and the compression in the SDR domain adopt models that conform to visual perception and perform brightness compression and saturation compression, it is possible to display a wide color gamut and high dynamic range image content with limited reproduction capabilities while reducing excessive loss of color gradation and reducing hue shift. This improves the consistency of visual perception and enhances the user experience.

[0101] Furthermore, in the first compression, a first luminance compression is performed first, followed by a first saturation compression based on the degree of luminance compression. This supplements visual recognizability. Since if saturation and hue remain unchanged after luminance compression, the compressed color will lack visual recognizability due to its overly dense luminance distribution. To prevent users from perceiving a change in color type due to hue changes, i.e., maintaining hue, saturation is reduced to compensate for the reduced recognizability caused by luminance compression, thereby further improving the consistency of visual perception.

[0102] Furthermore, in the second compression, similarly, a second luminance compression is performed first, followed by a second saturation compression based on the degree of luminance compression. This also helps to improve visual recognizability and further enhance the user experience.

[0103] Furthermore, in the first brightness compression, the higher the brightness of the image, the greater the degree of compression. This way, considering that a large amount of image content information is still in low-brightness areas, low-brightness content below a certain level is not compressed or compressed to a very low degree, while high-brightness image content is compressed more extensively, thus ensuring the uniformity of image brightness and preventing localized overly dark areas.

[0104] Furthermore, the degree of second luminance compression is determined based on at least one of saturation, the relationship between target luminance and signal source luminance, the luminance of the target color space boundary, and the relationship between the target color space boundary and the signal source color space boundary.

[0105] In this way, since the three-dimensional form of the color space after the first compression is usually quite different from the target color space, especially since the brightness of highly saturated colors is usually much higher than the brightness achievable under the same hue and saturation in the target color space, directly mapping to the target space would lead to a large amount of color value overflow. Therefore, by compressing the brightness of highly saturated colors, a large amount of color value overflow can be avoided. At the same time, the larger the ratio of the source brightness to the target brightness, the greater the degree of brightness compression. In addition, considering that the boundary brightness of different target color spaces will be different, the degree of brightness compression is determined by considering that the smaller the boundary brightness of the target color space, the greater the degree of second brightness compression; and the smaller the saturation ratio between the boundary of the target color space and the boundary of the source color space, the greater the degree of second brightness compression. This further improves the consistency of visual perception and enhances the user experience.

[0106] After the second luminance compression, the degree of second saturation compression is determined by the degree of second luminance compression; that is, the greater the degree of second luminance compression, the greater the degree of second saturation compression.

[0107] Furthermore, when the image after second compression contains portions that extend beyond the target color space boundary, a third compression is applied based on the second model. This third compression includes third saturation compression. For example, the more portions that extend beyond the target color space boundary, the greater the saturation compression. This allows for further saturation correction based on the target color space boundary, thereby improving visual perception consistency and enhancing the user experience. Additionally, under certain conditions, brightness compression can be further applied, but the primary focus is on saturation compression.

[0108] Furthermore, the first saturation compression adopts an approximate elliptic curve compression method, so that the degree of change of the initial compression point approaches 0, which allows the compression change to be continuous; and by adjusting the parameters, the elliptic curve can take on different shapes, thereby making the degree of compression controllable.

[0109] Furthermore, the first model used in the first compression includes the ICTP color space or the IPT color space. In this way, the EETF hue mapping of luminance is performed using the ICTP color space or the IPT color space that conforms to visual perception. Thus, apart from the luminance being compressed, the hue of the chromaticity information is not affected, which further improves the consistency of visual perception.

[0110] Furthermore, the second model used in the second compression includes the OKLab color space or the IPT color space. By utilizing the OKLab color space or the IPT color space that conforms to visual perception, the consistency of hue is perceived, and the change in tone (hue) is not perceived. Moreover, the calculation based on the OKLab color space or the IPT color space is simple and fast, which can improve the processing speed and reduce the requirements on processor hardware.

[0111] Example 2

[0112] This application also provides an image processing apparatus corresponding to the image processing method described in Embodiment 1. For details, please refer to the description in Embodiment 1, which will not be repeated here. Alternatively, this image processing apparatus can also be a display device.

[0113] Figure 6 is a schematic diagram of an image processing apparatus according to an embodiment of this application. As shown in Figure 6, the image processing apparatus 200 includes:

[0114] A first compression unit 201, in the HDR domain, performs a first compression on the input image based on a first model conforming to visual perception. The first compression includes first brightness compression and first saturation compression.

[0115] The second compression unit 202 performs a second compression on the image after the first compression in the SDR domain based on a second model that conforms to visual perception, to obtain an output image. The second compression includes second brightness compression and second saturation compression.

[0116] Figure 7 is a schematic diagram of a first compression section according to an embodiment of this application. As shown in Figure 7, the first compression section 201 includes:

[0117] The first luminance compression unit 2011 performs first luminance compression in the HDR domain based on a first model that conforms to visual perception.

[0118] The first saturation compression unit 2012 performs first saturation compression on the image that has undergone first brightness compression in the HDR domain based on a first model that conforms to visual perception.

[0119] Figure 8 is a schematic diagram of the second compression section according to an embodiment of this application. As shown in Figure 8, the second compression section 202 includes:

[0120] The second luminance compression unit 2021 performs second luminance compression in the SDR domain based on a second model that conforms to visual perception.

[0121] The second saturation compression unit 2022 performs second saturation compression on the image after the second brightness compression in the SDR domain based on a second model that conforms to visual perception.

[0122] In some embodiments, as shown in FIG8, the second compression unit 202 may further include:

[0123] The third saturation compression unit 2023 performs third saturation compression on the image based on the second model when there is a portion in the image that has undergone second saturation compression that exceeds the boundary of the target color space.

[0124] The specific functions of each of the above units can be found in the specific implementation of the corresponding steps in Embodiment 1, and will not be repeated here.

[0125] As can be seen from the above embodiments, in the HDR domain, the image is subjected to first compression, including first brightness compression and first saturation compression, based on a first model that conforms to visual perception. In the SDR domain, the image after the first compression is subjected to second compression, including second brightness compression and second saturation compression, based on a second model that conforms to visual perception. In this way, since both the compression in the HDR domain and the compression in the SDR domain adopt models that conform to visual perception and perform brightness compression and saturation compression, it is possible to display a wide color gamut and high dynamic range image content with limited reproduction capabilities while reducing excessive loss of color gradation and reducing hue shift, thereby improving the consistency of visual perception and enhancing the user experience.

[0126] Example 3

[0127] This application also provides a display device capable of performing the image processing method described in Embodiment 1. For details, please refer to the descriptions in Embodiments 2 and 1, which will not be repeated here.

[0128] In some embodiments, the display device is, for example, a display device with limited reproduction capability, such as a display in an endoscope. However, the embodiments of this application do not limit the type of display device, which may also be a display device with strong reproduction capability.

[0129] Figure 9 is a schematic block diagram of the system configuration of a display device according to an embodiment of this application. As shown in Figure 9, the display device 900 may include a processor 901 and a memory 902; the memory 902 is coupled to the processor 901. This figure is exemplary; other types of structures may also be used to supplement or replace this structure to implement telecommunications functions or other functions.

[0130] In some embodiments, processor 901 includes at least one of a central processing unit (CPU) and a graphics processing unit (GPU).

[0131] In some embodiments, processor 901 is a processor with 3D-LUT functionality.

[0132] As shown in Figure 9, the display device 900 may also include: an input device 903, a display panel 904, and a power supply 905.

[0133] In one embodiment, the function of the image processing apparatus described in Embodiment 2 can be integrated into the processor 901, that is, the processor 901 executes the image processing method described in Embodiment 1.

[0134] The processor 901 is configured to: in the HDR domain, perform a first compression on an image based on a first model conforming to visual perception, the first compression including a first brightness compression and a first saturation compression; and in the SDR domain, perform a second compression on the image after the first compression based on a second model conforming to visual perception, the second compression including a second brightness compression and a second saturation compression.

[0135] In some embodiments, in the first compression, a first luminance compression is performed first, and then a first saturation compression is performed according to the degree of the first luminance compression.

[0136] In some embodiments, a second luminance compression is performed first, and then a second saturation compression is performed based on the degree of the second luminance compression.

[0137] In some embodiments, in the first brightness compression, the degree of brightness compression is greater for the brighter parts of the image.

[0138] In some embodiments, the degree of second luminance compression is determined based on at least one of saturation, the relationship between target luminance and signal source luminance, the luminance of the target color space boundary, and the relationship between the target color space boundary and the signal source color space boundary.

[0139] In some embodiments, in the second brightness compression, the degree of brightness compression is greater for the parts of the image with higher saturation.

[0140] In some embodiments, in the second brightness compression, the greater the ratio of the signal source brightness to the target brightness, the greater the degree of the second brightness compression; the smaller the brightness of the target color space boundary, the greater the degree of the second brightness compression; and / or, the smaller the saturation ratio of the target color space boundary to the signal source color space boundary, the greater the degree of the second brightness compression, thereby determining the degree of brightness compression under different hues.

[0141] In some embodiments, in the second saturation compression, the degree of second saturation compression is determined by the degree of second luminance compression; that is, the greater the degree of second luminance compression, the greater the degree of second saturation compression.

[0142] In some embodiments, when there are portions in the second compressed image that extend beyond the target color space boundary, a third compression is performed on the second compressed image based on the second model, the third compression including a third saturation compression.

[0143] In some embodiments, in the third saturation compression, the more portions of the image that extend beyond the target color space boundary exist in the second compressed image, the greater the degree of saturation compression.

[0144] In some embodiments, the first saturation compression employs an approximately elliptic curve compression method.

[0145] In some embodiments, the first model includes the ICTCP color space or the IPT color space.

[0146] In some embodiments, the second model includes the OKLab color space or the IPT color space.

[0147] In this embodiment of the application, the display device 900 does not necessarily include all the components shown in FIG9.

[0148] As shown in Figure 9, the processor 901, sometimes also called a controller or operation control, may include a microprocessor or other processor device and / or logic device. The processor 901 receives input and controls the operation of various components of the display device 900.

[0149] The memory 902 may be, for example, one or more of a cache, flash memory, hard drive, removable media, volatile memory, non-volatile memory, or other suitable devices. The processor 901 can execute the program stored in the memory 902 to perform information storage or processing, etc. The functions of other components are similar to those in existing systems and will not be described further here. The various components of the display device 900 can be implemented using dedicated hardware, firmware, software, or a combination thereof, without departing from the scope of this application.

[0150] As can be seen from the above embodiments, in the HDR domain, the image is subjected to first compression, including first brightness compression and first saturation compression, based on a first model that conforms to visual perception. In the SDR domain, the image after the first compression is subjected to second compression, including second brightness compression and second saturation compression, based on a second model that conforms to visual perception. In this way, since both the compression in the HDR domain and the compression in the SDR domain adopt models that conform to visual perception and perform brightness compression and saturation compression, it is possible to display a wide color gamut and high dynamic range image content with limited reproduction capabilities while reducing excessive loss of color gradation and reducing hue shift, thereby improving the consistency of visual perception and enhancing the user experience.

[0151] This application also provides a computer-readable program, wherein when the program is executed in an image processing apparatus or display device, the program causes the computer to perform the image processing method described in Embodiment 1 in the image processing apparatus or display device.

[0152] This application also provides a storage medium storing a computer-readable program, wherein the computer-readable program causes a computer to perform the image processing method described in Embodiment 1 in an image processing apparatus or display device.

[0153] This application also provides a computer program product, which includes a computer program, wherein the program, when executed by a processor, implements the image processing method described in Embodiment 1.

[0154] The image processing apparatus or display device described in conjunction with the embodiments of this application can directly embody the image processing method in hardware, a software module executed by a processor, or a combination of both. For example, one or more and / or one or more combinations of functional block diagrams shown in FIG6 can correspond to various software modules in a computer program flow, or to various hardware modules. These software modules can correspond to the various steps shown in FIG1. ​​These hardware modules can, for example, be implemented by embedding these software modules using a field-programmable gate array (FPGA).

[0155] The software module can reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art. A storage medium can be coupled to the processor, enabling the processor to read information from and write information to the storage medium; or the storage medium can be an integral part of the processor. The processor and storage medium can reside in an ASIC. The software module can be stored in the memory of the mobile terminal or in a memory card that can be inserted into the mobile terminal. For example, if the display device uses a high-capacity MEGA-SIM card or a high-capacity flash memory device, the software module can be stored in the MEGA-SIM card or the high-capacity flash memory device.

[0156] One or more of the functional block diagrams and / or combinations thereof described in FIG6 can be implemented as a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, or any suitable combination thereof for performing the functions described herein. One or more of the functional block diagrams and / or combinations thereof described in FIG6 can also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in communication with a DSP, or any other such configuration.

[0157] The present application has been described above with reference to specific embodiments. However, those skilled in the art should understand that these descriptions are exemplary and not intended to limit the scope of protection of the present application. Those skilled in the art can make various modifications and variations to the present application based on its spirit and principles, and these modifications and variations are also within the scope of the present application.

Claims

1. An image processing method, the method comprising: In the HDR domain, the image is first compressed based on a first model that conforms to visual perception. The first compression includes first brightness compression and first saturation compression. as well as In the SDR domain, a second compression is performed on the image after the first compression based on a second model that conforms to visual perception. The second compression includes a second brightness compression and a second saturation compression.

2. The method according to claim 1, wherein, In the first compression, the first brightness compression is performed first, and then, based on the degree of the first brightness compression, the first saturation compression is performed, and / or, In the second compression, the second brightness compression is performed first, and then the second saturation compression is performed according to the degree of the second brightness compression.

3. The method according to claim 1 or 2, wherein, In the first brightness compression, the higher the brightness of the portion of the image, the greater the degree of the first brightness compression; In the first saturation compression, the greater the degree of the first brightness compression, the greater the degree of the first saturation compression, and / or, the smaller the brightness of the signal source color space boundary, the greater the degree of the first saturation compression, and / or, the smaller the saturation ratio between the target color space boundary and the signal source color space boundary, the greater the degree of the first saturation compression.

4. The method according to claim 1 or 2, wherein, The degree of the second brightness compression is determined based on at least one of saturation, the relationship between the target brightness and the signal source brightness, the brightness of the target color space boundary, and the relationship between the target color space boundary and the signal source color space boundary.

5. The method according to claim 4, wherein, In the second brightness compression, the higher the saturation of the portion of the image, the greater the degree of the second brightness compression; And / or, In the second brightness compression, the greater the ratio of signal source brightness to target brightness, the greater the degree of second brightness compression; And / or, In the second brightness compression, the smaller the brightness of the target color space boundary, the greater the degree of the second brightness compression. The greater the degree; And / or, In the second brightness compression, the smaller the saturation ratio between the target color space boundary and the signal source color space boundary, the greater the degree of the second brightness compression.

6. The method of claim 1 or 2, wherein, In the second saturation compression, the greater the degree of the second brightness compression, the greater the degree of the second saturation compression.

7. The method of any one of claims 1-6, wherein, The method further includes: When there are portions in the image that have been compressed by the second method that extend beyond the boundary of the target color space, a third compression is performed on the image that has been compressed by the second method based on the second model. The third compression includes a third saturation compression.

8. The method according to claim 7, wherein, In the third saturation compression, the more parts of the image that have exceeded the target color space boundary after the second compression, the greater the degree of the third saturation compression.

9. The method according to any one of claims 1-8, wherein, The first saturation compression adopts an approximate elliptic curve compression method.

10. The method according to any one of claims 1-9, wherein, The first model includes the ICTCP color space or the IPT color space, and / or, The second model includes the OKLab color space or the IPT color space.

11. An image processing apparatus, the apparatus comprising: The first compression unit performs a first compression on the image in the HDR domain based on a first model that conforms to visual perception. The first compression includes a first brightness compression and a first saturation compression. as well as The second compression unit, in the SDR domain, performs a second compression on the image after the first compression based on a second model that conforms to visual perception. The second compression includes a second brightness compression and a second saturation compression.

12. A display device, the device comprising: A memory that stores computer programs; as well as A processor that, when executing the computer program, implements the method described in any one of claims 1-10.

13. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method of any one of claims 1-10.

14. A computer program product comprising a computer program that, when executed by a processor, implements the method of any one of claims 1-10.

Images