Image processing device, image processing method and storage medium

Abstract

The present invention discloses an image processing device, an image processing method and a storage medium. The image processing device comprises: a first obtaining unit configured to obtain HDR data representing a high dynamic range (HDR) image; a second obtaining unit configured to obtain printing information to perform printing based on the HDR data obtained by the first obtaining unit; a third obtaining unit configured to obtain display information of a display device to be displayed based on the HDR data; and a conversion unit configured to convert the dynamic range of brightness of the HDR data obtained by the first obtaining unit into a dynamic range for performing printing according to the printing information obtained by the second obtaining unit based on the display information obtained by the third obtaining unit.

Description

Technical Field

[0001] The present invention relates to an image processing device capable of processing high dynamic range data, an image processing method and a non-transitory computer-readable storage medium storing a program. Background Art

[0002] International Publication No. 2018 / 092711 discloses converting the brightness dynamic range of HDR data of a high dynamic range (HDR) static image into static image data having a narrower dynamic range determined by the reflection brightness of a printed sheet. HDR data is used as imaging data for moving images, static images, and the like. Recently, the maximum brightness that can be displayed in a display that displays HDR data has been improved, and HDR data from the highlight side to the shadow side of an image can be displayed simultaneously with high image quality.

[0003] For example, in ITU-R (International Telecommunication Union Radiocommunication Sector) Recommendation BT.2100-2 (07 / 2018) Picture parameter values ​​for high dynamic range television for use in production and international project communication, Hybrid Log-Gamma (HLG) and Perceptual Quantization (PQ) are defined as two image transfer functions for HDR data. In the transfer function, the transfer function and bit count are defined so that tonal discontinuities are not visually detected in the image transfer.

[0004] The imaging method is defined by OETF (Optical-Electrical Transfer Function) on the imaging side, EOTF (Electro-Optical Transfer Function) on the display side, and OOTF (Optical-Optical Transfer Function) representing the overall characteristics of conversion from scene light to display light.

[0005] The HLG method is a method of defining the OETF on the camera side described above by processing the range from black to white as relative tones. The EOTF on the display side is formed by the inverse function of the OETF and the OOTF representing the overall characteristics of the conversion from scene light to display light. In the HLG method, the system gamma for determining the characteristics of the OOTF is applied only on the luminance component. In addition, by considering how the image quality varies between displays with different maximum displayable luminance values, the system gamma is determined according to the luminance of each display. In addition, the PQ method is a method of defining the EOTF on the camera side ... EOTF at a maximum of 10,000 cd / m 2 The method of defining the EOTF on the display side by using the absolute value of to represent the display side brightness. The OETF on the camera side is formed by the inverse function of the OOTF and the EOTF.

[0006] On the other hand, the dynamic range of print output tends to be narrower than that of HDR data. In recent years, before the emergence of HDR displays with a wide display brightness range, SDR (Standard Dynamic Range) displays have been the mainstream. Conventionally, the maximum brightness value to be used when displaying SDR data on a display is usually regarded as a fixed 100cd / m 2 Conversely, the maximum value of the brightness to be used when displaying the HDR data on the display may be changed according to the brightness value defined by the HDR data or the maximum brightness value of the HDR display. Summary of the invention

[0007] The present invention provides an image processing apparatus, an image processing method, and a non-transitory computer-readable storage medium storing a program that allow print output corresponding to display information of a display device.

[0008] A first aspect of the present invention provides an image processing device, comprising: a first obtaining unit configured to obtain HDR data representing a high dynamic range (HDR) image; a second obtaining unit configured to obtain printing information to perform printing based on the HDR data obtained by the first obtaining unit; a third obtaining unit configured to obtain display information of a display device to perform display based on the HDR data; and a conversion unit configured to convert the dynamic range of brightness of the HDR data obtained by the first obtaining unit into a dynamic range for performing printing according to the printing information obtained by the second obtaining unit based on the display information obtained by the third obtaining unit.

[0009] A second aspect of the present invention provides an image processing method to be performed in an image processing device, the image processing method comprising: an image processing method to be performed in an image processing device, the method comprising: obtaining HDR data representing a high dynamic range (HDR) image; obtaining printing information to print based on the obtained HDR data; obtaining display information of a display device to perform display based on the HDR data; and based on the obtained display information, converting the dynamic range of brightness of the obtained HDR data into a dynamic range used to perform printing based on the obtained printing information.

[0010] A third aspect of the present invention provides a non-transitory computer-readable storage medium storing a program to enable a computer to perform the following operations: obtain HDR data representing a high dynamic range (HDR) image; obtain printing information for printing based on the obtained HDR data; obtain display information of a display device to perform display based on the HDR data; and, based on the obtained display information, convert the dynamic range of brightness of the obtained HDR data into a dynamic range used to perform printing based on the obtained printing information.

[0011] According to the present invention, it is possible to perform print output corresponding to display information of a display device.

[0012] Further features of the present invention will become apparent from the following detailed description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 is a block diagram showing the overall configuration of a printing system;

[0014] Figure 2 is a block diagram showing the configuration of an image processing unit;

[0015] Figure 3 is a flow chart showing the steps of image processing;

[0016] Figure 4A and Figure 4B are diagrams showing conversion curves respectively;

[0017] Figure 5A and Figure 5B are diagrams showing conversion curves respectively;

[0018] Figure 6 is a graph showing a conversion curve;

[0019] Figure 7 is a process flow diagram illustrating the use of dynamic range conversion processing;

[0020] Figure 8 is a flowchart showing the steps of a dynamic range conversion process;

[0021] Fig. 9A , Fig. 9B and Fig. 9C are graphs respectively showing conversion curves; and

[0022] Fig.10 is a flowchart showing the steps of image processing. DETAILED DESCRIPTION

[0023] The embodiments will be described in detail below with reference to the accompanying drawings. Note that the following embodiments are not intended to limit the scope of the present invention. Although multiple features are described in the embodiments, it is not limited to the invention that satisfies all the features, and multiple such features can be appropriately combined. In addition, in the accompanying drawings, the same reference numerals are given to the same or similar configurations, and their repeated descriptions are omitted.

[0024] In the case where the above-mentioned HDR data is HDR data of the HLG method, it will be data whose hue is defined based on a relative brightness value. Therefore, when the data is to be displayed on multiple displays, as described above, the brightness of the image to be displayed will change according to the maximum brightness value of each display. In the case of the PQ method, the data is defined by an absolute brightness value. Therefore, when the data is to be displayed on multiple displays, for a brightness range that can be reproduced in common between displays, the brightness of the image to be displayed in each display will be the same. However, for a brightness range such as a highlight that cannot be reproduced in common between displays, the brightness of the image to be displayed will change according to the maximum brightness display of each display. In the case where the brightness dynamic range of the data and the displayable brightness range of the display are the same, the data will be displayed faithfully. On the other hand, in a display with a narrow brightness range relative to the data, the highlight that cannot be displayed, for example, is completely displayed as white at the maximum brightness value. On the other hand, as described in the International Publication No. 2018 / 092711, when the HDR data is input to a printing device, the HDR data will be output as a unified print corresponding to the data converted into a printable dynamic range.

[0025] In this way, although the display device will adaptively change the display of the same HDR data according to conditions such as the maximum brightness value, the printing device will be required to output the HDR data corresponding to the display device. Therefore, even if displays (display devices) with different maximum brightness values ​​appear recently, the printing device cannot obtain a print output corresponding to the display of each display.

[0026] According to one aspect of the present invention, a printout corresponding to display information of a display device can be obtained.

[0027] [First embodiment]

[0028] [System Layout]

[0029] Figure 1 101 is a block diagram showing the overall arrangement of a printing system to which an image processing apparatus is applied according to the present embodiment. The printing system includes a personal computer apparatus (information processing apparatus) 101 (hereinafter also referred to as “PC”), a display apparatus 102 and an output apparatus 103 .

[0030] The display device 102 is connected to the PC 101 via a display I / F. The display device 102 is an HDR (high dynamic range) display and is connected to the PC 101 via an HDMI interface. The connection between the PC 101 and the display device 102 is not limited to the HDMI interface, and may be other connection methods as long as it complies with a standard that can transmit HDR (high dynamic range) data. In addition, the display information transmitted between the PC 101 and the display device 102 may be transmitted via a display I / F 113 using a USB (Universal Serial Bus) that uses a transmission path different from the HDMI interface (described later). However, the display information transmission method is not limited to a USB cable as long as information can be communicated bidirectionally between the display device 102 and the PC 101 or the output device 103.

[0031] In addition, the output device 103 is connected to the PC 101 via an interface such as a network, a USB cable, or a local bus. In this embodiment, a structure using an inkjet printer (image processing device) as an example of the output device 103 will be described. The PC 101 performs operations such as issuing a print control instruction to the output device 103, transmitting necessary information and data, etc. The storage device 105 stores and manages the OS, system programs, various application software, parameter data required for this embodiment, etc. The storage device 105 is formed by, for example, a hard disk or a flash ROM. The CPU 104 uses the working memory 107 to read out the software or program stored in the storage device 105 and executes the processing. The operation unit 106 (hereinafter also referred to as "UI") used as a user interface accepts input related to the execution of the processing from the user and displays it to the user. The operation unit 106 includes input devices such as a keyboard, a mouse, etc. In addition, the data input / output device 108 inputs / outputs data to / from an external storage medium such as an SD card, and can input / output data to / from an external storage medium such as a storage device for storing data of a camera. In addition, by directly connecting an imaging device (not shown) to the data input / output apparatus 108 or the data transfer unit 109, data can be input / output from / to the imaging device without the intervention of an external storage medium.

[0032] The output device 103 includes a data transmission unit 109, a printer control unit 112, an image processing unit 110, a printing unit 111, and receives print data from the PC 101. In the present embodiment, the print data includes HDR data as input image data, display information of the display device 102, image processing parameters and printer control data as unique data of a storage medium, and print information such as print quality, print medium, etc. selected by a user on the operation unit 106. In this case, the print medium is, for example, a paper medium such as a print sheet.

[0033] The data transfer unit 109 obtains HDR data, display information of the display device 102, image processing parameters, and print information from the print data received by the PC 101, and transmits the obtained information and data to the image processing unit 110 to obtain printer control data, and then sends the obtained printer control data to the printer control unit 112. In the present embodiment, the HDR data stored in the storage device 105 of the PC 101 is used as input image data received by the output device 103. In addition, although the image processing unit 110 is formed in the display device 102 in the present embodiment, the image processing unit 110 may be formed in the PC 101.

[0034] In addition, image processing parameters and printer control data are stored in the storage device 105 of the PC 101 or the storage device (hard disk, ROM, etc.) of the output device 103 (not shown). It can be arranged so that these information will be selected based on the printing information included in the print data, and these information will be sent to the image processing unit 110 and the printer control unit 112. The print control unit 112 controls the operation of the print unit 111 according to the printer control data. The print unit 111 performs printing according to the inkjet printing method. Although the inkjet printing method is exemplified as a method used in the printing performed by the print unit 111 in the present embodiment, other printing methods such as an electrophotographic method may also be used. The display device 102 includes a display controller 114 that controls image display, and the display controller 114 generates, for example, display data.

[0035] Figure 2 1 is a block diagram showing the structure of the image processing unit 110 according to the present embodiment. In the image processing unit 110 according to the present embodiment, HDR data, display information of the display device 102, and print information are input to the dynamic range conversion unit 201. As described later, the dynamic range conversion unit 201 converts the HDR data into image data of a dynamic range that can be input to the output image generation unit 202 using each piece of input information. The dynamic range of the image data to be input to the output image generation unit 202 is narrower as a brightness range than the high dynamic range of the input HDR data. The dynamic range to be input to the output image generation unit 202 is, for example, 100 cd / m with a maximum brightness of SDR data. 2 In addition, a dynamic range whose maximum value is set by the reflection brightness specified by the sheet information set by the user may also be used. In this embodiment, it is assumed that the dynamic range conversion unit 201 converts the dynamic range of HDR data into the dynamic range of SDR data.

[0036] Next, the output image generation unit 202 generates data to be used for printing by the print head of the printing unit 111 for the image data (RGB data) output from the dynamic range conversion unit 201 .

[0037] Figure 3 is a flowchart showing image processing according to the present embodiment. In step S301, the dynamic range conversion unit 201 obtains RGB data of HDR data. The HDR data according to the present embodiment is stored according to the above-mentioned HLG method, which defines the hue based on the relative brightness value. At this stage, the hue is defined for each of the R, G and B elements based on the relative brightness value. Although the HLG method is adopted in the present embodiment, other methods can be adopted as long as the hue is defined relatively. In the present embodiment, the HDR data obtained in step S301 is also used in the display operation performed by the display device 102 as an HDR display.

[0038] In step S302, the dynamic range conversion unit 201 obtains the display information of the display device 102. In the present embodiment, an image represented by HDR data is displayed on the display device 102. At the time point when the user who wants to perform printing through the output device 103 presses the print instruction button on the print application or the print button on the panel of the output device 103, the dynamic range conversion unit 201 obtains the display information storing the display state of the display device 102 at that time via the display device 102 or the PC 101. In addition, in the case where the display information is stored in the PC 101 at an appropriate time synchronized with the display, the dynamic range conversion unit 201 can obtain the display information from the PC 101. In addition, it can be arranged so that the display information will be obtained at the time point when the print application or the output device 103 has been activated, or the display information will be re-obtained at the moment when the display setting has been changed on the display device 102 side. That is, as long as the display state of the display (display device 102) viewed by the user at the time of printing can be obtained. In addition, although a single display is connected to the PC 101 in the present embodiment, a plurality of displays may be simultaneously connected to a single PC and used in some cases. In the case of performing extended display through multiple displays (display across multiple displays), the display that is displaying the print preview of HDR data will be identified, and the display information of the playback will be obtained. In addition, in the case of repeated display between multiple displays (the same display between multiple displays), the user can select the display from which to obtain display information or can pre-set the display from which to preferably obtain display information.

[0039] In this embodiment, the dynamic range information and system gamma information of the display are obtained as display information. The dynamic range information includes the maximum brightness value (maximum brightness value) that can be displayed when the HDR data is displayed on the HDR display and the brightness value for displaying black on the display. In this embodiment, for example, 1000 cd / m 2 and 0cd / m 2 As the maximum brightness value and the brightness value of black of the display, respectively. When the display setting of the display device 102 is changed, the dynamic range of the display changes, and the dynamic range of brightness that the display itself can display can change according to the performance of the display. In addition, the system gamma information includes a gamma value γ and information indicating that the display is a display that complies with BT.2100. Since the display complies with BT.2100, the gamma value γ of the OOTF of the HLG method defined by BT.2100 can be calculated by the following equation:

[0040] γ=1.2+0.42Log 10 (L w / 1000)...(1)

[0041] BT.2100 is defined by equation (1), and for example, at a maximum luminance value of 1,000 cd / m 2 However, the user may change the gamma value γ via display settings depending on an object to be displayed, and in this case, the gamma value γ set by the display settings is used as display information.

[0042] In step S303, the dynamic range conversion unit 201 obtains printing information. The printing information is information for specifying the dynamic range after the dynamic range conversion. In this embodiment, for example, printing mode information is obtained as the printing information, and whether the input to the output image generation unit 202 is SDR data of sRGB values ​​is specified based on the printing mode information. Thus, the maximum brightness value 100 cd / m 2 The brightness dynamic range after conversion to SDR data is specified. Printing information can be obtained by obtaining information indicating the sheet type, and the printing information is sufficient as information that can specify the reflection brightness specified according to the sheet type. Other information can be used as long as the information is information from which the brightness dynamic range information at the time of printing can be obtained. In addition, the printed matter can be illuminated with illumination of various illuminances based on the observation environment. If the printed matter is to be illuminated with illumination, the brightness dynamic range of the sheet will be expanded, especially due to the increase in the brightness of the whiteness of the paper. Therefore, the reflection brightness of the sheet when the sheet is illuminated with illumination can be obtained as the brightness dynamic range information at the time of printing.

[0043] In step S304, the dynamic range conversion unit 201 generates image data obtained by converting the luminance dynamic range from the HDR data based on the display information obtained in step S302 and the print information obtained in step S303. That is, in the present embodiment, the dynamic range conversion process is performed to obtain the image data obtained from the display information and based on the obtained display information having a maximum luminance value of 1,000 cd / m 2 The brightness dynamic range is converted to the value obtained from the printed information and has a maximum brightness value of 100cd / m 2 As described above, the luminance dynamic range after the luminance dynamic range conversion becomes a luminance dynamic range represented by an absolute luminance value specified by the print information.

[0044] As described above, in, for example, ITU-R (International Telecommunication Union Radiocommunication Sector) BT.2100 Recommendation, hybrid log gamma (HLG) and perceptual quantization (PQ) are defined as two image transfer functions of HDR data. The HDR data according to the present embodiment is data converted by the transfer function OETF of the above-mentioned HLG method. Therefore, the dynamic range conversion unit 201 performs brightness conversion on the HDR data so that the transfer function EOTF (inverse function of OETF) and the transfer function OOTF of the HLG method determined by the following equation convert the brightness signal level x of the HDR data into the brightness signal level y:

[0045] y=OOTF[OETF -1 [max0,(1-β)E′+β)]]

[0046]

[0047]

[0048] OOTF[E]=αY s γ-1 E…(2)

[0049] Where γ is the value calculated by equation (1), L W is the maximum brightness of the display, L B is the brightness value of black on the display. In this embodiment, it is assumed that L B is 0. E' is the HLG method signal, and x is a signal normalized to the range between 0 and 1. α is the variable of user gain, Y S is the standard brightness value, and E is a linear light signal normalized to the range from 0 to 1.

[0050] Next, the dynamic range conversion unit 201 sets the data obtained from the OOTF process to a maximum luminance value of 1000 cd / m 2and performs dynamic range conversion processing to convert the data into a dynamic range corresponding to the value obtained from the print information and having a maximum brightness value of 100 cd / m 2 In this embodiment, the dynamic range conversion unit 201 converts the RGB data obtained from the OOTF process into data having a brightness value Y and a color difference CbCr by the following equation:

[0051] Y=0.29900×R+0.58700×G+0.114400×B...(3)

[0052] Cb=-0.16874×R-0.33126×G+0.50000×B...(4)

[0053] Cr=0.50000×R-0.41869×G-0.081×B...(5)

[0054] The dynamic range conversion unit 201 performs brightness dynamic range conversion based on Figure 4A The conversion curve represented by the solid line in the graph shown in FIG. 1 (the horizontal axis represents the input brightness value, and the vertical axis represents the output brightness value) converts the data of the converted brightness value Y into the brightness value Y′. Figure 4A The short dashed line shown in FIG. 1 indicates that the input and output are linear. Figure 4A As shown by the solid line, a linear conversion is performed from a dark part to a part of a certain brightness, and a conversion using the Log characteristic is performed in a bright part. Figure 4A As shown, preferably, a dynamic range compression process is performed while reproducing the highlight area of ​​the HDR data to maintain the contrast of the image. In addition, Figure 4B It adopts Figure 4A A similar dynamic range compression method is shown in the graph, and the maximum brightness of the display is 2,000 cd / m 2 The brightness relationship before and after dynamic range conversion in the case of . Figure 4A and Figure 4B As shown, the dynamic range conversion can be performed according to the displayable dynamic range of the display based on the display information obtained in step S302 and the print information obtained in step S303. Subsequently, the dynamic range conversion unit 201 combines the converted brightness value Y' and the color difference component to perform conversion to RGB data through the following equation:

[0055] R=Y+1.40200×Cr...(6)

[0056] G=Y–0.34414×Cb–0.71414×Cr...(7)

[0057] B=Y+1.77200×Cb...(8)

[0058] Although luminance dynamic range compression has been described above, color gamut compression processing can also be performed on the color gamut to convert the wide color gamut space of HDR data (such as ITU-R BT.2020) into the color gamut of SDR data (such as ITU-R BT.709).

[0059] Next, in step S305, the output image generation unit 202 generates output image data to be output to the printing unit 111. For example, the output image generation unit 202 performs a color conversion process to convert the SDR data (RGB data) output in step S304 into device-dependent RGB data. The output image generation unit 202 performs an ink color resolution process to convert the device-dependent RGB data into ink color data, and performs a tone correction process to perform a tone correction so that the ink color data will be linearly associated with the tone characteristics of the printing device. In addition, the output image generation unit 202 performs a halftone process for converting the ink color data into ink dot ON / OFF information, a mask data conversion process for generating binary data to be printed in each print scan of the print head, and the like.

[0060] Next, in step S306 , the output image generation unit 202 sends the generated output image data to the printing unit 111 , and then outputs the image on a printing medium.

[0061] As described above, the HDR data of the HLG method is displayed differently according to the performance of the display device 102. According to the present embodiment, the display information of the display device 102 viewed by the user and the printing information of the output device 103 can be used when the HDR data of the HLG method is printed, so that the dynamic range conversion corresponding to the display state of the display can be performed. Therefore, the print output corresponding to the display of each of the plurality of display devices 102 having different performances can be obtained.

[0062] [Second embodiment]

[0063] Hereinafter, points different from the first embodiment are described in the second embodiment. This embodiment will describe a case where HDR data of the PQ method is input.

[0064] In this embodiment, the PQ method of defining hue by absolute brightness value is used to record Figure 3 In this embodiment, it is assumed that 10,000 cd / m 2 information.

[0065] In step S302, the dynamic range conversion unit 201 obtains the dynamic range information of the display and the brightness dynamic range conversion method as display information. The description of the dynamic range information of the display is similar to that in the first embodiment. The brightness dynamic range conversion method is to convert the maximum brightness value 10,000 cd / m 2 The information about the method of converting the luminance dynamic range of the defined PQ method into the luminance dynamic range of the display is provided. This information is sufficient as information that allows the conversion to be performed in the process of step S304, and the information may be a conversion method name in the case where the conversion is to be performed by a known method, or may be information storing a path of a conversion module (for example, a conversion curve to be described later).

[0066] Figure 5A and Figure 5B is a graph showing two conversion curves for luminance dynamic range conversion of a display. Figure 5A The conversion method is shown, wherein the value is higher than 1,000 cd / m 2 Each data of the high brightness portion that cannot be displayed on the display included in HDR is displayed as 1,000cd / m 2 .on the other hand, Figure 5B Shown with Figure 4A and 4B A similar method is used in which the dark part data is linearly converted to a specific brightness part, and the highlight part data is converted by the Log characteristic. In this embodiment, since it is HDR data of the PQ method, although Figure 5A The conversion and Figure 5B The conversion will not result in Figure 5A and Figure 5B The difference between the brightness range 501 and the display operation performed by the display can be Figure 5A and Figure 5B Differences in display operation are observed between the brightness range 502.

[0067] The print information obtaining process of step S303 is similar to that according to the first embodiment.

[0068] In step S304, the dynamic range conversion unit 201 generates data in which the luminance dynamic range is converted according to the luminance dynamic range of the HDR data based on the display information obtained in step S302 and the print information obtained in step S303. In this embodiment, the dynamic range conversion unit 201 performs dynamic range conversion processing to convert the maximum luminance value to 10,000 cd / m 2 The HDR data is converted into a dynamic range information from a maximum brightness value of 100cd / m 2 The dynamic range of brightness obtained by printing information.

[0069] The HDR data according to the present embodiment is data converted by a transfer function (an inverse function of EOTF) according to the above-mentioned PQ method. Therefore, the dynamic range conversion unit 201 performs brightness conversion on the HDR data, thereby converting the brightness signal level x of the HDR data into a brightness signal level y by using the EOTF of the PQ method defined by the following equation:

[0070] y=EOTF[E′]=10000Y

[0071]

[0072] In this embodiment, L B is 0. E' is the PQ method signal, and x is a signal normalized to the range between 0 and 1.

[0073] Next, the dynamic range conversion unit 201 sets the data converted from the EOTF process to a maximum luminance value of 10,000 cd / m 2 The data is converted into a luminance dynamic range with a maximum luminance value of 100 cd / m 2 In this embodiment, the dynamic range conversion unit 201 converts the RGB data obtained by the EOTF processing into the data of the brightness value Y and the color difference CbCr by equations (3), (4) and (5). Then, the dynamic range conversion unit 201 converts the RGB data obtained by the EOTF processing into the data of the brightness value Y and the color difference CbCr. Figure 5A The conversion curve shown in FIG. 1 converts the data of the converted brightness value Y, and further converts the data of the converted brightness value Y to the data of the converted brightness value Y. Figure 4A The conversion curve shown is converted to obtain SDR data as the conversion result. Figure 4A and Figure 5A Each conversion operation represented by the conversion curve shown in FIG. 1 can therefore be performed by using the 1DLUT obtained by combining the 1DLUTs of the respective conversion curves. Subsequently, the converted brightness values ​​and color differences can be combined to perform conversion to RGB data by equations (6), (7), and (8). In addition, although the above description has been used Figure 5A , but you can also use Figure 5B .

[0074] The processes of steps S305 and S306 are similar to those according to the first embodiment.

[0075] As described above, in the HDR data of the PQ method, the highlight part of the data will be displayed differently according to the performance of the display device 102. According to the present embodiment, the display information of the display device 102 viewed by the user and the printing information of the output device 103 can be used when printing the HDR data of the PQ method, so that the dynamic range conversion corresponding to the display state of the display can be performed. Thereby, the print output corresponding to the display of each of the plurality of display devices 102 having different performances can be obtained.

[0076] [Third embodiment]

[0077] The differences from the first and second embodiments will be described in the third embodiment below. This embodiment will describe a case where HDR data of the PQ method is input and luminance dynamic range conversion is performed on the display device 102 side to be displayed on an SDR display. In this embodiment, the display device 102 is an SDR display.

[0078] In this embodiment, Figure 3 In step S301, HDR data of the PQ method is obtained.

[0079] In step S302, the dynamic range information of the display and the brightness dynamic range conversion method are obtained in a manner similar to the second embodiment. Since the display device 102 is an SDR display in this embodiment, the 100 cd / m 2 and 0cd / m 2 The description of the method for converting the dynamic range of brightness is similar to that according to the second embodiment.

[0080] Figure 6 : is a diagram showing a conversion curve of the luminance dynamic range conversion of the display according to the present embodiment. Figure 6 Corresponds to Figure 5B The maximum brightness value of the display's brightness dynamic range is set to 100cd / m 2 Although in this embodiment, the following Figure 6 The luminance dynamic range conversion shown is shown, but the dynamic range conversion method will vary depending on the manufacturer or model of the display as shown in the conversion curves 601 and 602.

[0081] The obtaining of the printing information at step S303 is performed in a manner similar to that according to the first embodiment.

[0082] In step S304, the dynamic range conversion unit 201 generates image data obtained by converting the luminance dynamic range of the HDR data based on the display information obtained in step S302 and the print information obtained in step S303. In the present embodiment, the dynamic range conversion unit 201 performs a dynamic range conversion process to convert the information of the luminance dynamic range held by the HDR data into an image data whose luminance dynamic range has a maximum luminance value of 100 cd / m corresponding to the value obtained from the print information. 2 data.

[0083] As described in the second embodiment, the HDR data is converted into information corresponding to the brightness signal level of the display. Subsequently, a dynamic range conversion process is performed on the converted data to convert the brightness dynamic range into a value corresponding to the maximum brightness value of 100 cd / m obtained from the print information. 2 In this embodiment, the converted brightness dynamic range of the display side and the printing side is equal to 100cd / m 2 Therefore, in step S304 of this embodiment, by using Figure 6 The conversion curve shown performs brightness dynamic range conversion on the converted brightness value Y data, and combines the converted brightness and color difference components to perform conversion to SDR RGB data through formulas (6), (7) and (8).

[0084] The processes of steps S305 and S306 are similar to those of the first embodiment.

[0085] As described above, in the case of displaying on an SDR display by performing a luminance dynamic range conversion process on the display device 102 side, the display may be different according to the fact that the luminance dynamic range conversion method is different according to the display device 102. According to the present embodiment, by obtaining the display information of the SDR display and the printing information of the output device 103 when printing HDR data, a dynamic range conversion corresponding to the display state of the display can be performed. Thereby, a print output corresponding to the display of each of a plurality of SDR displays having different performances can be obtained.

[0086] [Fourth embodiment]

[0087] In the following, the fourth embodiment will describe the differences from the first to third embodiments. In the first to third embodiments, processing is performed to convert data into a brightness signal level of a display by equations (2) and (9). This embodiment will describe an arrangement for obtaining data that has been converted into a brightness signal level of a display. In this embodiment, Figure 3 The HDR data obtained in the process of step S301 may conform to the HLG method or the PQ method.

[0088] In step S302, the dynamic range conversion unit 201 obtains the brightness dynamic range information of the display and the data converted into the brightness signal level of the display and in which the hue is defined by the absolute value as the display information. That is, in the case of obtaining HDR data of the HLG method, the OOTF-processed RGB data according to the first embodiment will be obtained. In addition, in the case of obtaining HDR data of the PQ method, the RGB data obtained by using the Figure 5A RGB data obtained by converting the RGB data that has been subjected to the EOTF processing according to the second embodiment according to the conversion curve of . Data obtained by performing the OETF processing (inverse function of EOTF) according to the PQ method on these data can also be obtained. These data can be generated by the display controller 114 during the image processing process of the display processing in the display device 102, and can be generated in the display device 102 or in a device such as PC 101 outside the display device 102. These data are stored in, for example, a storage unit of the display device 102 or a storage unit of a device such as PC 101 outside the display device 102. Data corresponding to the display state of the display at the time point when the user presses the print button can be obtained from the storage unit.

[0089] The print information obtaining process of step S303 is similar to that according to the first embodiment.

[0090] In step S304, the dynamic range conversion unit 201 generates image data in which the luminance dynamic range is converted from the HDR data based on the display information obtained in step S302 and the print information obtained in step S303. Figure 4A The dynamic range conversion unit 201 converts the maximum brightness value to 1000 cd / m 2 The information of the brightness dynamic range held by the data is converted into a brightness dynamic range having a maximum brightness value of 100 cd / m corresponding to the value obtained from the printing information. 2 data.

[0091] The processing of steps S305 and S306 is similar to that of the first embodiment.

[0092] As described above, data converted into the brightness signal level of the display and whose tone is defined by the absolute brightness value is obtained as display information, and brightness dynamic range conversion is performed. Therefore, a print output corresponding to the display of each of the plurality of display devices 102 having different performances can be obtained. In addition, since the dynamic range conversion unit 201 will use the HDR data that has been processed by OOTF or EOTF, the processing load on the output device 103 side can be reduced.

[0093] Although the brightness dynamic range conversion is performed by using YCbCr in this embodiment, the brightness dynamic range conversion may be performed by using ICtCp. ICtCp is a color space with a high dynamic range and a wide color gamut signal as a target. Here, let I be the brightness component and CtCp be the color difference component. The brightness component I is information that takes into account the human visual characteristics within a wide brightness range. Therefore, using the ICtCp color space enables the brightness dynamic range conversion to be performed by taking into account the human visual characteristics.

[0094] In this embodiment, according to Figure 7 and Figure 8 The process shown in the flowchart performs dynamic range conversion instead of based on Figure 4A and Figure 4B The characteristics shown in perform luminance dynamic range conversion.

[0095] exist Figure 7 In step S701, the dynamic range conversion unit 201 separates the RGB data obtained from the OOTF processing by equations (3), (4) and (5) into a luminance component and a color difference component.

[0096] In step S702, the dynamic range conversion unit 201 performs processing to separate the low-frequency component and the high-frequency component of the data converted into the brightness component. This is because based on the Retinex theory, the processing changes between the low-frequency component and the high-frequency component. The Retinex theory is a theory that simulates how the human brain perceives light and color. According to the theory, the intensity of light entering the eye can be represented by the product of the reflectivity of the object and the illumination light that illuminates the object, and the brightness and color perceived by a person depend more on the relative change from the surrounding environment than the absolute light amount. Here, the absolute light amount is the illumination light that illuminates the object, and the relative change is the reflectivity of the object.

[0097] In step S702, the dynamic range conversion unit 201 extracts the low-frequency component of the image data as the illumination light component that illuminates the object. A low-pass filter is applied to create the low-frequency component. As a processing method, a spatial filter may be applied, or the target frequency component may be temporarily converted into a spatial frequency by FFT and converted back into a frequency component by IFFT after filtering processing. In consideration of human visual characteristics, the frequency to be targeted may be determined based on the sheet size or observation distance of the sheet on which the print is to be observed. In order to obtain the high-frequency component, a high-pass filter which is the opposite of the low-pass filter may be applied, or the obtained low-frequency component may be subtracted from the original image.

[0098] In step S703, the dynamic range conversion unit 201 performs dynamic range conversion processing on the low-frequency component based on the input luminance dynamic range information and the output luminance dynamic range information. Figure 8 The processing of step S703 is described in detail.

[0099] In step S704, the dynamic range conversion unit 201 performs contrast correction processing on the high-frequency component. The contrast correction processing is a process of multiplying the obtained image by a coefficient k. In the case where the printed product is to be faithfully close to the input data, k is approximately equal to 1. In the case where deterioration such as ink bleeding occurs on the printed product, a value equal to or greater than 1 is set as the coefficient k.

[0100] In step S705, the dynamic range conversion unit 201 combines the image data for which the low-frequency components have been dynamically range converted with the image data for which the high-frequency components have been contrast corrected, thereby compressing the image data into a predetermined dynamic range and obtaining a brightness image with corrected contrast.

[0101] In step S706, the dynamic range conversion unit 201 combines the brightness component and the color difference component to perform conversion to RGB data through equations (6), (7) and (8). After the processing of step S706, Figure 7 The processing is finished.

[0102] Will be used by Figure 8 The flowchart describes the dynamic range conversion processing of step S703.

[0103] In step S801, the dynamic range conversion unit 201 calculates the compression range. In this embodiment, the dynamic range conversion process is performed to convert the maximum luminance value obtained from the display information to 1,000 cd / m 2 The luminance dynamic range is converted to a value with a maximum luminance value of 100 cd / m obtained from the printed information. 2In addition, the dynamic range conversion unit 201 obtains the exposure brightness value Ya from the metadata of the HDR data. This is where the user sets the exposure during the camera operation. In this embodiment, it is assumed that the exposure brightness value Ya is 18 cd / m 2 .

[0104] In step S802, the dynamic range conversion unit 201 divides the image of the HDR data into a plurality of regions. The regional division of the image can be performed by dividing the image into predetermined rectangular size regions or by creating groups of similar brightness pixels based on information of the brightness data. In the latter case, the contrast of a specific brightness range after regional division can be restored, and an image with better contrast can be obtained. In addition, not only brightness data but also RGB data can be used. Thus, image recognition can be performed by RGB data, and a method of restoring contrast according to each type of recognition region can be adopted.

[0105] In step S803 , the dynamic range conversion unit 201 creates a conversion curve for each area divided in step S802 . Fig. 9A , Fig. 9B and Fig. 9C is a diagram showing an example of a conversion curve. Fig. 9A is a graph showing the conversion curve for a given area. The horizontal axis represents the input brightness value, the vertical axis represents the output brightness value, and the thick line represents the conversion curve. The bar graph shows the brightness distribution of the area, and the bar corresponds to the degree of the predetermined brightness range (corresponding to the vertical axis on the right). Fig. 9A , Fig. 9B and Fig. 9C In , Di represents the brightness dynamic range before conversion, and Do represents the brightness dynamic range after conversion. When the slope is 1, that is, 45°, the input brightness value and the output brightness value will match, and no image change will occur in this part. That is, the contrast before conversion of the dynamic range will be maintained. As the slope decreases (to a degree less than 45°), the contrast after conversion decreases compared to the contrast before conversion. The contrast needs to be maintained to obtain a suitable converted image, and the slope is preferably set to 1. Since low-frequency components are being processed in this case, it is necessary to perform as many conversions as possible to set the slope to 1 to maintain the contrast of the low-frequency components. Fig. 9B This represents the conversion curve of another area, but the brightness distribution is biased toward the high brightness side. Fig. 9A In this way, slopes close to 1 are assigned to high-degree brightness groups according to the degree of distribution. Fig. 9CA conversion curve representing an area with uniform brightness distribution. In this case, the slope cannot be assigned to 1 even if there is a high degree brightness group. This is because, since the dynamic range Do of brightness after conversion is narrow, assigning a slope of 1 to a specific brightness group will cause the slope of another brightness group to be set closer to 0. In this case, the slope will be evenly distributed between the brightness groups, and the slope will be assigned according to each degree, so the slope of the brightness group without brightness in the brightness group will be set very close to 0. In addition, the slopes representing different areas of the image are not assigned to the brightness group. 9A to 9C The common part is, for example, the exposure brightness value Ya obtained in step S801, and each conversion curve is created so that the brightness value after conversion will always be a predetermined value Ya'. Thus, the exposure brightness set by the user when shooting can be maintained while reproducing the tone on the high brightness side.

[0106] In step S804, the dynamic range conversion unit 201 determines whether a conversion curve has been created for each divided area. If it is determined that a conversion curve has not been created for each divided area, the process is repeated from step S803. Otherwise, the process proceeds to step S805.

[0107] In step S805, the dynamic range conversion unit 201 performs dynamic range compression processing on each pixel using each created conversion curve. At this time, the processing is performed in consideration of the information of the surrounding areas so as not to create a position where the tone becomes discontinuous between the areas. More specifically, a window of the same degree as the area can be assigned so that weighting can be performed based on the area included in the window, and dynamic range compression processing can be performed based on the ratio. In addition, since a simple area ratio can cause image defects such as halos to be generated on the boundary, the weight can be changed based on the average brightness value of the target area. That is, compared to the target pixel, the image defect can be suppressed by reducing the weight according to the increase in the change in the average brightness value of each surrounding area.

[0108] In step S806, the dynamic range conversion unit 201 determines whether the processing of step S805 has been performed on all pixels. If it is determined that the processing has not been performed on all pixels, the processing is repeated from step S805. Otherwise, Fig.10 The processing is finished.

[0109] In this way, by separating image data into high-frequency components and low-frequency components based on the Retinex theory, and by performing brightness dynamic range conversion on the low-frequency components using conversion curves for each area of ​​the image, a high-contrast image can be generated that takes into account human visual characteristics.

[0110] In the first and second embodiments, display information may not be obtained in some cases, such as when display information cannot be obtained from the display device 102 or when HDR data is directly input to the output device 103 without a display function. Fig.10 The processing performed in this case is shown in FIG. Fig.10 In the process of step S301, it is determined whether the display information can be obtained. If it is determined that the display information cannot be obtained, the process of step S302 is skipped, and image data is generated by luminance dynamic range conversion using the HDR data and print information in the process of step S304 after step S303. In the luminance dynamic range conversion performed in this case, in the case of HDR data whose hue is defined by relative brightness, after performing EOTF (inverse function of OETF) processing, the luminance dynamic range defined by the relative values ​​held by the data is made to correspond to the luminance dynamic range of the output device 103. On the other hand, in the case of HDR data whose hue is defined by absolute brightness values ​​(such as HDR data of the PQ method), after EOTF processing, the luminance dynamic range defined by the relative values ​​held by the data is made to correspond to the luminance dynamic range of the output device 103. Figure 4A and Figure 4B The conversion curve shown in , etc. converts the luminance dynamic range of data defined by absolute values ​​into the luminance dynamic range when printing. In this way, the print output can be adaptively changed according to whether display information is obtained from the display device 102.

[0111] In addition, it can be arranged so that the user can select via the UI whether to perform luminance dynamic range compression by using display information or without using display information. By allowing the user to select whether the printout is a printout corresponding to the display of the display device 102 or a printout based on HDR data, the user's intention can be reflected.

[0112] The present embodiment shows an example of displaying and printing HDR data that complies with the BT.2100 standard. However, the present invention is not limited to the BT.2100 standard. Processing that complies with other standards may be performed, or only transfer through the OETF and EOTF may be performed. For example, although the HLG method and the PQ method are exemplified as transfer functions, another method may be used as long as a transfer function that processes HDR data by defining the hue by a relative luminance value or an absolute luminance value is used. In this case, the transfer function and the system gamma represented by equations (1) and (2) will have a form that complies with the corresponding standards. In addition, for example, a piece of conversion information that is unique to the display, such as a gamma value processed with a gamma conversion, may be obtained as display information.

[0113] Although the luminance dynamic range conversion method according to the embodiment is exemplified Figure 4A and Figure 4B , Figure 5A and Figure 5B as well as Figure 6 , but the present invention is not limited to these methods. Any kind of method may be used as long as it is a method of performing conversion of the luminance dynamic range. In addition, the input / output luminance dynamic range of the dynamic range conversion process is not limited to the luminance dynamic range (1,000 cd / m 2 In addition, the display device 102 according to the embodiment may be any form of device, not limited to a display, as long as it is a device that can display information such as a tablet or a touch panel installed on a smartphone or device.

[0114] The present invention is not limited to the above embodiments, and various changes and modifications can be made within the spirit and scope of the present invention. Therefore, in order to make the public aware of the scope of the present invention, the following claims are made.

[0115] Other embodiments

[0116] The embodiments of the present invention may also be implemented by providing software (program) for performing the functions of the above-described embodiments to a system or device via a network or various storage media, and a computer or a central processing unit (CPU) or a microprocessing unit (MPU) of the system or device reads and executes the program.

[0117] While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.

Claims

1. An image processing device, comprising: A first obtaining unit, configured to obtain HDR data representing a high dynamic range HDR image, wherein the HDR data is obtained by using a transfer function; a second obtaining unit configured to obtain printing information to perform printing based on the HDR data obtained by the first obtaining unit; a third obtaining unit configured to obtain display information of a display device, in which a print preview is displayed based on the obtained HDR data; as well as a conversion unit configured to determine a luminance range of the display device represented by the obtained display information as a dynamic range of luminance of the obtained HDR data, and, based on the obtained printing information, convert the dynamic range of luminance determined for the HDR data into a luminance range based on the printing information, The display information includes dynamic range information and system gamma information of the display device, and The print information is information for specifying a dynamic range after dynamic range conversion.

2. The device according to claim 1, wherein: The print information is information for specifying a dynamic range to be obtained by conversion by the conversion unit.

3. The device according to claim 2, wherein: The printing information includes information indicating the type of sheet to be printed.

4. The device according to claim 1, wherein: The conversion unit performs dynamic range conversion by using conversion information corresponding to the display device and for converting a dynamic range.

5. The device according to claim 1, wherein: The conversion unit converts the dynamic range of brightness of the HDR data obtained by the first obtaining unit into a dynamic range for printing to be performed so that input brightness and output brightness match within a predetermined brightness range.

6. The device according to claim 1, further comprising: a dividing unit configured to divide the image represented by the HDR data into regions; The conversion unit performs dynamic range conversion by using conversion information for converting the dynamic range and set for each area divided by the division unit.

7. The device according to claim 6, wherein: The division unit performs region division on a low-frequency component of an image represented by the HDR data.

8. The device according to claim 7, further comprising: A unit is configured to perform contrast correction on a high frequency component of an image represented by the HDR data.

9. The device according to any one of claims 1 to 8, further comprising: A printing unit is configured to perform printing based on the data subjected to the dynamic range conversion by the conversion unit.

10. The device according to any one of claims 1 to 8, wherein: The display device is a HDR display.

11. The device according to any one of claims 1 to 8, wherein: The display device is an SDR display.

12. An image processing method to be performed in an image processing apparatus, the method comprising: obtaining HDR data representing a high dynamic range HDR image, the HDR data being obtained by using a transfer function; obtaining printing information to print based on the obtained HDR data; Obtaining display information of a display device, and displaying a print preview in the display device based on the obtained HDR data; determining the luminance range of the display device represented by the obtained display information as the dynamic range of the luminance of the obtained HDR data, and, Based on the obtained printing information, converting the dynamic range of brightness determined for the HDR data into a brightness range based on the printing information, The display information includes dynamic range information and system gamma information of the display device, and The print information is information for specifying a dynamic range after dynamic range conversion.

13. A non-transitory computer-readable storage medium storing a program to enable a computer to perform the following operations: obtaining HDR data representing a high dynamic range HDR image, the HDR data being obtained by using a transfer function; obtaining printing information to print based on the obtained HDR data; Obtaining display information of a display device, and displaying a print preview in the display device based on the obtained HDR data; determining the luminance range of the display device represented by the obtained display information as the dynamic range of the luminance of the obtained HDR data, and, Based on the obtained printing information, converting the dynamic range of brightness determined for the HDR data into a brightness range based on the printing information, The display information includes dynamic range information and system gamma information of the display device, and The print information is information for specifying a dynamic range after dynamic range conversion.

Images