Image processing device and control method therefor, imaging device, program, and storage medium
The image processing device addresses the challenge of transmitting HDR and SDR images on small-scale devices by performing OETF conversions based on display characteristics, ensuring high-quality mixed display on EVFs.
Patent Information
- Application Number
- JP2023080363
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-15
- Publication Date
- 2024-11-27
- Estimated Expiration
- Not applicable · inactive patent
Smart Images

Figure 2024164693000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to an image processing apparatus that generates and transmits image data. [Background technology]
[0002] In recent years, digital cameras have been shifting from SLR to mirrorless cameras, and as a result, eyepiece viewfinders have been changing from optical viewfinders (hereinafter referred to as OVF) to electronic viewfinders (hereinafter referred to as EVF).
[0003] With an OVF, you see the light that comes in through the lens itself, whereas with an EVF, you see the image processed by the image processing engine and displayed on the EVF display, so there can be differences in how the subject appears between an OVF and an EVF.
[0004] In recent years, there has been an increasing demand for image content with a wide dynamic range, known as HDR (High Dynamic Range), whereas image content prior to HDR is known as SDR (Standard Dynamic Range).
[0005] HDR is defined in the international standard SMPTE STANDARD 2084 (hereafter referred to as SMPTE ST 2084) as requiring 10-bit or more image data for display, while SDR is defined in the international standard RECOMMENDATION ITU-R BT.709 (hereafter referred to as ITU BT.709) as requiring 8-bit or more image data for display.
[0006] Here, there are cases where it is desired to display a mixture of HDR and SDR images on a display unit. Patent Document 1 discloses a technique for converting the dynamic range of an output image according to the dynamic range that can be displayed at the output destination when displaying multiple images with different dynamic ranges side by side on a display unit. [Prior art documents] [Patent documents]
[0007] [Patent Document 1] JP 2020-136737 A Summary of the Invention [Problem to be solved by the invention]
[0008] When the display device is a large-scale device such as a commercial monitor, there are fewer restrictions in terms of hardware such as power consumption and heat generation, so it is easy to ensure that the bit precision of the arithmetic circuit is 10 bits or more, making it possible to perform HDR display that complies with SMPTE ST 2084.
[0009] On the other hand, in a small device such as an EVF, it is difficult to build a circuit with a calculation precision of 10 bits or more due to issues such as power consumption and heat generation. In addition, since priority is given to the resolution and frame rate of the image data when allocating the data input interface, the data input interface is less than 10 bits. In this case, quantization errors occur during calculation, making it difficult to maintain the gradation of HDR images, especially in the low brightness range.
[0010] In addition, EVFs are equipped with an input interface that accepts a single signal such as MIPI (registered trademark) (Mobile Industry Processor Interface) or Sub-LVDS (Sub-Low Voltage Differential Signal), making it currently difficult to simultaneously transmit and display HDR and SDR images with reduced image degradation through a single port.
[0011] The present invention has been made in consideration of the above-mentioned problems, and its purpose is to provide technology that enables the simultaneous transmission of HDR images and SDR images with reduced degradation in image quality over a single port, enabling the mixed display of HDR images and SDR images. [Means for solving the problem]
[0012] The image processing device according to the present invention is characterized by comprising a first conversion means for performing at least one of a first conversion for converting an OETF (Opto-Electro Transfer Function) of an HDR image (High Dynamic Range image) into an OETF based on the light-emitting characteristics of a display means, and a second conversion for converting an OETF of an SDR image (Standard Dynamic Range image) into an OETF of the light-emitting characteristics of the display means, and a transmission means for transmitting the HDR image converted by the first conversion and the SDR image converted by the second conversion to the display means. Effect of the Invention
[0013] According to the present invention, HDR images and SDR images with reduced degradation in image quality can be transmitted simultaneously through a single port, enabling the mixed display of HDR images and SDR images. [Brief description of the drawings]
[0014] [Figure 1] FIG. 1 is a block diagram of a general imaging device. [Diagram 2] A diagram showing an EOTF graph. [Diagram 3] A graph showing OETF. [Figure 4] A diagram showing the allocation of OETF code values. [Diagram 5] 1 is a block diagram of an imaging apparatus according to a first embodiment. [Figure 6] FIG. 4 is a block diagram of a light emission characteristic inverse converter. [Figure 7] Flowchart of image processing in the first embodiment [Figure 8] FIG. 4 is a view showing an image processing result in the first embodiment. [Figure 9] A diagram showing thumbnail display that mixes HDR and SDR images. [Figure 10] FIG. 11 is a block diagram of an imaging apparatus according to a second embodiment. [Figure 11] 10 is a flowchart of image processing according to the second embodiment. [Figure 12] FIG. 10 is a view showing an image processing result in the second embodiment. [Figure 13] FIG. 11 is a block diagram of an imaging apparatus according to a third embodiment. [Figure 14] 13 is a flowchart of image processing according to the third embodiment. [Figure 15] 13A to 13C are views showing image processing results according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0015] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.
[0016] (First embodiment) FIG. 1 is a block diagram showing a process from image input to display in a conventional general imaging device.
[0017] An AD (Analog Digital) converter 1202 converts the image data input from the image input unit 1201 from an analog signal to a digital signal.
[0018] An OETF (Opto-Electro Transfer Function) conversion unit 1203 converts the digital signal into an OETF.
[0019] A DA (Digital Analog) converter 1204 converts the OETF converted image data into an analog signal.
[0020] An EOTF (Electro-Optical Transfer Function) converter 1205 converts the analog signal into an EOTF signal. A display material 1206 displays the EOTF converted image data.
[0021] The display material 1206 is made up of a cathode ray tube, liquid crystal, OLED (Organic Light Emitting Diode), etc., and has its own light-emitting characteristics depending on the material and configuration. Display materials such as cathode ray tubes and liquid crystal have light-emitting characteristics close to the EOTF defined in ITU BT.709. On the other hand, the light-emitting characteristics of OLED depend on the switching characteristics of transistors, and its light-emitting characteristics are close to the EOTF defined in SMPTE ST 2084. The EOTF converter 1205 performs a process in which the analog signal is converted into the light-emitting brightness of the display material 1206 according to the light-emitting characteristics of the display material 1206.
[0022] The OETF converter 1203 performs conversion with the opposite characteristics to those of the EOTF converter 1205, which is based on the light-emitting characteristics of the display material 1206. By applying the conversion by the OETF converter 1203 to the image data in advance, it cancels out the characteristics of the EOTF converter 1205. As a result, the image displayed on the display material 1206 appears the same as the image input by the image input unit 1201.
[0023] Figure 2 is a graph of the EOTF showing an example of the light emission characteristics of SMPTE ST 2084, ITU BT.709, and OLED material. The vertical axis represents the ratio to the maximum light emission brightness, and the horizontal axis represents the code value input to the DA converter 1204 in 10 bits. The light emission characteristics of the OLED material are close to the EOTF of SMPTE ST 2084.
[0024] Figure 3 shows an example of the OETF graph for SMPTE ST 2084, ITU BT.709, and OLED material emission characteristics. The vertical axis shows the converted code value, and the horizontal axis shows the ratio to the maximum emission brightness. OETF is an inverse function of EOTF. The OETF of SMPTE ST 2084 is defined so that the human eye cannot see the quantization steps due to the change in brightness.
[0025] Figure 4 compares data that expresses the luminance of an input image as a percentage of its maximum luminance, with 8-bit (256 gradations) code values assigned in accordance with the OETFs of SMPTE ST 2084, ITU BT.709, and the luminance characteristics of OLED materials. SMPTE ST 2084 and the luminance characteristics of OLED materials assign many code values to the low luminance range, but ITU BT.709 assigns fewer code values to the low luminance range. In this case, it is difficult to maintain the gradation in the low luminance range of the input image, resulting in degradation of image quality such as tone jumps.
[0026] FIG. 5 is a block diagram illustrating an example of the configuration of an imaging device 1 which is a first embodiment of an image processing device of the present invention.
[0027] The imaging device 1 of the first embodiment includes an image generating unit 100, a data transmission path 120 that transmits digital signals with less than 10-bit precision, and a display unit .
[0028] The image generating unit 100 includes a sensor unit (imaging element) 101, an AD conversion unit 102, a sensor correction unit 103, a development unit 104, an OETF conversion unit 105, a recording and reproducing unit 106, a recording medium 107, an image synthesis unit 108, an area division unit 109, a first OETF conversion unit 110, a second OETF conversion unit 111, a transmission unit 112, a CPU 113, and an operation unit 114. The first OETF conversion unit 110 is an OETF conversion unit with HDR (High Dynamic Range) emission characteristics, and the second OETF conversion unit 111 is an OETF conversion unit with SDR (Standard Dynamic Range) emission characteristics.
[0029] The sensor unit 101 is composed of a CMOS (Complementary Metal Oxide Semiconductor) image sensor and has a plurality of pixels arranged two-dimensionally. Each pixel is provided with three color filters of red (R), green (G), and blue (B). The sensor unit 101 performs a photoelectric conversion process to convert the light transmitted through the color filters into an analog electrical signal.
[0030] The AD conversion unit 102 performs AD conversion processing to convert an analog electrical signal obtained by photoelectric conversion processing into image data consisting of a digital signal.
[0031] The sensor correction unit 103 performs restoration processing on the image data obtained by the AD conversion processing. For example, for missing pixels in the sensor unit 101 or pixels whose acquired pixel values are maximum (minimum) and are determined to be unreliable, the sensor correction unit 103 interpolates pixel values using surrounding pixel values. The sensor correction unit 103 also performs processing to subtract a predetermined offset value, such as dark current correction.
[0032] The development unit 104 performs development processes on the restored image data, such as conversion to a color space consisting of luminance and color difference, removal of noise, and correction of optical distortion.
[0033] The OETF converter 105 converts the developed image data into an OETF that complies with generalized standards such as ITU BT.709 and SMPTE ST 2084. Furthermore, the OETF converter 105 adds to the image data, as OETF information, which standard of OETF the image data has been converted into.
[0034] The recording / playback unit 106 plays back image data read from the recording medium 107. The recording medium 107 is a device capable of reading and writing image data OETF converted by the OETF conversion unit 105. An SD card (registered trademark), a microSD card (registered trademark), or the like is used as the recording medium 107. The recording medium 107 is removable from the image generation unit 100, and can be attached to a device (such as a personal computer) other than the image generation unit 100. Note that the recording medium 107 may be a built-in recording medium that cannot be attached to or detached from the image generation unit 100.
[0035] The image synthesis unit 108 holds background images and GUI images as data, and synthesizes them with image data read through the recording and playback unit 106. When multiple image data are read, the multiple image data are synthesized into one image. Furthermore, the OETF information of the overlapping portion is updated during synthesis.
[0036] The region division unit 109 divides the image data into an HDR display region and an SDR display region based on the image data combined by the image combination unit 108 and the OETF information that the image data has for each region. When displaying the entire image in HDR or SDR, division may not be performed.
[0037] The first OETF conversion unit 110 detects the HDR display area divided by the area division unit 109, and converts the OETF from SMPTE ST 2084 to an OETF based on the light-emitting characteristics of the display material 134.
[0038] The second OETF conversion unit 111 detects the SDR display area divided by the area division unit 109 , and converts the OETF from ITU BT.709 to an OETF based on the light emission characteristics of the display material 134 .
[0039] Here, detailed processing of the first OETF conversion unit 110 will be described with reference to Fig. 6. Fig. 6 shows an example in which the pixels of the display material 134 are composed of three colors, RGB, but the same applies to cases in which other colors, such as cyan, magenta, and yellow, are used.
[0040] FIG. 6A is a diagram for explaining the processing in the first OETF converter 110 when the input image data to the first OETF converter 110 is RGB data.
[0041] Since the light emission characteristics of the display unit 130 are different for each of RGB, the first OETF conversion units 110-01, 110-02, and 110-03 perform inverse conversion of the light emission characteristics of each of RGB. The WB adjustment unit 110-04 adjusts the white balance of the RGB data that has undergone the inverse conversion of the light emission characteristics, and outputs the adjusted data. The order of the light emission characteristic inverse conversion and the white balance processing may be reversed, or may be performed simultaneously. The second OETF conversion unit 111 performs the same processing.
[0042] FIG. 6(b) is a diagram for explaining the processing in the first OETF conversion unit 110 when the input image data to the first OETF conversion unit 110 is YCC data. In the YCC / RGB conversion unit 110-05, the input YCC data is converted into RGB data. The processing after the RGB conversion is the same as that in FIG. 6(a). In the RGB / YCC conversion unit 110-06, the RGB data that has been white-balanced is converted into YCC data and output. The second OETF conversion unit 111 performs the same processing.
[0043] The transmitting unit 112 transmits the converted image data to the display unit 130 via the data transmission path 120 .
[0044] The CPU 113 is connected to the above-mentioned blocks and other blocks of the image generating unit 100 via an internal bus (not shown). The CPU 113 controls the overall processing of the image generating unit 100. In this embodiment, the CPU 113 also controls the blocks of the display unit 130.
[0045] The operation unit 114 is connected to the CPU 113 via an internal bus (not shown). The operation unit 114 has various operation members as an input unit that accepts user operations. The operation unit 114 includes a shooting start button for starting shooting, a playback start button for playing back an image recorded on the recording medium 107, and the like. The operation unit 114 also has function buttons that are displayed as a GUI (Graphical User Interface) on the display unit 130 and are used to execute various functions in response to user operations. The function buttons include a start button for a DR comparison display function that displays an HDR image and an SDR image side by side when playing back an image recorded on the recording medium 107, and a start button for a thumbnail display function that displays a list of multiple images recorded on the recording medium 107, and the like.
[0046] The data transmission line 120 uses a transmission standard such as MIPI or Sub-LVDS. The data transmission line 120 in this embodiment has 8-bit precision. Bit precision does not mean a physical bit width, but a substantial bit width. For example, when a 10-bit width transmission line is configured with 8 bits of valid data and 2 bits of 0 padding, it has 8-bit precision.
[0047] The display unit 130 is configured to include a receiving unit 131, a DA conversion unit 132, an EOTF conversion unit 133, and a display material 134.
[0048] The receiving unit 131 receives image data from the image generating unit 100 via the data transmission path 120 .
[0049] The DA conversion unit 132 performs DA conversion to convert image data consisting of digital signals into analog electrical signals. The DA conversion unit 132 transmits the converted electrical signals to the display material 134. Based on the intensity of the transmitted electrical signals, the luminescence intensity of the luminescent material of each pixel of the display material 134 is determined.
[0050] The EOTF conversion unit 133 performs EOTF conversion based on the light emission characteristics of the display material 134. The display material 134 emits light to display an image based on the electrical signal for each pixel transmitted from the DA conversion unit 132, but the EOTF conversion unit 133 performs nonlinear conversion from the electrical signal to light emission luminance based on the electrical characteristics of the display material 134.
[0051] The display material 134 comprises a plurality of pixels arranged two-dimensionally. The light-emitting material corresponding to each pixel emits light to display an image. The display material in this embodiment may be a cathode ray tube, liquid crystal, OLED, or the like.
[0052] Next, a process for implementing a DR comparison display function for displaying an SDR image and an HDR image stored in the recording medium 107 side-by-side on the display unit 130 will be described with reference to the flowchart of Fig. 7. Fig. 8 shows the image processing results of each functional block in the flow of Fig. 7.
[0053] Each process in the flowchart of FIG. 7 is started when the CPU 113 detects that the operation unit 114 has received a user operation to start the DR comparison function, and is realized by the CPU 113 controlling each block of the image generation unit 100 and the display unit 130.
[0054] In step S302, the CPU 113 controls the recording and playback unit 106 to read out the HDR image data from the recording medium 107. The read out image data is played back as image data through the recording and playback unit 106.
[0055] In step S303, the CPU 113 controls the image synthesis unit 108 to synthesize the read image data with background image data that the image synthesis unit 108 holds as data in advance.
[0056] In step S304, the CPU 113 determines whether or not the reading and composition of image data to be simultaneously displayed has been completed. If the reading and composition of image data to be simultaneously displayed has not been completed (S304: NO), the CPU 113 repeats the processing of steps S302 to S304. In this embodiment, in steps S302 to S304, the CPU 113 reads SDR image data to be displayed simultaneously with the HDR image data, and generates an HDR-SDR composite image by combining the SDR image data with the background image data.
[0057] At this time, a desired GUI, such as a menu or a scroll bar, which is held as data by the image synthesis unit 108, may be synthesized. Here, as an example of GUI synthesis, synthesis of an icon for identifying the display format of the image will be considered.
[0058] The processing results of steps S302 to S304 are shown in Fig. 8(a). Area 401 is background image data, area 402 is HDR image data, area 403 is SDR image data, area 404 is an icon indicating that the image is displayed in HDR, and area 405 is an icon indicating that the image is displayed in SDR. The OETFs of areas 401, 403, 404, and 405 are the OETF of ITU BT.709, and the OETF of area 402 is the OETF of SMPTE ST 2084.
[0059] On the other hand, in step S304, if the reading and composition of image data to be displayed simultaneously has been completed (S304: YES), the HDR / SDR combined image, which is the composition result, is transmitted to the area division unit 109.
[0060] In step S305, the CPU 113 controls the area dividing unit 109 to divide the HDR-SDR combined image generated by the image combining unit 108 into an HDR display area and an SDR display area.
[0061] In this embodiment, the image synthesis unit 108 assigns coordinate information and synthesis order information when synthesizing each image, and updates the OETF information of the overlapping portion based on this information. The area division unit 109 determines that the area is an HDR display area if the OETF is for HDR display based on the OETF information for each pixel.
[0062] The processing result is shown in Fig. 8(b). Area 406 indicates the HDR display area, and area 407 indicates the SDR display area. Therefore, in this embodiment, the HDR image is assigned to the HDR display area, and the SDR image and GUI image are assigned to the SDR display area. At this time, the OETF of area 406 is determined to be SMPTE ST 2084, and the OETF of area 407 is determined to be ITU BT.709.
[0063] Then, the image data after region division and region information indicating the determination result as to whether the display region is an HDR display region or an SDR display region are transmitted to the first OETF conversion unit 110.
[0064] In step S306, the CPU 113 controls the first OETF conversion unit 110 to detect an HDR display area based on the image data and area information transmitted from the area division unit 109. Then, for the HDR display area, conversion is performed from the OETF of SMPTE ST 2084 to an OETF based on the light emission characteristics of the display material 134. Then, the converted data is transmitted to the second OETF conversion unit 111.
[0065] In step S307, the CPU 113 controls the second OETF conversion unit 111 to detect the SDR display area based on the image data and area information transmitted from the first OETF conversion unit 110. Then, for the SDR display area, conversion is performed from the OETF of ITU BT.709 to an OETF based on the light emission characteristics of the display material 134.
[0066] 8C shows the processing results of steps S306 and S307. The OETFs of the regions 406 and 407 after conversion are OETFs based on the light emission characteristics of the display material 134.
[0067] In step S308, the CPU 113 controls the second OETF conversion unit 111 to transmit the converted image data to the display unit 130 via the transmission unit 112 and the data transmission path 120.
[0068] In step S 309 , the CPU 113 controls the receiving unit 131 to receive the image data transmitted through the data transmission path 120 .
[0069] In step S310, the CPU 113 controls the DA conversion unit 132 to convert the image data, which is a digital signal, into an analog signal.
[0070] In step S311, the CPU 113 controls the EOTF conversion unit 133 to apply EOTF conversion based on the light emission characteristics of the display material 134 to the image data, and determines the light emission luminance for each pixel on the display material 134.
[0071] In step S312, CPU 113 displays an image on display unit 130 by causing the display material to emit light.
[0072] In the explanation so far, the EOTF for HDR has been described as SMPTE ST 2084, but the EOTF for HDR is not limited to SMPTE ST 2084. If there is a larger allocation of dark gradations compared to SDR, the EOTF can be the EOTF for HDR.
[0073] In the above, as an example, the process for realizing an image display state in which an SDR image and an HDR image are displayed side by side on the left and right of the display unit 130 as shown in Fig. 8(a) has been described. However, a thumbnail display in which HDR images and SDR images are mixed as shown in Fig. 9 can also be realized by using a similar process.
[0074] Furthermore, the image data to be read from the recording medium 107 may be single or multiple, and HDR images and SDR images may or may not be mixed together.
[0075] As described above, according to this embodiment, by dividing the HDR / SDR combined image data into an HDR display area and an SDR display area and converting each area into an OETF based on the light-emitting characteristics of the display material, it becomes possible to transmit an HDR image and an SDR image through a single port and display them together.
[0076] Second Embodiment 10 is a block diagram for explaining an example of the configuration of an imaging device 2 which is a second embodiment of an image processing device of the present invention. The same components as those in the first embodiment are given the same reference numerals and the description thereof will be omitted.
[0077] The imaging device 2 of the second embodiment includes an image generating unit 600, a data transmission path 120 that transmits digital signals with less than 10-bit precision, and a display unit .
[0078] In the second embodiment, the image generation unit 600 does not have the area division unit 109, and compared to the first embodiment, the connection order of the first OETF conversion unit 110, the second OETF conversion unit 111, and the image synthesis unit 108 is changed.
[0079] In the second embodiment, the read image data is converted into an OETF based on the light emission characteristics of the display material 134 by a first OETF conversion unit 601 and a second OETF conversion unit 602. After that, an image synthesis unit 603 synthesizes the background image, HDR image, and SDR image.
[0080] As in the first embodiment, a process for implementing a DR comparison display function for displaying an SDR image and an HDR image stored in the recording medium 107 side-by-side on the display unit 130 will be described with reference to the flowchart of Fig. 11. Also, Fig. 12 shows the image processing result in the flow of Fig. 11.
[0081] The processing shown in the flowchart of FIG. 11 is started when the CPU 113 detects that the operation unit 114 has received a user operation to start the DR comparison function, and is realized by the CPU 113 controlling each block of the image generation unit 600 and the display unit 130.
[0082] In step S702, the CPU 113 controls the recording and playback unit 106 to read out the HDR image data from the recording medium 107. The read out image data is played back as image data through the recording and playback unit 106.
[0083] In step S703, the CPU 113 determines whether the OETF of the read image data is HDR. If the OETF of the read image data is HDR (S703: YES), the CPU 113 proceeds to step S704 and sends the image data to the first OETF conversion unit 601.
[0084] In step S704, the CPU 113 controls the first OETF conversion unit 601 to convert the image data from the OETF of SMPTE ST 2084 to an OETF based on the light emission characteristics of the display material 134. Thereafter, the image data after the OETF conversion is sent to the image synthesis unit 603.
[0085] In step S706, the CPU 113 controls the image synthesis unit 603 to synthesize the HDR image data after OETF conversion by the first OETF conversion unit 601 with background image data previously held as data by the image synthesis unit 603. It is assumed that the OETF of the background image data has been converted into an OETF based on the light emission characteristics of the display material 134 in advance.
[0086] 12(a) shows the processing result. An area 801 shows background image data, and an area 802 shows an HDR image after conversion into the light emission characteristic OETF of the display material 134.
[0087] In step S707, the CPU 113 determines whether or not the reading and composition of the image data to be simultaneously displayed has been completed. If the reading and composition of the image data to be simultaneously displayed has not been completed (S707: NO), the CPU 113 repeats the processing of steps S702 to S706. In this embodiment, in steps S702 to S706, the SDR image data to be displayed simultaneously with the HDR image data is read, converted into an OETF based on the light emission characteristics of the display material 134, and then composed with the background image data.
[0088] In step S702, the CPU 113 controls the recording and reproducing unit 106 to read the SDR image data from the recording medium 107. The read image data is reproduced as image data through the recording and reproducing unit 106.
[0089] In step S703, the CPU 113 determines whether the OETF of the read image data is HDR. If it is determined that the OETF of the read image data is SDR (S703: NO), the CPU 113 proceeds to step S705 and sends the image data to the second OETF conversion unit 602.
[0090] In step S705, the CPU 113 controls the second OETF converter 602 to convert the image data from the OETF of ITU BT.709 to an OETF based on the light emission characteristics of the display material 134. Thereafter, the image data after the OETF conversion is sent to the image synthesizer 603.
[0091] In step S706, the CPU 113 controls the image synthesis unit 603 to synthesize the SDR image data after OETF conversion by the second OETF conversion unit 602 with the background image data, thereby generating an HDR / SDR synthesis image.
[0092] The processing result is shown in Fig. 12(b). Region 803 shows the SDR image after conversion to an OETF based on the luminescence characteristics of the display material 134.
[0093] At this time, a desired GUI image such as a menu or a scroll bar may be synthesized, which is stored as data in the image synthesis unit 603. Note that the OETF of the GUI image is assumed to be converted into an OETF based on the light emission characteristics of the display material 134 in advance.
[0094] In step S707, CPU 113 determines whether or not the reading and composition of image data to be simultaneously displayed has been completed. If the reading and composition of all image data to be displayed on display unit 130 has been completed (S707: YES), the process proceeds to step S708, where the image data is sent to display unit 130 via transmitter 112 and data transmission path 120. If the reading and composition of all image data has not been completed (S707: NO), steps S702 to S708 are repeated.
[0095] In step S 709 , the CPU 113 controls the receiving unit 131 to receive the image data transmitted through the data transmission path 120 .
[0096] In step S710, the CPU 113 controls the DA conversion unit 132 to convert the image data, which is a digital signal, into an analog signal.
[0097] In step S711, the CPU 113 controls the EOTF conversion unit 133 to apply EOTF conversion based on the light emission characteristics of the display material 134 to the image data, and determines the light emission luminance for each pixel on the display material 134.
[0098] In step S712, CPU 113 displays an image on display unit 130 by causing the display material to emit light.
[0099] As a result, with a configuration different from that of the first embodiment, it is possible to transmit HDR images and SDR images through one port and display them together.
[0100] (Third embodiment) 13 is a block diagram for explaining an example of the configuration of an imaging device 3 which is a third embodiment of an image processing device of the present invention. The same components as those in the first embodiment are given the same reference numerals and the description thereof will be omitted.
[0101] The imaging device 3 of the third embodiment includes an image generating unit 900, a data transmission path 120 that transmits digital signals with less than 10-bit precision, and a display unit .
[0102] In the third embodiment, the image generating unit 600 does not have the second OETF conversion unit 111, and an SDR / HDR conversion unit 901 is added to the first embodiment. Also, the connection order of the first OETF conversion unit 110 and the image synthesis unit 108 is changed.
[0103] The SDR / HDR conversion unit 901 converts the OETF of an SDR image from the ITU BT.709 OETF to the SMPTE ST 2084 OETF.
[0104] In the third embodiment, when a mixture of HDR and SDR images is displayed on the display unit 130, an SDR / HDR conversion unit 901 converts the OETF of the SDR image into an OETF of HDR once. Then, an image synthesis unit 902 synthesizes the HDR image, the SDR image converted into the OETF of HDR, and a background image. Then, a first OETF conversion unit 903 converts the synthesized image data into an OETF based on the light emission characteristics of the display material 134.
[0105] The process for implementing a DR comparison display function in which the light-emitting characteristics of the display material 134 are close to the EOTF of HDR and an SDR image and an HDR image stored in the recording medium 107 are displayed side-by-side on the display unit 130 will be described with reference to the flowchart in Fig. 14. Also, Fig. 15 shows the image processing results in the flow in Fig. 14.
[0106] The processing represented by the flowchart in FIG. 14 is initiated when the CPU 113 detects that the operation unit 114 has received a user operation to start the DR comparison function, and is realized by the CPU 113 controlling each block of the image generation unit 900 and the display unit 130.
[0107] Here, the luminance characteristics of the display material 134 being close to the HDR EOTF means that the sum of squares of the difference between the luminance characteristics of the display material 134 and the HDR EOTF is smaller than the sum of squares of the difference between the luminance characteristics of the display material 134 and the SDR EOTF.
[0108] In step S1002, the CPU 113 controls the recording and playback unit 106 to read out the HDR image data from the recording medium 107. The read out image data is reproduced as image data through the recording and playback unit 106.
[0109] In step S1003, it is determined whether the OETF of the read image data is HDR. If it is determined that the OETF of the read image data is HDR (S1003: YES), the process proceeds to step S1005, and the image data is sent to the image synthesis unit 902.
[0110] In step S1005, the CPU 113 controls the image synthesis unit 902 to synthesize the transmitted image data with background image data previously held as data by the image synthesis unit 902. It is assumed that the OETF of the background image data has been converted into the OETF of SMPTE ST 2084 in advance.
[0111] The result of this processing is shown in Fig. 15(a). Area 1101 indicates the background image data, and area 1102 indicates the HDR image data read from the recording medium.
[0112] In step S1006, the CPU 113 determines whether or not the reading and composition of image data to be simultaneously displayed has been completed. If the reading and composition of image data to be simultaneously displayed has not been completed (S1006: NO), the CPU 113 repeats the processing of steps S1002 to S1006. In this embodiment, in steps S1002 to S1006, SDR image data to be displayed simultaneously with HDR image data is read and composed with background image data.
[0113] In step S1002, the CPU 113 controls the recording and reproducing unit 106 to read the SDR image data from the recording medium 107. The read image data is reproduced as image data through the recording and reproducing unit 106.
[0114] In step S1003, if it is determined that the OETF of the read image data is SDR (S1003: YES), the recording and reproducing unit 106 sends the image data to the SDR / HDR conversion unit.
[0115] In step S1004, the CPU 113 controls the SDR / HDR conversion unit 901 to convert the read SDR image data from the OETF of ITU BT.709 to the OETF of SMPTE ST 2084. The SDR / HDR conversion unit 901 transmits the converted image data to the image synthesis unit 902.
[0116] In step S1005, the CPU 113 controls the image synthesis unit 902 to synthesize the converted SDR image data with the background image data to generate an HDR / SDR synthesis image. The result of this processing is shown in Figure 15(b). An area 1103 shows the converted SDR image.
[0117] At this time, a desired GUI image such as a menu or scroll bar may be synthesized, which is held as data by the image synthesis unit 902. Note that the OETF of the GUI image is assumed to have been converted into the OETF of SMPTE ST 2084 in advance.
[0118] In step S1006, the CPU 113 determines whether or not the reading and composition of image data to be simultaneously displayed has been completed. When the reading and composition of all image data to be displayed on the display unit 130 has been completed (S1006: YES), the image data is transmitted to the first OETF conversion unit 903.
[0119] In step S1007, the CPU 113 controls the first OETF converter 903 to convert the OETF of the transmitted image data from SMPTE ST 2084 to an OETF based on the light emission characteristics of the display material 134. The result of this processing is shown in FIG.
[0120] In step S 1008 , CPU 113 transmits the converted image data to display unit 130 via transmission unit 112 and data transmission path 120 .
[0121] In step S 1009 , the CPU 113 controls the receiving unit 131 to receive the image data transmitted through the data transmission path 120 .
[0122] In step S1010, the CPU 113 controls the DA conversion unit 132 to convert the image data, which is a digital signal, into an analog signal.
[0123] In step S1011, CPU 113 controls EOTF conversion unit 133 to apply EOTF conversion based on the light emission characteristics of display material 134 to the image data, and determines the light emission luminance for each pixel on display material 134.
[0124] In step S1012, CPU 113 displays an image on display unit 130 by causing the display material to emit light.
[0125] As a result, with a configuration different from that of the first and second embodiments, it is possible to transmit an HDR image and an SDR image through one port and display them together.
[0126] The disclosure of this specification includes the following image processing device, its control method, program, and storage medium.
[0127] (Item 1) a first conversion means for performing at least one of a first conversion for converting an OETF (Opto-Electro Transfer Function) of an HDR image (High Dynamic Range image) into an OETF based on the light emission characteristics of a display means, and a second conversion for converting an OETF of an SDR image (Standard Dynamic Range image) into an OETF of the light emission characteristics of the display means; A transmission means for transmitting an HDR image converted by the first conversion and an SDR image converted by the second conversion to the display means; An image processing device comprising:
[0128] (Item 2) 2. The image processing device according to item 1, wherein the first conversion and the second conversion involve performing an inverse conversion of the light emission characteristics of the display means.
[0129] (Item 3) 3. The image processing device according to item 1 or 2, further comprising a display means for simultaneously displaying an HDR image converted by the first conversion and an SDR image converted by the second conversion.
[0130] (Item 4) 4. The image processing device according to any one of items 1 to 3, further comprising a synthesis unit that synthesizes the HDR image and the SDR image before converting them by the first and second conversions.
[0131] (Item 5) 5. The image processing device according to item 4, further comprising a division means for dividing a composite image synthesized by the synthesis means into an HDR display area and an SDR display area.
[0132] (Item 6) 6. The image processing device according to item 5, characterized in that in the first conversion, the HDR display area is converted into an OETF based on the light emission characteristics of the display means.
[0133] (Item 7) 6. The image processing device according to item 5, wherein in the second conversion, the SDR display area is converted into an OETF based on the light emission characteristics of the display means.
[0134] (Item 8) 4. The image processing device according to any one of items 1 to 3, further comprising a synthesis means for synthesizing an HDR image converted by the first conversion and an SDR image converted by the second conversion.
[0135] (Item 9) 4. The image processing device according to any one of items 1 to 3, further comprising a second conversion means for converting the SDR image into an HDR OETF.
[0136] (Item 10) 10. The image processing device according to item 9, further comprising a synthesis means for synthesizing the SDR image converted by the second conversion means and the HDR image.
[0137] (Item 11) 11. The image processing device according to item 10, wherein the first conversion means performs the first conversion on the image combined by the combination means.
[0138] (Item 12) The image processing device according to any one of items 1 to 11, characterized in that the display means is composed of an OLED (Organic Light Emitting Diode), the HDR EOTF (Electro-Optical Transfer Function) is based on SMPTE STANDARD 2084, and the SDR EOTF is based on RECOMMENDATION ITU-R BT.709.
[0139] (Item 13) 13. The image processing device according to any one of items 1 to 12, wherein the transmitting means is configured with one port, and the standard of the transmitted signal complies with MIPI or Sub-LVDS.
[0140] (Item 14) An imaging means; An image processing device according to any one of items 1 to 13, An imaging device comprising:
[0141] (Item 15) a first conversion step of converting an OETF (Opto-Electro Transfer Function) of an HDR image (High Dynamic Range image) into an OETF based on the light emission characteristics of a display means, and a second conversion step of converting an OETF of an SDR image (Standard Dynamic Range image) into an OETF of the light emission characteristics of the display means; A transmission step of transmitting an HDR image converted by the first conversion and an SDR image converted by the second conversion to the display means; 13. A method for controlling an image processing apparatus comprising the steps of:
[0142] (Item 16) Item 16. A program for causing a computer to execute each step of the control method described in item 15.
[0143] (Item 17) A computer-readable storage medium storing a program for causing a computer to execute each step of the control method described in item 15.
[0144] (Other embodiments) The present invention can also be realized by a process in which a program for realizing one or more functions of the above-mentioned embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) for realizing one or more functions.
[0145] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]
[0146] 1, 2, 3 Imaging device, 100: image generating section, 101: sensor section, 102: AD conversion section, 103: sensor correction section, 104: developing section, 105: OETF conversion section, 106: recording and reproducing section, 107: recording medium, 108: image synthesis section, 109: area division section, 110: first OETF conversion section, 111: second OETF conversion section, 112: transmitting section, 113: CPU, 114: operation section, 120: data transmission path, 130: display section, 131: receiving section, 132: DA conversion section, 133: EOTF conversion section, 134: display material
Claims
1. a first conversion means for performing at least one of a first conversion for converting an OETF (Opto-Electro Transfer Function) of a HDR image (High Dynamic Range image) into an OETF based on the light emission characteristics of a display means, and a second conversion for converting an OETF of a SDR image (Standard Dynamic Range image) into an OETF of the light emission characteristics of the display means; a transmitting means for transmitting the HDR image converted by the first conversion and the SDR image converted by the second conversion to the display means; An image processing device comprising:
2. 2. The image processing apparatus according to claim 1, wherein the first conversion and the second conversion involve performing an inverse conversion of the light emission characteristics of the display means.
3. 2. The image processing apparatus according to claim 1, further comprising a display unit that simultaneously displays the HDR image converted by the first conversion and the SDR image converted by the second conversion.
4. 2. The image processing apparatus according to claim 1, further comprising a synthesis unit that synthesizes the HDR image and the SDR image before the first and second conversions are performed.
5. 5. The image processing device according to claim 4, further comprising a division unit that divides the composite image generated by the composition unit into an HDR display area and an SDR display area.
6. The image processing device according to claim 5 , wherein the first conversion converts the HDR display area into an OETF based on a light emission characteristic of the display means.
7. 6. The image processing apparatus according to claim 5, wherein the second conversion converts the SDR display area into an OETF based on a light emission characteristic of the display means.
8. The image processing device according to claim 1 , further comprising a synthesis unit that synthesizes an HDR image converted by the first conversion and an SDR image converted by the second conversion.
9. 2. The image processing apparatus according to claim 1, further comprising a second conversion unit for converting the SDR image into an HDR OETF.
10. 10. The image processing apparatus according to claim 9, further comprising a synthesis unit that synthesizes the SDR image converted by the second conversion unit with the HDR image.
11. 11. The image processing apparatus according to claim 10, wherein the first conversion means performs the first conversion on the image synthesized by the synthesis means.
12. 2. The image processing device according to claim 1, wherein the display means is composed of an OLED (Organic Light Emitting Diode), an EOTF (Electro-Optical Transfer Function) of HDR is based on SMPTE STANDARD 2084, and an EOTF of SDR is based on RECOMMENDATION ITU-R BT.
709.
13. 2. The image processing apparatus according to claim 1, wherein said transmitting means is configured with one port, and the standard of the transmitted signal complies with MIPI or Sub-LVDS.
14. An imaging means; An image processing device according to any one of claims 1 to 13, An imaging device comprising:
15. a first conversion step of converting an OETF (Opto-Electro Transfer Function) of a HDR image (High Dynamic Range image) into an OETF based on the light emission characteristics of a display means, and a second conversion step of converting an OETF of a SDR image (Standard Dynamic Range image) into an OETF of the light emission characteristics of the display means; a transmitting step of transmitting the HDR image converted by the first conversion and the SDR image converted by the second conversion to the display means; 13. A method for controlling an image processing apparatus comprising the steps of:
16. A program for causing a computer to execute each step of the control method according to claim 15.
17. A computer-readable storage medium storing a program for causing a computer to execute each step of the control method according to claim 15.
Citation Information
Patent Citations
Image output device, control method of them, program, and storage medium
JP2020136737A
