IMAGE PROCESSING APPARATUS, IMAGING APPARATUS, CONTROL METHOD, AND PROGRAM

Abstract

To create an image for display allowing a browser to perceive homogeneous visibility, for example, a contrast feel.SOLUTION: An image processing apparatus for creating an image to be displayed on display means capable of changing settings of the maximum display luminance, has: first acquisition means that acquires an image signal; second acquisition means that acquires the settings of the maximum display luminance of the display means; creation means that performs image processing including gradation conversion on the image signal acquired by the first acquisition means to create an image to be displayed; and setting means that, based on the settings of the maximum display luminance acquired by the second acquisition means, sets gradation characteristics used for the gradation conversion performed by the creation means. The gradation characteristics are to allocate a signal value of the image signal to a luminance area according to the settings of the maximum display luminance, and for at least a part of the luminance area, a logarithmic value of the signal value of the image signal and an absolute luminance indicate substantially the same linear relation of a proportional constant regardless of the settings of the maximum display luminance.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to an image processing device, an imaging device, a control method and a program, and more particularly to a tone conversion technique for generating an image for display. [Background technology]

[0002] A display has its own gamma characteristic (display characteristic) that determines the display brightness at which the gradation of an input image is output. Therefore, in order to display an image on a display with a desired gradation expression, it is necessary to adjust the input image taking into consideration the display characteristic. Patent Document 1 discloses that an input image with improved visibility is generated by using gradation characteristics that are adjusted to be the inverse characteristic of the display characteristic of an electronic viewfinder of an imaging device. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2010-245924 A Summary of the Invention [Problem to be solved by the invention]

[0004] In Patent Document 1, in order to ensure visibility in an image displayed on an electronic viewfinder, an input image is generated from a captured image using gradation characteristics configured by multiplying the inverse characteristic of the display characteristic by a power corresponding to the brightness of the imaging scene (visual environment). More specifically, when the imaging scene is bright, a through image with increased contrast is displayed by increasing the multiplier, and when the imaging scene is dark, a through image with reduced contrast is displayed by decreasing the multiplier.

[0005] On the other hand, changing the gradation characteristics by applying a power to the whole depending on the brightness of the image capture scene as in Patent Document 1 causes a user looking at an electronic viewfinder to perceive a different sense of contrast at each brightness, which can cause discomfort. Furthermore, such perceptual discomfort can also occur in an embodiment in which the maximum display luminance of the display is dynamically changed depending on the brightness of the surroundings, or the maximum display luminance is changed arbitrarily.

[0006] The present invention has been made in consideration of the above-mentioned problems, and aims to provide an image processing device, an imaging device, a control method, and a program for generating a display image that provides the viewer with uniform visibility, for example, a sense of contrast. [Means for solving the problem]

[0007] In order to achieve the above-mentioned object, the image processing device of the present invention is an image processing device that generates an image to be displayed on a display means configured to be able to change the setting of the maximum display luminance, and has a first acquisition means for acquiring an image signal, a second acquisition means for acquiring the setting of the maximum display luminance of the display means, a generation means for performing image processing including gradation conversion on the image signal acquired by the first acquisition means to generate an image to be displayed, and a setting means for setting gradation characteristics to be used for the gradation conversion performed by the generation means based on the setting of the maximum display luminance acquired by the second acquisition means, wherein the gradation characteristics assign signal values ​​of the image signal to a luminance range according to the setting of the maximum display luminance, and are characterized in that for at least a part of the luminance range, the logarithmic value of the signal value of the image signal and the absolute luminance show a linear relationship with approximately the same proportional constant regardless of the setting of the maximum display luminance. Effect of the Invention

[0008] With this configuration, the present invention makes it possible to generate a display image that allows the viewer to perceive uniform visibility, for example, a sense of contrast. [Brief description of the drawings]

[0009] [Figure 1]FIG. 1 is a diagram illustrating a functional configuration of a digital camera 100 according to an embodiment and a modification of the present invention. [Diagram 2] FIG. 13 is a diagram illustrating input / output characteristics in a setting priority mode according to an embodiment and a modification of the present invention; [Diagram 3] FIG. 13 is a diagram illustrating the relationship between an input signal and display luminance in a setting priority mode according to an embodiment and a modification of the present invention; [Figure 4] FIG. 13 is a diagram illustrating absolute input / output characteristics in a setting priority mode according to an embodiment and a modification of the present invention; [Diagram 5] FIG. 13 is a diagram illustrating absolute input / output characteristics in a uniform perception mode according to the first embodiment of the present invention; [Figure 6] FIG. 11 is another diagram illustrating the absolute input / output characteristics of the uniform perception mode according to the first embodiment of the present invention; [Figure 7] FIG. 13 is a diagram illustrating a functional configuration of an image processing unit 105 in a setting priority mode according to an embodiment and a modified example of the present invention. [Figure 8] FIG. 13 is a diagram illustrating a functional configuration of an image processing unit 105 relating to a uniform perception mode according to an embodiment and a modification of the present invention. [Figure 9] FIG. 13 is a diagram illustrating absolute input / output characteristics in a uniform perception mode according to a second embodiment of the present invention; DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] [Embodiment 1] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0011] In the embodiment described below, an example in which the present invention is applied to a digital camera having a display capable of operating as an electronic viewfinder as an example of an image processing device will be described. However, the present invention can be applied to any device capable of performing tone conversion to generate an image for display. Any device includes digital cameras, digital video cameras, personal computers, tablet terminals, mobile phones, game consoles, and, for example, see-through goggles used for presenting AR (Augmented Reality) and MR (Mixed Reality).

[0012] <Digital camera configuration> FIG. 1 is a block diagram showing the functional configuration of a digital camera 100 according to an embodiment of the present invention.

[0013] The control unit 101 is a control unit including at least one processor or circuit. The control unit 101 reads out an operation program for each block of the digital camera 100 from a recording medium 102, and loads the program into a memory 103 and executes the program to control the operation of the digital camera 100.

[0014] The recording medium 102 is a non-volatile recording device such as a Flash-ROM that is electrically erasable and recordable. The recording medium 102 records constants required for the operation of each block in addition to the operation programs of each block of the digital camera 100. The recording medium 102 may also serve as a semiconductor memory card for recording images (RAW data, developed images, etc.) obtained by imaging. On the other hand, the memory 103 is a volatile recording device such as a RAM or DRAM, and is used as an area for loading the operation programs of each block. The memory 103 is also used as a VRAM when displaying images on a display 106, which will be described later.

[0015] Lens 110 is a unit equipped with a group of imaging lenses, and is configured to be detachable from digital camera 100. Lens 110 is usually composed of multiple lenses, but for simplicity's sake, only one lens is shown in Fig. 1. Lens 110 has a control circuit (not shown), and its state is controlled based on a drive signal input from control unit 101.

[0016] The imaging unit 104 is an imaging element such as a CCD or CMOS sensor, and obtains an analog image signal by converting an optical image formed on an imaging surface by the lens 110 into an electrical signal. The obtained analog image signal is converted into a digital image signal (hereinafter referred to as RAW data) by an A / D converter (not shown). In this embodiment, the imaging unit 104 is described as a single-plate color imaging element equipped with a general primary color filter. Here, the primary color filter is configured by arranging three types of color filters having main transmission wavelength bands in the vicinity of 650 nm, 550 nm, and 450 nm in a mosaic pattern (Bayer array). By applying the primary color filter, each pixel of the single-plate color imaging element captures a color plane corresponding to one of the bands of R (red), G (green), or B (blue). In other words, the photoelectric conversion elements constituting the single-plate color imaging element can only obtain light intensity related to a single color plane. The imaging unit 104 may also include peripheral circuits such as an amplifier circuit that processes signals obtained from the pixels.

[0017] The image processing unit 105 performs various image processing such as pixel interpolation, resizing, and color conversion on the RAW data or the RAW data read from the recording medium 102. Details will be described later. The image processing unit 105 also performs arithmetic processing on the RAW data obtained by imaging to derive information required for exposure control and distance measurement control. In the digital camera 100 of this embodiment, TTL (through-the-lens) type AF (autofocus) processing, AE (autoexposure) processing, and EF (flash pre-flash) processing are performed based on the information. The image processing unit 105 also performs TTL type AWB (auto white balance) processing by performing arithmetic processing on the image data obtained by imaging.

[0018] The display 106 is, for example, a display device such as a liquid crystal display, and displays information such as settings of the digital camera 100, messages, GUIs such as menu screens, captured images, etc. The display 106 may be an electronic viewfinder (EVF) or a rear liquid crystal display built into the digital camera 100, or an external display detachably connected to the digital camera 100. In the following explanation, in order to explain display control of the electronic viewfinder while the digital camera 100 is capturing images, the display 106 is referred to as an electronic viewfinder, but the embodiment of the present invention is not limited to this.

[0019] The display 106 includes a display control circuit (not shown) and is configured to be able to change the maximum display luminance. The maximum display luminance of the display 106 may be dynamically controlled according to a BV value obtained by photometry of the image capture scene, or may be set by estimating the scene brightness according to the set image capture mode. Here, the photometry of the image capture scene may be performed based on an image signal obtained by the image capture unit 104, or may be performed based on the output of a separately provided photometry sensor, etc. In this embodiment, information on the maximum display luminance of the display 106 may be obtained, and the information is supplied to at least the image processing unit 105.

[0020] The operation unit 107 is a user interface that is provided in the digital camera 100 of this embodiment and that accepts various operational inputs. When the operation unit 107 detects that an operational input has been made to each user interface, it outputs a corresponding control signal to the control unit 101. The operation unit 107 includes a release switch for instructing the start of a shooting preparation operation and the start of shooting (main shooting), an imaging mode selection switch for selecting an imaging mode, direction keys, an enter key, and the like.

[0021] In this embodiment, the processing according to the invention will be described as being realized by circuits and processors corresponding to each block of the hardware provided in the digital camera 100. However, the embodiment of the present invention is not limited to this, and the processing of each block may be realized by a program that performs the same processing as that of each block.

[0022] <<Control of generation of images for electronic viewfinder>> Next, the generation of an image (display image) to be displayed on the electronic viewfinder in the digital camera 100 of this embodiment will be described in detail. The digital camera 100 of this embodiment has two display modes for the electronic viewfinder during image capture: a setting priority mode as a first mode according to the present invention, and a uniform perception mode as a second mode. These display modes are intended to cause the captured images acquired by the image capture unit 104 to be displayed sequentially on the display 106, and to differ in the generation method (development method) of the presented image when functioning as an electronic viewfinder, and can be switched independently of the image capture mode.

[0023] The digital camera 100 of this embodiment is configured to allow the user to set in advance the contrast correction, exposure correction, saturation correction, etc., performed during the development process in at least some of the imaging modes in which the developed image is recorded. The setting priority mode is a mode that provides an electronic viewfinder display so that the user can check the contents of the image correction applied by the development process before the actual shooting, and is suitable for shooting while grasping the image to be recorded. On the other hand, the uniform perception mode is a mode that provides a display of a sense of contrast similar to that when the user looks at the subject directly with the naked eye or when the user looks at the subject through a lens via an optical viewfinder (OVF). The uniform perception mode of this embodiment, which will be described in detail later, enables the display of a perceptually uniform gradation expression regardless of the maximum display luminance of the display.

[0024] The generation of a display image by the image processing unit 105 includes at least tone conversion that assigns tone values ​​to signal intensities indicated in the RAW data. The tone conversion is performed based on input / output characteristics that indicate the relationship between the signal values ​​of the RAW data and the tone values ​​after development processing. For example, tone conversion converts the signal values ​​of RAW data with a resolution of 14 bits into 8-bit tone values ​​after development processing. The input / output characteristics referenced in tone conversion differ between the setting priority mode and the uniform perception mode, which results in a difference in tone expression.

[0025] In this embodiment, the electronic viewfinder is described as being provided with two switchable display modes, a setting priority mode and a uniform perception mode, in any imaging mode, but the present invention is not limited to this. For example, only the uniform perception mode may be used in at least some imaging modes.

[0026] Also, the maximum display luminance of the display 106 is controlled to be different depending on whether the imaging scene is a sunny daytime scene or a nighttime scene, taking into consideration the visibility of the electronic viewfinder. Specifically, when the imaging scene is a bright scene such as a sunny daytime scene, the maximum display luminance is set to 450 nit, and when the imaging scene is a dark scene such as a nighttime scene, the maximum display luminance is set to 50 nit. However, the value of the maximum display luminance in each scene is not limited to this, and other values ​​may be set.

[0027] <Settings priority mode> First, generation of a display image in the setting priority mode will be described.

[0028] In the image processing unit 105 of this embodiment, in the tone conversion related to the setting priority mode, common input / output characteristics are used regardless of the brightness of the imaging scene, for example, as shown in Fig. 2. In the example of Fig. 2, as described above, an 8-bit tone value of the display image (developed image) is assigned to a signal value of 14-bit RAW data.

[0029] On the other hand, the display 106 has its own display characteristics (for example, γ2.2 in this embodiment), and is configured to perform control to switch the maximum display luminance according to the brightness of the imaging scene. Therefore, when a display image is displayed on the display 106, the relationship between the signal value of the RAW data and the display luminance on the display 106 is as shown in FIG. 3. In the figure, FIG. 3(a) shows the relationship when the imaging scene is a sunny daytime scene, and FIG. 3(b) shows the relationship when the imaging scene is a night scene. As shown in the figure, the vertical axis of the graph in FIG. 3 indicates the display luminance [nit] on the display 106, and the luminance range to which the signal value of the RAW data is assigned is different between a sunny daytime scene and a night scene. In other words, it is preferable to change the maximum display luminance of the display 106 according to the brightness of the imaging scene, but this also changes the range of the display luminance that realizes gradation expression.

[0030] Here, the difference in human visual perception that appears between a clear daytime sky and a night scene, which have different maximum display luminance in the setting priority mode, will be described with reference to the drawings.

[0031] According to the Weber-Fechner law, the amount of sensation perceived by humans is proportional to the logarithm of the amount of stimulation given to the sensory receptors. In this specification, based on such a relationship between the amount of sensation and the amount of stimulation, the input / output relationship for converting the signal value of RAW data to the final display luminance is evaluated on the scale of human perception to compare the input / output characteristics. That is, the signal value according to the amount of light appearing in the RAW data is treated as the amount of stimulation, and the final display luminance taking into account the maximum display luminance of the display 106 is treated as the amount of sensation to evaluate the relationship, thereby explaining the difference in the sense of contrast perceived through the electronic viewfinder. Note that the input / output characteristics referred to here are different from the gradation characteristics related to the conversion from the signal value of RAW data to the gradation value of the developed image shown in FIG. 2, and are gradation characteristics related to the conversion from the signal value of RAW data to absolute luminance when displayed on an arbitrary display. In the following, in order to distinguish between these, the gradation characteristics related to the conversion from the signal value of RAW data to absolute luminance are referred to as absolute input / output characteristics.

[0032] In the following, to show the relationship between the signal value of RAW data and the absolute input / output characteristics of absolute luminance, the former is used as the logarithmic value of the signal value, and the latter is used as the luminance component value of a perceptually uniform color space conforming to human visual characteristics. In this embodiment, the I value of the ICtCp color space defined in ITU-R BT.2100 is adopted as the luminance component value of the perceptually uniform color space. The I value can be derived from the RGB value using the inverse characteristic (Inverse EOTF) of the EOTF of the PQ (Perceptual Quantization) method standardized in SMPTE ST 2084. The PQ method determines absolute luminance independent of the display characteristics specific to the display device, and is suitable for defining a sensory amount because bits are efficiently allocated based on the human visual characteristics.

[0033] Based on this, the absolute input / output characteristics in two types of imaging scenes in the setting priority mode have a relationship as shown in Fig. 4(a). In the figure, the horizontal axis indicates the logarithmic value of the signal value of RAW data (hereinafter simply referred to as the logarithmic value of RAW data), which is normalized by the value of the proper exposure (shown in steps with the logarithmic value of the proper exposure set to 0). The vertical axis indicates the I value. In the example of Fig. 4(a), a curve (solid line) 401 indicates the absolute input / output characteristics during sunny daytime, and a curve (dashed line) 402 indicates the absolute input / output characteristics during nighttime. As shown in the figure, it can be seen that the range of I values ​​(luminance range) assigned to the logarithmic value of RAW data differs due to the difference in maximum display luminance between sunny daytime and nighttime.

[0034] If there is a linear relationship between input and output, i.e., the logarithmic value of the RAW data and the I value, then it follows the Weber-Fechner law and can be said to represent an even sense of contrast that matches human perception characteristics. In the example of Figure 4(a), both the absolute input and output characteristics during sunny daytime hours and the absolute input and output characteristics during night scenes show a linear relationship in the value range (step range) near the correct exposure. Hereinafter, a graph like Figure 4(a), which shows the logarithmic value of the RAW data on the horizontal axis and the I value on the vertical axis, will be referred to as a "perceptual linear graph."

[0035] According to the Weber-Fechner law, the increment (magnification) of the amount of stimulation is proportional to the increment of the amount of sensation, so if the increment of the logarithmic value of the RAW data is equal, the increment of the I value will not change regardless of the reference logarithmic value. In other words, if it is perceptually uniform, the difference in brightness (increment of the I value) when the logarithmic value of the RAW data increases from the reference value by an arbitrary multiple should be perceived to be approximately the same regardless of the reference value. In other words, if it is perceived as a perception (vision) by the same sensory receptor, it is necessary to show the same slope in the perceptual linear graph, that is, the same proportional constant (stimulus constant), regardless of the maximum display luminance of the display 106.

[0036] However, the two absolute input / output characteristics shown in the perceptual linear graph of Fig. 4(a) have different slopes in the linearly related sections, as shown by the straight line (dashed line) 403 and the straight line (dashed line) 404 in Fig. 4(b). This means that in the setting priority mode, absolute input / output characteristics that cause different senses of contrast to be perceived are adopted for two imaging scenes that differ in the maximum display luminance of the display 106. In other words, this means that the difference in brightness perceived in the electronic viewfinder is different between day and night and is not uniform.

[0037] The difference in the slope in the perceptual linear graph is manifested in such a manner that, for example, when the imaging scene is a night scene, the contrast is appropriate compared to how it appears to the naked eye, whereas when it is a sunny day, the contrast is higher than actual and dark areas appear crushed. In other words, in a mode in which developed images obtained with the same input / output characteristics as shown in Fig. 2 are displayed on the display 106, the degree of discrepancy in the sense of contrast between when the subject is viewed with the naked eye and when viewed through an electronic viewfinder changes according to the maximum display luminance of the display 106. More specifically, when the input / output characteristics are fixed and the maximum display luminance of the display 106 is changed, the higher the maximum display luminance, the higher the contrast becomes (the slope in the perceptual linear graph becomes steeper), leading to a discrepancy in the sense of contrast between imaging scenes.

[0038] <Uniform Perception Mode> In contrast, in the uniform perception mode, the absolute input / output characteristics are set so as to show a constant stimulation constant in a step range or luminance range that has a linear relationship on the perceptual linear graph, regardless of the maximum display luminance of the display 106. Specifically, for an image capture scene of a sunny day with the maximum display luminance set to 450 nit, the absolute input / output characteristics shown by curve 501 on the perceptual linear graph in Fig. 5 are used. And, for an image capture scene of a night scene with the maximum display luminance set to 50 nit, the absolute input / output characteristics shown by curve 601 on the perceptual linear graph in Fig. 6 are used.

[0039] As shown in the figure, the two absolute input / output characteristics match the dashed line 502 or 602, which shows a common slope (stimulation constant), in at least a part of the step range or brightness range, and show a mode in which the slope is gradually reduced toward the dark and saturated areas. In this embodiment, the slope of the dashed lines 502 and 602 is set to 75. In addition, the I value at the time of proper exposure is set to the same value as in the setting priority mode so that the difference in brightness atmosphere between the two absolute input / output characteristics and the setting priority mode for the same imaging scene is reduced.

[0040] Here, the slope value 75 in the range where the linear relationship appears (matches the dashed line) is a provisional optimum value derived through experiments so that the difference in brightness between when the subject is viewed with the naked eye and when viewed through an electronic viewfinder is small and the sense of contrast is uniform. In this embodiment, the I-intercepts of the curve 501, dashed line 502, curve 601, and dashed line 602 are all set to the same value as the I-value at the correct exposure at the highest display luminance in the setting priority mode. This makes it possible to make the display luminance at the correct exposure be a suitable value like the setting priority mode, while making the difference from the signal value of the RAW data at the correct exposure appear as a uniform difference in display luminance in any imaging scene.

[0041] Therefore, in the uniform perception mode, by adopting gradation characteristics of absolute input / output characteristics that show the same slope on a perceptual linear graph, it is possible to provide an electronic viewfinder with a uniform sense of contrast, regardless of the maximum display luminance of the display 106. In other words, because the difference in brightness at least within the range where a linear relationship appears appears constant, regardless of the imaging scene or the maximum display luminance of the display 106, a homogeneous sense of contrast can be perceived.

[0042] <Functional configuration of image processing unit 105> The functional configuration of the image processing unit 105 related to the generation of display images for each display mode will be described below with reference to the drawings. In this embodiment, in order to easily understand the difference between the development processes performed in relation to the generation of display images in the setting priority mode and the uniform perception mode, the functional configuration will be described using different diagrams for each mode, but these do not need to be implemented as separate hardware. In other words, at least a part of the functional configuration may be configured to be capable of generating display images for two display modes by controlling whether or not to use the functional configuration depending on the switching of the display mode, or by changing the operation, etc. These functional configurations are illustrated as different blocks that respectively represent the processes performed by the image processing unit 105 during the development process.

[0043] <Settings priority mode> The development process for the display image in the setting priority mode is realized by the functional configuration shown in Fig. 7. As described above, a primary color filter in which three types of color filters are arranged in a mosaic pattern is applied to the imaging unit 104. Therefore, the RAW data 701 is a color mosaic image. The image processing unit 105 reads the RAW data 701 from the memory 103 and applies the development process related to the setting priority mode to generate a display image 708.

[0044] The white balance unit 702 performs white balance processing on the RAW data 701, which converts the image of the subject, which is originally white, into white. More specifically, the white balance unit 702 plots the RGB data of each pixel constituting the RAW data 701 in a predetermined color space, such as an xy color space. The white balance unit 702 then integrates R, G, and B of the data plotted in the color space near the locus of blackbody radiation, which is likely to be the light source color, and derives white balance coefficients (G / R and G / B) for the R and B components from the integral value. The white balance unit 702 performs white balance processing using the obtained white balance coefficients, thereby correcting color casts caused by the light source and reproducing white.

[0045] The color interpolation unit 703 performs noise reduction processing and processing to interpolate pixel values ​​of color components not included in each pixel on the image data converted by the white balance unit 702. This processing generates a synchronized image in which R, G, and B color information (pixel values ​​of color components) are complete for all pixels.

[0046] The synchronized image generated by the color interpolation unit 703 is converted into a color image that is the basis of processing by a matrix conversion unit 704 performing matrix conversion processing. Furthermore, a color brightness adjustment unit 706 applies adjustment processing for adjusting color and brightness to the color image, and a gradation conversion unit 707 applies gradation conversion (gamma conversion) processing related to the input / output characteristics of Fig. 2, thereby generating an image for display 708.

[0047] Here, the adjustment by the color luminance adjustment unit 706 includes adjustment performed by referring to the color luminance adjustment parameters 705 describing the settings of contrast correction, exposure correction, saturation correction, etc. applied to the recorded image. Also, the gradation conversion unit 707 applies the gradation conversion process using the input / output characteristics shown in Fig. 2 uniformly regardless of the imaging scene.

[0048] The display image 708 thus generated is displayed on the display 106 with the maximum display luminance controlled according to the captured scene, thereby realizing an electronic viewfinder in the setting priority mode. That is, between the RAW data 701 and the displayed display image 708, the absolute input / output characteristic relationship shown in the perceptual linear graph of FIG. 4 holds true.

[0049] <Uniform Perception Mode> The development process for the display image in the uniform perception mode is realized by the functional configuration shown in Fig. 8. In Fig. 8, the functional configuration that performs the same operations as in the setting priority mode is given the same reference numerals, and only the functional configuration that performs different operations will be described below.

[0050] In the uniform perception mode, the color and luminance adjustment performed by the color and luminance adjustment unit 801 on the color image obtained by the matrix conversion unit 704 does not include adjustment according to the recording settings. Then, a gradation conversion unit 805 applies gradation conversion processing to the image adjusted by the color and luminance adjustment unit 801 to generate an image for display 806.

[0051] The gradation characteristics configured by the gradation characteristics configuration unit 804 are used in the gradation conversion process performed by the gradation conversion unit 805. When configuring the gradation characteristics, the gradation characteristics configuration unit 804 acquires the maximum display luminance 802 of the display device 106 and an adjustment curve 803 (one of the curves in FIG. 5 and FIG. 6) of the absolute input / output characteristics corresponding to the maximum display luminance 802. Here, since the adjustment curve 803 indicates the relationship between the logarithm value of the RAW data 701 and the I value which is the absolute luminance, the gradation characteristics configuration unit 804 of this embodiment performs conversion so that the display luminance corresponds to the absolute luminance on the display device 106. That is, the adjustment curve 803 configures the gradation characteristics used for the gradation conversion of the gradation conversion unit 805 by multiplying the absolute input / output characteristics indicated by the adjustment curve 803 by the inverse characteristics of the display characteristics. Using the gradation characteristics configured in this way, the gradation conversion unit 805 performs the gradation conversion process of the image subjected to the color luminance adjustment.

[0052] The display image 806 thus generated is displayed on the display 106 whose maximum display luminance is controlled according to the captured scene, thereby realizing an electronic viewfinder related to the uniform perception mode. That is, for the RAW data 701 and the displayed display image 806, the absolute input / output characteristic relationship shown in the perceptual linear graph of Fig. 5 or Fig. 6 holds true, and the perception of a uniform sense of contrast is realized regardless of the maximum display luminance.

[0053] [Variation 1] In the uniform perception mode of the above embodiment, the inverse characteristic of the display characteristic of the display device 106 is multiplied so that the absolute luminance shown in the adjustment curve 803 is guaranteed. However, the present invention is not limited to this. As described above, the I value conforms to the PQ method that determines the absolute luminance independent of the display characteristic specific to the display device. Therefore, in the case of using a display device capable of displaying an image developed in the PQ method, it is not necessary to multiply the inverse characteristic of the display characteristic when configuring the gradation characteristic from the adjustment curve 803. The present invention can be realized as long as it is a device that generates an image that presents a sense of contrast that is perceived equally regardless of the change in the maximum display luminance of the display device that can be changed. At this time, the information on the maximum display luminance that is set may be obtained from the display device via a signal formulated in accordance with the standard adopted for connection with the display device, or may be obtained from another device that controls the operation of the display device. That is, it is not necessary to determine the maximum display luminance based on the photometric result of the image capture scene as in the first embodiment.

[0054] [Variation 2] In the above-described embodiment, the absolute input / output characteristics of the uniform perception mode shown in Figs. 5 and 6 are described with the slope of the dashed lines 502 and 602 set to 75, but the present invention is not limited to this. For example, even if the slope is 65 to 85, the sense of contrast when viewing the subject with the naked eye is expressed. Therefore, for the two absolute input / output characteristics, the stimulus constants related to the step range or luminance range in which a linear relationship appears do not need to be strictly the same, and may be set to approximately the same value included in the above range (65 to 85).

[0055] [Variation 3] 5 and 6, the maximum display luminance of the display 106 differs between sunny daytime and nighttime, so there is a difference in the step range or luminance range where the linear relationship appears. In the example shown, the linear relationship appears in the step range or luminance range that includes proper exposure, from approximately -5 steps to approximately +2 steps during sunny daytime and from approximately -3 steps to approximately +1 step during nighttime. This difference is due to the fact that the luminance range that can be assigned to the signal value of the RAW data is expanded according to the maximum display luminance of the display 106.

[0056] However, the embodiment of the present invention is not limited to this, and the step range or luminance range in which the linear relationship appears may be changeable according to the luminance range in which the subject to be easily viewed exists, such as the high luminance side in a night scene or the low luminance side in a sunny day. Also, the width of the step range or luminance range in which the linear relationship appears is not necessarily limited to those shown in Figures 5 and 6, although a wider range gives the user an impression of perceptual uniformity. Presentation of a uniform sense of contrast that is not dependent on the maximum display luminance of the display 106 may be changeable so as to be realized in a specific step range or luminance range.

[0057] [Variation 4] 5 and 6, the I-intercept of each absolute input / output characteristic is set to the same value as the I-value in the correct exposure in the setting priority mode, but the present invention is not limited to this. For example, in a night scene, the I-intercept may be set to a higher value in order to achieve both a sense of contrast close to that of the naked eye and improved visibility of dark areas. Also, in a night scene, the I-intercept may be set to a lower value in order to achieve a brightness close to that of the subject when viewed with the naked eye. In addition, the I-intercept may be configured to be changeable to any value, not limited to a night scene, as long as it is guaranteed that the slope related to the absolute input / output characteristic is approximately the same regardless of the maximum display luminance. The change of the I-intercept may be performed according to a user setting, a scene recognition result, or the like.

[0058] [Variation 5] In the above embodiment, the maximum display luminance of the display 106 is set to two levels, and two types of absolute input / output characteristics are exemplified for the uniform perception mode, but it will be easily understood that the present invention is not limited to this. In other words, the present invention only requires that the gradation characteristics used to generate an image for display include a step range or luminance range that shows a linear relationship with approximately the same proportional constant regardless of the maximum display luminance between the logarithm value of the input image signal and the absolute luminance when displaying a developed image of the image signal. Therefore, it is sufficient that such gradation characteristics are provided for the number of maximum display luminances that can be set on the display 106, and that the proportional constant of the range in which the linear relationship appears is approximately the same between them.

[0059] [Embodiment 2] Incidentally, in an aspect in which the correction settings applied to the recorded image can be confirmed on the electronic viewfinder, such as in the setting priority mode, a display image that causes burn-in on the display 106 may be output depending on the contents of the correction settings and the maximum display luminance of the display 106. For example, in an imaging scene with a wide dynamic range, such as during sunny days, it is preferable to increase the maximum display luminance of the display 106 to expand the dynamic range of the electronic viewfinder in order to improve visibility. However, depending on the correction settings applied to the recorded image, the display image may be saturated all over, and the display 106 may be damaged by displaying it for a long time. In addition, displaying such an image with an increased maximum display luminance also leads to increased power consumption. For this reason, the maximum display luminance that can be set on the display 106 in the setting priority mode needs to be limited.

[0060] On the other hand, in the uniform perception mode, no adjustment is made according to the correction settings applied to the recorded image, and the absolute luminance on the display 106 is determined by the absolute input / output characteristics, so there is a lower risk of damage to the display 106 than in the setting priority mode. For this reason, the maximum display luminance of the display 106, which is set according to the photometry results, may be configured to be set to a higher value in the uniform perception mode than in the setting priority mode.

[0061] For example, when the image capture scene is a sunny daytime scene, the maximum display luminance is set to 450 nit in the setting priority mode, and gradation conversion is performed so that the I value falls within 700 as shown in FIG. 4, whereas the uniform perception mode may be set to a higher 800 nit. In this case, the dynamic range of the electronic viewfinder is expanded by the amount of the increased maximum display luminance, so that the absolute input / output characteristics related to the uniform perception mode can raise the maximum I value, for example, as shown by the solid line 901 in FIG. 9. Here, the dashed line 902 shows the absolute input / output characteristics of the setting priority mode related to a sunny daytime scene, and it can be understood that the absolute input / output characteristics of the uniform perception mode are mapped to a higher I value in the high step range. Therefore, as shown in the figure, the luminance range showing a linear relationship (the luminance range that matches the dashed line 903 with a constant slope) can be expanded, and a display image that has high visibility and allows the perception of a uniform contrast feeling over a wider range can be generated.

[0062] [Other embodiments] The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0063] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0064] 100: digital camera, 101: control unit, 104: imaging unit, 105: image processing unit, 106: display unit, 701: RAW data, 705: color luminance adjustment parameters, 706: color luminance adjustment unit, 707: gradation conversion unit, 708: image for display, 801: color luminance adjustment unit, 802: maximum display luminance, 803: adjustment curve, 804: gradation characteristic configuration unit, 805: gradation conversion unit, 806: image for display

Claims

1. An image processing device that generates an image to be displayed on a display means configured to be able to change a maximum display luminance setting, A first acquisition means for acquiring an image signal; a second acquisition means for acquiring a setting of the maximum display luminance of the display means; a generating means for performing image processing including tone conversion on the image signal acquired by the first acquiring means to generate the image to be displayed; a setting means for setting a gradation characteristic used in the gradation conversion performed by the generating means, based on the setting of the maximum display luminance acquired by the second acquiring means; having The gradation characteristic is a signal value of the image signal is assigned to a luminance range according to a setting of the maximum display luminance, For at least a part of the luminance range, the logarithmic value of the signal value of the image signal and the absolute luminance show a linear relationship with approximately the same proportional constant regardless of the setting of the maximum display luminance.

13. An image processing device comprising:

2. The image processing device according to claim 1 , wherein the width of the at least a part of the luminance range varies depending on a setting of the maximum display luminance.

3. 2. The image processing apparatus according to claim 1, wherein the setting means changes the width of the at least a part of the luminance range.

4. 4. The image processing apparatus according to claim 1, wherein the setting means changes the range of the absolute luminance included in the at least a portion of the luminance range.

5. 5. The image processing apparatus according to claim 1, wherein the setting means changes the absolute luminance with respect to a predetermined logarithmic value.

6. 6. The image processing apparatus according to claim 1, wherein the absolute luminance is a luminance component in a perceptually uniform color space.

7. 7. The image processing apparatus according to claim 1, wherein the absolute luminance is an I value in an ICtCp color space.

8. 8. The image processing apparatus according to claim 7, wherein the approximately equal proportionality constant is 65 to 85.

9. An image processing device according to any one of claims 1 to 8, an imaging means for imaging an imaging scene and outputting the image signal; The display means; A change means for changing the setting of the maximum display luminance of the display means; An imaging device comprising:

10. 10. The imaging apparatus according to claim 9, wherein the change means changes the setting of the maximum display luminance in accordance with the brightness of the imaging scene.

11. a developing means for developing the image signal to generate a recorded image; a recording means for recording the image generated by the developing means; and the display modes of the display means include a first mode for displaying an image generated by applying a correction related to the recorded image to the image signal, and a second mode for displaying the image to be displayed that is generated by the generation means, 11. The imaging apparatus according to claim 9, wherein the absolute luminance for a given logarithmic value for the same imaging scene is equivalent between the first mode and the second mode.

12. 12. The imaging apparatus according to claim 11, wherein the change unit sets the maximum display luminance higher in the second mode than in the first mode for the same imaging scene.

13. 13. The image pickup apparatus according to claim 9, wherein the display means is an electronic viewfinder.

14. A method for controlling an image processing device that generates an image to be displayed on a display means configured to be able to change a maximum display luminance setting, comprising: A first acquisition step of acquiring an image signal; a second acquisition step of acquiring the setting of the maximum display luminance of the display means; a generating step of performing image processing including tone conversion on the image signal acquired in the first acquiring step to generate the image to be displayed; a setting step of setting a gradation characteristic used in the gradation conversion performed in the generating step, based on the setting of the maximum display luminance acquired in the second acquiring step; having The gradation characteristic is a signal value of the image signal is assigned to a luminance range according to a setting of the maximum display luminance, For at least a part of the luminance range, the logarithmic value of the signal value of the image signal and the absolute luminance show a linear relationship with approximately the same proportional constant regardless of the setting of the maximum display luminance. A control method comprising:

15. A program for causing a computer to function as each of the means of the image processing apparatus according to any one of claims 1 to 8.

Images