Image processing device, imaging device, control method, and storage medium

Abstract

The present disclosure provides an image processing device, an imaging device, a control method, and a storage medium. An image processing device for generating an image to be displayed on a display component, the display component being configured to be able to change a setting for maximum display brightness, the image processing device comprising: a first acquisition component for acquiring an image signal; a second acquisition component for obtaining a setting for maximum display brightness of the display component; a generation component for generating the image to be displayed by performing image processing including tone conversion on the image signal obtained by the first acquisition component; and a setting component for setting a tone characteristic used in the tone conversion performed by the generation component based on the setting for maximum display brightness obtained by the second acquisition component.

Description

Technical Field

[0001] The present invention relates to an image processing device, an imaging device, a control method and a computer-readable storage medium, and more particularly to a tone conversion technology related to generating an image for display. Background Art

[0002] Display devices already have inherent gamma characteristics (display characteristics) that determine the display brightness at which the hue of an input image is output. Therefore, in order to display an image with a desired hue on the display device, it is necessary to adjust the input image taking into account the display characteristics. Japanese Patent Application Laid-Open No. 2010-245924 describes the use of a hue characteristic, including an adjusted inverse characteristic of the display characteristics of an electronic viewfinder of an imaging device, to generate an input image with improved visibility.

[0003] As described in Japanese Patent Application Laid-Open No. 2010-245924, in order to ensure the visibility of the image displayed in the electronic viewfinder, an input image is generated from the captured image using a tone characteristic configured by multiplying the inverse characteristic of the display characteristic by a power in accordance with the brightness of the shooting scene (viewing environment). Specifically, when the shooting scene is bright, the multiplier is increased and a through-the-lens image with increased contrast is displayed, while when the shooting scene is dark, the multiplier is decreased and a through-the-lens image with reduced contrast is displayed.

[0004] However, as described in Japanese Patent Application Laid-Open No. 2010-245924, the tonal characteristics change according to the overall power of the brightness of the shooting scene, causing the user looking through the electronic viewfinder to perceive different contrasts at different brightnesses, giving the user a feeling that something is off. This perceived discomfort may also be caused by the maximum display brightness of the display device being dynamically changed according to the brightness of the surrounding environment, or by the maximum display brightness being arbitrarily changed. Summary of the Invention

[0005] The present invention has been made in view of the above-mentioned problems, and provides an image processing apparatus, an imaging apparatus, a control method, and a computer-readable storage medium for generating a display image that gives a viewer uniform visibility (eg, contrast perception).

[0006] In its first aspect, the present invention provides an image processing device for generating an image to be displayed on a display component, the display component being configured to be able to change the setting of the maximum display brightness, the image processing device comprising: a first acquisition component for obtaining an image signal; a second acquisition component for obtaining the setting of the maximum display brightness of the display component; a generation component for generating an image to be displayed by performing image processing including tone conversion on the image signal obtained by the first acquisition component; and a setting component for setting a tone characteristic used in the tone conversion performed by the generation component based on the setting of the maximum display brightness obtained by the second acquisition component, wherein the tone characteristic is used to assign the signal value of the image signal to a brightness range according to the setting of the maximum display brightness; and the tone characteristic shows the following relationship, in which, regardless of the setting of the maximum display brightness, the logarithmic value and the absolute brightness of the signal value of the image signal are approximately the same in at least a portion of the brightness range.

[0007] In its second aspect, the present invention provides a camera device, comprising: an image processing device according to the first aspect; a camera component for capturing an image of a shooting scene and outputting the image signal; a display component; and a changing component for changing the setting of the maximum display brightness of the display component.

[0008] In its third aspect, the present invention provides a control method for an image processing device, wherein the image processing device is used to generate an image to be displayed on a display component, and the display component is configured to be able to change the setting of the maximum display brightness, the control method comprising: obtaining an image signal; obtaining the setting of the maximum display brightness of the display component; generating the image to be displayed by performing image processing including tone conversion on the obtained image signal; and setting a tone characteristic used in the tone conversion performed in the generation based on the setting of the obtained maximum display brightness, wherein the tone characteristic is used to assign the signal value of the image signal to a brightness range according to the setting of the maximum display brightness; and the tone characteristic shows the following relationship, in which, regardless of the setting of the maximum display brightness, the logarithmic value and the absolute brightness of the signal value of the image signal are approximately the same in at least a portion of the brightness range.

[0009] The present invention, in its fourth aspect, provides a computer-readable storage medium storing a program for causing a computer to function as a component of the image processing apparatus according to the first aspect.

[0010] Further features of the present invention will become apparent from the following description of exemplary embodiments (with reference to the attached drawings). BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 11 is a diagram showing a functional configuration of a digital camera 100 according to an embodiment and modifications of the present invention.

[0012] Figure 2 : is a diagram showing input / output characteristics of a setting priority mode according to the embodiment and modification examples of the present invention.

[0013] Figure 3A and 3B : is a diagram showing the relationship between input and display brightness in the setting priority mode according to the embodiment and the modification of the present invention.

[0014] Figure 4A and 4B : is a graph showing absolute input / output characteristics of a setting priority mode according to the embodiment and modification examples of the present invention.

[0015] Figure 5 is a graph showing absolute input / output characteristics of a uniform perception pattern according to the first embodiment of the present invention.

[0016] Figure 6 is another graph showing the absolute input / output characteristics of the uniform perception pattern according to the first embodiment of the present invention.

[0017] Figure 7 : is a diagram showing a functional configuration of the image processing unit 105 setting a priority mode according to the embodiment and the modification examples of the present invention.

[0018] Figure 8 : is a diagram showing a functional configuration of the image processing unit 105 in the uniform perception mode according to the embodiment and the modification examples of the present invention.

[0019] Figure 9 is a graph showing absolute input / output characteristics of a uniform perception pattern according to the second embodiment of the present invention. DETAILED DESCRIPTION

[0020] First embodiment

[0021] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments are not intended to limit the scope of the claimed invention. A number of features are described in the embodiments, but the invention is not limited to requiring all of these features, and a plurality of these features may be appropriately combined. Furthermore, in the accompanying drawings, the same reference numerals are assigned to the same or similar configurations, and redundant descriptions thereof are omitted.

[0022] The embodiments described below are examples of the present invention applied to a digital camera, which is an example of an image processing device that is provided with a display device capable of operating as an electronic viewfinder. However, the present invention can be applied to any device capable of performing tone conversion and generating an image for display. Examples of such devices include digital cameras or digital video cameras, personal computers, tablet terminals, mobile phones, game consoles, and transparent glasses for presenting augmented reality (AR) or mixed reality (MR).

[0023] Digital camera configuration

[0024] Figure 1 is a block diagram showing the functional configuration of the digital camera 100 according to an embodiment of the present invention.

[0025] The control unit 101 includes at least one processor or circuit. The control unit 101 reads an operation program of each block of the digital camera 100 from the storage medium 102, loads the operation program into the memory 103, and executes the operation program to control the operation of the digital camera 100.

[0026] The storage medium 102 is an electrically erasable and writable nonvolatile storage device, such as a flash ROM. The storage medium 102 stores the operating programs of the various blocks of the digital camera 100 and constants required for the operation of each block. Furthermore, the storage medium 102 may also be configured to store images (raw data, developed images, etc.) obtained by imaging (e.g., semiconductor memory card). The memory 103 is a volatile storage device such as RAM or DRAM, for example, and serves as a loading area for the operating programs of the various blocks. Furthermore, when displaying images on the display device 106 described below, the memory 103 functions as a VRAM.

[0027] The lens 110 is a unit on which an image pickup lens is mounted, and is configured to be detachable from the digital camera 100. The lens 110 generally includes a plurality of lenses, but this is not the case in Figure 1 The lens 110 includes a control circuit (not shown) and controls the state of the lens 110 based on a drive signal from the control unit 101.

[0028] The imaging unit 104 is an image sensor, such as a CCD or CMOS sensor, and obtains analog image signals by, for example, converting the optical image formed on the imaging surface by the lens 110 into electrical signals. The obtained analog image signals are converted into digital image signals (hereinafter referred to as raw data) by an A / D converter (not shown). In the embodiment described herein, the imaging unit 104 is a single-plate color image sensor equipped with typical primary color filters. In this example, the primary color filters include three types of filters with primary transmission wavelengths near 650 nm, 550 nm, and 450 nm, arranged in a mosaic pattern (Bayer array). By using the primary color filters, each pixel of the single-plate color image sensor captures a color plane corresponding to one of the R (red), G (green), and B (blue) bands. In other words, the photoelectric conversion elements that constitute the single-plate color image sensor can only obtain the light intensity of each monochromatic plane. Furthermore, the imaging unit 104 may include peripheral circuits, such as amplifier circuits, to process the signals obtained from the pixels.

[0029] The image processing unit 105 performs various types of image processing, such as pixel interpolation, resizing, and color conversion, on the raw data or the raw data read from the storage medium 102. Details will be described below. Furthermore, the image processing unit 105 derives information required for exposure control and distance measurement control by performing computational processing on the raw data obtained by image capture. In the digital camera 100 of this embodiment, through-the-lens (TTL) autofocus (AF) processing, automatic exposure (AE) processing, and pre-flash emission (EF) processing are performed. Furthermore, the image processing unit 105 performs TTL-based automatic white balancing (AWB) by performing computational processing on image data obtained by image capture.

[0030] The display device 106 is a display device such as a liquid crystal display, and displays information such as setting values of the digital camera 100, a GUI such as a message or menu screen, captured images, and the like. The display device 106 may be an electronic viewfinder (EVF) or a rear liquid crystal display built into the digital camera 100, or an external display detachably connected to the digital camera 100. The following describes control of the display of the electronic viewfinder while the digital camera 100 is capturing an image, using the electronic viewfinder as an example of the display device 106. However, the present invention is not limited thereto.

[0031] Furthermore, the display device 106 includes a display control circuit (not shown) and is configured to be able to change the maximum display brightness. For example, the maximum display brightness of the display device 106 can be dynamically controlled based on a BV value obtained by measuring the light of the captured scene, or the scene brightness can be estimated and set based on the selected imaging mode. Here, for example, the light of the captured scene can be measured based on an image signal obtained by the imaging unit 104 or based on the output of a separately provided light metering sensor. In this embodiment, information indicating the maximum display brightness of the display device 106 can be obtained, and this information is supplied to at least the image processing unit 105.

[0032] The operation unit 107 is a user interface provided in the digital camera 100 of this embodiment, and receives inputs for various operations. When the operation unit 107 detects that an operation has been input to the user interface, the operation unit 107 outputs a corresponding control signal to the control unit 101. The operation unit 107 includes a release switch for instructing the start of an image capture preparation operation and the start of image capture (actual image capture), an image capture mode selection switch for selecting an image capture mode, a direction key, an enter key, and the like.

[0033] In the present embodiment described here, the processing related to the present invention is implemented as hardware by circuits or processors corresponding to the blocks of the digital camera 100. However, the present invention is not limited thereto, and the processing of the blocks may be implemented by a program that executes processing similar to that of the blocks.

[0034] Electronic viewfinder image generation control

[0035] Next, the digital camera 100 of this embodiment generates images to be displayed on the electronic viewfinder (images for display) in detail. The digital camera 100 of this embodiment has two types of electronic viewfinder display modes for capturing images: a setting priority mode, which is a first mode according to the present invention, and a uniform perception mode, which is a second mode according to the present invention. These display modes change the generation method (development method) used to present images when captured images obtained by the imaging unit 104 are sequentially displayed on the display device 106 serving as the electronic viewfinder, and can independently switch the imaging mode.

[0036] The digital camera 100 of this embodiment is configured so that, in at least one imaging mode for storing developed images, contrast correction, exposure correction, colorimetry correction, and the like performed during the development process can be preset. The setting priority mode provides an electronic viewfinder display that allows the details of image correction applied by the development process to be checked before actual imaging, and is suitable for capturing images while viewing the image to be stored. The uniform perception mode provides a display with a contrast perception similar to that of a user looking directly at a subject or viewing a subject through an optical viewfinder (OVF) via a lens. Details are described below, but the uniform perception mode of this embodiment allows a display with perceptually uniform tonal expression to be achieved regardless of the maximum display brightness of the display device.

[0037] The image processing unit 105 generates an image for display by at least including tone conversion, in which tone values are assigned to signal intensities indicated in the raw data. Tone conversion is performed based on an input / output characteristic that indicates the relationship between the signal values of the raw data and the tone values of the post-development processing. For example, in tone conversion, the signal values of raw data with 14-bit resolution are converted to 8-bit tone values after development processing. The setting priority mode and the uniform perception mode have different input / output characteristics that are referenced in tone conversion, resulting in differences in tone expression.

[0038] Note that in the embodiment described here, two types of electronic viewfinder display modes, namely, setting priority mode and uniform perception mode, can be switched in any imaging mode. However, the present invention is not limited to this. For example, a configuration may be used in which only uniform perception mode can be used in one or more imaging modes.

[0039] Furthermore, considering the visibility of the electronic viewfinder, the maximum display brightness of the display device 106 is controlled to be different when the scene being photographed is a bright daytime scene and when the scene being photographed is a night scene. Specifically, 450 nits is used when the scene being photographed is a bright scene (such as a bright daytime scene), and 50 nits is used when the scene being photographed is a dark scene (such as a night scene). However, the maximum display brightness value for each scene is not limited to this, and the maximum display brightness can be set to another value.

[0040] Setting Priority Mode

[0041] First, generation of an image for display in the setting priority mode will be described.

[0042] With the image processing unit 105 of this embodiment, in tone conversion according to the setting priority mode, for example, Figure 2As shown, a common input / output characteristic is used regardless of the brightness of the shooting scene. Figure 2 In the illustrated example, as described above, the tone values of an 8-bit image (developed image) for display are assigned to the signal values of 14-bit raw data.

[0043] The display device 106 has inherent display characteristics (for example, γ2.2 in this embodiment) and is configured to perform control so as to switch the maximum display brightness according to the brightness of the shooting scene. Therefore, when an image for display is displayed on the display device 106, the relationship between the signal value of the raw data and the display brightness of the display device 106 is as follows: Figure 3A and 3B shown. Figure 3A The relationship is shown in the case where the shooting scene is a sunny day, and Figure 3B The relationship is shown in FIG. 1 when the shooting scene is a night scene. Figure 3A and 3B As shown in the graph, the vertical axis represents the display luminance (nits) of display device 106. The luminance range of the signal value assigned to the raw data differs depending on whether it is a daytime, sunny day or a night scene. In other words, the maximum display luminance of display device 106 preferably varies depending on the brightness of the shooting scene, and this results in a different range of display luminance representing tonal expression.

[0044] Here, the difference in human visual perception exhibited in a daytime sunny scene and a night scene having different maximum display brightness in a setting priority mode will be described with reference to the accompanying drawings.

[0045] According to the Weber-Fechner law, the intensity of sensation perceived by humans is proportional to the logarithm of the intensity of stimulation given to the sensory receptors. In this specification, based on this relationship between the intensity of sensation and the intensity of stimulation, the input / output characteristics are compared by evaluating the relationship between the input and output when converting from the original data signal value to the final display brightness in terms of human perception. In other words, using the signal value corresponding to the amount of light indicated in the original data corresponding to the stimulation intensity, and the final display brightness obtained by taking into account the maximum display brightness of the display device 106 as the intensity of sensation, the relationship is evaluated, and the difference in perceived contrast via the electronic viewfinder is described. Note that the input / output characteristics described here are different from those described above. Figure 2 The tonal characteristics shown relate to the conversion from raw data signal values to tonal values of a developed image, and are also to the conversion from raw data signal values to absolute luminance when displayed on an arbitrary display device. Hereinafter, to distinguish between the two, the tonal characteristics relating to the conversion from raw data signal values to absolute luminance are referred to as the absolute input / output characteristics.

[0046] Hereinafter, when discussing the absolute input / output characteristic relationship between the original data signal value and the absolute luminance, the logarithmic value of the signal value will be used for the former, and the luminance component value of the perceptually uniform color space based on the human visual characteristics will be used. In this embodiment, as the luminance component value of the perceptually uniform color space, the I value of the ICtCp color space specified in ITU-R BT.2100 will be used. The I value can be derived from the RGB value using the inverse characteristics of the EOTF (inverse EOTF) of the perceptual quantization (PQ) method standardized in SMPTE ST 2084. The PQ method determines absolute luminance that is independent of the display characteristics inherent to the display device and is suitable for defining perceived intensity (due to the efficient allocation of bits based on human visual characteristics).

[0047] Using this, the absolute input / output characteristics of the two types of shooting scenes in setting priority mode correspond to Figure 4A In the figure, the horizontal axis represents the logarithmic value of the raw data signal value (hereinafter, simply referred to as the logarithmic value of the raw data), in which the appropriate exposure value is normalized (indicated by the level at which the appropriate exposure logarithmic value is set to level 0). The vertical axis represents the I value. Figure 4A In the example shown in FIG. 4 , a curve (solid line) 401 indicates the absolute input / output characteristics of a daytime sunny scene, and a curve (dashed line) 402 indicates the absolute input / output characteristics of a night scene. As can be seen from the figure, due to the difference in maximum display brightness between a daytime sunny scene and a night scene, the assigned value range (brightness range) of the I value is different relative to the raw data logarithm value.

[0048] Here, if the input / output, i.e., the raw data logarithm value, has a linear relationship with the I value, then according to the Weber-Fechner law, this can be said to represent a perceptually uniform contrast that matches the human perception characteristics. Figure 4A In the example of , both the absolute input / output characteristics from a clear daytime scene and the absolute input / output characteristics at the time of a night scene are expressed by a linear relationship in a value range (level range) in and around the appropriate exposure. Figure 4A As shown, the graph with the original data logarithm on the horizontal axis and the I value on the vertical axis is called a perceptual linear graph.

[0049] However, according to the Weber-Fechner law, the increment (multiplier) of stimulus intensity is proportional to the increment of sensation intensity. Therefore, if the increment of the raw data logarithm is uniform, the increment of the I value does not change regardless of the reference logarithm. In other words, if it is perceptually uniform, the difference in brightness (I value increment) when the raw data logarithm is increased by any multiple from the reference value should be perceived as being roughly the same, regardless of the reference value. In other words, if there is the same (visual) perception via the sensory receptor, then regardless of the maximum display brightness of the display device 106, the perceptual linear graph needs to show the same tilt, that is, the same proportional constant (stimulus constant).

[0050] However, in Figure 4A The two absolute input / output characteristics indicated in the perceptual linear diagram are given by Figure 4B The slopes of the cross sections of the linear relationship indicated by straight line (dash-dotted line) 403 and straight line (dash-double-dotted line) 404 in FIG. This means that in setting priority mode, for two shooting scenes with different maximum display brightnesses of display device 106, absolute input / output characteristics are used to perceive different contrasts. In other words, the brightness difference perceived in the electronic viewfinder differs depending on whether it is daytime or nighttime, indicating non-uniformity.

[0051] The difference in the slope of the perceptual linear graph indicates that, when the photographed scene is a night scene, there is an appropriate contrast perception compared to the appearance with the naked eye, whereas, in the case of a daytime sunny scene, the contrast is higher than it actually is and the dark parts appear to be a loss of shadow detail. In other words, in how the image is displayed on the display device 106 using the same Figure 2 When display images are obtained using the same input / output characteristics as shown, the degree of difference in contrast perception between the case where the subject is viewed with the naked eye and the case where the subject is viewed through the electronic viewfinder varies depending on the maximum display brightness of the display device 106. Specifically, when the input / output characteristics are fixed and the maximum display brightness of the display device 106 is changed, as the maximum display brightness increases, the contrast also increases (the slope of the perceptual linear graph becomes steeper), resulting in a difference in contrast perception between shooting scenes.

[0052] Uniform sensing mode

[0053] In the uniform perception mode, in the level range or luminance range in the linear relationship of the perceptual linearity diagram, the absolute input / output characteristic is set to show a constant stimulus constant regardless of the maximum display luminance of the display device 106. Specifically, in a daytime sunny shooting scene with the maximum display luminance set to 450 nits, the absolute input / output characteristic is set to show a constant stimulus constant regardless of the maximum display luminance of the display device 106. Figure 5The absolute input / output characteristic indicated by curve 501 in the perceptual linear graph of FIG. In addition, in a night scene shooting scene where the maximum display brightness is set to 50 nits, the image is displayed using the image obtained by Figure 6 The curve 601 in the perceptual linearity graph indicates the absolute input / output characteristics.

[0054] As shown in the figure, the two absolute input / output characteristics match the dashed line 502 or 602 indicating a common slope (stimulus constant) in at least a portion of the level range or brightness range, and show a gentle, stable slope in the dark and saturated parts. In this embodiment, the slope of the dashed lines 502 and 602 is set to 75. In addition, in order to reduce the difference in brightness atmosphere between the setting priority modes for the same shooting scene, the I value for the two absolute input / output characteristics under proper exposure is set to the same value as that in the setting priority mode.

[0055] Here, the value of 75, which is the slope within the range represented by the linear relationship (matching the dotted line), is a temporary optimal value derived from experiments, resulting in a reduction in the difference in brightness between when viewing the subject with the naked eye and when viewing the subject through the electronic viewfinder, and a uniform perception of contrast. Furthermore, in this embodiment, the I-intercepts of curve 501, dotted line 502, curve 601, and dotted line 602 are each set in setting priority mode to the same value as the I-value for proper exposure with maximum display brightness. Therefore, the display brightness for proper exposure can be set to an appropriate value, as in setting priority mode, and for any shooting scene, the difference from the original data signal value for proper exposure can be represented by a uniform display brightness difference.

[0056] Therefore, in the uniform perception mode, an electronic viewfinder with perceptually uniform contrast can be provided by using the tonal characteristics of the absolute input / output characteristics indicating a uniform slope in the perceptual linearity graph, regardless of the maximum display brightness of the display device 106. In other words, regardless of the shooting scene or the maximum display brightness of the display device 106, at least the brightness difference within the range represented by the linear relationship is constant, and thus a uniform contrast can be perceived.

[0057] Functional configuration of the image processing unit 105

[0058] The functional configuration of the image processing unit 105 related to generating an image for display in each display mode will be described below with reference to the accompanying drawings. In this embodiment, in order to determine the difference between the setting priority mode and the uniform perception mode in the development processing performed to generate the image for display, the functional configuration will be described with reference to the diagram divided by mode. However, these do not need to be implemented as different hardware. In other words, images for display in two display modes can be generated by controlling whether at least a part of the functional configuration is used or changing the operation according to switching the display mode. The functional configuration is described using different blocks corresponding to the processing items performed by the image processing unit 105 in the development processing.

[0059] Setting Priority Mode

[0060] pass Figure 7 The functional configuration shown implements development processing related to images for display in settings priority mode. As described above, imaging unit 104 uses a primary color filter in which three types of filters are arranged in a mosaic pattern. Therefore, raw data 701 is a color mosaic image. Image processing unit 105 reads raw data 701 from memory 103, applies development processing related to settings priority mode, and generates image 708 for display.

[0061] The white balance unit 702 performs white balance processing on the raw data 701 to convert the image color of the subject, which was originally white, to white. Specifically, for example, the white balance unit 702 plots the RGB data of each pixel constituting the raw data 701 in a predetermined color space, such as an xy color space. The white balance unit 702 then integrates the R, G, and B values of the data plotted at or near the locus of blackbody radiation, which has a high probability of being the color of the light source in the color space, and derives white balance coefficients (G / R and G / B) for the R and B components from the integrated values. The white balance unit 702 performs white balance processing using the obtained white balance coefficients, and can correct for coloration caused by the light source and reproduce white.

[0062] The color interpolation unit 703 performs noise reduction processing and pixel values of color components not included in the pixels on the image data obtained through conversion by the white balance unit 702. Through this processing, a synchronized image having a complete set of color information (pixel values of color components) of R, G, and B for all pixels is generated.

[0063] The synchronous image generated by the color interpolation unit 703 is converted into a color image as a basis for processing via matrix conversion processing performed by the matrix conversion unit 704. Furthermore, adjustment processing for adjusting color and brightness is performed on the color image by the color and brightness adjustment unit 706, and the color image is converted into a color image via a color and brightness adjustment processing performed by the tone conversion unit 707. Figure 2 An image 708 for display is generated by performing tone conversion (gamma conversion) processing related to the input / output characteristics of the image.

[0064] Here, the adjustment of the color and brightness adjustment unit 706 includes the adjustment made with reference to the color and brightness adjustment parameters 705, which describe the settings of contrast correction, exposure correction, color saturation correction, etc. to be applied to the stored image. In addition, the tone conversion unit 707 uniformly uses the Figure 2 The input / output characteristics shown perform tone conversion processing.

[0065] By displaying the image 708 for display generated in this manner on the display device 106 with the maximum display brightness controlled according to the shooting scene, an electronic viewfinder related to the setting priority mode is realized. In other words, the relationship of the absolute input / output characteristics shown in the perceptual linearity diagram of FIG4 is satisfied with respect to the raw data 701 and the display image 708 for display.

[0066] Uniform sensing mode

[0067] pass Figure 8 The functional configuration shown implements the development processing related to the image for display in the uniform perception mode. Functional configurations for performing operations similar to those in the setting priority mode will be given the same reference numerals and will be referred to below. Figure 8 Only the functional configurations for performing different operations are described.

[0068] In uniform perception mode, the color and brightness adjustment unit 801 performs adjustments other than those based on the settings for storage on the color image obtained by the matrix conversion unit 704. Furthermore, the tone conversion unit 805 applies tone conversion processing to the image adjusted by the color and brightness adjustment unit 801 to generate an image for display 806.

[0069] In the tone conversion process performed by the tone conversion unit 805, the tone characteristic configured by the tone characteristic configuration unit 804 is used. When configuring the tone characteristic, the tone characteristic configuration unit 804 obtains the maximum display brightness 802 of the display device 106 and the adjustment curve 803 of the absolute input / output characteristic corresponding to the maximum display brightness 802 ( Figure 5 and Figure 6). Here, the adjustment curve 803 indicates the relationship between the logarithmic value of the raw data 701 and the I value (i.e., absolute brightness). Therefore, the tone characteristic configuration unit 804 of this embodiment performs conversion so that the display brightness corresponds to the absolute brightness of the display device 106. In other words, by multiplying the absolute input / output characteristic indicated by the adjustment curve 803 by the inverse characteristic of the display characteristic, the tone characteristic used in the tone conversion of the tone conversion unit 805 is configured. The tone conversion unit 805 uses the tone characteristic configured in this manner to perform tone conversion processing on the image that has been subjected to color and brightness adjustment.

[0070] By displaying the image 806 for display generated in this manner on the display device 106 with the maximum display brightness controlled according to the shooting scene, an electronic viewfinder related to the uniform perception mode is realized. In other words, with respect to the raw data 701 and the display image 806 for display, Figure 5 or Figure 6 The absolute input / output characteristic relationship shown in the perceptual linearity graph can achieve a perceptually uniform contrast ratio that is independent of the maximum display brightness.

[0071] First Modification

[0072] Note that in the uniform perception mode of the above-described embodiment, to ensure the absolute luminance indicated by adjustment curve 803, the inverse characteristic of the display characteristics of display device 106 is used in the multiplication. However, the present invention is not limited to this. As described above, because the I value is based on the PQ method, which determines absolute luminance independent of the inherent display characteristics of the display device, as long as a display device capable of displaying images developed using the PQ method is used, it is not necessary to use the inverse characteristic of the display characteristics in the multiplication when configuring the tone characteristics from adjustment curve 803. The present invention can be implemented as long as a display device capable of changing the maximum display luminance can generate an image with uniform perceptible contrast regardless of changes in the maximum display luminance. Here, information on the set maximum display luminance can be obtained from the display device via a standard-defined signal for connecting to the display device, or it can be obtained from another device that controls the operation of the display device. In other words, the result of measuring the light of the captured scene does not need to be used in the determination as in the first embodiment.

[0073] Second Modification

[0074] In the above embodiments Figure 5 and Figure 6In the case of the indicated absolute input / output characteristics of the uniform perception mode, the inclination of the dashed lines 502 and 602 is set to 75. However, the present invention is not limited to this. For example, even with an inclination of 65 to 85, there is contrast perception when viewing the subject with the naked eye. Therefore, with respect to the two absolute input / output characteristics, the stimulus constant related to the level range or brightness range in which a linear relationship exists does not need to be strictly the same, and the value only needs to be set to a substantially identical value included in the above range (from 65 to 85).

[0075] Third Modification

[0076] exist Figure 5 and Figure 6 In the case of the absolute input / output characteristics of the indicated uniform perception mode, since the maximum display brightness of the display device 106 differs depending on whether it is a daytime sunny scene or a night scene, the level range or brightness range in which the linear relationship exists is different. In the example shown in the figure, a linear relationship exists within the level range or brightness range that includes appropriate exposure (from approximately -5 levels to approximately +2 levels for daytime sunny scenes, and from approximately -3 levels to approximately +1 levels for night scenes). This difference is due to the expansion of the brightness range that can be assigned to the raw data signal value according to the maximum display brightness of the display device 106.

[0077] However, the present invention is not limited to this, and it is possible to change the level range or brightness range in which a linear relationship exists to match the brightness range in which the subject is easy to see, such as high brightness in the case of night scenes and low brightness in the case of sunny scenes during the day. In addition, from the perspective of giving the user an impression of uniform perception, the greater the breadth (width) of the level range or brightness range in which a linear relationship exists, the better. However, the present invention is not necessarily limited to Figure 5 and Figure 6 The illustrated embodiment may be capable of altering the presentation of a perceptually uniform contrast ratio independent of the maximum display brightness of the display device 106 to occur within a specific level range or brightness range.

[0078] Fourth Modification

[0079] In addition, Figure 5 and Figure 6In the example of , the I intercept of each absolute input / output characteristic is set to be equal to the I value for proper exposure in the setting priority mode. However, the present invention is not limited to this. For example, when the shooting scene is a night scene, the I intercept can be set to a higher value because contrast perception close to that of the naked eye can be achieved and the visibility of the dark part can be improved in a compatible manner. In addition, for example, when the shooting scene is a night scene, in order to make the brightness closer to the brightness of the subject when viewed with the naked eye, the I intercept can be set to a lower value. In addition, as long as it can be ensured that the inclination with respect to the absolute input / output characteristic is approximately the same regardless of the maximum display brightness, the I intercept can be changed to an arbitrary value in more than just the case where the shooting scene is a night scene. The I intercept can be changed according to user settings, scene recognition results, etc.

[0080] Fifth Modification

[0081] In the above embodiment, there are two levels in the setting of the maximum display brightness of the display device 106, and there are two types of absolute input / output characteristics for the uniform perception mode. However, it should be easily understood that the present invention is not limited to this. In other words, it is only necessary to include the following level range or brightness range in the tone characteristics used to generate the image for display: it indicates that the logarithmic value of the image signal (i.e., input) and the absolute brightness when the developed image of the image signal is displayed have approximately the same proportional constant linear relationship regardless of the maximum display brightness. Therefore, the number of such tone characteristics provided is the same as the number of maximum display brightnesses that the display device 106 can set, and the proportional constant only needs to be approximately the same in the portion where the linear relationship exists within this range.

[0082] Second embodiment

[0083] With a configuration such as being able to check the correction settings applied to a stored image in the electronic viewfinder in the setting priority mode, it is possible to output an image for display that causes burn-in on the display device 106, depending on the contents of the correction settings and the maximum display brightness of the display device 106. For example, for shooting scenes with a wide dynamic range (such as daytime sunny scenes), in order to improve visibility, it is desirable to increase the maximum display brightness of the display device 106 to extend the dynamic range of the electronic viewfinder. However, depending on the correction settings applied to the stored image, the entire image for display may be saturated, wherein display over an extended period of time may cause damage to the display device 106. In addition, displaying such an image with an increased maximum display brightness results in increased power consumption. Therefore, it is necessary to limit the maximum display brightness that can be set for the display device 106 in the setting priority mode.

[0084] However, in uniform perception mode, since no adjustment is made based on the correction settings applied to the stored image, and the absolute brightness of the display device 106 is set using the absolute input / output characteristics, the possibility of damage to the display device 106 is lower than in the setting priority mode. Therefore, the maximum display brightness of the display device 106, which is set based on the light measurement results, can be set to a higher value in uniform perception mode than in the setting priority mode.

[0085] For example, in the case where the shooting scene is a sunny day, the maximum display brightness in the setting priority mode can be set to 450 nits, and the maximum display brightness in the uniform perception mode can be set to a higher 800 nits, even if the tone conversion is performed to maintain the same brightness as above. Figure 4A and 4B Here, the dynamic range of the electronic viewfinder is extended corresponding to the increase in the maximum display brightness, and the absolute input / output characteristics related to the uniform perception mode are as follows. Figure 9 The maximum value of I can be increased, as indicated by the solid line 901 in FIG. The dashed line 902 indicates the absolute input / output characteristics of the setting priority mode associated with daytime sunny weather, and it should be understood that the absolute input / output characteristics of the uniform perception mode are mapped to a higher I value in a higher-order range. Therefore, as shown in the figure, the luminance range indicating a linear relationship (the luminance range matching the dot-dash line 903 with a constant slope) can be expanded, visibility can be improved, and an image for display with a perceptually uniform contrast over a wide range can be generated.

[0086] Other embodiments

[0087] The embodiments of the present invention can also be implemented by the following method, that is, providing software (program) that performs the functions of the above-mentioned embodiments to a system or device through a network or various storage media, and the computer or central processing unit (CPU) or microprocessing unit (MPU) of the system or device reads and executes the program.

[0088] While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.

Claims

1. An image processing apparatus for generating an image to be displayed on a display component, the display component being configured to be able to change a setting of a maximum display brightness, the image processing apparatus comprising: A first obtaining component, configured to obtain an image signal; a second obtaining component, configured to obtain a setting of the maximum display brightness of the display component; generating means for generating an image to be displayed by performing image processing including tone conversion on the image signal obtained by the first obtaining means; as well as Setting means for setting a tone characteristic used in tone conversion performed by said generating means based on the setting of said maximum display brightness obtained by said second obtaining means, wherein the tone characteristic is for distributing a signal value of the image signal to a brightness range according to a setting of the maximum display brightness; and the tone characteristic shows a relationship in which, regardless of the setting of the maximum display brightness, the logarithmic value of the signal value of the image signal and the absolute brightness are in a linear relationship with a proportionality constant of 65 to 85 in at least a portion of the brightness range, The width of the at least a portion of the brightness range varies according to the setting of the maximum display brightness.

2. The image processing apparatus according to claim 1, wherein The setting section changes the width of the at least a portion of the brightness range.

3. The image processing apparatus according to claim 1, wherein The setting section changes the range of absolute luminance included in the at least a portion of the luminance range.

4. The image processing apparatus according to claim 1, wherein The setting section changes the absolute brightness with respect to a predetermined logarithmic value. The image processing apparatus according to claim 1 , wherein: The absolute luminance is the brightness component of a perceptually uniform color space. The image processing apparatus according to claim 1 , wherein: The absolute brightness is the I value in the ICtCp color space.

7. A camera device comprising: The image processing device according to any one of claims 1 to 6; A camera component, configured to capture an image of a scene and output the image signal; Display components; as well as A changing component is used to change the setting of the maximum display brightness of the display component.

8. The imaging device according to claim 7, wherein The changing section changes the setting of the maximum display brightness according to the brightness of the photographic scene.

9. The imaging device according to claim 7, further comprising: a developing component for developing the image signal and generating a stored image; as well as A storage component is used to store the stored image generated by the imaging component, wherein: The display mode of the display means includes a first mode for displaying an image generated by applying a correction related to a stored image to the image signal and a second mode for displaying the image to be displayed generated by the generating means; as well as For the same photographic scene, the absolute brightness relative to the predetermined logarithmic value is equal in the first mode and the second mode.

10. The imaging device according to claim 9, wherein For the same photographic scene, the changing section sets a higher maximum display brightness in the second mode than in the first mode.

11. The imaging apparatus according to claim 7, wherein The display component is an electronic viewfinder.

12. A method for controlling an image processing device for generating an image to be displayed on a display component, the display component being configured to be able to change a setting of a maximum display brightness, the method comprising: obtaining an image signal; obtaining a setting of the maximum display brightness of the display component; generating an image to be displayed by performing image processing including tone conversion on the obtained image signal; as well as The tone characteristic used in the tone conversion performed in the generation is set based on the obtained setting of the maximum display brightness, wherein, the tone characteristic is for distributing a signal value of the image signal to a brightness range according to a setting of the maximum display brightness; and the tone characteristic shows a relationship in which, regardless of the setting of the maximum display brightness, the logarithmic value of the signal value of the image signal and the absolute brightness are in a linear relationship with a proportionality constant of 65 to 85 in at least a portion of the brightness range, The width of the at least a portion of the brightness range varies according to the setting of the maximum display brightness.

13. A computer-readable storage medium storing a program, wherein when the program is executed by a computer, the program implements the steps of the control method according to claim 12.

Images