Imaging device and method for controlling the imaging device

Abstract

To express brightness by taking advantage of the display brightness of a display device by controlling exposure by considering the brightness performance of the display device.SOLUTION: An imaging device includes acquisition means for acquiring representative brightness of a high brightness area in an image, setting means for setting target brightness of the high brightness area on the basis of the display brightness of a display device that displays the image, and exposure control means for controlling exposure on the basis of the representative brightness and the target brightness of the high brightness area.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an imaging device and a method for controlling an imaging device. [Background technology]

[0002] SDR (Standard Dynamic Range) is a display technology that displays images on monitors and displays using a standard dynamic range. Compared to SDR, HDR (High Dynamic Range) is a display technology that can express images with a wider dynamic range. Dynamic Range (DDR) allows for the representation of differences in brightness and color that are difficult to express with SDR.

[0003] When capturing HDR images, highlight-weighted metering is a known metering method for imaging devices that prioritizes measuring the brightness of the high-luminance areas of the screen. In highlight-weighted metering, blown-out highlights are reduced by automatically measuring the brighter parts of the screen. Furthermore, Patent Document 1 discloses a technique for controlling appropriate exposure without being affected by high-luminance light in scenes where high-luminance light exists in addition to the main subject in the screen. [Prior art documents] [Patent Documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2015-166767 [Overview of the project] [Problems that the invention aims to solve]

[0005] The display brightness of devices that display HDR images varies. Therefore, even if the high-brightness areas (hereinafter referred to as highlight areas) or the area containing the subject in the image are captured to the target brightness, it may be difficult for the user to display the image at the brightness intended, depending on the brightness performance of the display device.

[0006] An object of the present invention is to provide an imaging device that enables luminance expression that makes use of the display luminance of a display device by controlling exposure in consideration of the luminance performance of the display device. **Means for Solving the Problems**

[0007] The imaging device according to the present invention includes an acquisition unit that acquires a representative luminance of a high-luminance region in an image, a setting unit that sets a target luminance of the high-luminance region based on the display luminance of a display device that displays the image, an exposure control unit that controls exposure based on the representative luminance and the target luminance of the high-luminance region 、 Correction means for performing gradation correction based on the dynamic range of the scene in which the image was taken and the target brightness. and has death, The aforementioned dynamic range is expressed by a number of steps that increase in stages according to the width of the dynamic range. The correction means performs the tone correction when the dynamic range is equal to or greater than a predetermined number of steps, and does not perform the tone correction when the dynamic range is less than a predetermined number of steps. This is a feature. **Advantages of the Invention**

[0008] According to the present invention, by controlling exposure in consideration of the luminance performance of the display device, luminance expression that makes use of the display luminance of the display device becomes possible. **Brief Description of the Drawings**

[0009] [Figure 1] It is a block diagram illustrating the configuration of a digital camera. [Figure 2] It is a flowchart illustrating imaging processing by highlight priority photometry. [Figure 3] It is a diagram for explaining the representative luminance of a highlight portion. [Figure 4] It is a diagram showing an example of determining a highlight portion by screen division. [Figure 5] It is a diagram for explaining an example of detecting a subject and determining a highlight portion. [Figure 6] It is an example of a screen in which a user designates a target luminance. [Figure 7]This is an example of a lookup table for determining the amount of brightness change. [Figure 8] This is a flowchart illustrating the exposure control process. [Figure 9] This figure illustrates histograms before and after exposure compensation. [Figure 10] This diagram explains the gradation correction in dark areas. [Modes for carrying out the invention]

[0010] The embodiments of the present invention will be described below with reference to the drawings. The embodiments described below are merely examples of methods for realizing the present invention, and may be modified or changed as appropriate depending on the configuration of the apparatus to which the present invention is applied and various conditions. Furthermore, the embodiments can be combined as appropriate.

[0011] <Digital Camera Configuration> Figure 1 is a block diagram illustrating the configuration of a digital camera 100 as an imaging device according to an embodiment of the present invention. The digital camera 100 comprises an imaging mechanism including a shooting lens 101, an aperture and shutter 102, an automatic exposure (AE) processing unit 103, a focus lens 104, an autofocus (AF) processing unit 105, an image sensor 106, and an A / D conversion unit 107.

[0012] The photographic lens 101 has a zoom mechanism. The aperture and shutter 102 control the amount of incident light, which is reflected light from the subject, and the charge accumulation time to the image sensor 106 according to instructions from the AE processing unit 103. The AE processing unit 103 measures the metering area according to the metering mode and controls the exposure by controlling the operation of the aperture and shutter 102. The AE processing unit 103 also controls the A / D conversion unit 107. The focus lens 104 focuses on the light-receiving surface of the image sensor 106 and forms an optical image according to a control signal from the AF processing unit 105. The AF processing unit 105 also calculates distance information from the digital camera 100 to the subject.

[0013] The image sensor 106 converts the optical image formed on the light-receiving surface into an electrical signal using photoelectric conversion means such as a CCD element or CMOS element, and outputs it to the A / D conversion unit 107. The A / D conversion unit 107 converts the received electrical signal (analog signal) into a digital signal (RAW signal). The A / D conversion unit 107 includes a CDS circuit to remove noise from the received electrical signal and a nonlinear amplification circuit to nonlinearly amplify the received electrical signal before converting it into a RAW signal.

[0014] The digital camera 100 also includes an image processing unit 108, an image recognition unit 109, a format conversion unit 110, and a DRAM (Dynamic RAM) 111. The image processing unit 108 performs development processing on the RAW signal input from the A / D conversion unit 107, including predetermined pixel interpolation, resizing processing such as image reduction, and color conversion processing, to output image data.

[0015] The image processing unit 108 has a gamma function that converts the signal level of the RAW signal input from the A / D conversion unit 107 according to the magnitude of the original signal level. The A / D conversion unit 107 can also convert the analog signal converted by the gamma function back into a RAW signal.

[0016] The image processing unit 108 adjusts the image quality of the captured image by performing gradation correction, such as increasing or decreasing the brightness level of the image. Based on the recognition results from the image recognition unit 109, the image processing unit 108 can perform gradation correction according to the scene.

[0017] The image recognition unit 109 receives image data that has been appropriately processed by the image processing unit 108. The image recognition unit 109 has a photometering function and can measure the brightness of the input image. The AE processing unit 103 performs exposure control based on the photometering result from the image recognition unit 109. To reveal.

[0018] The image recognition unit 109 can recognize a scene using known techniques. For example, the image recognition unit 109 can detect the face of a person who is the subject and recognize that it is a scene in which a person is being photographed. The image recognition unit 109 can also acquire information on the dynamic range of the shooting scene. The photometric results and scene recognition information from the image recognition unit 109 are output to the AE processing unit 103.

[0019] The format conversion unit 110 converts the format of the image data generated by the image processing unit 108 in order to store the image data in the DRAM 111. The DRAM 111 is an internal memory and is used as a buffer for temporary storage of image data, or as working memory for image data compression / decompression processing.

[0020] The digital camera 100 comprises an image recording unit 112, a system control unit 113, VRAM 114 (Video RAM), a display unit 115, an operation unit 116, a main switch (main SW) 117, and a shooting switch (shooting SW) 118. The image recording unit 112 has a recording medium such as a memory card and an interface for recording captured images (still images and videos).

[0021] The system control unit 113 includes a CPU (processor), ROM, and RAM. The CPU controls the overall operation of the digital camera 100 by loading programs stored in ROM into the RAM's work area and executing them. The system control unit 113 can perform processing for each component of the digital camera 100 by executing programs stored in ROM. The system control unit 113 controls which of the multiple imaging drive modes of the image sensor 106 is used. VRAM 114 is memory for image display.

[0022] The display unit 115 is, for example, an LCD (Liquid Crystal Display). The display unit 115 displays images, provides information to assist with operation, displays the status of the digital camera 100, and displays the shooting screen and the distance measurement area when shooting.

[0023] The control unit 116 is a component that allows the user to operate the digital camera 100 from the outside. The user can use the control unit 116 to make various settings, such as exposure compensation, aperture value setting, and image playback settings. The control unit 116 includes a menu switch, a zoom lever for instructing the zoom operation of the shooting lens, and an operation mode switch for switching between shooting mode and playback mode.

[0024] The main switch 117 is a switch for turning on the power to the digital camera 100. The shooting switch 118 is a switch that performs two stages of operation depending on how far it is pressed. When the shooting switch 118 is pressed halfway (SW1 operation), the system control unit 113 performs shooting preparation operations such as AE processing and AF processing. When the shooting switch 118 is pressed all the way (SW2 operation), the system control unit 113 performs the shooting process.

[0025] A series of processes performed by the digital camera 100 will now be described. When the main switch 117 is pressed and the power is turned on, the system control unit 113 of the digital camera 100 performs imaging processing at a predetermined period (for example, a 33ms period) using the image sensor 106. The digital camera 100 enters a shooting standby state in which it sequentially displays the captured images on the display unit 115.

[0026] When the system control unit 113 receives a shooting instruction by pressing the shooting switch 118 (SW2 operation), it executes the main shooting process using the image sensor 106. The image processing unit 108 performs image processing on the captured image and records the processed image data in the image recording unit 112. The digital camera 100 returns to the shooting standby state. When the main switch 117 is pressed again, the power to the digital camera 100 is turned off.

[0027] Here, we will explain the brightness performance of the display device (display unit 115). As a standard for HDR display devices, the US VESA (Video Electronics Standards Association) has established Display HDR. HDR is a standard that clarifies the HDR-related performance of display devices, such as brightness and contrast ratio, by dividing them into several grades. For example, 400 cd / m² 2 , 600 cd / m² 2 , 1000 cd / m² 2 As display devices capable of high-brightness display, each is called "Display "HDR400," "Display HDR600," and "Display HDR1000" are defined. As such, the display brightness of display devices that show HDR images varies.

[0028] Furthermore, the signal characteristics representing the relationship between the video signal level and display brightness in HDR are defined by the EOTF (Electro-Optical Transfer Function). There are two methods for EOTF: PQ (Perceptual Quantization), which is standardized as SMPTE ST 2084, and HLG (Hybrid Log Gamma), which is standardized as ARIB STD-B67. The HLG method treats display brightness as a relative value, while the PQ method treats display brightness up to 10,000 cd / m². 2 It differs from the HLG method in that it treats it as the absolute value of (or nits).

[0029] The PQ method defines peak brightness as an absolute value, allowing for display at a constant brightness and offering higher reproducibility than the HLG method, where peak brightness varies depending on the display device. On the other hand, the PQ method can sometimes have difficulty expressing brightness according to the display brightness of the display device.

[0030] For example, a "Display HDR400" display device has a peak brightness of 400 cd / m². 2 Therefore, highlight-weighted metering is used to capture highlights at 600 cd / m².2 When taking an HDR image so that it becomes [a certain value], a display device with "Display HDR400" will clip (round down) a signal value of 400 cd / m 2 or higher and display it as white.

[0031] Also, a display device with "Display HDR1000" can express up to 1000 cd / m 2 When taking an HDR image so that the highlight part becomes 400 cd / m with spot metering on the highlight, a display device with "Display HDR1000" will 2 clip (round down) a signal value of 400 cd / m 2 ~1000 cd / m 2 and hardly use the area. In this case, the displayed image will not make full use of the brightness performance of the display device with "Display HDR1000".

[0032] <Shooting process flow by spot metering on the highlight> Referring to FIG. 2, the shooting process flow by spot metering on the highlight will be described. FIG. 2 is a flowchart illustrating the shooting process by spot metering on the highlight.

[0033] <"0000196">In step S201, the system control unit 113 sets the target brightness of the highlight part. The highlight part is an area with relatively high brightness in the image and is also referred to as a high - brightness area. The target brightness is a target value that determines the brightness at which the highlight part is photographed. The system control unit 113 can set the target brightness based on an instruction (selection operation, input operation, etc.) from the user via the operation unit 116.

[0034] The user sets the target brightness considering the display brightness of the display device. The user sets the target brightness By setting the display brightness to that of the display device, it is possible to achieve brightness expression that takes advantage of the display brightness of the display device. The display device here may be an external display device connected to the digital camera 100, or it may be the display unit 115 provided by the digital camera 100. The system control unit 113 may also acquire information on the display brightness of the display device from the display device that displays the captured image, and set the acquired display brightness as the target brightness.

[0035] In step S202, the system control unit 113 measures the brightness based on the target brightness set in step S201 and performs exposure control in highlight-weighted metering. The system control unit 113 converts the target brightness determined in step S201 into a signal value before A / D conversion using the image processing unit 108 and outputs it to the AE processing unit 103.

[0036] The system control unit 113 measures the brightness of the highlight area using the image processing unit 108 and the image recognition unit 109. The image processing unit 108 further converts the brightness of the highlight area into a signal value before A / D conversion and outputs it to the AE processing unit 103. The AE processing unit 103 determines the exposure stop difference between the target brightness and the brightness of the highlight area, and controls the operation of the aperture and shutter 102 to determine the exposure. Details of the exposure control process by the AE processing unit 103 will be explained in Figure 8 below.

[0037] In step S203, the system control unit 113 performs scene-specific tone correction processing. The system control unit 113 performs tone correction processing on the image captured with the exposure determined in step S202, using the image processing unit 108. Scene-specific tone correction processing is a process that corrects the tone curve based on the dynamic range and target brightness of the image.

[0038] In step S204, the system control unit 113 executes the main imaging process. The system control unit 113 performs image processing on the captured image using the image processing unit 108 and records the image data after image processing in the image recording unit 112.

[0039] (Setting the target brightness) The setting of the target brightness of the highlight area in step S201 of Figure 2 will be explained in detail. The AE processing unit 103 controls the exposure so that the brightness of the highlight area approaches the target brightness set in step S201. The highlight area refers to the region in the image that is relatively bright. The brightness of the highlight area is a representative brightness (hereinafter also referred to as representative brightness) determined based on the brightness of the pixels included in the highlight area.

[0040] First, we will explain the representative brightness of the highlight area by referring to Figures 3 to 5. The representative brightness of the highlight area can be determined in various ways. For example, the representative brightness of the highlight area can be determined based on the brightness histogram (hereinafter referred to as the histogram) of the image shown in Figures 3(a) and 3(b). In the histogram, the vertical axis is the number of pixels and the horizontal axis is the brightness.

[0041] Figure 3(a) shows an example where the luminance at the peak (maximum value) of the high-luminance side of the histogram is used as the representative luminance of the highlight area. Alternatively, the representative luminance of the highlight area may be the luminance at the centroid of the high-luminance side of the histogram (the region enclosed by the high-luminance convex portion of the histogram). The high-luminance side of the histogram can be, for example, the region enclosed by the range from the high-luminance edge to the minimum value and the corresponding luminance coordinate axis. Note that the peak in the histogram only needs to be defined within the range that includes the vertex with the maximum number of pixels, and can be, for example, the region between the minimum values ​​on both sides of the vertex.

[0042] Figure 3(b) shows an example where the representative brightness of the highlight area is the average brightness of pixels included in the brightness range Da where the cumulative frequency from the high-brightness side in the histogram is less than or equal to a predetermined percentage (e.g., 5%). Alternatively, the representative brightness of the highlight area may be the brightness at the centroid of the region where the cumulative frequency from the high-brightness side in the histogram is less than or equal to a predetermined percentage. The predetermined percentage is 1% or more. It is preferable that it be 0% or less.

[0043] The region from which to obtain the histogram may be the entire image, or it may be the inner region excluding the edges of the image. For example, the system control unit 113 may generate a histogram from a predetermined percentage (e.g., 80%) of the inner region of the image.

[0044] The reason for generating a luminance histogram from the area excluding the edges of the image is explained below. In the area around the edges of the image, luminance tends to be lower due to lens vignetting. As a result, the histogram of the area around the image will be shifted towards the low-luminance side and will not be detected properly. Therefore, it is preferable not to include the area around the image in the detection range.

[0045] Furthermore, the area surrounding the image may not contain a subject, and since users will not focus on an area without a subject, it is preferable to exclude this area from the detection range. Additionally, there are cases where it is desirable to capture both a still image and a video of the same scene and make the tone correction effect similar. Since the aspect ratio of images is 3:2 or 4:3 for still images and 16:9 for videos, excluding the top and bottom edges of still images makes it possible to make the tone correction effect similar for both still images and videos.

[0046] Figure 4 shows an example of determining the highlight area by screen division. The highlight area can be determined by dividing the image into multiple blocks, for example, by selecting the block with the highest average brightness within each block. Alternatively, the highlight area may be the block with the highest average, maximum, minimum, median, or mode (hereinafter also referred to as block brightness) within each block.

[0047] In the example in Figure 4, 80% of the inner area of ​​the image is divided into multiple blocks. The image contains clouds and trees, and the highlight area is determined to be the block with the highest block brightness within the cloud area. The representative brightness of the highlight area can be the block brightness of the block determined to be the highlight area.

[0048] Figure 5 illustrates an example of detecting a subject and determining the highlight area. The highlight area may be determined by the image recognition unit 109 based on the brightness of the subject, such as the face of a person or animal, detected in the image.

[0049] Figures 5(a) and 5(b) show examples of detecting a person's face and torso as the subject. The system control unit 113 acquires the average brightness within a rectangle that encloses the entire face or part of the face as the brightness of the face. The system control unit 113 acquires the brightness of the torso in the same way as the face. Note that the brightness of the face and torso are not limited to the average brightness within the rectangle, but may also be the maximum, minimum, median, mode, etc. of the brightness within the rectangle.

[0050] In Figure 5(a), the brightness of the face is higher than that of the torso, so the highlight area can be the face region. In this case, the representative brightness of the highlight area is the brightness of the face. In Figure 5(b), the brightness of the torso is higher than that of the face, so the highlight area can be the torso region. In this case, the representative brightness of the highlight area is the brightness of the torso. In this way, the system control unit 113 can acquire the brightness of multiple regions included in the subject and acquire the representative brightness of the region with the highest brightness as the highlight area.

[0051] Next, the target brightness will be explained. When performing highlight-weighted photometry, the system control unit 113 controls the exposure so that the representative brightness of the highlight area approaches the target brightness. The target brightness is set according to the display brightness of the display device and specified by the user.

[0052] Referring to Figure 6, the screen on which the user specifies the target brightness will be explained. Figure 6(a) is This is an example of a screen where the user selects one of three predetermined brightness levels, TH1, TH2, or TH3, to specify the target brightness. When displaying an HDR image with absolute brightness, such as in the PQ method, TH1 is set to 400 cd / m² in accordance with the Display HDR standard. 2 TH2 is 600 cd / m² 2 TH3 has a noise level of 1000 cd / m².2 This can be done as follows.

[0053] Figure 6(b) shows an example of a screen in which the user specifies the target brightness by inputting a numerical value for the desired brightness. The system control unit 113 may set the user-specified brightness TH specified by the user on the screen in Figure 6(b) as the target brightness, or it may adjust (change) the target brightness by performing a predetermined calculation on the user-specified brightness TH. In the following description, the predetermined calculation will be described in the case where it is applied to the user-specified brightness TH, but it may also be applied to the display brightness of the display device obtained from the display device.

[0054] The predetermined calculation is, for example, a process of increasing or decreasing a certain luminance value relative to the user-specified luminance TH. If the amount of change in luminance from the user-specified luminance TH is α, the target luminance TargetLum can be expressed as shown in (Equation 1). TargetLum = TH - α ... (Equation 1) The target brightness is set lower than the user-specified brightness TH if α > 0, and higher than the user-specified brightness TH if α < 0. For example, if the user-specified brightness TH is 400 cd / m² 2 Let's assume the brightness change amount α is 20 cd / m². 2 In that case, the target brightness is 380 cd / m². 2 It will be set to this.

[0055] Furthermore, the predetermined calculation can be a process that increases or decreases the amount of brightness change α, which changes according to the user-specified brightness TH, relative to the user-specified brightness TH. Figure 7 shows an example of a lookup table that determines the amount of brightness change α from the user-specified brightness TH. By preparing a lookup table in advance, the system control unit 113 can change the amount of brightness change α according to the user-specified brightness TH.

[0056] The user-specified brightness TH is set based on the display brightness of the display device, and by changing the brightness change amount α, the target brightness is set to a brightness corresponding to the display brightness of the display device. In other words, the brightness change amount α in the lookup table should be set to a value corresponding to the display brightness of the display device.

[0057] In the example in Figure 7, the user-specified luminance TH is 400 cd / m². 2 , 600 cd / m² 2 , 1000 cd / m² 2 In this case, the luminance change α is 10 cd / m². 2 50 cd / m² 2 , 100 cd / m² 2 This is how it works. The lookup table in Figure 7 is a table that assumes the user selects a target brightness from a predetermined range of brightness levels, as shown in Figure 6(a). If the user inputs a target brightness value, as shown in Figure 6(b), the lookup table should be a table that determines the brightness change amount α for each input brightness value (user-specified brightness TH).

[0058] The method for changing the luminance change amount α according to the user-specified luminance TH is not limited to using a lookup table, but may also be a method using a function Rate(TH) that uniquely determines the luminance change amount α according to the user-specified luminance TH. The target luminance TargetLum is expressed as shown in (Equation 2). TargetLum=TH-Rate(TH) (Formula 2) The function Rate(TH) can be, for example, a function that multiplies the user-specified brightness TH by a predetermined coefficient.

[0059] (Exposure control) For exposure control in highlight-weighted metering at step S202 in Figure 2, please refer to Figure 8. Let me explain. Figure 8 is a flowchart illustrating the exposure control process.

[0060] In step S801, the system control unit 113 converts the representative brightness of the highlight area into a RAW signal value Ya. The RAW signal value is the signal value after A / D conversion processing and before gamma processing, and can be uniquely determined based on the setting value of the gamma function.

[0061] In step S802, the system control unit 113 converts the target brightness set in step S201 into a RAW signal value Yb, similar to step S801. In step S803, the system control unit 113 obtains the exposure stop difference. The exposure stop difference can be calculated as Log(Ya / Yb) using a known calculation. In step S804, the system control unit 113 uses the exposure stop difference obtained in step S803 to perform exposure correction so that the representative brightness of the highlight area approaches the target brightness.

[0062] Refer to Figure 9 to explain the changes in the histogram before and after exposure compensation. Figure 9(a) shows an example of the histogram of an image before exposure compensation. The representative brightness of the highlight area is defined as the brightness at the peak of the first high-brightness side of the histogram. The representative brightness of the highlight area is lower than the target brightness. Also, there is a small distribution of pixels on the high-brightness side, and the image before exposure compensation is generally darker than the target brightness.

[0063] Figure 9(b) shows the histogram after exposure compensation, as illustrated in Figure 9(a). The system control unit 113 performs exposure compensation so that the representative brightness of the highlight area shifts to the higher brightness side of the histogram by the amount of the exposure stop difference obtained in step S803. As a result, the representative brightness of the highlight area after exposure compensation almost matches the target brightness. The distribution of the histogram shifts to the higher brightness side overall, and the image after exposure compensation becomes brighter than the image before exposure compensation.

[0064] Figure 9 shows an example of compensation where the representative brightness of the highlight area is lower than the target brightness, and the exposure is increased. However, if the representative brightness of the highlight area is higher than the target brightness, the system control unit 113 can bring the representative brightness of the highlight area closer to the target brightness by lowering the exposure.

[0065] (Scene-specific tone correction processing) The scene-specific tone correction process in step S203 of Figure 2 will be explained. In step S202, exposure control is performed so that the representative brightness of the highlight area approaches the target brightness. However, if the dynamic range of the shooting scene is greater than a predetermined number of stops, the resulting image may be darker than expected, potentially resulting in an undesirable image with blown-out highlights or crushed blacks. Therefore, in step S203, if the dynamic range of the shooting scene is greater than a predetermined number of stops, the system control unit 113 performs tone correction so that the dark areas (low brightness areas) excluding the highlight areas (high brightness areas) become brighter.

[0066] The dynamic range of a shooting scene is the difference in brightness (the difference between the highest and lowest brightness levels) of the shooting scene, and is expressed in steps, such as an EV value, which increases in stages according to the width of the dynamic range. The predetermined number of steps can be, for example, 8 to 12 steps, and may be set to any other number of steps depending on the image quality desired by the user.

[0067] It has a dynamic range of 15 stops and a maximum brightness of 5000 cd / m². 2 The representative brightness of the highlight area is 4500 cd / m². 2 This section explains the tone correction process when shooting a scene with a target brightness of 400 cd / m². 2 When set to this value, the system control unit 113 sets the representative brightness of the highlight area to 4500 cd / m². 2 The actual brightness is lower at 400 cd / m². 2 The exposure is controlled to approximate the target. By lowering the exposure, the image after exposure compensation will be darker than the image before exposure compensation, and the tonal range in dark areas will be worse.

[0068] Target brightness: 400 cd / m² 2 If setting the exposure to a certain level results in poor tonal gradation in dark areas, the system control unit 113 adjusts the exposure to a lower level while performing tonal correction to brighten the dark areas. Tonal correction can be achieved using known methods such as tone curve correction. By performing tonal correction to brighten the dark areas, the image after tonal correction becomes brighter overall, making the dark areas easier to see.

[0069] When performing tonal correction in dark areas, since exposure control was performed in step S202 to match the representative brightness of the highlight area to the target brightness, the system control unit 113 increases the brightness of the dark areas to avoid affecting the representative brightness of the highlight area. Specifically, the system control unit 113 detects the peak on the high-brightness side of the image histogram and refrains from performing tonal correction on pixels in the peak area and pixels on the high-brightness side of the peak.

[0070] The peak on the high-brightness side can be, for example, a predetermined range (e.g., from RL-β to RL+β) that includes the brightness at the position where the number of pixels is maximum on the high-brightness side (reference brightness RL). Alternatively, the peak on the high-brightness side may be the region enclosed in the histogram by the range from the edge of the high-brightness side to the minimum value and the corresponding brightness coordinate axis.

[0071] Dark area gradation correction may be performed when the proportion of the image's blacked-out area exceeds a predetermined percentage. This predetermined percentage can be set appropriately according to the user's desired image quality.

[0072] Refer to Figure 10 to explain the gradation correction of dark areas. Figure 10(a) shows the histogram after adjusting the representative brightness of the highlight area to the target brightness by controlling the exposure in step S202. In Figure 10(a), the peak on the high-brightness side is based on the representative brightness of the highlight area and is in the range from YamaStart to YamaEnd. In the histogram of Figure 10(a), the pixel distribution is also concentrated on the low-brightness side, forming a peak, and the dark areas of the image are completely blacked out.

[0073] The system control unit 113 corrects the tone curve to compensate for the brightness of dark areas and suppress black clipping. Figure 10(b) shows an example of tone curve correction. The system control unit 113 detects a peak on the low-luminance side from the histogram and obtains the position of the peak's apex, the centroid of the peak, and the position of the average luminance of the pixels included in the peak as DarkPointB, which represents the dark area luminance before correction. In Figure 10(b), DarkPointB is set to the position of the peak on the low-luminance side.

[0074] A lookup table is prepared in advance to associate the dark area brightness before correction (input brightness) with the dark area brightness after correction (output brightness), and the system control unit 113 obtains the output brightness DarkPointA for the input brightness DarkPointB.

[0075] The tone curve is corrected at the low-luminance end, for example, by spline interpolation to form a curve passing through (0,0), (DarkPointB,DarkPointA), and (YamaStart,YamaStart). Note that the method is not limited to spline interpolation; the tone curve may also be corrected to a polyline connecting the three points with straight lines.

[0076] Furthermore, the tone curve is defined as a straight line connecting (YamaStart,YamaStart) and (YamaEnd,YamaEnd) on the high-luminance side. The system control unit 113 does not perform gradation correction on the high-luminance side, but performs gradation correction on the low-luminance side to make it brighter, thereby maintaining the brightness of the highlight area while correcting the low-luminance side to make it brighter.

[0077] The system control unit 113 may, in step S202, control the exposure so that the representative brightness on the high-brightness side falls below the target brightness, and in step S203, correct the exposure so that both the dark and highlight areas become brighter. For example, the system control unit 113 corrects the tone curve to a curve passing through (0,0), (DarkPointB,DarkPointA), and (YamaEnd,YamaEnd) by spline interpolation. In this case, the system control unit 113 should lower the exposure in step S202 so that the amount of tone curve correction in the highlight area cancels out. The system control unit 113 can control the exposure by, for example, adjusting the brightness change amount α in (Equation 1).

[0078] The dynamic range of the shooting scene is above a specified number of stops, and the target brightness is 1000 cd / m². 2 Let's explain the case where the target brightness is 1000 cd / m². 2Therefore, due to the brightness contrast effect, users may perceive dark areas as relatively darker. The target brightness is 1000 cd / m². 2 In this case, the system control unit 113 sets the dark area correction amount Dark when the target brightness is 400 cd / m². 2 You should set it to a larger value than in the previous case.

[0079] The dark area correction amount for the reference target brightness is defined as DarkRef. The reference target brightness is 400 cd / m². 2 Therefore, DarkRef = DarkPointA - DarkPointB. If TargetLum is the target brightness and F(TargetLum) is the dark area correction ratio determined according to TargetLum, then the dark area correction amount Dark with respect to TargetLum can be expressed as (Equation 3). Dark=DarkRef×F(TargetLum) (Formula 3)

[0080] This section explains the case where the dynamic range of the shooting scene is less than a predetermined number of stops. When the dynamic range of the shooting scene is less than a predetermined number of stops, the brightness and gradation of the image will be well maintained even if the representative brightness of the highlight area is matched to the target brightness. For example, when the dynamic range is 5 stops and the maximum brightness is 400 cd / m². 2 The representative brightness of the highlight area is 350 cd / m². 2 In this case, even if the representative brightness of the highlight area is adjusted to the target brightness, good gradation is maintained. Therefore, if the dynamic range is less than a predetermined number of steps, the system control unit 113 may choose not to perform step S203.

[0081] According to the above embodiment, the digital camera 100 can set a target brightness based on the display brightness of the display device and control the exposure based on the representative brightness of the highlight area and the target brightness, thereby realizing brightness expression that makes the most of the display brightness of the display device.

[0082] <Other Embodiments> The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0083] This embodiment includes the following configurations and methods. (Composition 1) An acquisition means for obtaining representative brightness of high-brightness regions in an image, A setting means for setting the target brightness of the high-brightness region based on the display brightness of the display device that displays the aforementioned image, An imaging device characterized by having exposure control means that controls the exposure based on the representative brightness of the high-brightness region and the target brightness. (Configuration 2) The target brightness is set by user specification. The imaging apparatus according to configuration 1, characterized by the features described above. (Composition 3) The target brightness is set based on the display brightness of the display device obtained from the display device. The imaging apparatus according to configuration 1, characterized by the features described above. (Composition 4) The target brightness is changed based on the amount of brightness change corresponding to the display brightness of the display device. An imaging apparatus according to any one of configurations 1 to 3, characterized by the above. (Composition 5) The system further includes correction means for performing gradation correction based on the dynamic range of the image and the target brightness. An imaging device according to any one of configurations 1 to 4, characterized by the above. (Composition 6) The aforementioned dynamic range is expressed by a number of steps that increase in stages according to the width of the dynamic range. The correction means performs the tone correction when the dynamic range of the image is equal to or greater than a predetermined number of steps, and does not perform the tone correction when the dynamic range of the image is less than a predetermined number of steps. The imaging apparatus according to configuration 5, characterized in that it is a device. (Composition 7) The correction means performs the gradation correction so that the low-luminance areas of the image become brighter when the dynamic range of the image is greater than or equal to a predetermined number of steps. The imaging apparatus according to configuration 6, characterized in that... (Composition 8) The correction means performs the gradation correction not only on the low-luminance region but also on the high-luminance region. The exposure control means controls the exposure to be lowered so that the tone correction for the high-luminance region is canceled out. The imaging device according to configuration 7, characterized by the features described above. (Composition 9) The representative brightness of the high-brightness region is the brightness at the position of the maximum value on the high-brightness side of the brightness histogram of the image, or the brightness at the centroid position of the region enclosed by the convex portion on the high-brightness side of the brightness histogram. An imaging device according to any one of configurations 1 to 8, characterized by the above. (Composition 10) The region enclosed by the high-luminance convex portion is the region enclosed by the range from the high-luminance end of the luminance histogram to the minimum value and the corresponding luminance coordinate axis. The imaging apparatus according to configuration 9, characterized by the features described herein. (Composition 11) The representative brightness of the high-brightness region is the average brightness of pixels whose cumulative frequency from the high-brightness side of the brightness histogram of the image is less than or equal to a predetermined percentage, or the brightness at the centroid of the region enclosed by the range in which the cumulative frequency from the high-brightness side of the brightness histogram of the image is less than or equal to a predetermined percentage, and the corresponding brightness coordinate axis. An imaging device according to any one of configurations 1 to 8, characterized by the above. (Composition 12) The acquisition means divides the image into multiple blocks and acquires the maximum value of the block brightness, which is the average, maximum, minimum, median, or mode of brightness within each block, as the representative brightness of the high-brightness region. An imaging device according to any one of configurations 1 to 8, characterized by the above. (Composition 13) The representative brightness of the high-brightness region is the average brightness within the area of ​​the subject's face detected in the image. The mean, maximum, minimum, median, or mode. An imaging device according to any one of configurations 1 to 8, characterized by the above. (method) An acquisition step to obtain representative brightness of high-brightness regions in an image, A setting step of setting the target brightness of the high-brightness region based on the display brightness of the display device that displays the aforementioned image, An exposure control step that controls the exposure based on the representative brightness of the high-brightness region and the target brightness. A control method for an imaging device, characterized by having the following features. (program) A program for causing a computer to function as one of the means of the imaging apparatus described in any one of configurations 1 to 13. (medium) A computer-readable storage medium containing a program for causing the computer to function as one of the means of the imaging apparatus described in any one of configurations 1 to 13. [Explanation of symbols]

[0084] 100: Imaging device, 103: AE processing unit, 108: Image processing unit, 109: Image recognition unit, 113: System control unit, 115: Display unit

Claims

1. An acquisition means for obtaining representative brightness of high-brightness regions in an image, A setting means for setting the target brightness of the high-brightness region based on the display brightness of the display device that displays the aforementioned image, Exposure control means that controls exposure based on the representative brightness of the high-brightness region and the target brightness, Correction means for performing gradation correction based on the dynamic range of the scene in which the image was taken and the target brightness. It has, The aforementioned dynamic range is expressed by a number of steps that increase in stages according to the width of the dynamic range. The correction means performs the tone correction when the dynamic range is equal to or greater than a predetermined number of steps, and does not perform the tone correction when the dynamic range is less than a predetermined number of steps. An imaging device characterized by the following features.

2. The target brightness is set by user specification. The imaging apparatus according to feature 1.

3. The target brightness is set based on the display brightness of the display device obtained from the display device. The imaging apparatus according to feature 1.

4. The target brightness is changed based on the amount of brightness change corresponding to the display brightness of the display device. The imaging apparatus according to feature 1.

5. The correction means performs the gradation correction so that the low-luminance areas of the image become brighter when the dynamic range is greater than or equal to a predetermined number of steps. The imaging apparatus according to feature 1.

6. The correction means performs the gradation correction not only on the low-luminance region but also on the high-luminance region. The exposure control means controls the exposure such that the gradation correction for the high-luminance region is offset. Control to lower The imaging apparatus according to feature 5.

7. The representative brightness of the high-brightness region is the brightness at the position of the maximum value on the high-brightness side of the brightness histogram of the image, or the brightness at the centroid position of the region enclosed by the convex portion on the high-brightness side of the brightness histogram. The imaging apparatus according to feature 1.

8. The region enclosed by the high-luminance convex portion is the region enclosed by the range from the high-luminance end of the luminance histogram to the minimum value and the corresponding luminance coordinate axis. The imaging apparatus according to feature 7.

9. The representative brightness of the high-brightness region is the average brightness of pixels whose cumulative frequency from the high-brightness side of the brightness histogram of the image is less than or equal to a predetermined percentage, or the brightness at the centroid of the region enclosed by the range in which the cumulative frequency from the high-brightness side of the brightness histogram of the image is less than or equal to a predetermined percentage, and the corresponding brightness coordinate axis. The imaging apparatus according to feature 1.

10. The acquisition means divides the image into multiple blocks and acquires the maximum value of the block brightness, which is the average, maximum, minimum, median, or mode of brightness within each block, as the representative brightness of the high-brightness region. The imaging apparatus according to feature 1.

11. The representative brightness of the high-brightness region is the mean, maximum, minimum, median, or mode of brightness within the facial region of the subject detected in the image. The imaging apparatus according to feature 1.

12. An acquisition step to obtain representative brightness of high-brightness regions in an image, A setting step of setting the target brightness of the high-brightness region based on the display brightness of the display device that displays the aforementioned image, An exposure control step that controls the exposure based on the representative brightness of the high-brightness region and the target brightness, A correction step that performs gradation correction based on the dynamic range of the scene in which the image was taken and the target brightness. It has, The aforementioned dynamic range is expressed by a number of steps that increase in stages according to the width of the dynamic range. In the correction step, if the dynamic range is equal to or greater than a predetermined number of steps, the tone correction is performed; if the dynamic range is less than a predetermined number of steps, the tone correction is not performed. A control method for an imaging device, characterized by the following:

13. A program for causing a computer to function as one of the means of an imaging apparatus according to any one of claims 1 to 11.

14. A computer-readable storage medium storing a program for causing a computer to function as one of the means of an imaging apparatus according to any one of claims 1 to 11.

Images