Imaging device and control method for the imaging device

Abstract

To provide a technique of improving the contrast of a high luminance region of an image while maintaining brightness intended by a user.SOLUTION: An imaging apparatus comprises: exposure control means for performing exposure control such that the representative luminance of a high luminance region of an image becomes a target luminance set by a user; and gradation correction means for performing gradation correction of increasing the contrast of the high luminance region by extending a luminance range of the high luminance region of the image captured after the exposure control is performed. The gradation correction means changes the extension degree of the luminance range of the high luminance region on the basis of the target brightness.SELECTED DRAWING: Figure 7

Description

Technical Field

[0001] The present invention relates to an image capturing apparatus and a control method for the image capturing apparatus.

Background Art

[0002] A technique for performing gradation correction on an image is known in order to obtain an image with preferable brightness and contrast. Patent Document 1 discloses a technique that performs gradation processing for converting a luminance signal such that the maximum reference luminance detected in the image corresponds to a predetermined luminance signal value.

[0003] Patent Document 2 discloses a technique for controlling appropriate exposure without being affected by high-intensity light in a scene where high-intensity light exists in addition to the main subject in an image when capturing an image with a highlight-priority photometry method that performs photometry with emphasis on the high-intensity region of the image.

Prior Art Literature

Patent Literature

[0004]

Patent Literature 1

Patent Literature 2

Summary of the Invention

Problem to be Solved by the Invention

[0005] In highlight-priority photometry, exposure is controlled such that the luminance of the high-intensity region in the image approaches a target luminance specified by a user, or is equal to or lower than the target luminance. As a result, the gradation is compressed, which may lower the contrast of the image.

[0006] For example, when shooting an 8-bit image using highlight-weighted metering, suppose the user specifies a target brightness of 150 for the high-brightness areas. The exposure will be adjusted so that the brightness of the high-brightness areas approaches the target brightness of 150, resulting in the image brightness being as intended by the user. On the other hand, by bringing the brightness of the high-brightness areas closer to the target brightness, the tonal range is compressed, and the image contrast will be lowered. Thus, prioritizing the target brightness may result in the image contrast not being as intended by the user. Also, if the exposure is increased to increase the image contrast, the image brightness may not be as intended by the user.

[0007] The present invention aims to provide a technology that improves the contrast of high-brightness areas in an image while maintaining the brightness intended by the user. [Means for solving the problem]

[0008] The imaging apparatus according to the present invention includes an exposure control means that controls the exposure so that the representative brightness of the high-brightness region of an image becomes a target brightness set by the user, and a tone correction means that performs tone correction to increase the contrast of the high-brightness region by extending the brightness range of the high-brightness region of the image captured after the exposure control has been performed, wherein the tone correction means The extent of the extended luminance range is determined according to the extent of the luminance range of the high-luminance region, and the extent of the extended luminance range of the high-luminance region after extension is determined by multiplying the extent of the extended luminance range by an extension rate corresponding to the target luminance. It is characterized by the following: [Effects of the Invention]

[0009] According to the present invention, it is possible to improve the contrast of high-luminance areas in an image while maintaining the brightness intended by the user. [Brief explanation of the drawing]

[0010] [Figure 1] This is a block diagram illustrating the configuration of a digital camera. [Figure 2] This is an example screen where the user selects a target brightness for a high-brightness area. [Figure 3] This flowchart illustrates the shooting process using highlight-weighted metering. [Figure 4] This is a flowchart illustrating the gradation correction process. [Figure 5] This is an example of an image histogram. [Figure 6] This is an example of a lookup table that determines the width of the peaks in a histogram. [Figure 7] This is an example of a histogram showing the peak portion before and after elongation. [Figure 8] This is an example of a tone curve corresponding to the histogram after stretching the peak portion. [Figure 9] This diagram illustrates the determination of correction target values ​​according to the target brightness. [Figure 10] This flowchart illustrates a gradation correction process that takes into account the brightness of the main subject. [Figure 11] This is an example of a histogram before and after stretching, taking into account the brightness of the main subject. [Modes for carrying out the invention]

[0011] Embodiments of the present invention will be described below with reference to the drawings. The embodiments described below are merely examples of methods for realizing the present invention, and may be modified or changed as appropriate depending on the configuration of the apparatus to which the present invention is applied and various conditions. Furthermore, the embodiments can be combined as appropriate.

[0012] <Digital Camera Configuration> Figure 1 is a block diagram illustrating the configuration of a digital camera 100 as an imaging device according to an embodiment of the present invention. The digital camera 100 comprises an imaging mechanism including a shooting lens 101, an aperture and shutter 102, an automatic exposure (AE) processing unit 103, a focus lens 104, an autofocus (AF) processing unit 105, an image sensor 106, and an A / D conversion unit 107.

[0013] The photographing lens 101 has a zoom mechanism. The aperture and shutter 102 control the amount of incident light, which is reflected light from a subject, entering the image sensor 106 and the charge accumulation time in accordance with instructions from the AE processing unit 103. The AE processing unit 103 performs photometry on a photometric region according to a photometry mode, and controls exposure by controlling the operation of the aperture and shutter 102. The AE processing unit 103 also controls the A / D conversion unit 107. The focus lens 104 focuses on the light-receiving surface of the image sensor 106 to form an optical image in accordance with a control signal from the AF processing unit 105. The AF processing unit 105 also calculates distance information from the digital camera 100 to the subject.

[0014] The image sensor 106 converts an optical image formed on the light-receiving surface into an electrical signal by photoelectric conversion means such as a CCD element or a CMOS element, and outputs the electrical signal to the A / D conversion unit 107. The A / D conversion unit 107 converts the received electrical signal (analog signal) into a digital signal (RAW signal). The A / D conversion unit 107 includes a CDS circuit that removes noise from the received electrical signal, and a non-linear amplification circuit that performs non-linear amplification on the received electrical signal before converting it into a RAW signal.

[0015] The digital camera 100 further includes an image processing unit 108, an image recognition unit 109, a format conversion unit 110, and a DRAM (Dynamic RAM) 111. The image processing unit 108 performs development processing for outputting image data by performing predetermined pixel interpolation, resizing processing such as image reduction, and color conversion processing on the RAW signal input from the A / D conversion unit 107.

[0016] The image processing unit 108 adjusts the image quality of a captured image by adjusting white balance (WB) for the RAW signal input from the A / D conversion unit 107, and performing gradation correction by increasing or decreasing the luminance level of the image or the like. For example, the image processing unit 108 has, with respect to the luminance level of image data, a function of increasing or decreasing the luminance level of the entire image at a uniform increase / decrease rate, and a tone curve (gamma) function that converts a signal level according to the magnitude of an original signal level. The image processing unit 108 implements gradation correction processing through these functions.

[0017] The image recognition unit 109 receives input of image data appropriately processed by the image processing unit 108. As photometric processing, the image recognition unit 109 can recognize the brightness condition of an input image. The image recognition unit 109 divides the image data into a plurality of regions and performs photometry. Accordingly, the image recognition unit 109 can determine a high-luminance region in an image and acquire a photometric result in the high-luminance region.

[0018] The high-luminance region may be, for example, a luminance range corresponding to a peak on the high-luminance side of a luminance histogram of an image. The luminance range corresponding to the peak on the high-luminance side may range from a minimum value (inflection point) on the lower-luminance side relative to the peak vertex to a minimum value (inflection point) on the higher-luminance side relative to the peak vertex in the luminance histogram, or may be specified by a user.

[0019] The image recognition unit 109 can recognize a scene by a known technique. For example, when detecting a face, the image recognition unit 109 recognizes that the scene is a scene where a person is photographed. The photometric result and scene recognition information obtained by the image recognition unit 109 are output to the AE processing unit 103.

[0020] A user can select a photometry mode via an operation unit 116. The operation unit 116 outputs the photometry mode selected by the user to the AE processing unit 103. The AE processing unit 103 performs automatic exposure based on the photometric result and the scene recognition information output by the image recognition unit 109, and information such as the photometry mode output by the operation unit 116.

[0021] The image recognition unit 109 can recognize a focus state of an input image. A recognition result of the focus state is output to the AF processing unit 105. The AF processing unit 105 implements AF control based on the recognition result of the focus state.

[0022] The format conversion unit 110 converts the format of the image data generated by the image processing unit 108 in order to store the image data in the DRAM 111. The DRAM 111 is an internal memory and is used as a buffer for temporary storage of image data, or as working memory for image data compression / decompression processing.

[0023] The digital camera 100 comprises an image recording unit 112, a system control unit 113, a VRAM (Video RAM) 114, a display unit 115, an operation unit 116, a main switch (main SW) 117, and a shooting switch (shooting SW) 118. The image recording unit 112 has a recording medium such as a memory card and an interface for recording captured images (still images, videos).

[0024] The system control unit 113 includes a CPU (processor), ROM, and RAM. The CPU controls the overall operation of the digital camera 100 by loading programs stored in ROM into the RAM's work area and executing them. The system control unit 113 can perform processing for each component of the digital camera 100 by executing programs stored in ROM. The system control unit 113 controls which of the multiple imaging drive modes of the image sensor 106 is used. VRAM 114 is memory for image display.

[0025] The display unit 115 is, for example, an LCD (Liquid Crystal Display). The display unit 115 displays images, provides operational assistance, and displays the status of the digital camera 100. In addition, during shooting, it displays the shooting screen and the distance measurement area. Furthermore, as illustrated in Figure 2, the display unit 115 displays a screen for the user to select a target brightness for the high-brightness area.

[0026] The control unit 116 is a component that allows the user to operate the digital camera 100 from the outside. The user can use the control unit 116 to, for example, adjust exposure compensation, set the aperture value, and configure image playback settings. Various settings such as these can be configured. The control unit 116 includes a menu switch, a zoom lever for instructing the zoom operation of the shooting lens, and an operation mode switch for switching between shooting mode and playback mode.

[0027] The user can select a metering mode by operating the control unit 116. The metering modes selectable by the user include, for example, evaluative metering mode, partial metering mode, and highlight-weighted metering mode.

[0028] Evaluative metering mode measures light in multiple areas set on the screen and determines the final exposure based on information such as the subject's brightness distribution, color, distance, and composition. Evaluative metering mode is suitable for general shooting, including backlit photography. Partial metering mode measures light in the central area of ​​the screen. Partial metering mode is effective when there is strong light around the subject, such as in backlit situations. Highlight-weighted metering mode determines the exposure so that the high-brightness areas of the screen are at the correct brightness. Highlight-weighted metering mode prioritizes metering in high-brightness areas. Generally, when controlling the exposure so that the high-brightness areas of the screen are at the correct brightness, the exposure will be darker in highlight-weighted metering mode than in evaluative metering mode.

[0029] In highlight-weighted metering mode, the user can select the target brightness TH for high-brightness areas by operating the control unit 116 on the screen shown in Figure 2. The target brightness TH for high-brightness areas selected by the user is output to the AE processing unit 103 for exposure control and to the image processing unit 108 for image processing control.

[0030] The main switch 117 is a switch for turning on power to the digital camera 100. The shooting switch 118 is a switch configured to perform two-step operation depending on the pressing depth. In response to a half-press operation (SW1 operation) that presses the shooting switch 118 halfway down, the system control unit 113 executes shooting preparation operations such as AE processing and AF processing. In response to a full-press operation (SW2 operation) that presses the shooting switch 118 all the way down, the system control unit 113 executes shooting processing.

[0031] A series of processing executed by the digital camera 100 will be described. When the main switch 117 is pressed to turn on the power, the system control unit 113 of the digital camera 100 causes the image sensor 106 to execute imaging processing at a predetermined cycle (e.g., a 33 ms cycle). The digital camera 100 enters a main shooting standby state in which captured images are sequentially displayed on the display unit 115. Upon receiving a shooting instruction via pressing of the shooting switch 118 (SW2 operation), the system control unit 113 executes main shooting processing by the image sensor 106. The system control unit 113 causes the image processing unit 108 to execute image processing on the captured image, and records the image data after image processing in the image recording unit 112. The digital camera 100 returns to the main shooting standby state again. When the main switch 117 is pressed again, the power of the digital camera 100 is turned off.

[0032] Figure 2 is an example of a screen for a user to select a target luminance for the luminance of a high-luminance area. The luminance of the high-luminance area is a representative luminance determined based on the luminance of pixels included in the high-luminance area (hereinafter also referred to as representative luminance). The representative luminance can be, for example, an average value, a maximum value, a minimum value, a median value, or a mode value of luminances of pixels included in the high-luminance area. The target luminance is a target value that defines what luminance the high-luminance area should have in shooting.

[0033] In the screen of Figure 2(A), the user selects one of TH1, TH2, and TH3 (TH1 < TH2 < TH3) as the target luminance for a highlight portion (high-luminance area). For example, in an 8-bit image, TH1 can be 120, TH2 can be 150, and TH3 can be 200.

[0034] Figure 2(A) shows an example where TH3 is specified as the target brightness. If the subject, which is the highlight area 201, is photographed brighter than intended, the user will darken the subject by changing the target brightness TH in the order of TH3 → TH2 → TH1 and taking the picture again. Figure 2(B) shows an example where TH1 is specified as the target brightness. By lowering the target brightness, the user can darken the image and improve the tonal range of the subject.

[0035] Note that TH1, TH2, and TH3 may be pre-set luminance values, or they may be changeable by the user. Furthermore, the user interface for setting the target luminance is not limited to the example in Figure 2. The target luminance may be set, for example, by the user inputting a luminance value.

[0036] <Processing flow using highlight-weighted metering> Referring to Figure 3, the flow of the shooting process using highlight-weighted metering performed by the digital camera 100 will be explained. Figure 3 is a flowchart illustrating the shooting process using highlight-weighted metering.

[0037] In step S301, the system control unit 113 sets the target brightness of the highlight area. The highlight area is a high-brightness region in the image. The target brightness is a target value that determines what brightness level the highlight area should be captured at. The system control unit 113 can set the target brightness based on instructions from the user via the operation unit 116 (selection operation or input operation, etc.) on the screen shown in Figure 2(A) or Figure 2(B) on the display unit 115.

[0038] In step S302, the system control unit 113 performs exposure control in highlight-weighted metering. The system control unit 113 converts the target brightness set in step S301 into a signal value before A / D conversion using the image processing unit 108, and transmits the converted signal value to the AE processing unit 103. The system control unit 113 also measures the brightness of the highlight area using the image processing unit 108 and the image recognition unit 109. The system control unit 113 further converts the brightness of the highlight area into a signal value before A / D conversion using the image processing unit 108, and transmits the converted signal value to the AE processing unit 103. The AE processing unit 103 determines the exposure by finding the difference in exposure stops between the target brightness and the brightness of the highlight area, and controlling the operation of the aperture and shutter 102.

[0039] In step S303, the system control unit 113 performs imaging. The system control unit 113 executes the main imaging process using the image sensor 106. In step S304, the system control unit 113 performs gradation correction. The system control unit 113 performs gradation correction on the image captured in step S303 using the image processing unit 108.

[0040] <Flow of tone correction> Referring to Figure 4, the details of the gradation correction process in step S304 of Figure 3 will be explained. In step S401, the image processing unit 108 generates an image histogram (luminance histogram). The image processing unit 108 detects the luminance of the image data captured in step S303 of Figure 3 and generates a histogram. Figure 5 shows an example of the generated histogram. The horizontal axis represents luminance, and the vertical axis represents the number of pixels with the luminance shown on the horizontal axis. In an 8-bit image, the luminance on the horizontal axis ranges from 0 to 255.

[0041] When generating a histogram, the range for detecting brightness is the entire image data, but it may also be the area excluding the edges (periphery) of the image. The reason for generating the histogram from the area excluding the edges of the image is that brightness tends to be lower in the area around the image due to lens vignetting. In the area around the image, brightness may be shifted to the low-brightness side and may not be detected properly, so the histogram is generated from the area excluding the edges of the image. This is preferable. Furthermore, it is preferable not to include the area surrounding the image in the detection range, as there may be no subject present, and if there is no subject, the user will not focus on it.

[0042] In step S402, the image processing unit 108 acquires histogram features. Histogram features are explained with reference to Figure 5. Histogram features are values ​​that represent the characteristics of the high-luminance peak Mt in the histogram. The histogram features include the starting position Ms of the peak Mt, the ending position Me of the peak Mt, and the position of the reference point Ref, which serves as the reference for the peak Mt. The image processing unit 108 controls the luminance at the reference point Ref so that it does not change due to gradation correction. The luminance at the reference point Ref is also called the reference luminance.

[0043] The image processing unit 108 acquires the peak portion Mt on the high-brightness side of the histogram generated in step S401 as a high-brightness region. The peak portion Mt can be acquired, for example, by searching the histogram from the high-brightness side and including a range in which the increase in the number of pixels continues above a predetermined threshold and a range in which the decrease in the number of pixels continues above a predetermined threshold.

[0044] The starting position Ms of the peak Mt can be, for example, the position on the low-luminance side of the peak where the number of pixels changes from decreasing to increasing when the histogram is searched from the high-luminance side. The ending position Me of the peak Mt can be, for example, the position on the high-luminance side of the peak where the increase in the number of pixels begins when the histogram is searched from the high-luminance side. Note that the peak Mt may be obtained by other methods. For example, the peak Mt may be the portion (luminance range) of the high-luminance peak of the histogram where the number of pixels is greater than or equal to a predetermined number.

[0045] Furthermore, the image processing unit 108 sets a reference point Ref, which is a point in the image where the brightness does not fluctuate within the brightness range of the peak portion Mt. In Figure 5, the reference point Ref is set at the position of the peak (maximum value) of the peak portion Mt. The reference point Ref is not limited to the position of the peak portion Mt, but may also be the centroid position of the region corresponding to the peak portion Mt in the histogram. The region corresponding to the peak portion Mt is, for example, the region enclosed by the histogram of the peak portion Mt, the horizontal axis (brightness), a straight line parallel to the vertical axis at the starting position Ms, and a straight line parallel to the vertical axis at the ending position Me. Alternatively, the reference point Ref may be the position of the representative brightness of the high-brightness region, for example, the average brightness of the brightness of the peak portion Mt.

[0046] Furthermore, the histogram features used to specify the peak portion Mt may be specified by the user via the operation unit 116. The user can specify the peak portion Mt by selecting the range of the peak portion Mt in the histogram, or by specifying the start position Ms, the end position Me, and the reference point Ref.

[0047] In step S403, the image processing unit 108 determines the correction target value. The correction target value is the width (extension) of the brightness range of the high-brightness region after stretching the high-brightness region in the horizontal axis (brightness) direction in the histogram in order to increase the contrast of the high-brightness region.

[0048] In highlight-weighted metering mode, the tonal range of the image is compressed, and the contrast in high-luminance areas (peaks Mt) may decrease. Therefore, the image processing unit 108 increases the contrast in high-luminance areas by stretching the peaks Mt of the histogram in the horizontal direction. The image processing unit 108 determines how much to stretch the high-luminance peaks Mt of the histogram generated in step S402 in the horizontal direction, that is, the width of the stretched peaks Mt, as the correction target value. Based on the determined correction target value, the image processing unit 108 determines the start and end positions of the stretched peaks Mt.

[0049] The correction target value, which is the width of the elongated peak portion Mt (the width of the elongated luminance range), is predetermined in a lookup table, for example, according to the width of the luminance range of the high-luminance region before elongation. Figure 6 is an example of a lookup table that determines the width of the peak portion Mt in the histogram. The lookup table in Figure 6 associates the width of the peak portion Mt before stretching with the width after stretching.

[0050] In an 8-bit image, if the width of the peak portion Mt before stretching is 10 counts, the gradation may be compressed and the contrast may be reduced. Therefore, in the lookup table in Figure 6, the width after stretching is set to 30 counts. On the other hand, if the width of the peak portion Mt before stretching is 50 counts, the gradation is sufficient, so the width after stretching remains unchanged at 50 counts, and the histogram is not stretched.

[0051] Referring to Figure 7, the histograms before and after stretching of the peak portion Mt will be explained. Figure 7(A) shows an example of the histogram before stretching. Figure 7(B) shows an example of the histogram after stretching. The width of the peak portion Mt is stretched so that the luminance at the reference point Ref (reference luminance) does not change. The width W1 of the peak portion Mt before stretching is stretched to W2 in the histogram after stretching, with the reference point Ref as the center of the peak portion Mt.

[0052] The stretched width W2 of the peak portion Mt (the width of the stretched brightness range) is not limited to being determined by referring to a lookup table, but may also be determined using a function (stretched width determination function) that uniquely determines the rate at which it changes according to the width of the peak portion Mt before stretching. Let W1 be the width of the peak portion Mt before stretching, and Rate(W1)(≧1) be the stretched width determination function, then the stretched width W2 of the peak portion Mt is expressed as shown in (Equation 1) below.

number

[0053] Whether or not to stretch the peaks may be determined by setting a threshold in advance for the width of the peak portion Mt. For example, the image processing unit 108 determines not to stretch the width of the peak portion Mt if the width of the peak portion Mt is greater than or equal to the threshold. The threshold for determining whether or not to stretch the peaks may be a unique value and may be changed according to the target brightness TH of the highlight portion specified by the user in step S301. For example, when the target brightness is TH1, the target brightness is compressed more than when the target brightness is TH3, so the threshold when the target brightness is TH1 may be smaller than the threshold when the target brightness is TH3.

[0054] Furthermore, the image processing unit 108 may not extend the width of the peak portion Mt, not only if the cumulative number of pixels included in the peak portion Mt or the ratio of the cumulative number of pixels is below a threshold. For example, the image processing unit 108 may not extend the width of the peak portion Mt if the ratio of the cumulative number of pixels in the peak portion Mt to the total number of pixels is 5% or less.

[0055] Furthermore, when the image processing unit 108 stretches the width of the peak portion Mt, it may stretch it so that the brightness of the end position Me after stretching (the highest brightness in the brightness range of the peak portion Mt) does not exceed the target brightness TH. Also, the image processing unit 108 may change the degree of stretching of the peak portion Mt based on the relationship between the target brightness TH and the reference point Ref determined and set in step S402. For example, if the difference in brightness between the target brightness TH and the reference point Ref is less than or equal to a predetermined threshold (e.g., 10 counts), the image processing unit 108 can reduce the degree of stretching of the peak portion Mt by bringing Rate(W1) in (Equation 1) closer to 1. By reducing the degree of stretching, the gradation correction is reduced. Reducing the degree of stretching includes the case where Rate(W1) is set to 1 and the width of the peak portion Mt is not stretched.

[0056] The starting position Ms and ending position Me of the elongated peak portion Mt are determined, for example, based on the elongated width (correction target value) of the peak portion Mt and the position of the reference point Ref. If the low-luminance side is on the left and the high-luminance side is on the right, the starting position Ms and ending position Me are determined, for example, so that they are symmetrical with respect to the luminance at the reference point Ref. That is, the luminance of the starting position Ms after elongation ( The minimum brightness and the brightness at the end position Me after extension (maximum brightness) are determined such that the brightness at the reference point Ref (reference brightness) is the midpoint.

[0057] Furthermore, the starting position Ms and ending position Me may be determined asymmetrically, with the low-luminance side on the left and the high-luminance side on the right, by varying the degree of stretching on the low-luminance side and the high-luminance side, respectively, relative to the reference point Ref of the peak portion Mt. In other words, the luminance of the stretched starting position Ms (minimum luminance) and the luminance of the stretched ending position Me (maximum luminance) are determined to be different from the luminance at the reference point Ref (reference luminance). The image processing unit 108 may stretch one of the peak portion Mt's low-luminance or high-luminance side more strongly, or leave the other side unstretched.

[0058] Furthermore, the image processing unit 108 may determine the start position Ms and end position Me based on the target brightness TH. For example, the image processing unit 108 can set the end position Me of the peak portion Mt to coincide with the target brightness TH, and determine the start position Ms so that the width of the peak portion Mt after elongation becomes the corrected target value.

[0059] In step S404, the image processing unit 108 determines the tone curve based on the correction target value determined in step S403. Specifically, the image processing unit 108 determines the control points for changing the shape of the tone curve and the amount of correction for the output brightness at the control points. In the following description, the reference point acquired in step S402 is Ref, the start position of the peak portion Mt before stretching is Ms1, and the end position is Me1. Also, the start position of the peak portion Mt after stretching, determined in step S403, is Ms2, and the end position is Me2.

[0060] Referring to Figure 8, the tone curve corresponding to the histogram after stretching of the peak portion Mt will be explained. The horizontal axis of the tone curve is the input luminance, and the vertical axis is the output luminance. Figure 8(A) shows an example of a polylinear tone curve connecting three control points: the start position (Ms1, Ms2), the reference point (Ref, Ref), and the end position (Me1, Me2) of the peak portion Mt.

[0061] Figure 8(B) shows an example of a curved tone curve obtained by spline interpolation connecting (0,0), (Ms1,Ms2), (Ref,Ref), (Me1,Me2), and (255,255). The tone curve is determined to be monotonically increasing to prevent tone inversion.

[0062] Figure 8(C) shows an example of a polylinear tone curve in which control points that do not change brightness are added in addition to the reference point Ref. The added control points are, for example, points on the lower brightness side of the starting position Ms2 after the extension of the peak portion Mt (Y_low, Y_low), and points on the higher brightness side of the ending position Me2 after the extension of the peak portion Mt (Y_high, Y_high). By adding control points, the gradations from 0 to Y_low and from Y_high to 255 in the brightness range other than the peak portion Mt can be maintained.

[0063] Y_low should be set to a lower brightness level than the starting position Ms2 after the extension of the peak portion Mt. Alternatively, Y_low may be set to a lower brightness level than the peak portion detected in the histogram on the low brightness side.

[0064] Y_high should be set to a higher brightness than the end position Me2 after the extension of the peak portion Mt. Y_high may be set according to the target brightness TH, and may be set to the same brightness as the target brightness TH, or to a brightness that is a predetermined number of counts greater than the target brightness TH.

[0065] In step S405, the image processing unit 108 applies the tone curve determined in step 404 to the image. The tone curve determined in step S404 is a function f: x → y (0 ≤ x, y ≤ 25 for 8-bit images) that associates the output luminance value y with the input luminance value x. 5) The image processing unit 108 converts the grayscale of the image using function f.

[0066] <Regarding the corrected target values> The correction target value is determined by the method described in step S403 of Figure 4, but may be further modified by taking into account the target luminance TH or the luminance of the main subject. Two variations of the method for determining the correction target value are described with reference to Figures 9 to 11.

[0067] (Method for determining the corrected target value 1) Method 1 for determining the correction target value is to change the elongation rate (degree of elongation) of the width of the peak portion Mt, taking into account the target brightness TH. In highlight-weighted metering, where the user can select the target brightness TH for the highlight area, the contrast will be lower when the user selects TH1 as the target brightness TH than when they select TH3, which is brighter than TH1.

[0068] If the correction target value is the same for both the target brightness TH (TH1) and TH3, the contrast may improve when the target brightness TH is TH3, but when the target brightness TH is TH1, the stretching may be insufficient, and the contrast may not be as high as the user intended. In determination method 1, the image processing unit 108 determines the correction target value by changing the histogram stretching rate based on the target brightness TH selected by the user. The image processing unit 108 increases the stretching rate of the peak portion Mt of the histogram as the target brightness TH decreases, thereby enabling contrast adjustment according to the target brightness TH.

[0069] Referring to Figure 9, the determination of the correction target value according to the target brightness TH will be explained in detail. The correction target value is changed using an elongation rate corresponding to the target brightness TH. The following examples illustrate how to determine the correction target value when the target brightness TH is 200 and 150 (both 8 bits).

[0070] Figure 9(A) shows an example of histogram stretching when the target brightness TH is 200. The brightness range of the high-brightness peak Mt before stretching is 150-170, and the width of the peak Mt is 20 counts. According to the lookup table shown in Figure 6, the width of the peak Mt after stretching is 40 counts. The image processing unit 108 stretches the width of the peak Mt to 40 counts while maintaining the brightness at the reference point Ref(160). The brightness range of the peak Mt after stretching becomes 140-180.

[0071] Figure 9(B) shows an example of histogram stretching when the target brightness TH is 150. The brightness range of the high-brightness peak Mt before stretching is 100-120, and the width of the peak Mt is 20 counts, the same as when the target brightness TH is 200. According to the lookup table shown in Figure 6, the width of the peak Mt after stretching is 40 counts. When the target brightness TH is 150, the gradation is more compressed than when the target brightness TH is 200, so stretching to the same width as when the target brightness TH is 200 may not improve the contrast. For this reason, in the example in Figure 9(B), the image processing unit 108 stretches the width of the peak Mt to 60 counts while maintaining the brightness at the reference point Ref(110). The brightness range of the peak Mt after stretching is 80-140.

[0072] Let S be the elongation rate of the peak portion Mt considering the target brightness TH, and W2 be the width of the peak portion Mt before considering the target brightness TH. W2 is the corrected target value determined in step S403. The width Ws of the peak portion Mt in the histogram after considering the target brightness TH is expressed as (Equation 2).

number

[0073] In the examples in Figure 9(A) and Figure 9(B), W2 is 40 counts in both cases. Figure 9(A) In Figure 9(B), when the target brightness TH is 200, the elongation rate S in (Equation 2) is 1.0, and the width Ws of the elongated peak Mt is 1.0 × 40 = 40 counts. In Figure 9(B), when the target brightness TH is 150, the elongation rate S in (Equation 2) is 1.5, and the width Ws of the elongated peak Mt is 1.5 × 40 = 60 counts.

[0074] The elongation rate S with respect to the target brightness TH is expressed as shown in (Equation 3) using the maximum brightness THmax and minimum brightness THmin of the target brightness that the user can specify, and the elongation rate Smax with respect to the maximum brightness THmax and the elongation rate Smin with respect to the minimum brightness THmin.

number

[0075] The image processing unit 108 may change the elongation rate S based on the relationship between the target brightness TH and the reference point Ref determined and set in step S402. For example, if the difference in brightness between the target brightness TH and the reference point Ref is less than or equal to a predetermined count (e.g., 10 counts), the image processing unit 108 can reduce the degree to which the peak portion Mt is elongated by bringing the elongation rate S in (Equation 2) closer to 1.

[0076] (Method for determining the corrected target value 2) Method 2 for determining the correction target value involves changing the elongation rate (degree of elongation) of the width of the mountain portion Mt, taking into account the brightness of the main subject. When shooting with highlight-weighted metering, it is difficult to determine whether the user expects improved contrast in high-brightness areas. The digital camera 100 can provide a suitable gradation correction for the main subject by combining highlight-weighted metering with main subject detection processing.

[0077] Referring to Figure 10, the process of determining a correction target value that takes into account the brightness of the main subject and performing gradation correction based on the correction target value will be explained. Figure 10 is a flowchart illustrating the gradation correction process that takes into account the brightness of the main subject. The same reference numerals are used for the same processes as in Figure 4, and detailed explanations are omitted. The image processing unit 108 generates an image histogram in step S401 and acquires the features of the histogram in step S402.

[0078] In step S1003, the image processing unit 108 determines whether or not a main subject has been detected in the image. The main subject can be, for example, a person, a person's face, an animal, or a plant. If a main subject is detected, the process proceeds to step S1006; otherwise, the process proceeds to step S403.

[0079] In step S1006, the image processing unit 108 determines whether the main subject has been detected in the high-brightness region, that is, whether the brightness of the main subject overlaps with the high-brightness peak Mt of the histogram. Whether the brightness of the main subject overlaps with the high-brightness peak Mt of the histogram can be determined, for example, by whether the difference between the brightness of the main subject and the brightness of the peak Mt or reference point Ref is ±5 counts or less. The brightness of the main subject may be, for example, the average brightness or the highest brightness within the region of the main subject. The image processing unit 108 determines that the brightness of the main subject overlaps with the high-brightness peak Mt of the histogram if the difference is ±5 counts or less.

[0080] As another determination method, the image processing unit 108 generates a histogram (luminance histogram) of the main subject, and if a predetermined percentage (for example, 70%) or more of the histogram of the main subject is... This can be determined by whether or not it is included in the peak portion Mt. The image processing unit 108 determines that the histogram of the main subject overlaps with the high-luminance peak portion Mt if a predetermined proportion or more of the histogram of the main subject is included in the peak portion Mt.

[0081] If the histogram of the main subject overlaps with the high-luminance peak Mt, the process proceeds to step S1007. If the histogram of the main subject does not overlap with the high-luminance peak Mt, the process proceeds to step S1008.

[0082] In step S1007, the image processing unit 108 determines the correction target value. The image processing unit 108 obtains a histogram of the main subject, and if it determines that the main subject has low contrast, it determines the correction target value in the same way as in step S403. On the other hand, if the image processing unit 108 determines that the main subject has high contrast, it either lowers the correction target value determined in step S403 to reduce the degree of stretching, or does not correct the width of the peak portion Mt.

[0083] The contrast of the main subject can be determined, for example, by the standard deviation of the histogram of the main subject. The image processing unit 108 sets a first threshold for the standard deviation, and if the standard deviation is less than the first threshold, it determines that the main subject has low contrast. If the image processing unit 108 determines that the contrast is low, it uses the elongation range determination function Rate(W1) defined in step S403 to determine the correction target value according to (Equation 1) and performs gradation correction.

[0084] On the other hand, if the standard deviation is greater than or equal to the first threshold, the image processing unit 108 determines that the main subject has high contrast. If the image processing unit 108 determines that the main subject has high contrast, it reduces the degree of stretching of the brightness range of the peak portion Mt. Reducing the degree of stretching includes cases where no stretching occurs. The image processing unit 108 can determine the correction target value using (Equation 1) with Rate(W1), which is defined to approach 1 as the difference between the standard deviation and the first threshold (standard deviation - first threshold) increases. Furthermore, if the standard deviation reaches a second threshold which is greater than the first threshold, the image processing unit 108 determines that the contrast of the main subject is sufficient and sets Rate(W1)=1, so no stretching is performed.

[0085] In step S1007, the contrast of the main subject may be determined not only by the standard deviation, but also by whether the width of the peak portion of the histogram of the main subject is below a predetermined threshold. For example, the image processing unit 108 can determine that the contrast is low if the width of the peak portion of the histogram of the main subject is 20 counts or less, and that the contrast is high if it is greater than 20 counts.

[0086] In step S1008, the histogram of the main subject does not overlap with the high-luminance peak Mt, resulting in a lower contrast for the main subject, which may not be what the user intended. Therefore, in step S1008, the image processing unit 108 extends the luminance range of the high-luminance region through the processing in step S403, and also extends the peak portion of the luminance histogram of the main subject.

[0087] The image processing unit 108 can determine a correction target value for the width of the peak portion of the main subject in the same manner as in step S403, and extend the peak portion of the main subject. A lookup table that defines the width of the extended peak portion (width of the extended brightness range) and a threshold for determining whether or not to extend the peak portion may be set separately from the peak portion Mt of the high-brightness region for the histogram of the main subject. In addition, the method for determining the start and end positions of the peak portion of the histogram of the main subject may differ from that for the peak portion Mt of the high-brightness region.

[0088] Furthermore, to avoid affecting the peak portion Mt of the high-brightness region, the histogram is set such that (brightness at the end of the peak portion of the main subject) ≤ (brightness at the start of the peak portion Mt of the high-brightness region). Make it so that it becomes ).

[0089] Figure 11 shows an example of histograms before and after stretching, considering the brightness of the main subject when the main subject does not overlap with the high-brightness region. Figure 11(A) shows the histogram before stretching, where the brightness range of 50 to 70 is the peak portion including the brightness of the main subject. The brightness range of 120 to 140 represents the peak portion Mt of the high-brightness region. Figure 11(B) shows the histogram after stretching, where the peak portion including the brightness of the main subject has been stretched to the brightness range of 40 to 80. On the other hand, the peak portion Mt of the high-brightness region has been stretched to 100 to 160 by the processing in step S403. In this way, the image processing unit 108 stretches the histogram while maintaining the relationship (brightness at the end position of the peak portion of the main subject = 80) ≤ (brightness at the start position of the peak portion Mt of the high-brightness region = 100).

[0090] As another example, if the brightness range of the peak portion of the main subject after stretching in Figure 11(B) is 70 to 110, the relationship (brightness at the end of the peak portion of the main subject = 110) ≤ (brightness at the start of the peak portion Mt in the high-brightness area = 100) will no longer be maintained. In this case, the image processing unit 108 can reduce the degree of stretching by setting the brightness range of the peak portion Mt in the high-brightness area to 110 to 160. When the brightness range of the peak portion of the main subject after stretching and the brightness range of the peak portion Mt in the high-brightness area overlap, the image processing unit 108 can appropriately correct the contrast of the main subject by reducing the degree of stretching of the peak portion Mt in the high-brightness area.

[0091] If the main subject is not detected in step S1003, i.e., in step S403, there is no object for which the user intentionally determines the contrast, and in highlight-weighted metering mode, the entire screen may have low contrast. In this case, the image processing unit 108 can appropriately correct the image contrast by determining the correction target value in the same way as in step S403 in Figure 4.

[0092] According to the above embodiment, the digital camera 100 can improve the contrast in high-brightness areas as intended by the user, even when the tonal range is compressed and the image becomes darker in highlight-weighted metering mode compared to other metering modes.

[0093] <Other Embodiments> The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0094] This embodiment includes the following configurations, methods, and programs. (Composition 1) An exposure control means that controls the exposure so that the representative brightness of the high-brightness region of the image becomes a target brightness set by the user, A tone correction means that performs tone correction to increase the contrast of the high-brightness region by extending the brightness range of the high-brightness region of the image captured after the exposure control described above has been performed. It has, The gradation correction means changes the degree of extension of the luminance range of the high-luminance region based on the target luminance. An imaging device characterized by the following features. (Configuration 2) The gradation correction means determines the width of the extended luminance range according to the width of the luminance range of the high-luminance region, and multiplies the width of the extended luminance range by an extension rate corresponding to the target luminance. This determines the extent of the extended luminance range of the high-luminance region. The imaging apparatus according to configuration 1, characterized by the features described above. (Composition 3) The extent of the extended luminance range is determined using a lookup table that associates the extent of the luminance range of the high-luminance region with the extent of the extended luminance range, or using a predetermined function that calculates the extent of the extended luminance range corresponding to the extent of the luminance range of the high-luminance region. The imaging apparatus according to configuration 2, characterized in that... (Composition 4) The aforementioned gradation correction means performs the gradation correction so as not to change the luminance in the reference luminance within the luminance range of the high-luminance region. An imaging device according to any one of configurations 1 to 3, characterized by the above. (Composition 5) The reference luminance is the luminance at the maximum value of the peak portion corresponding to the high-luminance region in the luminance histogram of the image, the luminance at the centroid position of the region corresponding to the peak portion of the luminance histogram, or the representative luminance of the high-luminance region. The imaging apparatus according to configuration 4, characterized by the features described above. (Composition 6) The aforementioned reference brightness is specified by the user. The imaging apparatus according to configuration 4 or 5, characterized by the features described herein. (Composition 7) In the luminance histogram of the aforementioned image, the minimum and maximum luminances after the extension of the luminance range of the high-luminance region are determined such that the reference luminance is the midpoint. An imaging device according to any one of configurations 4 to 6, characterized by the above. (Composition 8) In the brightness histogram of the aforementioned image, the minimum and maximum brightness levels after the extension of the brightness range of the high-brightness region are determined such that the difference from the reference brightness is different between the lower and higher brightness sides by weighting the degree of extension compared to the reference brightness. An imaging device according to any one of configurations 4 to 6, characterized by the above. (Composition 9) The gradation correction means reduces the degree of extension of the luminance range of the high-luminance region when the difference between the target luminance and the reference luminance is less than or equal to a predetermined threshold. An imaging device according to any one of configurations 4 to 8, characterized by the above. (Composition 10) In the brightness histogram of the aforementioned image, the brightness range of the high-brightness region is extended such that the highest brightness after the extension of the brightness range of the high-brightness region does not exceed the target brightness. An imaging apparatus according to any one of configurations 1 to 9, characterized by the above. (Composition 11) The gradation correction means detects the main subject in the high-brightness region of the image, If the standard deviation of the luminance histogram of the main subject is less than the threshold, the tone correction is performed. If the standard deviation of the luminance histogram of the main subject is greater than or equal to the threshold, the degree of extension of the luminance range of the high-luminance region is reduced. An imaging apparatus according to any one of configurations 1 to 10, characterized by the features described above. (Composition 12) An exposure control means that controls the exposure so that the representative brightness of the high-brightness region of the image becomes a target brightness set by the user, A tone correction means that performs tone correction to increase the contrast of the high-brightness region by extending the brightness range of the high-brightness region of the image captured after the exposure control described above has been performed. It has, When the main subject is detected in an area of ​​the image with a lower brightness than the high-brightness area, the gradation correction means extends the brightness range of the high-brightness area and also extends the brightness range of the area in which the main subject is detected. An imaging device characterized by the following features. (Composition 13) The tone correction means determines the width of the extended luminance range of the luminance range of the area where the main subject is detected, according to the width of the luminance range of the area where the main subject is detected. The imaging apparatus according to configuration 12, characterized in that... (Composition 14) The tone correction means extends the brightness range of the high-brightness region and the brightness range of the region where the main subject is detected so that the brightness range of the high-brightness region after extension and the brightness range of the region where the main subject is detected after extension do not overlap. The imaging apparatus according to configuration 12 or 13, characterized by the above. (Composition 15) The high-brightness region is specified by the user. An imaging device according to any one of configurations 1 to 14, characterized by the features described above. (Composition 16) The representative brightness is the average, maximum, minimum, median, or mode of the brightness of the pixels included in the high-brightness region. An imaging apparatus according to any one of configurations 1 to 15, characterized by the features described herein. (Method 1) An exposure control step that controls the exposure so that the representative brightness of the high-brightness region of the image becomes the target brightness set by the user, A tone correction step is performed to increase the contrast of the high-brightness region by extending the brightness range of the high-brightness region of the image captured after the exposure control described above has been performed. It has, In the aforementioned gradation correction step, the degree of extension of the luminance range of the high-luminance region is changed based on the target luminance. A control method for an imaging device, characterized by the following: (Method 2) An exposure control step that controls the exposure so that the representative brightness of the high-brightness region of the image becomes the target brightness set by the user, A tone correction step is performed to increase the contrast of the high-brightness region by extending the brightness range of the high-brightness region of the image captured after the exposure control described above has been performed. It has, In the tone correction step, if the main subject is detected in an area of ​​the image with a lower brightness than the high-brightness area, the brightness range of the high-brightness area is extended, and the brightness range of the area in which the main subject was detected is also extended. A control method for an imaging device, characterized by the following: (program) A program for causing a computer to perform each step of the method for controlling an imaging device as described in Method 1 or 2. [Explanation of symbols]

[0095] 100: Imaging device (digital camera), 103: AE processing unit, 108: Image processing unit, 109: Image recognition unit, 113: System control unit

Claims

1. An exposure control means that controls the exposure so that the representative brightness of the high-brightness region of the image becomes a target brightness set by the user, A tone correction means that performs tone correction to increase the contrast of the high-brightness region by extending the brightness range of the high-brightness region of the image captured after the exposure control described above has been performed. It has, The gradation correction means determines the width of the extended luminance range according to the width of the luminance range of the high-luminance region, and determines the width of the luminance range of the high-luminance region after extension by multiplying the width of the extended luminance range by an extension rate corresponding to the target luminance. An imaging device characterized by the following features.

2. The elongation rate is set to be higher as the target brightness decreases. The imaging apparatus according to feature 1.

3. The extent of the extended luminance range is determined using a lookup table that associates the extent of the luminance range of the high-luminance region with the extent of the extended luminance range, or using a predetermined function that calculates the extent of the extended luminance range corresponding to the extent of the luminance range of the high-luminance region. The imaging device according to feature 2.

4. The tone correction means performs tone correction so as not to change the brightness at the reference brightness within the brightness range of the high-brightness region. The imaging apparatus according to any one of claims 1 to 3.

5. The reference luminance is the luminance at the maximum value of the peak portion corresponding to the high-luminance region in the luminance histogram of the image, the luminance at the centroid position of the region corresponding to the peak portion of the luminance histogram, or the representative luminance of the high-luminance region. The imaging apparatus according to feature 4.

6. The aforementioned reference brightness is specified by the user. The imaging apparatus according to feature 4.

7. In the luminance histogram of the aforementioned image, the minimum and maximum luminances after the extension of the luminance range of the high-luminance region are determined such that the reference luminance is the midpoint. The imaging apparatus according to feature 4.

8. In the brightness histogram of the aforementioned image, the minimum and maximum brightness levels after the extension of the brightness range of the high-brightness region are determined such that the difference from the reference brightness is different between the lower and higher brightness sides by weighting the degree of extension compared to the reference brightness. The imaging apparatus according to feature 4.

9. The gradation correction means reduces the degree of extension of the luminance range of the high-luminance region when the difference between the target luminance and the reference luminance is less than or equal to a predetermined threshold. The imaging apparatus according to feature 4.

10. In the brightness histogram of the aforementioned image, the brightness range of the high-brightness region is extended such that the highest brightness after the extension of the brightness range of the high-brightness region does not exceed the target brightness. The imaging apparatus according to any one of claims 1 to 3.

11. The gradation correction means detects the main subject in the high-brightness region of the image, If the standard deviation of the luminance histogram of the main subject is less than the threshold, the tone correction is performed. If the standard deviation of the luminance histogram of the main subject is greater than or equal to the threshold, the degree of extension of the luminance range of the high-luminance region is reduced. The imaging apparatus according to any one of claims 1 to 3.

12. An exposure control means that controls the exposure so that the representative brightness of the high-brightness region of the image becomes a target brightness set by the user, A tone correction means that performs tone correction to increase the contrast of the high-brightness region by extending the brightness range of the high-brightness region of the image captured after the exposure control described above has been performed. It has, When the main subject is detected in an area of ​​the image with a lower brightness than the high-brightness area, the gradation correction means extends the brightness range of the high-brightness area and also extends the brightness range of the area where the main subject is detected. An imaging device characterized by the following features.

13. The tone correction means determines the width of the extended luminance range of the luminance range of the area in which the main subject is detected, according to the width of the luminance range of the area in which the main subject is detected. The imaging apparatus according to claim 12, characterized in that

14. The tone correction means extends the brightness range of the high-brightness region and the brightness range of the region where the main subject is detected so that the brightness range of the high-brightness region after extension and the brightness range of the region where the main subject is detected after extension do not overlap. The imaging apparatus according to claim 12 or 13.

15. The high-brightness region is specified by the user. The imaging apparatus according to any one of claims 1 to 3.

16. The representative brightness is the average, maximum, minimum, median, or mode of the brightness of the pixels included in the high-brightness region. The imaging apparatus according to any one of claims 1 to 3.

17. An exposure control step that controls the exposure so that the representative brightness of the high-brightness region of the image becomes the target brightness set by the user, A tone correction step is performed to increase the contrast of the high-brightness region by extending the brightness range of the high-brightness region of the image captured after the exposure control described above has been performed. It has, In the aforementioned gradation correction step, the width of the extended luminance range is determined according to the width of the luminance range of the high-luminance region, and the width of the extended luminance range of the high-luminance region after extension is determined by multiplying the width of the extended luminance range by an extension rate corresponding to the target luminance. A control method for an imaging device, characterized by the following:

18. An exposure control step that controls the exposure so that the representative brightness of the high-brightness region of the image becomes the target brightness set by the user, A tone correction step is performed to increase the contrast of the high-brightness region by extending the brightness range of the high-brightness region of the image captured after the exposure control described above has been performed. It has, In the tone correction step, if the main subject is detected in an area of ​​the image with a lower brightness than the high-brightness area, the brightness range of the high-brightness area is extended, and the brightness range of the area in which the main subject was detected is also extended. A control method for an imaging device, characterized by the following:

19. A program for causing a computer to perform each step of the method for controlling an imaging device according to claim 17 or 18.

Images