Exposure control apparatus, method of operating the exposure control apparatus, program for operating the exposure control apparatus, focus control apparatus, and imaging apparatus

Abstract

To provide an exposure control apparatus, a method of operating the exposure control apparatus, a program for operating the exposure control apparatus, a focus control apparatus, and an imaging apparatus which can improve the accuracy of focusing.SOLUTION: The imaging apparatus includes a control unit serving as an exposure control apparatus for controlling exposure of an image sensor including normal pixels for capturing a subject and phase-difference detection pixels for detecting phase difference. A CPU of the control unit functions as a second exposure-condition setting unit. The second exposure-condition setting unit performs exposure correction processing according to a difference between the maximum luminance value of the phase-difference detection pixels within a focus-adjustment area and a saturation level.SELECTED DRAWING: Figure 11

Description

[Technical Field]

[0001] The technology of the present disclosure relates to an exposure control device, an operating method for an exposure control device, an operating program for an exposure control device, a focus control device, and an imaging device.

[0002] The imaging device described in Patent Document 1 includes an imaging element, focus detection means, saturated area detection means, and exposure control means. The imaging element has a plurality of imaging pixels, at least some of which are configured with pixels having focus detection functions. The focus detection means detects the amount of defocus using signals acquired from pixels having focus detection functions in a plurality of focus detection regions provided on the imaging element. The saturated area detection means detects the number of regions in which signals are saturated among the plurality of focus detection regions as a saturated area detection value. The exposure control means controls exposure when acquiring signals from pixels having focus detection functions, using the saturated area detection value detected by the saturated area detection means. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2014-178931 Summary of the Invention

[0004] One embodiment of the technique of the present disclosure provides an exposure control device, an operating method for an exposure control device, an operating program for an exposure control device, a focus control device, and an imaging device that can improve focusing accuracy. [Means for solving the problem]

[0005] The exposure control device disclosed herein is an exposure control device that controls the exposure of an imaging element that includes normal pixels for imaging a subject and phase difference detection pixels for detecting a phase difference, and is equipped with a processor that performs exposure correction processing according to the difference between the maximum luminance value and the saturation level of the phase difference detection pixels within a set area.

[0006] As the exposure correction process, the processor preferably calculates an exposure correction amount based on the difference and exposes the phase difference detection pixels based on the exposure correction amount.

[0007] The saturation level is preferably set to an upper limit value at which the maximum luminance value of the phase difference detection pixels in the set region does not saturate.

[0008] Preferably, the processor varies the saturation level depending on the subject.

[0009] Preferably, the processor varies the saturation level depending on the object class determined by the machine learning model.

[0010] It is preferable that the phase difference detection pixels are provided with filters of a specific color, and that the processor sets the saturation level higher when the proportion of subjects of the specific color is equal to or greater than a predetermined first threshold value than when the proportion of subjects of the specific color is less than the first threshold value.

[0011] It is preferable that the processor calculates the maximum luminance value of the phase difference detection pixels from the maximum luminance value of the normal pixels in the set region when the subject is imaged with the normal pixels at the set exposure.

[0012] It is preferable that the maximum luminance value of a normal pixel is determined by comparing the luminance value of a single normal pixel within the set region, the average luminance value of multiple adjacent or nearby normal pixels within the set region, or the maximum luminance value of multiple adjacent or nearby normal pixels within the set region.

[0013] It is preferable that the processor calculates the maximum luminance value of the phase difference detection pixel by multiplying the maximum luminance value of the normal pixel by the sensitivity ratio between the normal pixel and the phase difference detection pixel.

[0014] The sensitivity ratio is preferably changed depending on the position of the normal pixel having the maximum luminance value, and also depending on the position of the set region.

[0015] The processor preferably changes the set region when the scene is backlit. More specifically, the processor preferably sets the set region narrower when the scene is backlit than when the scene is not backlit.

[0016] When there are two peaks in the distribution of the luminance values ​​of the normal pixels within the set region, or when there are multiple inflection points in the distribution, it is preferable that the processor adopts the maximum luminance value at the peak on the lower luminance side of the distribution peaks as the maximum luminance value of the normal pixels to be used when calculating the maximum luminance value of the phase difference detection pixels.

[0017] In the case of a backlit scene or a night scene, it is preferable that the processor adopts the maximum luminance value at the peak on the low luminance side as the maximum luminance value of the normal pixel to be used when calculating the maximum luminance value of the phase difference detection pixel.

[0018] It is preferable that the processor adopts the median value of the maximum luminance values ​​of the normal pixels obtained in multiple consecutive frames as the maximum luminance value of the normal pixels to be used when calculating the maximum luminance value of the phase difference detection pixels.

[0019] It is preferable that the processor detects normal pixels that have reached a saturation level within the set area, and if the proportion of normal pixels that have reached the saturation level is equal to or less than a predetermined second threshold, perform exposure correction processing according to the difference.

[0020] The processor determines whether the distribution of normal pixels that have reached the saturation level is sparse or dense, and if the proportion is greater than a second threshold and the distribution is determined to be sparse, performs exposure compensation processing according to the difference; if the proportion is greater than the second threshold and the distribution is determined to be dense, and the proportion of normal pixels determined to be densely distributed is less than a predetermined third threshold, performs exposure compensation processing according to a predetermined set exposure compensation amount; and if the proportion is greater than the second threshold and the distribution is determined to be dense, and the proportion of normal pixels determined to be densely distributed is equal to or greater than the third threshold, preferably does not perform exposure compensation processing.

[0021] As an exposure correction process, the processor preferably calculates an exposure correction amount based on the difference, exposes the phase difference detection pixels based on the exposure correction amount, and, if the exposure time of the exposure conditions corresponding to the exposure correction amount is longer than a predetermined threshold time, corrects the exposure conditions so that the exposure time is equal to or shorter than the threshold time.

[0022] As an exposure correction process, the processor preferably calculates an exposure correction amount based on the difference, exposes the phase difference detection pixels based on the exposure correction amount, and, in continuous shooting in which the phase difference detection pixels are exposed between image recordings, if the exposure time of the exposure conditions corresponding to the exposure correction amount is longer than a preset threshold time, corrects the exposure conditions so that the exposure time is equal to or shorter than the threshold time.

[0023] The method of operating an exposure control device disclosed herein is a method of operating an exposure control device that controls the exposure of an imaging element that includes normal pixels for imaging a subject and phase difference detection pixels for detecting a phase difference, and includes performing exposure correction processing according to the difference between the maximum luminance value and the saturation level of the phase difference detection pixels within a set area.

[0024] The operating program of the exposure control device disclosed herein is an operating program of the exposure control device that controls the exposure of an imaging element that includes normal pixels for imaging a subject and phase difference detection pixels for detecting a phase difference, and causes a computer to execute processing that includes performing exposure correction processing according to the difference between the maximum luminance value and the saturation level of the phase difference detection pixels within a set area.

[0025] The focus control device of the present disclosure comprises the exposure control device described above.

[0026] The imaging device of the present disclosure includes the exposure control device described above. [Brief explanation of the drawings]

[0027] [Figure 1] FIG. 1 is a diagram illustrating a configuration of an imaging device. [Figure 2]FIG. 2 is a diagram showing the arrangement of pixels of an imaging element. [Figure 3] FIG. 2 is a diagram illustrating a configuration of a normal pixel. [Figure 4] FIG. 2 is a diagram illustrating a configuration of a first phase difference detection pixel. [Figure 5] FIG. 10 is a diagram illustrating a configuration of a second phase difference detection pixel. [Figure 6] 10 is a graph showing a phase difference between a first calculation signal and a second calculation signal. [Figure 7] FIG. 2 is a block diagram showing a detailed configuration of a control unit. [Figure 8] FIG. 2 is a block diagram showing a processing unit of a CPU. [Figure 9] 10 is a flowchart showing a procedure for calculating a maximum luminance value of a phase difference detection pixel. [Figure 10] 10A and 10B are graphs showing the sensitivity ratio of a normal pixel and a phase difference detection pixel with respect to position, and diagrams showing the sensitivity ratio according to the position of a normal pixel having a maximum luminance value. [Figure 11] 10A and 10B are diagrams illustrating a state in which the maximum luminance value of a phase difference detection pixel is calculated from the maximum luminance value of a normal pixel, and an exposure correction amount. [Figure 12] 10 is a flowchart showing a processing procedure of a control unit. [Figure 13] 10 is a flowchart illustrating another example of the procedure for calculating the maximum luminance value of the phase difference detection pixels. [Figure 14] 10 is a flowchart illustrating yet another example of the procedure for calculating the maximum luminance value of the phase difference detection pixels. [Figure 15] FIG. 10 is a diagram showing a mode in which the median value of maximum luminance values ​​of normal pixels obtained in a plurality of consecutive frames is adopted. [Figure 16] FIG. 10 is a diagram illustrating a processing unit of a second embodiment. [Figure 17] FIG. 10 is a diagram showing the saturation level of a dark green subject and the saturation level of a non-dark green subject. [Figure 18] FIG. 10 is a diagram illustrating a processing unit of a third embodiment. [Figure 19] 10A and 10B are diagrams showing areas for determining maximum luminance values ​​when the scene is not backlit and when the scene is backlit. [Figure 20] 10A and 10B are diagrams showing how to obtain the maximum luminance value of a normal pixel, where (A) shows a case where the scene is not a backlit scene or a night scene, and (B) shows a case where the scene is a backlit scene or a night scene. [Figure 21] FIG. 10 is a block diagram showing a processing unit of a fourth embodiment. [Figure 22] 10A and 10B show a method for determining whether the distribution of saturated pixels is sparse or dense. FIG. 10A shows a case where the saturated pixels in the determination area centered on the reference saturated pixel are less than 80% and the distribution is determined to be sparse. FIG. 10B shows a case where the saturated pixels in the determination area centered on the reference saturated pixel are 80% or more and the distribution is determined to be dense. [Figure 23] FIG. 10 is a diagram illustrating another example of a method for determining whether the distribution of saturated pixels is sparse or dense. [Figure 24] 10 is a flowchart showing a processing procedure according to a fourth embodiment. [Figure 25] 10 is a diagram showing how to deal with a case where the exposure time according to the calculated exposure correction amount is longer by a first threshold time than the exposure time according to the set exposure correction amount. FIG. [Figure 26] 10 is a timing chart for continuous shooting in which exposure for focus adjustment is performed between image recordings. [Figure 27] FIG. 27 is a diagram showing how to deal with the case where the exposure time corresponding to the calculated exposure correction amount is longer than the second threshold time in the continuous shooting shown in FIG. 26. [Figure 28] 10 is a timing chart of continuous shooting before and after correcting the second exposure condition. [Figure 29] 10A and 10B are diagrams illustrating a manner in which the maximum luminance value of a phase difference detection pixel is corrected using the difference between the photometric value of a normal pixel and the photometric value of a phase difference detection pixel. DETAILED DESCRIPTION OF THE INVENTION

[0028] [First embodiment] As an example, as shown in FIG. 1, an imaging device 10 is, for example, a mirrorless single-lens digital camera. The imaging device 10 includes an imaging optical system 11 and an imaging element 12. The imaging optical system 11 has multiple types of lenses for focusing subject light onto the imaging element 12. Specifically, the imaging optical system 11 includes an objective lens 13, a focus lens 14, and a zoom lens 15. These lenses 13 to 15 are arranged in this order from the object side (subject side) toward the image-forming side (image-forming element 12 side). Although simplified in FIG. 1, each of the lenses 13 to 15 is actually a lens group made up of a combination of multiple lenses. The imaging optical system 11 also includes an aperture 16. The aperture 16 is arranged closest to the image-forming side of the imaging optical system 11. The imaging device 10 may be of a type in which a lens barrel incorporating the imaging optical system 11 and the like and a main body incorporating the imaging element 12 and the like are integrated, or may be of a so-called interchangeable lens type in which the lens barrel and the main body are separate.

[0029] Focus lens 14 is provided with a focus lens drive mechanism 17, zoom lens 15 is provided with a zoom lens drive mechanism 18, and diaphragm 16 is provided with an diaphragm drive mechanism 19. Focus lens drive mechanism 17 holds focus lens 14 and includes a focus cam ring with cam grooves formed on its outer periphery, a focus motor that moves the focus cam ring along the optical axis OA by rotating the focus cam ring about the optical axis OA, and a focus motor driver, etc. Zoom lens drive mechanism 18 similarly holds zoom lens 15 and includes a zoom cam ring with cam grooves formed on its outer periphery, a zoom motor that moves the zoom cam ring along the optical axis OA by rotating the zoom cam ring about the optical axis OA, and a zoom motor driver, etc. Diaphragm drive mechanism 19 includes a diaphragm motor that opens and closes the multiple diaphragm blades of diaphragm 16, and a diaphragm motor driver, etc.

[0030] The focus motor, zoom motor, and diaphragm motor are, for example, stepping motors. In this case, the positions of focus lens 14 and zoom lens 15 on the optical axis OA and the opening of diaphragm 16 can be derived from the drive amounts of the focus motor, zoom motor, and diaphragm motor. Note that instead of the drive amounts of the focus motor and zoom motor, position sensors may be provided to detect the positions of focus lens 14 and zoom lens 15.

[0031] Electrical components such as motors or drivers of each of the drive mechanisms 17 to 19 are connected to the control unit 20. The electrical components of each of the drive mechanisms 17 to 19 are driven under the control of the control unit 20. More specifically, the control unit 20 issues drive signals in response to instructions from the user input via the operation unit 21, thereby driving the electrical components of each of the drive mechanisms 17 to 19. For example, when an instruction to change the angle of view to the telephoto side is input via the angle of view change switch of the operation unit 21, the control unit 20 issues a drive signal to a driver of a zoom motor of the zoom lens drive mechanism 18, thereby moving the zoom lens 15 to the telephoto side.

[0032] The focus motor, zoom motor, and diaphragm motor output their drive amounts to the control unit 20. The control unit 20 derives the positions of the focus lens 14 and zoom lens 15 on the optical axis OA and the opening of the diaphragm 16 from the drive amounts.

[0033] The imaging element 12 is, for example, a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor, and has an imaging surface 42 (see FIG. 2) that captures light from a subject. The imaging element 12 is arranged so that the center of the imaging surface 42 coincides with the optical axis OA and the imaging surface 42 is perpendicular to the optical axis OA. Note that the terms "coincidence" and "orthogonality" used here refer to perfect coincidence and orthogonality, as well as coincidence and orthogonality that include errors generally accepted in the technical field to which the technology of the present disclosure pertains.

[0034] An imaging element driver 22 is connected to the imaging element 12. The imaging element driver 22 is connected to the control unit 20. Under the control of the control unit 20, the imaging element driver 22 supplies a vertical scanning signal and a horizontal scanning signal to the imaging element 12, thereby controlling the imaging element 12 to convert the object light into a signal. Furthermore, the image sensor driver 22 adjusts the gain applied to the image signal 43 (see FIG. 2) output from the image sensor 12, thereby changing the ISO (International Organization for Standardization) sensitivity of the resulting image.

[0035] A shutter 23 is provided between the imaging optical system 11 and the image sensor 12. The shutter 23 is, for example, a focal plane shutter having front and rear curtains. A shutter drive mechanism 24 is connected to the shutter 23. The shutter drive mechanism 24 includes an electromagnet that holds the front and rear curtains and releases the hold to cause the front and rear curtains to move, as well as a driver for the electromagnet. The shutter drive mechanism 24 is driven under the control of the control unit 20 to open and close the shutter 23.

[0036] The control unit 20 is connected to various units, such as an image input controller 25, an image memory 26, and an image processing unit 27, via a bus line 28. Other units connected to the bus line 28 include a VRAM (Video Random Access Memory) 29, a display control unit 30, a media controller 31, and an instruction receiving unit 32. Although not shown in the figure, the bus line 28 is also connected to a strobe drive control unit that controls the drive of a strobe device, an external communication I / F (Interface) that communicates with an external device via a connection terminal such as a USB (Universal Serial Bus) terminal, or a wireless communication I / F.

[0037] Image data obtained by capturing an image of the subject light is input from the imaging element 12 to the image input controller 25. The image input controller 25 outputs the image data to the image memory 26. The image memory 26 is, for example, a Synchronous Dynamic Random Access Memory (SDRAM), and temporarily stores the image data.

[0038] The image processing unit 27 reads unprocessed image data from the image memory 26. The image processing unit 27 performs various image processing on the image data. The various image processing includes, for example, offset correction processing, sensitivity correction processing, pixel interpolation processing, white balance correction processing, gamma correction processing, demosaic processing, luminance signal and color difference signal generation processing, edge enhancement processing, color correction processing, etc. The image processing unit 27 writes the image data after the various image processing back to the image memory 26.

[0039] The VRAM 29 receives image data from the image memory 26, which has undergone various image processing and is to be displayed as a live view image (also called a through image). The VRAM 29 has an area for storing image data for two consecutive frames. The image data stored in the VRAM 29 is sequentially overwritten with new image data. The VRAM 29 sequentially outputs the newer image data of the two consecutive frames of image data to the display control unit 30.

[0040] The display control unit 30 functions as a so-called video encoder, converting image data from the VRAM 29 into video data and outputting it to either the viewfinder monitor 33 or the rear monitor 34. This allows the user to view a live view image through either the viewfinder monitor 33 or the rear monitor 34. The display frame rate of the live view image is, for example, 60 fps (frames per second).

[0041] Whether the video data is output to the finder monitor 33 or the rear monitor 34 is determined, for example, as follows: A pupil detection sensor is provided in the finder. If the pupil detection sensor detects that the user is looking into the finder, the video data is output to the finder monitor 33. On the other hand, if the pupil detection sensor detects that the user is looking into the finder, the video data is output to the finder monitor 33. If it is detected that the user is not looking through the viewfinder, the video data is output to the rear monitor 34.

[0042] When an instruction to start shooting a still image or a moving image is issued by fully pressing the release button on the operation unit 21, the image processing unit 27 performs compression processing on the image data in the image memory 26. In the case of a still image, the image processing unit 27 performs compression processing on the image data in, for example, the JPEG (Joint Photographic Experts Group) format. In the case of a moving image, the image processing unit 27 performs compression processing on the image data in, for example, the MPEG (Moving Picture Experts Group) format. The image processing unit 27 outputs the compressed image data to the media controller 31.

[0043] The media controller 31 records the compressed image data from the image processing unit 27 on a memory card 35. The memory card 35 is detachably attached to a memory card slot (not shown).

[0044] When the image playback mode is selected via the mode selector switch of the operation unit 21, the media controller 31 reads image data from the memory card 35 and outputs it to the image processing unit 27. The image processing unit 27 performs decompression processing on the image data from the memory card 35. The image processing unit 27 outputs the decompressed image data to the display control unit 30. The display control unit 30 converts the image data into video data and outputs it to the rear monitor 34. This allows the user to view the played-back image on the rear monitor 34.

[0045] The instruction receiving unit 32 receives various operation instructions input by the user via the operation unit 21 and a touch panel 36 provided integrally with the rear monitor 34. The instruction receiving unit 32 outputs the received operation instructions to the control unit 20 via the bus line 28.

[0046] As described above, the operation unit 21 includes a field angle change switch, a release button, and a mode change switch. The release button is a two-stage push button that can be pressed halfway or all the way. Pressing the release button halfway issues an instruction to prepare for shooting a still image or video, and pressing it all the way issues an instruction to start shooting a still image or video. In addition to these, the operation unit 21 also includes a menu button for displaying various setting menus on the rear monitor 34, a cross key used for setting numerical values ​​and switching between options, and a confirmation button operated to confirm settings, etc. The touch panel 36 is superimposed on the display surface of the rear monitor 34. The touch panel 36 recognizes various operation instructions from the user by detecting contact with a dedicated indicator such as the user's finger or a stylus pen.

[0047] Modes that can be switched using the mode selector switch include still image capture mode, video capture mode, image playback mode, and setting mode. Still image capture mode includes not only a normal capture mode in which one still image is captured, but also a continuous capture mode in which still images are captured continuously at a predetermined capture interval (for example, a frame rate of 5 fps to 10 fps). Continuous capture mode is activated, for example, when the release button is pressed all the way down for a predetermined period of time or more (for example, one second or more). Continuous capture mode ends when the release button is released from its fully pressed state. Note that in normal capture mode, it is also possible to perform so-called quick capture, in which after pressing the release button halfway, you press the release button all the way down to capture a photo without waiting for the completion of the capture preparation operation.

[0048] As an example, as shown in FIG. 2, the image sensor 12 is provided with a photoelectric conversion unit 40. The photoelectric conversion unit 40 is composed of a plurality of pixels 41 arranged two-dimensionally along the X and Y directions. The plurality of pixels 41 form an image pickup surface 42. As is well known, the pixels 41 each include a microlens 45, a color filter 46, and a photoelectric conversion element 47 such as a photodiode. (See FIGS. 3 to 5 for all of them.) The X and Y directions are the horizontal and vertical directions when the bottom surface of the imaging device 10 is placed on a horizontal surface.

[0049] Scanning lines parallel to the X direction are wired between rows of pixels 41. Furthermore, signal lines parallel to the Y direction are wired between columns of pixels 41. The pixels 41 (photoelectric conversion elements 47) are connected to the signal lines via amplifiers and switches. The switches are also connected to the scanning lines. In the case of a storage operation in which signal charges corresponding to subject light are stored in the pixels 41 (photoelectric conversion elements 47), an OFF signal is supplied as a vertical scanning signal through the scanning lines to turn the switches OFF. In the case of a readout operation in which an image signal (voltage signal) 43 corresponding to the signal charges is read from the pixels 41 (photoelectric conversion elements 47), an ON signal is supplied as a vertical scanning signal through the scanning lines to turn the switches ON. The ends of the signal lines are connected to a CDS (Correlated Double Sampling) circuit and an ADC (Analog to Digital Converter) circuit. The CDS circuit performs correlated double sampling on the image signal 43 input through the signal lines. The ADC circuit converts the correlated double sampled image signal 43 into a digital image signal 43.

[0050] Depending on the type of color filter 46, the pixels 41 are divided into three types: green pixels (denoted as "G" in FIG. 2) that are sensitive to light in the green wavelength band, red pixels (denoted as "R" in FIG. 2) that are sensitive to light in the red wavelength band, and blue pixels (denoted as "B" in FIG. 2) that are sensitive to light in the blue wavelength band. The three types of pixels 41 are regularly arranged in a predetermined array. As an example of the predetermined array, a so-called Bayer array is shown here, in which two green pixels, one blue pixel, and one red pixel are arranged in a 2×2 matrix.

[0051] The pixels 41 include normal pixels 41N and phase difference detection pixels 41P. The phase difference detection pixels 41P further include first phase difference detection pixels 411P and second phase difference detection pixels 412P. The normal pixels 41N include three types: green pixels, blue pixels, and red pixels, but the phase difference detection pixels 41P are only green pixels. Green is an example of a "specific color" according to the technology of the present disclosure.

[0052] The phase difference detection pixels 41P are arranged at predetermined intervals in the X direction and the Y direction. In FIG. 2, the phase difference detection pixels 41P are arranged at intervals of five pixels in the X direction and at intervals of two pixels in the Y direction. The phase difference detection pixels 41P are arranged so that the first phase difference detection pixels 411P and the second phase difference detection pixels 412P alternate in the X direction and the Y direction. For example, in the fourth row, the phase difference detection pixels 41P are arranged from left to right in the order of the second phase difference detection pixel 412P, the first phase difference detection pixel 411P, .... In addition, in the tenth column, the phase difference detection pixels 41P are arranged from top to bottom in the order of the second phase difference detection pixel 412P, the first phase difference detection pixel 411P, the second phase difference detection pixel 412P, the first phase difference detection pixel 411P, .... The first phase difference detecting pixel 411P and the second phase difference detecting pixel 412P adjacent to each other in the X direction and the Y direction form one pair for detecting the phase difference α (see FIG. 6).

[0053] As an example, as shown in FIGS. 3 to 5, the normal pixel 41N, the first phase difference detection pixel 411P, and the second phase difference detection pixel 412P have the same basic configuration, and are composed of a microlens 45, a color filter 46, and a photoelectric conversion element 47 arranged in this order from the object side.

[0054] 3, the photoelectric conversion element 47 of the normal pixel 41N outputs an image generation signal 43N corresponding to the subject light that has been collected by the microlens 45 and transmitted through the color filter 46 as the image signal 43. The image generation signal 43N is stored in the image memory 26 as part of the image data.

[0055] 4 and 5, a light-shielding member 49 is disposed between the color filter 46 and the photoelectric conversion element 47 of the first phase difference detection pixel 411P and the second phase difference detection pixel 412P. This light-shielding member 49 is not disposed in the normal pixel 41N. The light-shielding member 49 of the first phase difference detection pixel 411P shields the right half of the photoelectric conversion element 47 when viewed from the object side. In contrast, the light-shielding member 49 of the second phase difference detection pixel 412P shields the left half of the photoelectric conversion element 47 when viewed from the object side.

[0056] The photoelectric conversion element 47 of the first phase difference detection pixel 411P outputs, as the image signal 43, a first calculation signal 431P corresponding to subject light that is collected by the microlens 45, transmitted through the color filter 46, and whose right half is shielded by the light blocking member 49. On the other hand, the photoelectric conversion element 47 of the second phase difference detection pixel 412P outputs, as the image signal 43, a second calculation signal 432P corresponding to subject light that is collected by the microlens 45, transmitted through the color filter 46, and whose left half is shielded by the light blocking member 49. The first calculation signal 431P and the second calculation signal 432P are stored in the image memory 26 as part of the image data, similar to the image generation signal 43N. Note that, hereinafter, unless there is a need to particularly distinguish between them, the first calculation signal 431P and the second calculation signal 432P will be collectively referred to as a calculation signal 43P.

[0057] 6, a phase difference α appears between a first calculation signal 431P and a second calculation signal 432P output from a first phase difference detection pixel 411P and a second phase difference detection pixel 412P that are adjacent to each other in the X and Y directions. This phase difference α indicates in which direction and by how much the focus lens 14 needs to be moved to achieve the in-focus position. The imaging device 10 calculates the in-focus position of the focus lens 14 based on the phase difference α, and performs autofocus control to automatically move the focus lens 14 to the in-focus position.

[0058] The region for calculating the focus position (hereinafter referred to as the focus adjustment region) 90 (see FIG. 10 ) is a preset region in the center of the imaging surface 42. The focus adjustment region 90 may be a region specified by the user or a region surrounding a specific subject recognized by well-known subject recognition technology. The specific subject may be the pupil, face, or torso of a person, the pupil, face, or torso of an animal, or the front or torso of a vehicle such as an automobile, railroad car, or airplane. Here, the pupil of a person or animal refers to the pupil, or the black part of the eye. The face of a person or animal refers to the part of the body that includes the forehead, cheeks, chin, eyes, nose, mouth, ears, etc. The torso of a person or animal refers to the part excluding the head, neck, limbs, and tail. The front of a vehicle refers to the front body of an automobile, the part of the lead car that includes the destination display, front window, headlights, etc. of a railroad car, or the nose of an airplane that includes the radome, front window, etc. The fuselage of a vehicle refers to the entire body excluding the wheels in the case of an automobile, the entire body excluding the wheels in the case of a railway vehicle, regardless of whether it is the lead car, middle car, or last car, and the entire body excluding the front, main wings, tail, etc. Note that subject recognition technology can be exemplified by technology that uses machine learning models such as convolutional neural networks.

[0059] As the name suggests, the image generation signal 43N is used to generate an image such as a live view image. In contrast, the calculation signal 43P is used only to calculate the phase difference α and is not used to generate an image. Therefore, in the pixel interpolation process, the image processing unit 27 interpolates the pixel value of the phase difference detection pixel 41P using the image generation signals 43N of the normal pixels 41N surrounding the phase difference detection pixel 41P.

[0060] Here, the phase difference detection pixels 41P can capture less subject light than the normal pixels 41N due to the presence of the light blocking member 49. For this reason, when exposure is performed under exposure conditions set based on the normal pixels 41N, the amount of light is insufficient for the phase difference detection pixels 41P, and the signal component for calculating the phase difference α of the calculation signal 43P is buried in noise components. Therefore, in the technology of the present disclosure, as will be described later, exposure conditions are set based on the phase difference detection pixels 41P. By correcting the exposure condition and exposing under the corrected exposure condition, the amount of light taken in by the phase difference detection pixel 41P is increased, and the signal-to-noise (SN) ratio of the calculation signal 43P is improved. Note that "exposure" refers to the amount of light incident on the image sensor 12 through the imaging optical system 11. Therefore, for example, "increasing the exposure" means increasing the amount of light incident on the image sensor 12. On the other hand, "exposure" refers to the act of forming an image of subject light on the image sensor 12.

[0061] 7, the control unit 20 includes a storage 55, a CPU (Central Processing Unit) 56, and a memory 57. The storage 55, the CPU 56, and the memory 57 are interconnected via a bus line 58. The control unit 20 is an example of an "exposure control device," a "focus control device," and a "computer" according to the techniques of the present disclosure.

[0062] The storage 55 is a non-volatile storage device such as an EEPROM (Electrically Erasable Programmable Read-Only Memory). The storage 55 stores various programs and various data associated with the programs. Instead of the EEPROM, the storage 55 may be a Ferroelectric Random Access Memory (FeRAM) or a Magnetoresistive Random Access Memory (MRAM).

[0063] The memory 57 is a work memory for the CPU 56 to execute processing. The CPU 56 loads a program stored in the storage 55 into the memory 57 and executes processing in accordance with the program. In this way, the CPU 56 comprehensively controls each unit of the imaging device 10. The CPU 56 is an example of a "processor" according to the technology of the present disclosure. The memory 57 may be built into the CPU 56.

[0064] As an example, as shown in Fig. 8, an operating program 65 is stored in the storage 55. The operating program 65 is a program for causing the CPU 56 to perform autofocus control and the like. In other words, the operating program 65 is an example of an "exposure control device operating program" according to the technology of the present disclosure. In addition to the operating program 65, the storage 55 also stores sensitivity ratio information 66 and saturation level information 67.

[0065] When the operating program 65 is started, the CPU 56, in cooperation with the memory 57 and the like, functions as a photometry unit 70, a first exposure condition setting unit 71, a maximum brightness value derivation unit 72, an exposure correction amount calculation unit 73, a second exposure condition setting unit 74, and a focus calculation unit 75. In addition to these processing units 70 to 75, the CPU 56 also functions as various other processing units.

[0066] The photometry unit 70 reads the image generation signals 43N from the image memory 26. The photometry unit 70 measures the brightness of the subject before capturing a still image during the shooting preparation operation performed by half-pressing the release button. Hereinafter, the exposure for measuring the brightness of the subject before capturing a still image will be referred to as the photometry exposure. The photometry unit 70 calculates luminance values ​​from the image generation signals 43N obtained by the photometry exposure, which are obtained from the image generation signals 43N of a plurality of photometry normal pixels 41N evenly distributed across the imaging surface 42. The photometry unit 70 then calculates the average of the calculated luminance values ​​and derives the average luminance value as the brightness of the subject. The photometry unit 70 outputs the photometry result 80 of the subject brightness to the first exposure condition setting unit 71. The average value may be an arithmetic average or a weighted average.

[0067] The first exposure condition setting unit 71 sets the first exposure condition 81 according to the photometry result 80. The first exposure condition 81 is a condition for obtaining an image with the set exposure. The first exposure condition 81 is a condition related to exposure for calculating an exposure compensation amount corresponding to the phase difference detection pixel 41P, which is performed following the exposure for photometry (hereinafter referred to as exposure for calculating an exposure compensation amount). Furthermore, the first exposure condition 81 is a condition related to still image capture by fully pressing the release button (hereinafter referred to as main exposure). Specifically, the first exposure condition 81 is a combination of the shutter speed of the shutter 23, the aperture opening (f-number or F-stop) of the diaphragm 16, and the gain to be applied to the image signal 43. The first exposure condition setting unit 71 outputs the first exposure condition 81 to the diaphragm driving mechanism 19, the image sensor driver 22, and the shutter driving mechanism 24. The diaphragm driving mechanism 19, the image sensor driver 22, and the shutter driving mechanism 24 perform exposure for calculating an exposure compensation amount under the first exposure condition 81. The "set exposure" refers to an exposure set by an automatic exposure function provided in the imaging device 10. Alternatively, the "set exposure" refers to an exposure set by a user that the user deems appropriate.

[0068] The maximum luminance value derivation unit 72 reads out the image generation signal 43N obtained in the exposure for calculating the exposure compensation amount from the image memory 26. The maximum luminance value derivation unit 72 derives the maximum luminance value of the phase difference detection pixel 41P from the image generation signal 43N with reference to the sensitivity ratio information 66. The maximum luminance value derivation unit 72 outputs a derivation result 82 of the maximum luminance value of the phase difference detection pixel 41P to the exposure compensation amount calculation unit 73. Note that the maximum luminance value of the phase difference detection pixel 41P is the maximum value among the luminance values ​​of the phase difference detection pixel 41P within the focus adjustment region 90.

[0069] The exposure compensation amount calculation unit 73 calculates an exposure compensation amount according to the phase difference detection pixels 41P, based on the saturation level information 67 and the maximum luminance value of the phase difference detection pixels 41P in the derivation result 82. The exposure compensation amount is a setting that increases exposure more than the first exposure condition 81 in order to perform exposure according to the phase difference detection pixels 41P. The exposure compensation amount calculation unit 73 outputs a calculation result 83 of the exposure compensation amount to the second exposure condition setting unit 74.

[0070] The second exposure condition setting unit 74 is responsible for exposure compensation processing. More specifically, the second exposure condition setting unit 74 sets second exposure conditions 84 according to the calculation result 83 based on the first exposure conditions 81. The second exposure conditions 84 are conditions related to the exposure for adjusting the focus of the focus lens 14 (hereinafter referred to as the focus adjustment exposure) that follows the exposure for calculating the exposure compensation amount. Like the first exposure conditions 81, the second exposure conditions 84 are a combination of the shutter speed of the shutter 23, the aperture opening of the diaphragm 16, and the gain applied to the image signal 43. However, because the exposure compensation amount is set to increase the exposure as described above, the shutter speed of the second exposure conditions 84 is set to a slower value than that of the first exposure conditions 81 to lengthen the exposure time. The second exposure conditions 84 are an example of an "exposure condition according to the exposure compensation amount" according to the technology disclosed herein. The second exposure condition setting unit 74 outputs the second exposure conditions 84 to the diaphragm drive mechanism 19, the image sensor driver 22, and the shutter drive mechanism 24. The aperture drive mechanism 19, the image sensor driver 22, and the shutter drive mechanism 24 perform exposure for focus adjustment under the second exposure condition 84. The exposure for focus adjustment is an example of "exposure to phase difference detection pixels" according to the technology of the present disclosure. Note that the second exposure condition 84 may be obtained by changing the aperture opening of the aperture 16 and the gain applied to the image signal 43 in addition to or instead of the shutter speed, from the first exposure condition 81.

[0071] Although not shown, the focus calculation unit 75 receives the drive amount of the focus motor from the focus lens drive mechanism 17. The focus calculation unit 75 derives the current position of the focus lens 14 on the optical axis OA from the drive amount.

[0072] The focus calculation unit 75 also reads out the calculation signal 43P obtained in the exposure for focus adjustment from the image memory 26. Specifically, the calculation signal 43P is data obtained by arranging a plurality of first calculation signals 431P output from the first phase difference detection pixels 411P two-dimensionally in the X direction and the Y direction following the arrangement of the first phase difference detection pixels 411P, and data obtained by arranging a plurality of second calculation signals 432P output from the second phase difference detection pixels 412P two-dimensionally in the X direction and the Y direction following the arrangement of the second phase difference detection pixels 412P. The calculation signal 43P is data that is arranged two-dimensionally in the X and Y directions in accordance with the arrangement of the pixel elements 41, 42, 43A, 43B, 43C, 43D, 43E, 43F, 43G, 43H, 43I, 43J, 43K, 43K, 43KH, 43KJ, 43KL, 43KN, 43K

[0073] The focusing calculation unit 75 calculates the phase difference α shown in FIG. 6 from the calculation signal 43P of the focus adjustment region 90. Based on the phase difference α, the focusing calculation unit 75 calculates the in-focus position of the focus lens 14 when the focus lens 14 is at its current position. The focusing calculation unit 75 outputs the in-focus position calculation result 85 to the focus lens driving mechanism 17. The focus lens driving mechanism 17 moves the focus lens 14 to the in-focus position. Note that if the current position of the focus lens 14 and the in-focus position are the same, the focus lens driving mechanism 17 does nothing, and the focus lens 14 is not moved. Note that the method of calculating the in-focus position of the focus lens 14 based on the phase difference α is well known, so a detailed description thereof will be omitted here.

[0074] The maximum luminance value derivation unit 72 derives the maximum luminance value of the phase difference detection pixel 41P in the procedure shown in step ST150A of the flowchart in Fig. 9, for example. First, the maximum luminance value derivation unit 72 calculates a luminance value from the individual image generation signal 43N of the normal pixel 41N in the focus adjustment region 90. Then, by comparing the individual luminance values ​​of the normal pixel 41N in the focus adjustment region 90, the maximum luminance value of the normal pixel 41N in the focus adjustment region 90 is found (step ST1501A). The focus adjustment region 90 is an example of a "set region" according to the technology of the present disclosure.

[0075] 10 as an example, the maximum luminance value derivation unit 72 derives a sensitivity ratio SR between the normal pixel 41N and the phase difference detection pixel 41P according to the position of the normal pixel 41NM having the maximum luminance value, using the sensitivity ratio information 66 (step ST1502). The sensitivity ratio information 66 is information in which the sensitivity ratio between the normal pixel 41N and the first phase difference detection pixel 411P and the sensitivity ratio between the normal pixel 41N and the second phase difference detection pixel 412P corresponding to each position of the normal pixel 41N are registered. Two sensitivity ratios according to the position of the normal pixel 41NM having the maximum luminance value are obtained: the sensitivity ratio between the normal pixel 41N and the first phase difference detection pixel 411P and the sensitivity ratio between the normal pixel 41N and the second phase difference detection pixel 412P. The maximum luminance value derivation unit 72 derives the higher of the sensitivity ratio between the normal pixel 41N and the first phase difference detection pixel 411P and the sensitivity ratio between the normal pixel 41N and the second phase difference detection pixel 412P as the sensitivity ratio SR. In this way, the sensitivity ratio is changed depending on the position of the normal pixel 41NM having the maximum luminance value.

[0076] Furthermore, if the focus adjustment area 90 is an area designated by a user or an area surrounding a specific subject recognized by well-known subject recognition technology, the position of the focus adjustment area 90 is not fixed. In this case, as shown in the focus adjustment area 90X that is an area designated by a user or an area surrounding a specific subject recognized by well-known subject recognition technology, the sensitivity ratio SR also changes depending on the position of the focus adjustment area 90X. Note that the sensitivity ratio is the ratio between the amount of subject light incident on the normal pixels 41N and the amount of subject light incident on the phase difference detection pixels 41P, which changes due to light blocking by the light blocking member 49.

[0077] 11, for example, the maximum luminance value derivation unit 72 calculates the maximum luminance value of the phase difference detection pixel 41P by multiplying the maximum luminance value of the normal pixel 41N calculated in step ST1501A by the sensitivity ratio derived in step ST1502 (step ST1503). That is, the maximum luminance value derivation unit 72 calculates the maximum luminance value of the phase difference detection pixel 41P from the maximum luminance value of the normal pixel 41N in the focus adjustment region 90 when the subject is captured by the normal pixel 41N at the set exposure.

[0078] The exposure compensation amount calculation unit 73 calculates the exposure compensation amount based on the difference between the maximum luminance value and the saturation level of the phase difference detection pixel 41P. More specifically, the exposure compensation amount calculation unit 73 calculates the difference between the maximum luminance value and the saturation level of the phase difference detection pixel 41P as the exposure compensation amount. Since the second exposure condition 84 is set based on the exposure compensation amount, the second exposure condition setting unit 74 exposes the phase difference detection pixels 41P based on the exposure compensation amount. In other words, the second exposure condition setting unit 74 performs a process of increasing the exposure by the amount of the difference between the maximum luminance value and the saturation level of the phase difference detection pixels 41P as the exposure compensation process. Conversely to this example, the exposure compensation amount may be set to lower the exposure than the first exposure condition 81 in order to perform exposure according to the phase difference detection pixels 41P.

[0079] The saturation level is set to an upper limit value at which the maximum luminance value of the phase difference detection pixels 41P in the focus adjustment region 90 does not saturate. For example, if the luminance value is a 10-bit value (a value between 0 and 1023), the saturation level is set to a value of around 900, which is approximately 90% of 1023. Note that there are two types of "saturation" states: a state where the charge that can be stored in the pixel 41 is at its limit, and a state where the increase in the signal component due to increased exposure has reached its ceiling and the signal-to-noise ratio cannot be further improved. In the technology of the present disclosure, the latter case is defined as the "saturation" state.

[0080] Next, the operation of the above configuration will be described with reference to the flowchart shown in Fig. 12 as an example. As shown in Fig. 8, when the operating program 65 is started, the CPU 56 functions as a photometry unit 70, a first exposure condition setting unit 71, a maximum brightness value derivation unit 72, an exposure correction amount calculation unit 73, a second exposure condition setting unit 74, and a focus calculation unit 75.

[0081] When the release button is half-pressed in still image shooting mode and a still image shooting preparation instruction is received by the instruction receiving unit 32 (YES in step ST100), exposure for photometry is performed in the image sensor 12 under the control of the control unit 20 (step ST110). The image generation signal 43N obtained as a result is read from the image memory 26 to the photometry unit 70. The brightness of the subject is then derived in the photometry unit 70 (step ST120). The photometry result 80 of the subject brightness is output from the photometry unit 70 to the first exposure condition setting unit 71.

[0082] The first exposure condition setting unit 71 sets the first exposure condition 81 according to the photometry result 80 (step ST130). The first exposure condition 81 is output from the first exposure condition setting unit 71 to the aperture drive mechanism 19, the image sensor driver 22, and the shutter drive mechanism 24. Then, under the first exposure condition 81, exposure for calculating the exposure compensation amount is performed at the image sensor 12 (step ST140).

[0083] The image generation signal 43N obtained in the exposure for calculating the exposure compensation amount is read from the image memory 26 to the maximum luminance value derivation unit 72. As shown in FIGS. 9 to 11, the maximum luminance value derivation unit 72 derives the maximum luminance value of the phase difference detection pixel 41P (step ST150A). The derivation result 82 of the maximum luminance value of the phase difference detection pixel 41P is output from the maximum luminance value derivation unit 72 to the exposure compensation amount calculation unit 73.

[0084] 11, the exposure compensation amount calculation unit 73 calculates the difference between the maximum luminance value and the saturation level of the phase difference detection pixel 41P as the exposure compensation amount (step ST160). The exposure compensation amount calculation result 83 is output from the exposure compensation amount calculation unit 73 to the second exposure condition setting unit 74.

[0085] The second exposure condition setting unit 74 sets the second exposure condition 84 according to the calculation result 83 (step ST170). The second exposure condition 84 is output from the second exposure condition setting unit 74 to the aperture drive mechanism 19, the image sensor driver 22, and the shutter drive mechanism 24. Then, under the second exposure condition 84, exposure for focus adjustment is performed at the image sensor 12 (step ST180).

[0086] The focusing calculation unit 75 derives the current position of the focus lens 14 on the optical axis OA based on the drive amount of the focus motor from the focus lens drive mechanism 17. The calculation signal 43P obtained in the focus adjustment exposure is read from the image memory 26 to the focusing calculation unit 75. The focusing calculation unit 75 calculates a phase difference α from the calculation signal 43P in the focus adjustment region 90, and calculates the in-focus position of the focus lens 14 based on the phase difference α (step ST190). The calculation result 85 of the in-focus position is output from the focusing calculation unit 75 to the focus lens drive mechanism 17. The focus lens drive mechanism 17 then moves the focus lens 14 to the in-focus position (step ST200).

[0087] When the release button is fully pressed and an instruction to start shooting a still image is received by instruction receiving unit 32 (NO in step ST210, YES in step ST220), main exposure is performed in image sensor 12 under first exposure condition 81 (step ST230). This completes shooting one still image.

[0088] As described above, the imaging device 10 includes the control unit 20, which is an exposure control device that controls the exposure of the imaging element 12 including the normal pixels 41N for capturing an image of a subject and the phase difference detection pixels 41P for detecting the phase difference α. The CPU 56 of the control unit 20 functions as the second exposure condition setting unit 74. The second exposure condition setting unit 74 performs exposure correction processing according to the difference between the maximum luminance value and the saturation level of the phase difference detection pixels 41P within the focus adjustment region 90. This allows exposure for focus adjustment to be performed under exposure conditions (second exposure conditions 84) suitable for the phase difference detection pixels 41P. The signal-to-noise ratio of the calculation signal 43P can be improved compared to when exposure for focus adjustment is performed under exposure conditions (first exposure conditions 81) based on the normal pixels 41N. This allows for improved focusing accuracy.

[0089] 11, as the exposure correction process, the second exposure condition setting unit 74 calculates an exposure correction amount based on the difference between the maximum luminance value and the saturation level of the phase difference detection pixels 41P, and exposes the phase difference detection pixels 41P (exposure for focus adjustment) based on the exposure correction amount. This makes it possible to maximize the signal-to-noise ratio of the calculation signal 43P.

[0090] When the luminance value of the phase difference detection pixel 41P, and therefore the calculation signal 43P of the phase difference detection pixel 41P, becomes saturated, the signal component of the calculation signal 43P decreases accordingly, resulting in a decrease in the signal-to-noise ratio. Therefore, in the technology disclosed herein, as shown in FIG. 11 , the upper limit value at which the maximum luminance value of the phase difference detection pixel 41P within the focus adjustment region 90 does not become saturated is set as the saturation level. This makes it possible to prevent a decrease in the signal-to-noise ratio of the calculation signal 43P due to saturation.

[0091] As described above, the phase difference detection pixels 41P are all green pixels. Therefore, it is not possible to calculate a luminance value from the calculation signal 43P, and therefore it is not possible to derive the maximum luminance value of the phase difference detection pixels 41P from the calculation signal 43P. Therefore, in the technology disclosed herein, as shown in FIGS. 9 to 11 , the maximum luminance value derivation unit 72 calculates the maximum luminance value of the phase difference detection pixels 41P from the maximum luminance value of the normal pixels 41N in the focus adjustment region 90 when an image of the subject is captured by the normal pixels 41N at the set exposure. In this way, it is possible to derive the maximum luminance value of the phase difference detection pixels 41P, which cannot be derived from the calculation signal 43P.

[0092] 9, the maximum luminance value of the normal pixel 41N can be obtained by comparing the individual luminance values ​​of the normal pixel 41N within the focus adjustment area 90. Therefore, the maximum luminance value of the normal pixel 41N can be easily obtained.

[0093] As shown in FIGS. 9 and 11, the maximum luminance value derivation unit 72 multiplies the maximum luminance value of the normal pixel 41N by the sensitivity ratio SR between the normal pixel 41N and the phase difference detection pixel 41P. The maximum luminance value of the detection pixels 41P is calculated. Therefore, the maximum luminance value of the phase difference detection pixels 41P can be calculated with higher accuracy.

[0094] 10, the sensitivity ratio SR is changed according to the position of the normal pixel 41NM that has the maximum luminance value. Therefore, it is possible to calculate the maximum luminance value of the phase difference detection pixel 41P with high accuracy according to the position of the normal pixel 41NM that has the maximum luminance value. Furthermore, the sensitivity ratio SR is changed according to the position of the focus adjustment region 90. Therefore, it is possible to calculate the maximum luminance value of the phase difference detection pixel 41P according to the position of the focus adjustment region 90.

[0095] Although the maximum luminance value of the normal pixels 41N in the focus adjustment region 90 is obtained by comparing the luminance values ​​of the individual normal pixels 41N, this is not limiting. As an example, as in step ST1501B of step ST150B shown in FIG. 13, the maximum luminance value of the normal pixel 41N may be obtained by comparing the average values ​​of the luminance values ​​of multiple adjacent or nearby normal pixels 41N in the focus adjustment region 90. As an example, as in step ST1501C of step ST150C shown in FIG. 14, the maximum luminance value of the normal pixel 41N may be obtained by comparing the maximum values ​​of the luminance values ​​of multiple adjacent or nearby normal pixels 41N in the focus adjustment region 90. The processing of step ST150B or ST150C is performed instead of the processing of step ST150A.

[0096] 13 and 14 , the plurality of adjacent normal pixels 41N may be, for example, four normal pixels 41N arranged in a 2×2 matrix, nine normal pixels 41N arranged in a 3×3 matrix, or three normal pixels 41N arranged in a row. The plurality of adjacent normal pixels 41N may be, for example, five normal pixels 41N located at the center, upper left corner, upper right corner, lower left corner, and lower right corner of a 5×5 matrix of 25 normal pixels 41N, and positioned several pixels apart. Note that the plurality of adjacent or nearby normal pixels 41N may be normal pixels 41N of a specific color, for example, green pixels that contribute highly to the luminance value. In addition, in the case of FIG. 13 , the position of the normal pixel 41NM having the maximum luminance value when deriving the sensitivity ratio SR between the normal pixel 41N and the phase difference detection pixel 41P is the normal pixel 41N at the center of the plurality of adjacent or nearby normal pixels 41N.

[0097] In this way, the maximum luminance value of the normal pixel 41N may be found by comparing either the average luminance value of a plurality of adjacent or nearby normal pixels 41N within the focus adjustment area 90 or the maximum luminance value of a plurality of adjacent or nearby normal pixels 41N within the focus adjustment area 90. This can reduce the processing effort compared to comparing the luminance value of each of the normal pixels 41N within the focus adjustment area 90.

[0098] Although the exposure for calculating the exposure compensation amount is performed only once, this is not limiting. As an example, as shown in FIG. 15 , the exposure for calculating the exposure compensation amount may be performed over multiple consecutive frames. In this case, the maximum luminance value derivation unit 72 calculates the maximum luminance value of the normal pixel 41N for each of multiple consecutive frames. The median value of the multiple maximum luminance values ​​thus obtained is adopted as the maximum luminance value of the normal pixel 41N used when calculating the maximum luminance value of the phase difference detection pixel 41P. FIG. 15 illustrates an example in which the exposure for calculating the exposure compensation amount is performed over three consecutive frames. The median value is the value ranked in the middle of the multiple maximum luminance values. However, if the multiple maximum luminance values ​​are an even number, the median value is the arithmetic mean value of the two middle-ranked maximum luminance values. This prevents an outlier erroneously calculated due to noise or the like from being adopted as the maximum luminance value of the normal pixel 41N used when calculating the maximum luminance value of the phase difference detection pixel 41P, thereby preventing the maximum luminance value of the phase difference detection pixel 41P and, ultimately, the exposure compensation amount from being calculated correctly.

[0099] In order to use the median value, a minimum of three consecutive frames is required, as shown in the example. However, four or more consecutive frames may be used. However, the number of frames may be The more frames there are, the longer the time required for exposure to calculate the exposure correction amount, so about 3 to 5 frames are preferable.

[0100] [Second embodiment] As an example, as shown in FIG. 16 , the CPU 56 of the second embodiment functions as an object determination unit 95 and a saturation level setting unit 96 in addition to the processing units 70 to 75 of the first embodiment. The object determination unit 95 reads an image generation signal 43N obtained during exposure for calculating an exposure correction amount from the image memory 26. The object determination unit 95 also receives an object determination model 97 and a green threshold value GTH. The object determination model 97 and the green threshold value GTH are stored in the storage 55. The object determination model 97 is, for example, a semantic segmentation model that determines multiple object classes for each pixel 41. The object determination model 97 is an example of a "machine learning model" according to the technology of the present disclosure. The green threshold value GTH is an example of a "first threshold value" according to the technology of the present disclosure.

[0101] The object determination unit 95 uses an object determination model 97 to determine the object class appearing in the image represented by the image generation signal 43N. The object determination unit 95 calculates the proportion of the green object class among the determined object classes. Then, the object determination unit 95 compares the calculated proportion with a green threshold GTH and outputs the comparison result as a determination result 98 to the saturation level setting unit 96. The proportion of the green object class is an example of the "proportion of objects of a specific color" according to the technology of the present disclosure.

[0102] The saturation level setting unit 96 sets the saturation level based on the determination result 98. More specifically, as shown in FIG. 17 as an example, if the determination result 98 indicates that the proportion of the green object class is less than the green threshold value GTH (hereinafter referred to as a non-dark green object), the saturation level setting unit 96 sets a relatively low saturation level. On the other hand, if the determination result 98 indicates that the proportion of the green object class is equal to or greater than the green threshold value GTH (hereinafter referred to as a dark green object), the saturation level setting unit 96 sets a relatively higher saturation level than the saturation level for a non-dark green object. The saturation level for a non-dark green object is set to a value of approximately 900, which is approximately 90% of 1023. On the other hand, the saturation level for a dark green object is set to a value of approximately 1000, which is approximately 98% of 1023. The exposure compensation amount calculation unit 73 calculates the exposure compensation amount based on the saturation level set by the saturation level setting unit 96. Note that examples of dark green objects include forests, grasslands, and moss.

[0103] In this way, in the second embodiment, the saturation level is changed depending on the subject. Therefore, it is possible to set a saturation level that is suitable for the subject. Furthermore, in the second embodiment, the saturation level is changed depending on the subject class determined by the subject determination model 97. The subject determination model 97 can improve the accuracy of determining whether the subject is a dark green subject or a non-dark green subject.

[0104] As described above, the phase difference detection pixel 41P is provided with a green color filter 46. Therefore, in the case of a dark green subject, a higher calculation signal 43P is output compared to the case of a non-dark green subject. Therefore, in the second embodiment, in the case of a dark green subject, the saturation level is set relatively high so that a higher exposure compensation amount is calculated. This makes it possible to perform exposure compensation processing suitable for a dark green subject.

[0105] Note that it is also possible to determine whether the subject is a dark green subject or a non-dark green subject without using the subject determination model 97. For example, an integrated value of the pixel values ​​of the image generation signals 43N of the green pixels among the normal pixels 41N is calculated, and the calculated integrated value is compared with the green threshold value GTH. If the integrated value is less than the green threshold value GTH, the subject is determined to be a non-dark green subject, and if the integrated value is equal to or greater than the green threshold value GTH, the subject is determined to be a dark green subject. In this case, the integrated value is an example of the "proportion of subjects of a specific color" according to the technology of the present disclosure.

[0106] Although green is exemplified as the specific color, the specific color may be red or blue. However, as described above, since the green color filter 46 is arranged in the phase difference detection pixel 41P, it is preferable that the specific color is green.

[0107] [Third embodiment] As an example, as shown in FIG. 18 , the CPU 56 of the third embodiment functions as a photographic scene determination unit 100 in addition to the processing units 70 to 75 (excluding the maximum brightness value derivation unit 72, which are not shown) of the first embodiment. The photographic scene determination unit 100 reads the image generation signal 43N obtained during exposure for calculating the exposure correction amount from the image memory 26. The photographic scene determination unit 100 performs, for example, a light-dark distribution analysis or brightness value histogram analysis of the image generation signal 43N to determine whether the photographic scene is a backlit scene. The photographic scene determination unit 100 outputs a determination result 101 indicating whether the scene is a backlit scene to the maximum brightness value derivation unit 72. Note that a machine learning model may be used to determine whether the scene is a backlit scene. A semantic segmentation model, which distinguishes between multiple types of subjects in an image on a pixel-by-pixel basis, is preferred as the machine learning model, as it improves the accuracy of determining whether the scene is a backlit scene.

[0108] 19, when the determination result 101 of the photographic scene is not a backlit scene, the maximum luminance value derivation unit 72 sets the focus adjustment region 90 as the region for obtaining the maximum luminance value of the normal pixels 41N. On the other hand, when the determination result 98 of the photographic scene is a backlit scene, the maximum luminance value derivation unit 72 sets the reduced region 103 obtained by reducing the focus adjustment region 90 by a specified magnification, for example, 0.6 times, as the region for obtaining the maximum luminance value of the normal pixels 41N. The reduced region 103 is an example of a "set region" according to the technology of the present disclosure.

[0109] In a backlit scene, the center appears dark and the periphery appears bright. Furthermore, the main subject to be focused on is often located in the center. Therefore, if the maximum luminance value of the normal pixels 41N is calculated including the bright peripheral area, the normal pixels 41N in the peripheral area, where the main subject is unlikely to be present, will be determined to have the maximum luminance value as the normal pixels 41NM, resulting in an inappropriate exposure compensation amount being calculated. Therefore, the maximum luminance value derivation unit 72 changes the area for calculating the maximum luminance value of the normal pixels 41N when the scene is backlit. More specifically, the maximum luminance value derivation unit 72 narrows the area for calculating the maximum luminance value of the normal pixels 41N when the scene is backlit compared to when the scene is not backlit. This prevents an inappropriate exposure compensation amount from being calculated.

[0110] Instead of narrowing the area in which the maximum luminance value of the normal pixels 41N is obtained, the maximum luminance value of the normal pixels 41N may be obtained as shown in Fig. 20. In this case, the photographic scene determination unit 100 determines whether the scene is a night scene in addition to whether the scene is a backlit scene.

[0111] The maximum luminance value derivation unit 72 generates a luminance value histogram of the normal pixels 41N in the focus adjustment region 90. As shown in (A), if the distribution of luminance values ​​of the normal pixels 41N in the focus adjustment region 90 has one peak or one inflection point in the distribution, the photographic scene determination unit 100 outputs a determination result 101 that the scene is not a backlit scene or a night scene. In this case, the maximum luminance value derivation unit 72 uses the literally maximum luminance value in the luminance value histogram as the maximum luminance value of the normal pixels 41N to be used when calculating the maximum luminance value of the phase difference detection pixels 41P. On the other hand, as shown in (B), if the distribution of luminance values ​​of the normal pixels 41N in the focus adjustment region 90 has two peaks or multiple inflection points in the distribution, the photographic scene determination unit 100 outputs a determination result 101 that the scene is a backlit scene or a night scene. In this case, the maximum luminance value derivation unit 72 adopts the maximum luminance value at the peak P1 on the lower luminance side of the two peaks P1 and P2 in the distribution of luminance values ​​in the luminance value histogram as the maximum luminance value of the normal pixel 41N to be used when calculating the maximum luminance value of the phase difference detection pixel 41P.

[0112] The low-luminance peak P1 is a portion that appears relatively dark in a backlit scene or a night scene. Therefore, if there are two peaks in the distribution of luminance values ​​of the normal pixels 41N within the focus adjustment region 90, or if there are multiple inflection points in the distribution, that is, in the case of a backlit scene or a night scene, by adopting the maximum luminance value at the low-luminance peak P1, it is possible to prevent an inappropriate exposure correction amount from being calculated, just as in the case where a narrow region is set for determining the maximum luminance value of the normal pixels 41N.

[0113] [Fourth embodiment] As an example, as shown in FIG. 21 , the CPU 56 of the fourth embodiment functions as a detection unit 105 and a coarse / dense determination unit 106 in addition to the processing units 70 to 75 of the first embodiment (not shown except for the exposure compensation amount calculation unit 73 and the second exposure condition setting unit 74). The detection unit 105 reads out the image generation signal 43N obtained in the exposure for calculating the exposure compensation amount from the image memory 26. Saturation level information 67 is also input to the detection unit 105. The detection unit 105 calculates a luminance value from the image generation signal 43N. Then, by comparing the calculated luminance value with the saturation level, normal pixels 41N (hereinafter referred to as saturated pixels 41NS (see FIG. 22 )) that have reached the saturation level within the focus adjustment area 90 are detected. The detection unit 105 outputs a detection result 107 of the saturated pixels 41NS to the coarse / dense determination unit 106 and the second exposure condition setting unit 74.

[0114] The sparse / dense determination unit 106 determines whether the distribution of the saturated pixels 41NS is sparse or dense based on the detection result 107. If the sparse / dense determination unit 106 determines that the distribution of the saturated pixels 41NS is dense, it calculates the percentage of the saturated pixels 41NS that are determined to be densely distributed. The percentage is calculated by dividing the number of saturated pixels 41NS that are determined to be densely distributed by the number of all normal pixels 41N in the focus adjustment region 90. The sparse / dense determination unit 106 outputs a sparse / dense determination result 108 indicating whether the distribution of the saturated pixels 41NS is sparse or dense to the second exposure condition setting unit 74. If the distribution of the saturated pixels 41NS is determined to be dense, the sparse / dense determination result 108 also includes the percentage of the saturated pixels 41NS that are determined to be densely distributed.

[0115] Specifically, the determination of whether the distribution of saturated pixels 41NS is sparse or dense is performed as follows. Specifically, as shown in FIG. 22 as an example, the sparse / dense determination unit 106 first sets a determination region 115 centered on a reference saturated pixel 41NS, indicated by light hatching. The determination region 115 is a rectangular region having a width of 10 pixels above, below, left, and right from the reference saturated pixel 41NS. The sparse / dense determination unit 106 calculates the percentage of saturated pixels 41NS present in the determination region 115. The sparse / dense determination unit 106 sets the determination region 115 by considering all saturated pixels 41NS as the reference saturated pixels 41NS, and calculates the percentage of saturated pixels 41NS for all determination regions 115. Then, as shown in (A), if the saturated pixels 41NS present in all determination regions 115 are less than 80%, the distribution of saturated pixels 41NS is determined to be sparse. On the other hand, as shown in (B), if there is even one determination region 115 in which the saturated pixels 41NS are 80% or more, it is determined that the distribution of the saturated pixels 41NS is dense.

[0116] Alternatively, as an example, as shown in FIG. 23, the sparse / dense determination unit 106 first derives outliers (saturated pixels 41NS surrounded by dashed lines) that are separated from other saturated pixels 41NS by a threshold distance. Then, a determination region 115 is set whose center is the center of gravity of the cluster of saturated pixels 41NS excluding the outliers. If the determination region 115 contains all saturated pixels 41NS excluding the outliers, the sparse / dense determination unit 106 determines that the distribution of saturated pixels 41NS is dense. In all other cases, the sparse / dense determination unit 106 determines that the distribution of saturated pixels 41NS is sparse.

[0117] 21, the second exposure condition setting unit 74 receives the calculation result 83 from the exposure compensation amount calculation unit 73, the sparse / dense determination result 108 from the sparse / dense determination unit 106, as well as a set exposure compensation amount 109, a first determination threshold DTH1, and a second determination threshold DTH2. The first decision threshold DTH1 and the second decision threshold DTH2 are stored in the storage 55. The first decision threshold DTH1 is an example of a "second threshold" according to the technology of the present disclosure. The second decision threshold DTH2 is an example of a "third threshold" according to the technology of the present disclosure.

[0118] The second exposure condition setting unit 74 calculates the percentage of saturated pixels 41NS based on the detection result 107. The percentage is calculated by dividing the number of saturated pixels 41NS by the number of all normal pixels 41N in the focus adjustment region 90. The second exposure condition setting unit 74 compares the percentage of saturated pixels 41NS with a first decision threshold DTH1. If the percentage of saturated pixels 41NS is equal to or less than the first decision threshold DTH1, the second exposure condition setting unit 74 sets second exposure conditions 84 according to the calculation result 83 (calculated exposure compensation amount). Also, if the percentage of saturated pixels 41NS is greater than the first decision threshold DTH1 and the sparse / dense determination unit 106 determines that the distribution of saturated pixels 41NS is sparse, the second exposure condition setting unit 74 sets second exposure conditions 84 according to the calculation result 83. Setting the second exposure conditions 84 according to the calculation result 83 means, in other words, performing exposure compensation processing according to the difference between the maximum luminance value and the saturation level of the phase difference detection pixels 41P. The first decision threshold DTH1 is set to, for example, 1%.

[0119] If the proportion of saturated pixels 41NS is greater than the first decision threshold DTH1 and the sparse / dense determination unit 106 determines that the distribution of saturated pixels 41NS is dense, the second exposure condition setting unit 74 compares the proportion of saturated pixels 41NS determined to be densely distributed with the second decision threshold DTH2. If the proportion of saturated pixels 41NS determined to be densely distributed is less than the second decision threshold DTH2, the second exposure condition setting unit 74 sets second exposure conditions 84 according to the set exposure compensation amount 109. Setting the second exposure conditions 84 according to the set exposure compensation amount 109 means, in other words, performing exposure compensation processing according to the set exposure compensation amount 109.

[0120] The set exposure compensation amount 109 is +0.5EV (Exposure Value) to +1.0EV. The set exposure compensation amount 109 may be switched depending on the situation, such as when shooting quickly, in continuous shooting mode, or in moving object tracking AF (Auto Focus), in which case the exposure time for focus adjustment needs to be shortened, and when the set exposure compensation amount 109 is set to +0.5EV, and in other cases the set exposure compensation amount 109 is set to +1.0EV.

[0121] If the proportion of saturated pixels 41NS determined to have a dense distribution is equal to or greater than the second determination threshold DTH2, the second exposure condition setting unit 74 does not set the second exposure conditions 84 according to the calculation result 83 or the set exposure compensation amount 109, but instead sets the first exposure conditions 81 as the second exposure conditions 84. In other words, the second exposure condition setting unit 74 does not perform exposure compensation processing.

[0122] 24 shows an example of the processing procedure of the fourth embodiment. First, the image generation signal 43N obtained in the exposure for calculating the exposure correction amount is read from the image memory 26 to the detection unit 105. Then, the detection unit 105 detects normal pixels 41N that have reached a saturation level within the focus adjustment region 90, i.e., saturated pixels 41NS (step ST300). The detection result 107 of the saturated pixels 41NS is output from the detection unit 105 to the sparse / dense determination unit 106 and the second exposure condition setting unit 74.

[0123] The second exposure condition setting unit 74 compares the percentage of saturated pixels 41NS with the first decision threshold DTH1 (step ST310). If the percentage of saturated pixels 41NS is equal to or less than the first decision threshold DTH1 (YES in step ST310), the second exposure condition setting unit 74 sets the second exposure condition 84 according to the calculation result 83 (step ST320). On the other hand, if the percentage of saturated pixels 41NS is greater than the first decision threshold DTH1 (NO in step ST310), the process proceeds to step ST330.

[0124] In step ST330, the sparse / dense determination unit 106 determines whether the distribution of the saturated pixels 41NS is sparse or dense. If it is determined that the distribution of the saturated pixels 41NS is sparse (YES in step ST330), the second exposure condition setting unit 74 sets the second exposure condition 84 according to the calculation result 83 (step ST320), just as in the case where the proportion of saturated pixels 41NS is equal to or less than the first determination threshold DTH1. On the other hand, if it is determined that the distribution of the saturated pixels 41NS is dense (NO in step ST330), the process proceeds to step ST340.

[0125] In step ST340, the second exposure condition setting unit 74 compares the proportion of saturated pixels 41NS determined to have a dense distribution with the second decision threshold DTH2. If the proportion of saturated pixels 41NS determined to have a dense distribution is less than the second decision threshold DTH2 (YES in step ST340), the second exposure condition setting unit 74 sets the second exposure conditions 84 according to the set exposure compensation amount 109 (step ST350). On the other hand, if the proportion of saturated pixels 41NS determined to have a dense distribution is equal to or greater than the second decision threshold DTH2 (NO in step ST340), the second exposure condition setting unit 74 sets the first exposure conditions 81 to the second exposure conditions 84 (step ST360). That is, in this case, exposure compensation processing is not performed.

[0126] As described above, in the fourth embodiment, the detection unit 105 detects saturated pixels 41NS within the focus adjustment region 90. When the proportion of saturated pixels 41NS is equal to or less than the first decision threshold DTH1, the second exposure condition setting unit 74 performs exposure correction processing according to the difference between the maximum luminance value and the saturation level of the phase difference detection pixels 41P. Therefore, when the proportion of saturated pixels 41NS is equal to or less than the first decision threshold DTH1, it is possible to improve the focusing accuracy.

[0127] Furthermore, the sparse / dense determination unit 106 determines whether the distribution of saturated pixels is sparse or dense. If the sparse / dense determination unit 106 determines that the proportion of saturated pixels 41NS is greater than the first determination threshold DTH1 and that the distribution is sparse, the second exposure condition setting unit 74 performs exposure correction processing according to the difference between the maximum luminance value and the saturation level of the phase difference detection pixels 41P. Therefore, if the sparse / dense determination unit 106 determines that the proportion of saturated pixels 41NS is greater than the first determination threshold DTH1 and that the distribution is sparse, it is possible to improve focusing accuracy.

[0128] When the proportion of saturated pixels 41NS is greater than the first determination threshold DTH1 and the sparse / dense determination unit 106 determines that the distribution is dense, and the proportion of saturated pixels 41NS determined to be dense is less than the second determination threshold DTH2, performing exposure correction processing according to the difference between the maximum luminance value of the phase difference detection pixels 41P and the saturation level may cause the calculation signals 43P of the phase difference detection pixels 41P in the focus adjustment region 90 to exceed the saturation level. If the calculation signals 43P exceed the saturation level, signal components exceeding the saturation level are lost, resulting in a decrease in the signal-to-noise ratio of the calculation signals 43P. Therefore, the second exposure condition setting unit 74 performs exposure correction processing according to a preset exposure correction amount 109. This prevents the calculation signals 43P of the phase difference detection pixels 41P in the focus adjustment region 90 from exceeding the saturation level. This prevents a deterioration in focusing accuracy.

[0129] Furthermore, when the proportion of saturated pixels 41NS is greater than the first determination threshold DTH1 and the sparse / dense determination unit 106 determines that the distribution is dense, and the proportion of saturated pixels 41NS determined to be dense is equal to or greater than the second determination threshold DTH2, performing exposure correction processing according to the difference between the maximum luminance value and the saturation level of the phase difference detection pixels 41P or performing exposure correction processing according to the set exposure correction amount 109 may cause most of the calculation signals 43P of the phase difference detection pixels 41P within the focus adjustment region 90 to exceed the saturation level. Therefore, the second exposure condition setting unit 74 does not perform exposure correction processing. This prevents the calculation signals 43P of the phase difference detection pixels 41P within the focus adjustment region 90 from exceeding the saturation level. This prevents deterioration of focusing accuracy.

[0130] As an example, as shown in FIG. 25 , consider a case where exposure for focus adjustment is performed by setting second exposure conditions 84 according to calculation result 83. In this case, the second exposure condition setting unit 74 calculates a difference ΔT between the time required for exposure for focus adjustment when second exposure conditions 84 according to calculation result 83 are set (hereinafter referred to as the exposure time according to calculation result 83) and the time required for exposure for focus adjustment when second exposure conditions 84 according to set exposure compensation amount 109 are set (hereinafter referred to as the exposure time according to set exposure compensation amount 109). Then, the difference ΔT is compared with a preset first threshold time TTH1. The first threshold time TTH1 is stored in the storage 55. The first threshold time TTH1 is set to a time that the user perceives as a processing delay, for example, 0.5 seconds. In this case, the time obtained by adding the exposure time according to set exposure compensation amount 109 and the first threshold time TTH1 is an example of a “preset threshold time” according to the technology of the present disclosure.

[0131] If the difference ΔT is longer than the first threshold time TTH1, the second exposure condition setting unit 74 modifies the second exposure condition 84 so that the difference ΔT is equal to or shorter than the first threshold time TTH1. That is, if the difference ΔT is longer than the first threshold time TTH1, the exposure time according to the calculation result 83 is shortened to allow underexposure for the phase difference detection pixel 41P. Alternatively, the underexposure may be compensated for by increasing the gain applied to the image signal 43. However, since setting the gain too high increases noise components, it is preferable to set an upper limit on the gain setting. In this way, if the difference ΔT is longer than the first threshold time TTH1, modifying the second exposure condition 84 so that the difference ΔT is equal to or shorter than the first threshold time TTH1 can reduce the risk of the user perceiving a processing delay.

[0132] As an example, as shown in Figure 26, consider a case where through image output processing is performed between image recording for the Nth (N is a natural number greater than or equal to 1) continuous shooting in continuous shooting mode and image recording for the N+1th continuous shooting. The through image output processing is a process in which exposure for focus adjustment and focus control are repeated a specified number of times (six times in Figure 26), but image recording is not performed. The focus control in the through image output processing is a control that calculates the focus position of focus lens 14 from calculation signal 43P obtained in the focus adjustment exposure. However, the focus control in the through image output processing does not move focus lens 14 to the focus position.

[0133] In focus control for the N+1th continuous shooting, focus calculation unit 75 calculates the focus position by referring to calculation results 85 of the multiple focus positions obtained in focus control for the through-image output process after the Nth continuous shooting. In other words, focus calculation unit 75 predicts the current focus position from calculation results 85 of the multiple most recent focus positions. Focus lens drive mechanism 17 moves focus lens 14 to the focus position predicted by focus calculation unit 75.

[0134] As an example, as shown in FIGS. 27 and 28, consider a case where, in focus control for through image output processing, a second exposure condition 84 is set according to a calculation result 83 to perform exposure for focus adjustment. In this case, the second exposure condition setting unit 74 compares the exposure time according to the calculation result 83 with a preset second threshold time TTH2. The second threshold time TTH2 is stored in the storage 55. The second threshold time TTH2 is the longest exposure time of the image sensor 12 that can be set for one frame in the through image output processing. The second threshold time TTH2 is an example of a "preset threshold time" according to the technology of the present disclosure.

[0135] If the exposure time according to the calculation result 83 is longer than the second threshold time TTH2, the second exposure condition setting unit 74 corrects the second exposure condition 84 so that the exposure time according to the calculation result 83 is equal to or shorter than the second threshold time TTH2. That is, if the exposure time according to the calculation result 83 is longer than the second threshold time TTH2, the exposure time according to the calculation result 83 is rounded up to a shorter value, as in the case of FIG. 25, and underexposure for the phase difference detection pixel 41P is tolerated. Alternatively, the underexposure may be compensated for by setting the gain to be applied to the image signal 43 high. However, since setting the gain too high increases noise components, it is preferable to set an upper limit on the gain setting.

[0136] In this way, if the exposure time corresponding to the calculation result 83 is longer than the second threshold time TTH2, the second exposure condition 84 is modified so that the exposure time corresponding to the calculation result 83 is equal to or shorter than the second threshold time TTH2. This allows the exposure time corresponding to the calculation result 83 to be within the longest exposure time that can be set for one frame of the through-image output process by the image sensor 12. Therefore, as shown in FIG. 28, before the second exposure condition 84 is modified, the number of calculation results 85 obtained in the through-image output process is three. However, after the second exposure condition 84 is modified, the number of calculation results 85 obtained in the through-image output process doubles to six. Since the number of calculation results 85 that can be used to predict the focus position increases, the accuracy of predicting the focus position during focus control for the (N+1)th continuous shooting can be improved. Note that the image recording portion is not shown in FIG. 28.

[0137] In the first embodiment, photometry by the normal pixels 41N is performed based on the image generation signals 43N of the multiple normal pixels 41N for photometry that are equally distributed on the imaging surface 42. However, this is not limiting. Photometry by the phase difference detection pixels 41P may be performed based on the calculation signals 43P of the phase difference detection pixels 41P for photometry within the focus adjustment region 90. In this case, however, as shown in FIG. 29 as an example, the maximum luminance value of the phase difference detection pixels 41P is corrected by adding a difference ΔP between the brightness of the subject derived by the photometry by the normal pixels 41N and the brightness of the subject derived by the photometry by the phase difference detection pixels 41P to the maximum luminance value of the phase difference detection pixels 41P obtained by multiplying the maximum luminance value of the normal pixels 41N by the sensitivity ratio SR. The exposure compensation amount calculation unit 73 calculates the difference between the corrected maximum luminance value of the phase difference detection pixels 41P and the saturation level as the exposure compensation amount.

[0138] Exposure may be controlled by an electronic shutter instead of the shutter 23.

[0139] The imaging device according to the technique of the present disclosure is not limited to the exemplified mirrorless single-lens digital camera, but may also be a compact digital camera, a video camera, a surveillance camera, a smartphone, a tablet terminal, or even an endoscope.

[0140] Furthermore, the calculation signal 43P of the phase difference detection pixel 41P obtained by the exposure control device according to the technology of the present disclosure is not limited to use in the illustrated focus control device, but can also be used, for example, to create a distance map that indicates the distance from the imaging device to the subject, and to generate a three-dimensional image using information on the phase difference α.

[0141] In each of the above embodiments, the following various processors can be used as the hardware structure of the processing units that perform various processes, such as the image processing unit 27, the display control unit 30, the instruction receiving unit 32, the photometry unit 70, the first exposure condition setting unit 71, the maximum brightness value derivation unit 72, the exposure compensation amount calculation unit 73, the second exposure condition setting unit 74, the focus calculation unit 75, the object determination unit 95, the saturation level setting unit 96, the photographic scene determination unit 100, the detection unit 105, and the sparse / dense determination unit 106. The various processors include the CPU 56, which is a general-purpose processor that executes software (operation program 65) and functions as various processing units, as well as a programmable logic device (PLC) such as an FPGA (Field Programmable Gate Array), whose circuit configuration can be changed after manufacture. PLD), and / or ASIC (Application Specific This includes dedicated electrical circuits, such as processors with circuit configurations designed specifically to perform specific processing, such as a Serial Integrated Circuit (SCI).

[0142] A single processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (e.g., a combination of multiple FPGAs and / or a combination of a CPU and an FPGA). Also, multiple processing units may be configured with a single processor.

[0143] As an example of configuring multiple processing units in one processor, first, there is a form in which one processor is configured by combining one or more CPUs and software, as represented by computers such as client and server, and this processor functions as multiple processing units. Second, there is a form in which one processor is configured by combining one or more CPUs and software, as represented by computers such as system on chip (SoC), etc. As shown in the figure, one form of implementation uses a processor that realizes the functions of the entire system including multiple processing units on a single IC (Integrated Circuit) chip. In this way, the various processing units are configured as a hardware structure using one or more of the various processors described above.

[0144] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements.

[0145] From the above description, the technology described in the following supplementary paragraphs can be understood.

[0146] [Additional note 1] An exposure control device that controls exposure of an imaging element including normal pixels for capturing an image of a subject and phase difference detection pixels for detecting a phase difference, a processor; The processor: performing exposure correction processing according to a difference between a maximum luminance value and a saturation level of the phase difference detection pixel within a set region; Exposure control device. [Additional note 2] The processor: As the exposure compensation process, an exposure compensation amount based on the difference is calculated; 2. The exposure control device according to claim 1, wherein the phase difference detection pixels are exposed to light based on the exposure correction amount. [Additional note 3] The exposure control device according to supplementary item 1 or supplementary item 2, wherein the saturation level is set to an upper limit value at which the maximum luminance value of the phase difference detection pixel in the set region does not saturate. [Additional note 4] The processor: 4. The exposure control device according to any one of claims 1 to 3, wherein the saturation level is changed depending on the subject. [Additional note 5] The processor: 5. The exposure control device according to claim 4, wherein the saturation level is changed in accordance with a subject class determined by a machine learning model. [Additional note 6] a filter of a specific color is disposed in the phase difference detection pixel, The processor: An exposure control device as described in Appendix 4, which sets the saturation level higher when the proportion of subjects of the specific color is equal to or greater than a predetermined first threshold value than when the proportion of subjects of the specific color is less than the first threshold value. [Additional note 7] The processor: 7. The exposure control device according to claim 1, wherein a maximum luminance value of the phase difference detection pixels is calculated from a maximum luminance value of the normal pixels in the set area when the subject is imaged by the normal pixels at a set exposure. [Additional note 8] The maximum luminance value of the normal pixel is A single luminance value of the normal pixel within the set region; an average value of the luminance values ​​of a plurality of adjacent or nearby normal pixels within the set region; and 8. The exposure control device according to claim 7, wherein the maximum luminance value is obtained by comparing the maximum luminance values ​​of a plurality of adjacent or nearby normal pixels within the set region. [Additional note 9] The processor: 9. The exposure control device according to claim 7 or 8, wherein the maximum luminance value of the phase difference detection pixel is calculated by multiplying the maximum luminance value of the normal pixel by a sensitivity ratio between the normal pixel and the phase difference detection pixel. [Additional Note 10] 10. The exposure control device according to claim 9, wherein the sensitivity ratio is changed depending on the position of the normal pixel having the maximum luminance value. [Additional Note 11] The exposure control device according to supplementary item 9 or supplementary item 10, wherein the sensitivity ratio is changed depending on the position of the setting area. [Additional Note 12] The processor: 12. The exposure control device according to any one of claims 7 to 11, wherein the setting area is changed in the case of a backlit scene. [Additional Note 13] The processor: Item 13. The exposure control device according to item 12, wherein the setting area is set narrower in a backlit scene than in a non-backlit scene. [Additional Note 14] The processor: The exposure control device according to any one of Supplementary Items 7 to 13, wherein, when there are two peaks in the distribution of luminance values ​​of the normal pixels within the set region or when there are multiple inflection points in the distribution, a maximum luminance value at a peak on the lower luminance side of the peaks in the distribution is adopted as the maximum luminance value of the normal pixels used when calculating the maximum luminance value of the phase difference detection pixels. [Supplementary Item 15] The processor: 15. The exposure control device according to claim 14, wherein, in the case of a backlit scene or a night scene, a maximum luminance value at the peak on the low luminance side is adopted as a maximum luminance value of the normal pixels used in calculating a maximum luminance value of the phase difference detection pixels. [Additional Note 16] The processor: 16. The exposure control device according to claim 7, wherein a median value of maximum luminance values ​​of the normal pixels obtained in a plurality of consecutive frames is adopted as the maximum luminance value of the normal pixels used when calculating the maximum luminance value of the phase difference detection pixels. [Additional Note 17] The processor: Detecting the normal pixels that have reached the saturation level within the set region; An exposure control device according to any one of claims 1 to 16, wherein if the proportion of normal pixels that have reached the saturation level is equal to or less than a predetermined second threshold, exposure correction processing is performed according to the difference. [Additional Note 18] The processor: determining whether the distribution of the normal pixels that have reached the saturation level is sparse or dense; If it is determined that the ratio is greater than the second threshold value and the distribution is sparse, an exposure correction process is performed according to the difference; If the ratio is greater than the second threshold value and the distribution is determined to be dense, and if the ratio of the normal pixels determined to be dense in distribution is less than a preset third threshold value, performing exposure compensation processing according to a preset exposure compensation amount; An exposure control device as described in appended claim 17, which determines that the proportion of normal pixels whose distribution is determined to be dense is greater than the second threshold and that the proportion of normal pixels whose distribution is determined to be dense is greater than or equal to the third threshold, and does not perform exposure correction processing. [Additional Note 19] The processor: As the exposure compensation process, an exposure compensation amount based on the difference is calculated; exposing the phase difference detection pixels based on the exposure correction amount; An exposure control device according to any one of appendix 1 to appendix 18, wherein, if the exposure time of the exposure condition according to the exposure compensation amount is longer than a predetermined threshold time, the exposure condition is corrected so that the exposure time is equal to or shorter than the threshold time. [Additional Note 20] The processor: As the exposure compensation process, an exposure compensation amount based on the difference is calculated; exposing the phase difference detection pixels based on the exposure correction amount; 19. The exposure control device according to claim 1, wherein, in continuous shooting in which the phase difference detection pixels are exposed between image recordings, if an exposure time of an exposure condition according to the exposure compensation amount is longer than a predetermined threshold time, the exposure condition is corrected so that the exposure time is equal to or shorter than the threshold time. [Additional Note 21] A focus control device comprising the exposure control device according to any one of claims 1 to 20. [Supplementary Item 22] An imaging device comprising the exposure control device according to any one of supplementary items 1 to 20.

[0147] The technology of the present disclosure can be appropriately combined with the various embodiments and / or various modified examples described above. Furthermore, it is not limited to the above-described embodiments, and various configurations can be adopted without departing from the spirit of the present disclosure. Furthermore, the technology of the present disclosure extends not only to programs but also to storage media that non-temporarily store programs.

[0148] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[0149] In this specification, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0150] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference. [Explanation of symbols]

[0151] 10. Imaging device 11 Imaging optical system 12 Image sensor 13 Objective Lens 14 Focus Lens 15 Zoom Lens 16 apertures 17 Focus lens drive mechanism 18 Zoom lens drive mechanism 19 Aperture drive mechanism 20 Control Unit 21 Control section 22 Image sensor driver 23 Shutter 24 Shutter drive mechanism 25 Image Input Controller 26 Image Memory 27 Image processing section 28, 58 bus lines 29 VRAM 30 Display control unit 31 Media Controller 32 Instruction Reception Department 33 Finder Monitor 34 Rear monitor 35 Memory Card 36 Touch Panel 40 Photoelectric conversion unit 41 pixels 41N normal pixels 41NM Normal pixel with maximum brightness value 41NS Normal pixels that have reached the saturation level (saturated pixels) 41P Phase difference detection pixel 42 imaging surface 43 Image Signal 43N Image generation signal 43P Calculation signal 45 Microlenses 46 Color Filter 47 Photoelectric conversion element 49 Light blocking material 55 Storage 56 CPU 57 Memory 65 Operating Program 66 Sensitivity Ratio Information 67 Saturation Level Information 70 Photometry section 71 First exposure condition setting section 72 Maximum brightness value derivation section 73 Exposure compensation amount calculation section 74 Second exposure condition setting section 75 Focus calculation section 80 Photometry results 81 1st exposure condition 82 Derivation results 83 Calculation results 84 Second exposure condition 85 Calculation result 90, 90X focusing area 95 Subject determination section 96 Saturation level setting section 97 Subject Detection Model 98, 101 Judgment result 100 Shooting scene determination unit 103 Reduced area 105 Detector 106 Sparseness / Density Judgment Unit 107 Detection Results 108 Sparseness / Density Judgment Results 109 Setting exposure compensation amount 115 Judgment area 411P 1st phase difference detection pixel 412P Second phase difference detection pixel 431P 1st calculation signal 432P Second calculation signal α phase difference ΔP is the difference between the brightness of the subject determined by photometry using normal pixels and the brightness of the subject determined by photometry using phase difference detection pixels. ΔT is the difference between the time required for exposure for focus adjustment when the second exposure conditions are set according to the calculation result and the time required for exposure for focus adjustment when the second exposure conditions are set according to the set exposure compensation amount. DTH1 First decision threshold DTH2 Second decision threshold GTH Green Threshold OA optical axis P1, P2 Peaks in the distribution of brightness values SR sensitivity ratio ST100, ST110, ST120, ST130, ST140, ST150A, ST150B, ST150C, ST160, ST170, ST180, ST190, ST200, ST210, ST220, ST230, ST300, ST310, ST320, ST330, ST340, ST350, ST360, ST1501A, ST1501B, ST1501C, ST1502, ST1503 Step TTH1 First threshold time TTH2 Second threshold time

Claims

1. An exposure control device that controls exposure of an imaging element including normal pixels for capturing an image of a subject and phase difference detection pixels for detecting a phase difference, a processor; The processor: performing exposure correction processing according to a difference between a maximum luminance value and a saturation level of the phase difference detection pixel within a set region; Exposure control device.

2. The processor: As the exposure compensation process, an exposure compensation amount based on the difference is calculated; The exposure control device according to claim 1 , wherein the phase difference detection pixels are exposed based on the exposure correction amount.

3. The exposure control device according to claim 1 , wherein the saturation level is set to an upper limit value at which the maximum luminance value of the phase difference detection pixels in the set region does not saturate.

4. The processor:

2. The exposure control device according to claim 1, wherein the saturation level is changed depending on the subject.

5. The processor: The exposure control device according to claim 4 , wherein the saturation level is changed in accordance with a subject class determined by a machine learning model.

6. a filter of a specific color is disposed in the phase difference detection pixel, The processor:

5. The exposure control device according to claim 4, wherein when the proportion of subjects of the specific color is equal to or greater than a predetermined first threshold, the saturation level is set higher than when the proportion of subjects of the specific color is less than the first threshold.

7. The processor:

2. The exposure control device according to claim 1, wherein the maximum luminance value of the phase difference detection pixels is calculated from the maximum luminance value of the normal pixels in the set region when the subject is imaged by the normal pixels at a set exposure.

8. The maximum luminance value of the normal pixel is A single luminance value of the normal pixel within the set region; an average value of the luminance values ​​of a plurality of adjacent or nearby normal pixels within the set region; and 8. The exposure control device according to claim 7, wherein the maximum luminance value is obtained by comparing any one of the maximum luminance values ​​of a plurality of adjacent or nearby normal pixels within the set region.

9. The processor: The exposure control device according to claim 7 , wherein the maximum luminance value of the phase difference detection pixel is calculated by multiplying the maximum luminance value of the normal pixel by a sensitivity ratio between the normal pixel and the phase difference detection pixel.

10. The exposure control device according to claim 9 , wherein the sensitivity ratio is changed depending on the position of the normal pixel having the maximum luminance value.

11. 10. The exposure control device according to claim 9, wherein the sensitivity ratio is changed depending on the position of the setting area. 。

12. The processor:

8. The exposure control device according to claim 7, wherein the setting area is changed when the scene is backlit.

13. The processor: The exposure control device according to claim 12 , wherein the setting area is set narrower when the scene is a backlit scene than when the scene is not a backlit scene.

14. The processor:

8. The exposure control device according to claim 7, wherein, when there are two peaks in the distribution of luminance values ​​of the normal pixels in the set region or when there are a plurality of inflection points in the distribution, a maximum luminance value at a peak on a lower luminance side of the peaks in the distribution is adopted as the maximum luminance value of the normal pixels to be used when calculating the maximum luminance value of the phase difference detection pixels.

15. The processor:

15. The exposure control device according to claim 14, wherein, in the case of a backlit scene or a night scene, a maximum luminance value at the peak on the low luminance side is adopted as a maximum luminance value of the normal pixel used when calculating a maximum luminance value of the phase difference detection pixel.

16. The processor:

8. The exposure control device according to claim 7, wherein a median value of maximum luminance values ​​of the normal pixels obtained in a plurality of consecutive frames is adopted as the maximum luminance value of the normal pixels used when calculating the maximum luminance value of the phase difference detection pixels.

17. The processor: Detecting the normal pixels that have reached the saturation level within the set region; 2. The exposure control device according to claim 1, wherein, when the proportion of the normal pixels that have reached the saturation level is equal to or less than a second threshold value that is set in advance, exposure correction processing is performed in accordance with the difference.

18. The processor: determining whether the distribution of the normal pixels that have reached the saturation level is sparse or dense; If it is determined that the ratio is greater than the second threshold value and the distribution is sparse, an exposure correction process is performed according to the difference; if the proportion is greater than the second threshold and the distribution is determined to be dense, and if the proportion of the normal pixels determined to be dense in distribution is less than a third threshold set in advance, performing exposure compensation processing according to a preset exposure compensation amount; 18. The exposure control device according to claim 17, wherein the ratio is greater than the second threshold value and the distribution is determined to be dense, and when the ratio of the normal pixels determined to have the dense distribution is equal to or greater than the third threshold value, exposure correction processing is not performed.

19. The processor: As the exposure compensation process, an exposure compensation amount based on the difference is calculated; exposing the phase difference detection pixels based on the exposure correction amount; 2. The exposure control device according to claim 1, wherein, when an exposure time of an exposure condition according to the exposure compensation amount is longer than a predetermined threshold time, the exposure condition is corrected so that the exposure time is equal to or shorter than the threshold time.

20. The processor: As the exposure compensation process, an exposure compensation amount based on the difference is calculated; exposing the phase difference detection pixels based on the exposure correction amount; 2. The exposure control device according to claim 1, wherein, in continuous shooting in which the phase difference detection pixels are exposed between image recordings, if an exposure time of an exposure condition according to the exposure compensation amount is longer than a predetermined threshold time, the exposure condition is corrected so that the exposure time is equal to or shorter than the threshold time.

21. 1. A method for operating an exposure control device that controls exposure of an image sensor including normal pixels for capturing an image of a subject and phase difference detection pixels for detecting a phase difference, comprising: performing exposure correction processing according to a difference between a maximum luminance value and a saturation level of the phase difference detection pixel within a set region; A method for operating an exposure control device, comprising:

22. An operation program for an exposure control device that controls exposure of an image sensor including normal pixels for capturing an image of a subject and phase difference detection pixels for detecting a phase difference, Exposure correction according to the difference between the maximum luminance value and the saturation level of the phase difference detection pixel in the set region to carry out processing; An operating program for an exposure control device that causes a computer to execute a process including the steps of:

23. A focus control device comprising the exposure control device according to claim 1.

24. An imaging device comprising the exposure control device according to claim 1.

Images