Imaging device and method for calculating defocus amount

Abstract

An imaging device according to the present technology is provided with: an imaging element provided with a light blocking pixel and a photodiode division pixel; and a defocus amount calculation unit that, on the basis of an exposure amount, calculates a defocus amount using an output signal of the light blocking pixel and / or an output signal of the photodiode division pixel.

Description

[Technical field]

[0001] The present technology relates to an imaging device including an imaging element having a pixel group that outputs a phase difference signal, and a defocus amount calculation method. [Background technology]

[0002] Some imaging devices have a function of acquiring focus information about a subject in order to perform autofocus control. Some of these imaging devices have pixels for detecting a focus. For example, Patent Document 1 discloses a configuration in which pixels using a PD (Photodiode) division method (photodiode division pixels) and pixels using a light-shielding pixel method (light-shielding pixels) are provided as pixels for detecting a focus. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] WO2017 / 212909 publication Summary of the Invention [Problem to be solved by the invention]

[0004] These focus detection pixels output signals for detecting the phase difference, but in dark environments where the amount of light is likely to be insufficient, there is a problem that the S / N ratio of the phase difference signal deteriorates. Therefore, an object of the present technology is to obtain the defocus amount of a subject with high accuracy even in a dark environment where the amount of light tends to be insufficient. [Means for solving the problem]

[0005] The imaging device according to the present technology includes an imaging element having a light-shielding pixel and a photodiode divided pixel, and a defocus amount calculation unit that calculates a defocus amount using at least one of an output signal of the light-shielding pixel and an output signal of the photodiode divided pixel based on an amount of exposure. This makes it possible to suitably select either the output signal of the light-shielded pixel or the output signal of the photodiode divided pixel in the imaging device to calculate the defocus amount.

[0006] In the above-mentioned imaging device, the light-shielding pixel may have a pupil division function by being equipped with a light-shielding portion that blocks one of a pair of light beams that pass through a pair of partial regions that are biased in opposite directions in a predetermined direction at the exit pupil, and a light-receiving element that receives the other light beam. As a result, a light-shielded pixel is, for example, either a pixel into which only light that has passed through a left region defined by the light-shielding portion as the left half of the exit pupil is incident, or a pixel into which only light that has passed through a right region defined by the light-shielding portion as the right half of the exit pupil is incident.

[0007] In the above-described imaging device, the photodiode segment pixel may have a pupil division function by including segment pixels that receive each of a pair of light beams that have passed through the pair of partial regions. As a result, the split pixels of the photodiode split pixel are divided into a split pixel into one into which only light that has passed through a left region that is the left half of the exit pupil is incident, and a split pixel into which only light that has passed through a right region that is the right half of the exit pupil by the light-shielding portion is incident.

[0008] In the imaging device described above, each pixel included in the imaging element may be either one of the light-shielded pixels and the photodiode divided pixels. That is, all pixels of the image sensor are either light-shielded pixels or photodiode divided pixels.

[0009] The defocus amount calculation section in the above-described imaging device may calculate the defocus amount by using output signals of the light-shielded pixels when the exposure amount is equal to or greater than a threshold value. Since the light-shielding pixel has a light-shielding portion, the output signal level is made smaller than that of the PD division pixel.

[0010] When the exposure amount is less than a threshold value, the defocus amount calculation section in the above-described imaging device may calculate the defocus amount using at least one of the output signals of the light-shielding pixels and the output signals of the photodiode divided pixels, depending on whether the area is an on-axis area, which is an area including a central portion of the imaging element, or an off-axis area, which is an area other than the on-axis area of ​​the imaging element. This allows focus control to be performed even if the amount of exposure is less than the threshold value.

[0011] The defocus amount calculation section in the above-described imaging device may calculate the defocus amount using an output signal of the photodiode divided pixel in the on-axis region. Since the photodiode divided pixel does not have a light-shielding portion, the output signal level is made higher than that of the light-shielded pixel.

[0012] The defocus amount calculation section in the above-described imaging device may calculate the defocus amount by using output signals of the light-shielded pixels in the off-axis region. This allows a highly reliable defocus amount to be calculated.

[0013] The defocus amount calculation section in the above-described imaging device may calculate the defocus amount by using output signals of the photodiode divided pixels in the off-axis region. This allows a highly reliable defocus amount to be calculated.

[0014] The defocus amount calculation section in the above-described imaging device may calculate the defocus amount using one of the output signals of the light-shielded pixels and the output signals of the photodiode divided pixels, whichever has higher reliability. This allows a highly reliable defocus amount to be calculated.

[0015] The defocus amount calculation unit in the above-mentioned imaging device may perform autofocus control based on a contrast method in an off-axis region, which is a region other than an on-axis region, which is a region including a central portion of the imaging element, when the exposure amount is less than a threshold value. In the off-axis region where the amount of exposure is insufficient, the accuracy of the phase difference information based on the output signals of the light-shielded pixels and the output signals of the photodiode divided pixels may be low.

[0016] The imaging element in the imaging device described above may have a plurality of types of light-shielding pixels corresponding to pupil distances of exit pupils. This makes it possible to obtain an appropriate phase difference signal from any one of the multiple types of light-shielded pixels even if the pupil position of the exit pupil changes as the imaging optical system is driven.

[0017] In the imaging device described above, the plurality of types of light-shielding pixels may have different light-shielding regions of the light-shielding portions. For example, the closer the pupil distance, the larger the light blocking area is.

[0018] The defocus amount calculation section in the above-described imaging device may calculate the defocus amount using an output signal of the light-shielded pixel selected in accordance with a pupil distance of an exit pupil. This allows the selection of light-shielded pixels that receive light passing through one side area of ​​the exit pupil.

[0019] The imaging device described above may further include a camera control unit that acquires the pupil distance from a barrel control unit that is provided in the lens barrel. For example, in an imaging device equipped with an interchangeable lens barrel, a light-shielding pixel is selected according to the pupil distance.

[0020] In the above-described imaging device, the light-shielding pixels arranged in the same row on the imaging element may correspond to the same pupil distance. This prevents output signals from a plurality of types of light-shielded pixels from being mixed in the pixel signals read out for each pixel row.

[0021] The imaging device described above may further include a camera control unit that issues an instruction to drive a focus lens of the imaging optical system based on the defocus amount. In this way, the focus lens is controlled by a phase difference signal based on the amount of exposure.

[0022] The above-mentioned imaging device may further include a user interface control unit that performs display control based on the defocus amount. This makes it possible to provide information to inform the photographer of the focus state according to the current lens position, for example.

[0023] In the above-described imaging device, when the defocus amount is calculated using the output signals of the light-shielded pixels, the imaging element may add up and output the output signals of the photodiode divided pixels. This makes it possible to reduce the number of times the output from each divided pixel is read out, compared to reading the output from each divided pixel separately.

[0024] In the imaging device described above, when the defocus amount is calculated using the output signals of the photodiode divided pixels, the imaging element may output each of the output signals of the photodiode divided pixels. This allows the output signals of the divided pixels to be acquired without being added.

[0025] In the above-mentioned imaging device, a signal processing unit is provided that performs signal processing on the image signal output from the imaging element, and the signal processing unit may add the output signals of the photodiode divided pixels in the column direction when the exposure amount is less than a predetermined amount. This makes it possible to maintain the output level of the signal output from the photodiode divided pixel at a predetermined level or higher.

[0026] The defocus amount calculation method according to the present technology selects at least one of the phase difference signals from the output signals of the light-shielded pixels and the output signals of the photodiode divided pixels based on the exposure amount, and calculates the defocus amount. [Brief description of the drawings]

[0027] [Figure 1] 1 is a perspective view of an imaging device according to an embodiment of the present technology; [Diagram 2] FIG. [Diagram 3] FIG. 1 is a block diagram of an imaging device. [Figure 4] FIG. 1 is a schematic diagram of an imaging device. [Diagram 5] 2 is a configuration example of an imaging element. [Figure 6] 13 is a configuration example of light-shielding pixels arranged in an on-axis region. [Figure 7] 13 is a configuration example of PD divided pixels arranged in an on-axis region. [Figure 8] 13 is an example of an output signal of the first pixel row. [Figure 9] 13 is a diagram for explaining a difference integral value between a left-opening pixel output and a right-opening pixel output. FIG. [Figure 10] 13 is a diagram for explaining a difference integral value between a shifted left-opening pixel output and a shifted right-opening pixel output. FIG. [Figure 11] 13 is a diagram for explaining a difference integral value between a shifted left-opening pixel output and a shifted right-opening pixel output. FIG. [Figure 12] 13 is a diagram for explaining a difference integral value between a shifted left-opening pixel output and a shifted right-opening pixel output. FIG. [Figure 13] FIG. 13 is a diagram illustrating the relationship between a difference integral value and a shift amount. [Figure 14] FIG. 11 is a diagram showing the relationship between a shift amount and a defocus amount. [Figure 15] FIG. 2 is a diagram for explaining an on-axis region and an off-axis region. [Figure 16] FIG. 2 is an explanatory diagram of pixel positions on an imaging element. [Figure 17] 13 is a configuration example of light-shielded pixels arranged in an off-axis region. [Figure 18] This is a configuration example of PD divided pixels arranged in an off-axis region. [Figure 19] 13 is a configuration example of light-shielded pixels arranged in an off-axis region. [Figure 20] 13 is an example of the arrangement of light-shielding pixels according to pupil distance. [Figure 21] 11 is a diagram showing the difference in performance between a light-shielded pixel and a PD divided pixel. [Figure 22] 10A to 10C are diagrams for explaining the exposure amount and the AF error, and for explaining the phase difference signal selected under each condition. [Figure 23] 11A and 11B are diagrams for explaining a method of reading out signals from each pixel when a phase difference signal output from a light-shielded pixel is used. [Figure 24] 11 is a diagram for explaining the readout time of a signal from each pixel when a phase difference signal output from a light-shielded pixel is used. FIG. [Diagram 25] 11 is a diagram for explaining a method of reading out a signal from each pixel when a phase difference signal output from a PD division pixel is used. FIG. [Figure 26] 11 is a diagram for explaining the readout time of a signal from each pixel when a phase difference signal output from a PD division pixel is used. FIG. [Figure 27] 11 is a diagram for explaining a method of reading out a signal from each pixel when a phase difference signal output from a light-shielded pixel and a phase difference signal output from a PD division pixel are used. FIG. [Figure 28] 11 is a diagram for explaining the readout time of a signal from each pixel when a phase difference signal output from a light-shielded pixel and a phase difference signal output from a PD division pixel are used. FIG. [Figure 29] FIG. 4 is a block diagram relating to calculation of an exposure amount. [Diagram 30] 10 is a flowchart executed when a half-press of the shutter button is detected. [Diagram 31] 10 is a flowchart executed when a half-press of the shutter button is detected. [Diagram 32] 10 is a flowchart executed when a half-press of the shutter button is detected. [Diagram 33] 10 is a flowchart executed when a half-press of the shutter button is detected. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0028] Hereinafter, embodiments will be described in the following order with reference to the accompanying drawings. <1. Configuration of imaging device> 2. Image sensor configuration <3.AF control> <4. On-axis area and off-axis area> <5. Pupil distance and pixel configuration> <6. Differences in AF control due to differences in phase detection pixels> <7. Phase difference signal selection> <8. Control of exposure amount> <9. Processing flow> <9-1. First Example> <9-2. Second Example> <10. Variations> <11. Summary> <12. This Technology>

[0029] <1. Configuration of imaging device> FIG. 1 shows the appearance of an imaging device 1 according to the present embodiment. In the following examples, the subject side is described as the front and the photographer side is described as the rear, but these directions are for convenience of explanation, and the implementation of the present technology is not limited to these directions.

[0030] As shown in FIGS. 1 and 2, the imaging device 1 is configured with a camera housing 2 in which required parts are arranged inside and outside, and a lens barrel 3 attached to a front part 2a of the camera housing 2.

[0031] A rear monitor 4 is disposed on the rear surface 2b of the camera housing 2. On the rear monitor 4, a through image, a recorded image, and the like are displayed.

[0032] The rear monitor 4 is, for example, a display device such as a liquid crystal display (LCD) or an organic EL (Electro-Luminescence) display.

[0033] The rear monitor 4 is rotatable relative to the camera housing 2. For example, the rear monitor 4 is rotatable around an upper end portion of the rear monitor 4 as a rotation axis such that a lower end portion of the rear monitor 4 moves rearward. The right end or the left end of the rear monitor 4 may be the rotation axis. Furthermore, the rear monitor 4 may be rotatable about a plurality of axes.

[0034] An EVF (Electric Viewfinder) 5 is disposed on the top surface 2c of the camera housing 2. The EVF 5 includes an EVF monitor 5a and a frame-shaped surrounding portion 5b that protrudes rearward so as to surround the upper portion and the left and right sides of the EVF monitor 5a.

[0035] The EVF monitor 5a is formed using an LCD, an organic EL display, etc. Note that an optical view finder (OVF) may be provided instead of the EVF monitor 5a.

[0036] The rear surface 2b and the top surface 2c are provided with various operators 6. The operators 6 include, for example, a playback menu start button, a decision button, a cross key, a cancel button, a zoom key, a slide key, and a shutter button 6S (release button).

[0037] The various types of operators 6 include various types of buttons, dials, composite operators that can be pressed and rotated, etc. The various types of operators 6 enable, for example, menu operation, playback operation, mode selection / switching operation, focus operation, zoom operation, and parameter selection / setting of shutter speed, F-number, etc.

[0038] FIG. 3 is a block diagram of the imaging device 1. As shown in FIG. Inside and outside the camera housing 2 of the imaging device 1, an imaging element 7, a camera signal processing unit 8, a recording unit 9, a display unit 10, an output unit 11, an operation unit 12, a power supply unit 13, a camera control unit 14, a memory unit 15, etc. are provided.

[0039] The lens barrel 3 comprises an optical system 16, a driver section 17, a barrel control section 18, an operation section 19, a memory section 20, and the like.

[0040] As shown in Figures 3 and 4, the optical system 16 is composed of various lenses such as an entrance end lens, a zoom lens, a focus lens, and a collecting lens, an aperture mechanism that controls exposure by adjusting the opening amount of the lens or iris (aperture) so that sensing is performed in a state where the signal charge is not saturated and is within the dynamic range, and a shutter unit such as a focal plane shutter. It should be noted that a portion of each component of the optical system 16 may be provided in the camera housing 2.

[0041] The imaging element 7 is, for example, a charge coupled device (CCD) type or a complementary metal-oxide semiconductor (CMOS) type, and performs exposure control on light from a subject that is incident via an optical system 16 .

[0042] The sensor surface of the image sensor 7 is configured to have a sensing element in which a plurality of pixels are arranged two-dimensionally. Some of the pixels in the image sensor 7 are pixels that output a signal for calculating the defocus amount. In the following description, the pixels that output a signal for calculating the defocus amount are referred to as "image surface phase difference pixels 7a." In the present embodiment, all pixels of the image sensor 7 are image surface phase difference pixels 7a for detecting the phase difference of the optical image of the subject. However, the image sensor 7 may be configured to include pixels other than the image surface phase difference pixels 7a. A detailed configuration example of the image sensor 7 will be described later.

[0043] The imaging element 7 includes a processing unit that performs, for example, CDS (Correlated Double Sampling) processing, AGC (Automatic Gain Control) processing, and A / D (Analog / Digital) conversion processing on the electric signals photoelectrically converted by the pixels. Therefore, the imaging element 7 outputs a captured image signal as digital data to the camera signal processing unit 8 and the camera control unit 14.

[0044] The image plane phase difference pixel 7a outputs a signal used to calculate the defocus amount. In this embodiment, the image sensor 7 includes both a light-shielding pixel using a PD (photodiode) light-shielding method and a PD division pixel using a PD division method as the image plane phase difference pixel 7a. The signal output from the image plane phase difference pixel 7a is a signal obtained by photoelectric conversion, but is a signal from which phase difference information can be detected by subsequent processing. Therefore, in the following description, the signal output from the image plane phase difference pixel 7a and from which phase difference information can be detected may be referred to as a "phase difference signal".

[0045] The imaging element 7 outputs the phase difference signals obtained from these image plane phase difference pixels 7a to the downstream camera signal processing unit 8 and camera control unit 14. The phase difference signals are used in correlation calculations for calculating the defocus amount.

[0046] The camera signal processing unit 8 is configured by, for example, a microprocessor specialized for digital signal processing, such as a DSP (Digital Signal Processor), a microcomputer, or the like.

[0047] The camera signal processing unit 8 includes various units for performing various types of signal processing on the digital signal (captured image signal) sent from the imaging element 7.

[0048] Specifically, correction between the R, G, and B color channels, white balance correction, aberration correction, shading correction, and other processes are performed. In addition, the camera signal processing unit 8 performs various processes such as YC generation processing to generate (separate) a luminance (Y) signal and a color (C) signal from the R, G, and B image data, processing to adjust the luminance and color, knee correction, and gamma correction. Furthermore, the camera signal processing unit 8 performs conversion to a final output format by performing resolution conversion processing and codec processing for encoding for recording or communication. The image data converted to the final output format is stored in the memory unit 15. The image data is output to the display unit 10, whereby an image is displayed on the rear monitor 4 or the EVF monitor 5a. Furthermore, the image data is output from an external output terminal, whereby the image is displayed on a device such as a monitor provided outside the imaging device 1.

[0049] The camera signal processing unit 8 includes a defocus amount calculation unit 8a. The defocus amount calculation unit 8a detects phase difference information from the output signals of the image surface phase difference pixels 7a. Specifically, the defocus amount calculation unit 8a detects phase difference information from a group of signals output from a plurality of image surface phase difference pixels 7a and a group of signals output from another plurality of image surface phase difference pixels 7a. The camera signal processing unit 8 (or the camera control unit 14) has information on what kind of signal group is to be compared in order to detect the phase difference information. The specific details will be described in paragraph

[0079] . The defocus amount calculation unit 8a calculates the defocus amount based on the detected phase difference information. The calculated defocus amount may be used for an autofocus (AF) function by being used to drive a focus lens included in the optical system 16 via the lens barrel control unit 18. The defocus amount may also be used to present information regarding the focus state of the subject to the user.

[0050] The recording unit 9 is made up of, for example, a non-volatile memory, and functions as a storage unit for storing image files (content files) such as still image data and video data, attribute information of the image files, thumbnail images, and the like. Image files are stored in formats such as Joint Photographic Experts Group (JPEG), Tagged Image File Format (TIFF), and Graphics Interchange Format (GIF). There are various possible actual forms for the recording unit 9. For example, the recording unit 9 may be configured as a flash memory built into the imaging device 1, or may be configured as a memory card (e.g., a portable flash memory) that can be attached to and detached from the imaging device 1 and an access unit that accesses the memory card for storage and reading. Also, the recording unit 9 may be realized as a hard disk drive (HDD) built into the imaging device 1.

[0051] The display unit 10 executes a process for providing various displays to the photographer. The display unit 10 is, for example, the rear monitor 4 or the EVF monitor 5a. The display unit 10 executes a process for displaying image data that has been converted to an appropriate resolution and input from the camera signal processor 8. This causes a so-called through image, which is an image captured during release standby, to be displayed. Furthermore, the display unit 10 realizes the display of various operation menus, icons, messages, etc. as a GUI (Graphical User Interface) on the screen based on instructions from the camera control unit 14. Moreover, the display unit 10 is capable of displaying a reproduced image of image data read out from a recording medium in the recording unit 9.

[0052] In this example, both the EVF monitor 5a and the rear monitor 4 are provided, but the implementation of the present technology is not limited to this configuration, and only one of the EVF monitor 5a and the rear monitor 4 may be provided, or either one or both of the EVF monitor 5a and the rear monitor 4 may be configured to be detachable.

[0053] The output unit 11 performs data communication and network communication with external devices in a wired or wireless manner, for example, transmitting captured image data (still image files and video files) to an external display device, recording device, playback device, etc. The output unit 11 may also function as a network communication unit. For example, the output unit 11 may communicate via various networks such as the Internet, a home network, and a LAN (Local Area Network) and transmit and receive various data between the output unit 11 and a server or terminal on the network.

[0054] The operation unit 12 provided on the camera housing 2 includes not only the various operators 6 described above, but also a rear monitor 4 that employs a touch panel system, and outputs operation information to the camera control unit 14 in response to various operations such as tapping and swiping by the photographer. The operation unit 12 may function as a receiving unit for an external operation device such as a remote controller separate from the imaging device 1.

[0055] The power supply unit 13 generates a power supply voltage (Vcc) required for each unit, for example from a battery stored inside, and supplies it as an operating voltage. When the lens barrel 3 is attached to the imaging device 1, the power supply voltage Vcc from the power supply unit 13 is also supplied to the circuits within the lens barrel 3. The power supply unit 13 may be provided with a circuit for charging the battery and a circuit for generating a power supply voltage Vcc, using a DC voltage converted and input by an AC adapter connected to a commercial AC power supply as a power source.

[0056] The camera control unit 14 is configured by a microcomputer (arithmetic processing device) equipped with a CPU (Central Processing Unit), and performs overall control of the imaging device 1. For example, it controls the shutter speed in response to the operation of the photographer, issues instructions regarding various signal processes in the camera signal processing unit 8, performs imaging operations and recording operations in response to the user's operations, and performs playback operations of recorded image files. The camera control unit 14 switches between various shooting modes, etc. Examples of the various shooting modes include a still image shooting mode, a moving image shooting mode, and a continuous shooting mode in which still images are captured continuously.

[0057] The camera control unit 14 includes a user interface control unit (UI control unit) 14a for enabling the user to operate these functions. The UI control unit 14a performs a process for detecting operations on the operators 6 provided in the imaging device 1, a display process for the rear monitor 4, an operation detection process, and the like.

[0058] The UI control unit 14a performs display control for notifying the user of the defocus amount. By recognizing the notification about the defocus amount, the user can perform a manual focusing operation or set an arbitrary defocus state. The notification is performed by, for example, the display unit 10. Specifically, an icon image may be displayed, or characters or the like may be displayed.

[0059] In addition, the camera control unit 14 issues instructions to a lens barrel control unit 18 in order to control various lenses included in the optical system 16 . For example, it performs processing to specify an aperture value to ensure the amount of light required for AF control, and issues instructions to operate the aperture mechanism according to the aperture value.

[0060] The camera control unit 14 is capable of acquiring information on various lenses included in the optical system 16 via the lens barrel control unit 18. The lens information includes, for example, the model number of the lens, the position and F-number of the zoom lens, or the exit pupil position. The camera control unit 14 is also capable of acquiring the aperture value of the aperture mechanism included in the optical system 16.

[0061] The memory unit 15 stores information and the like used in the processing executed by the camera control unit 14. The illustrated memory unit 15 collectively represents, for example, a Read Only Memory (ROM), a Random Access Memory (RAM), a flash memory, and the like. The memory unit 15 may be a memory area built into a microcomputer chip serving as the camera control unit 14, or may be configured by a separate memory chip.

[0062] The ROM, flash memory, etc. of the memory unit 15 store programs and the like used by the camera control unit 14. The ROM, flash memory, etc. store an OS (Operating System) for the CPU to control each unit, content files such as image files, and application programs and firmware for various operations. The camera control unit 14 executes the program to control the entire imaging device 1 and lens barrel 3.

[0063] The RAM of the memory unit 15 is used as a working area for the camera control unit 14 by temporarily storing data, programs, and the like used when the CPU of the camera control unit 14 executes various data processes.

[0064] The lens barrel control unit 18 of the lens barrel 3 is formed by, for example, a microcomputer, and outputs control signals to a driver unit 17 to actually drive various lenses of the optical system 16 based on instructions from the camera control unit 14. In addition, information communication between the camera control unit 14 and the lens barrel control unit 18 may be possible only when the lens barrel 3 is attached to the camera housing 2, or it may be possible via wireless communication when the lens barrel 3 is not attached to the camera housing 2.

[0065] The lens barrel control unit 18 transmits information on the exit pupil position and the pupil distance of the exit pupil to the camera control unit 14 based on the types and drive positions of the various lenses included in the optical system 16. Specifically, the lens barrel control unit 18 obtains information on the pupil distance from information stored in a ROM serving as the memory unit 20, and transmits the information to the camera control unit 14.

[0066] The driver unit 17 includes, for example, a motor driver for a zoom lens drive motor, a motor driver for a focus lens drive motor, an aperture mechanism driver for a motor that drives an aperture mechanism, and the like. Each driver supplies a drive current to the corresponding drive motor in response to an instruction from the lens barrel control unit 18 .

[0067] Operation unit 19 of lens barrel 3 indicates an operator provided on the side of lens barrel 3. Operation information by operation unit 19 is supplied to barrel control unit 18, and notified to camera control unit 14 via barrel control unit 18. In response to the operation of the operation unit 19, the barrel control unit 18 controls the operation of the optical system 16, and the camera control unit 14 performs various settings and controls the operations.

[0068] The operation unit 19 may function as a receiving unit for an external operation device such as a remote controller separate from the lens barrel 3 .

[0069] The memory unit 20 is configured with a ROM, a flash memory, etc., and stores programs, data, etc. used by the lens barrel control unit 18. The memory unit 20 stores an OS (Operating System) for the CPU to control each unit, application programs for various operations, firmware, etc. The information stored in the memory unit 20 also includes information such as the pupil distance of the exit pupil of the optical system 16 .

[0070] 2. Image sensor configuration The configuration of the imaging element 7 will be described with reference to FIG. The image sensor 7 is configured with pixels arranged in a matrix in the row and column directions. Each pixel 21 is either a light-shielded pixel 21A provided with a light-shielding portion that blocks a portion of light incident on one pixel, or a PD split pixel 21B configured of two split pixels.

[0071] The imaging element 7 has a first pixel row 22A that includes light-shielded pixels 21A and thereby outputs a phase difference signal using the PD light-shielding method, and a second pixel row 22B that is composed only of PD divided pixels 21B and thereby outputs a phase difference signal using the PD division method.

[0072] The first pixel rows 22A are arranged discretely in the vertical direction, and a plurality of second pixel rows 22B are arranged between the first pixel rows 22A. The first pixel rows 22A may be arranged regularly or irregularly. However, a regular arrangement can reduce design costs and manufacturing costs related to the manufacture of the image sensor 7.

[0073] Each PD split pixel 21B included in the second pixel row 22B is covered with a Bayer array color filter, and depending on the type of color filter, it has either red (R) spectral sensitivity, green (G) spectral sensitivity, or blue (B) spectral sensitivity.

[0074] The configuration of the light-shielded pixel 21A will be described with reference to the schematic diagram of FIG. The light-shielding pixel 21A includes a PD 30, a light-shielding section 31 arranged in front of the PD 30 (on the subject side), an inner lens 32 arranged in front of the light-shielding section 31, a color filter (cyan) 33 arranged in front of the inner lens 32, and an on-chip microlens 34 arranged in front of the color filter 33. It is to be noted that the inner lens 32 and the color filter 33 may not be provided in the light-shielded pixel 21A.

[0075] The PD 30 is a light receiving element on which a portion of the light that has passed through the exit pupil EP is incident, but a light shielding portion 31 disposed in front of the PD 30 allows light to be received only in a portion of the light receiving region of the PD 30. That is, the light shielding portion 31 is formed so as to cover the left half area of ​​the PD 30. The light shielding portion 31 is formed with a right opening 35R.

[0076] The inner lens 32 and the on-chip microlens 34 are optical components provided for efficiently collecting, onto the PD 30, light that has passed through the exit pupil EP and is incident on one pixel. The color filter 33 is, for example, a filter having a spectral sensitivity of cyan (Cy).

[0077] 6, PD 30 is configured to receive only light passing through a left region (hereinafter referred to as "left pupil region EPL") which is the left half of the exit pupil EP. That is, light passing through a right region (hereinafter referred to as "right pupil region EPR") which is the right half of the exit pupil EP is blocked by light blocking portion 31 and does not reach PD 30. This achieves a pupil division function.

[0078] 6, the light-shielded pixel 21A configured to receive light passing through the left pupil region EPL receives light in a region biased to the right side on the light receiving surface, and is therefore referred to as a light-shielded pixel 21AR. That is, the light-shielded pixel 21AR has a right opening 35R formed therein. 6 is a light-shielded pixel 21A configured to receive light passing through a right pupil region EPR, and this pixel is referred to as a light-shielded pixel 21AL because it receives light in a region biased to the left side of the light receiving surface. A left opening 35L is formed in the light-shielding portion 31 included in the light-shielded pixel 21AL.

[0079] As shown in FIG. 5, the distance between the light-shielded pixels 21AR and 21AL is, for example, the distance of two pixels, and they are arranged alternately. The signal output from the light-shielding pixel 21AR and the signal output from the light-shielding pixel 21AL are treated as a pair of phase difference signals by the camera signal processing unit 8 (or the camera control unit 14). That is, the defocus amount calculation unit 8a of the downstream camera signal processing unit 8 calculates the defocus amount using the phase difference between the signal output from the light-shielding pixel 21AR and the signal output from the light-shielding pixel 21AL.

[0080] Next, the PD divided pixel 21B will be described. FIG. 7 is a diagram illustrating a schematic configuration of a PD divided pixel 21B as a pixel 21 in the imaging element 7. As shown in FIG.

[0081] The PD split pixel 21B includes two split pixels, that is, a left PD 40L that is the split pixel on the left side and a right PD 40R that is the split pixel on the right side, a pixel boundary metal 41 arranged in front of them, an inner lens 32, a color filter 33, and an on-chip microlens 34. The color filter 33 is any one of a color filter 33R having a spectral sensitivity of red (R), a color filter 33G having a spectral sensitivity of green (G), and a color filter 33B having a spectral sensitivity of blue (B). It should be noted that the inner lens 32 and the like may not be provided in the PD divided pixel 21B.

[0082] As shown in the figure, the left PD 40L receives light that has passed through a right pupil region EPR of the exit pupil EP. The right PD 40R receives light that has passed through a left pupil region EPL. This achieves a pupil division function.

[0083] <3.AF control> An AF control method for focusing using each of the above-mentioned pixels 21 will be described. First, the AF control using the phase difference signals output from the light-shielded pixels 21A and the PD division pixels 21B will be described.

[0084] The phase difference signal will be described by taking the light-shielded pixel 21A as an example. FIG. 8 is a graph showing the relationship between the position of the light-shielded pixel 21A and the output for a certain first pixel row 22A on the image sensor 7. The solid line indicates a signal output from a light-shielded pixel 21AL in which a left opening 35L is formed, and the dashed line indicates a signal output from a light-shielded pixel 21AR in which a right opening 35R is formed. The solid line graph indicates a left opening pixel output 50L, and the dashed line graph indicates a right opening pixel output 50R.

[0085] The shaded area in FIG. 9 shows the integral difference between the waveform of the left-opening pixel output 50L and the waveform of the right-opening pixel output 50R. Next, the waveform of the left-opening pixel output 50L is shifted a certain distance to the right on the graph and shown as waveform 50L1 in Fig. 10. The shaded area in Fig. 10 indicates the difference integral value between the waveform 50L1 and the right-opening pixel output 50R.

[0086] The waveform 50L1 is further shifted a certain distance to the right as waveform 50L2 in Fig. 11. The shaded area in Fig. 11 indicates the difference integral value between the waveform 50L2 and the right aperture pixel output 50R.

[0087] The waveform 50L2 is further shifted a certain distance to the right as waveform 50L3, which is shown in Fig. 12. The shaded area in Fig. 12 indicates the difference integral value between the waveform 50L3 and the right aperture pixel output 50R.

[0088] FIG. 13 shows a graph of the difference integral values ​​shown by the shaded areas in FIG. 9, FIG. 10, FIG. 11 and FIG. As shown in the figure, the more the shift amount is increased, the smaller the difference integral value becomes, and once the shift amount exceeds a certain amount, the more the shift amount is increased, the larger the difference integral value becomes again. The shift amount at which the difference integral value is smallest is the phase difference. That is, appropriate AF control can be performed by moving the focus lens so that the outputs of the light-shielded pixels 21AL and 21AR are shifted by the phase difference and the waveforms of the left-opening pixel output 50L and the right-opening pixel output 50R substantially overlap.

[0089] It should be noted that the so-called front focus and back focus can be distinguished depending on the direction in which the waveform of the left opening pixel output 50L is shifted. That is, in the state of Fig. 9, the difference integral value can be minimized by shifting the waveform of the left opening pixel output 50L to the right. This state is called the front focus state. On the other hand, when the difference integral value can be minimized by shifting the waveform of the left aperture pixel output 50L to the left, this is called a back focus state.

[0090] In more detail, since the light-shielded pixels 21AL and 21AR are separated by two pixels on the image sensor 7, the optimal focus state can be created by moving the focus lens so that the waveforms of the left-opening pixel output 50L and the right-opening pixel output 50R are shifted by two pixels.

[0091] FIG. 14 shows the relationship between the shift amount and the defocus amount. The shift amount is the amount at which the difference integral value shown in FIG. 13 becomes smaller than the original value, and can be said as the phase difference. The relationship between the shift amount and the defocus amount is expressed by a linear function. The larger the shift amount, the larger the defocus amount becomes, and a state where the shift amount is large is a state where the image is out of focus. The defocus amount can be calculated from the shift amount.

[0092] Although the AF control based on the phase difference signal output from the light-shielded pixel 21A has been described so far, the AF control based on the phase difference signal output from the PD division pixel 21B can also be performed in the same manner.

[0093] Specifically, the defocus amount can be obtained by comparing the waveform of the signal output from the left PD 40L of the PD split pixel 21B (corresponding to the left opening pixel output 50L in each figure) with the waveform of the signal output from the right PD 40R (corresponding to the right opening pixel output 50R in each figure), and calculating the shift amount to approximately match these waveforms.

[0094] When a defocus amount with high reliability cannot be calculated from the signals output from the light-shielded pixels 21A and the PD divided pixels 21B, it is possible to calculate the defocus amount using a contrast method. For example, the contrast is detected based on a luminance signal generated in the camera signal processing unit 8, and focus control is performed. It should be noted that the contrast method described here is merely an example, and a wide variety of other well-known methods can be used to calculate the defocus amount using the contrast method.

[0095] <4. On-axis area and off-axis area> The image sensor 7 may have different configurations of pixels 21 in a region near the center and in other regions. Specifically, the region near the center of the image sensor 7 will be described as an on-axis region ArC, and the other regions as off-axis regions ArM. 15 is an example of the on-axis region ArC and the off-axis region ArM, but the on-axis region ArC may be an approximately circular region or may have another shape. In addition, whether each pixel belongs to the on-axis region ArC or the off-axis region ArM is stored in advance in the memory unit 15. Therefore, the camera control unit 14 can determine whether a pixel or a region belongs to the on-axis region ArC or the off-axis region ArM by referring to the memory unit 15.

[0096] The pixel 21C arranged in the on-axis region ArC receives light incident from a direction substantially perpendicular to the light receiving surface, as shown in Fig. 16. On the other hand, the pixel 21M arranged in the off-axis region ArM receives light incident from an oblique direction to the light receiving surface, as shown in the figure.

[0097] The pixels 21 of the imaging element 7 are configured such that the more distant each portion is from the center, the more offset each portion is toward the center. The light-shielded pixel 21AR shown in FIG. 6 is, for example, an example of a pixel 21C arranged approximately in the center of the on-axis region ArC. The light-shielded pixel 21AR as the pixel 21M arranged in the off-axis region ArM will be described with reference to FIG.

[0098] As shown in the figure, an inner lens 32 and a color filter 33 are disposed offset toward the exit pupil EP with respect to the PD 30 and the light blocking portion 31. In addition, an on-chip microlens 34 is disposed offset toward the exit pupil EP with respect to the inner lens 32 and the color filter 33. With this configuration, the light obliquely incident from the exit pupil EP can be made to enter the PD 30 efficiently.

[0099] Next, FIG. 18 shows the PD divided pixel 21B as the pixel 21M arranged in the off-axis region ArM. As shown in the figure, a pixel boundary metal 41 is disposed in front of the left PD 40L and the right PD 40R, and an inner lens 32 and a color filter 33 are disposed in front of the pixel boundary metal 41, offset toward the exit pupil EP side with respect to the pixel boundary metal 41. Also, an on-chip microlens 34 is disposed offset toward the exit pupil EP side with respect to the inner lens 32 and the color filter 33.

[0100] Regardless of whether the pixel 21 is a light-shielded pixel 21A or a PD divided pixel 21B, the offset amount of each part, such as the inner lens 32 and the color filter 33, is determined according to the arrangement position of the pixel 21. This makes it easy to form each part on the imaging element 7. In addition, by forming each part with an offset, light incident obliquely from the exit pupil EP can be efficiently made incident on the left PD 40L and the right PD 40R, and light that has passed through the left pupil region EPL is incident on the right PD 40R, and light that has passed through the right pupil region EPR is incident on the left PD 40L. In other words, a pupil division function is realized.

[0101] <5. Pupil distance and pixel configuration> The angle of light incident on pixel 21 is also affected by the distance between the exit pupil and image sensor 7. The closer the distance between the exit pupil and image sensor 7, the larger the angle of incidence on pixel 21. When the angle of incidence changes, the light receiving area of ​​PD 30 that receives light that has passed through the left pupil area EPL changes. For example, FIG. 17 shows a configuration in which the pupil distance of the exit pupil EP is set to a predetermined distance, and the right aperture 35R is set to be larger than half the surface area of ​​the PD 30.

[0102] When the lens barrel 3 is an interchangeable lens, the pupil distance of the exit pupil EP changes depending on the type of optical system 16. When the exit pupil EP is closer than a predetermined distance, the light-shielded pixel 21AR shown in FIG 17 receives not only light that has passed through the left pupil region EPL of the exit pupil EP, but also light that has passed through the right pupil region EPR.

[0103] Therefore, in this embodiment, a plurality of types of light-shielded pixels 21AR and light-shielded pixels 21AL are provided according to the pupil distance of the exit pupil EP.

[0104] A specific description will be given with reference to Figures 17 and 19. Figure 17 shows an optical system 16 in which the pupil distance of the exit pupil EP is set to a predetermined distance, and a right opening 35R of the light-shielding portion 31 is formed so that light that has passed through the left pupil region EPL is received by the PD 30.

[0105] On the other hand, FIG. 19 shows an optical system 16 in which the pupil distance of the exit pupil EP is shorter than a predetermined distance, and a right opening 35R of the light-shielding portion 31 is formed so that light that passes through the left pupil region EPL is received by the PD 30. As can be seen from both figures, in order for the PD 30 to receive light that has passed through the left pupil region EPL in the optical system 16 with a short pupil distance, it is necessary to accommodate incident light that is incident at a greater angle. That is, in the configuration shown in FIG. 19, the right opening 35R is narrowed, and the opening end of the right opening 35R on the exit pupil EP side is positioned farther away from the exit pupil EP. By adopting such a configuration, the light-shielded pixels 21AR and 21AL can be adapted to a case where the pupil distance of the exit pupil EP is short, and the pupil division function can be performed.

[0106] In the light-shielded pixels 21AR shown in FIGS. 17 and 19, the offset amounts of the inner lens 32, the color filter 33, and the on-chip microlens 34 are the same. That is, only the shape of the right opening 35R of the light blocking portion 31 is different.

[0107] The light-shielded pixels 21AR and 21AL shown in FIG. 17 and the light-shielded pixels 21AR and 21AL shown in FIG. In this example, eight types of light-shielding pixels 21AR and light-shielding pixels 21AL corresponding to the pupil distance are arranged on the image sensor 7. That is, the eight types of light-shielding pixels 21AR and light-shielding pixels 21AL are provided with right openings 35R and left openings 35L having eight different opening areas. In the following description, the eight types of pupil distances are designated pupil distances S0 to S7 in order from the shortest pupil distance. The light-shielded pixels 21AR and 21AL corresponding to pupil distance S0 are designated as light-shielded pixels RS0 and LS0. Similarly, the light-shielded pixels 21AR and 21AL corresponding to pupil distance S1 are designated as light-shielded pixels RS1 and LS1, the light-shielded pixels 21AR and 21AL corresponding to pupil distance S2 are designated as light-shielded pixels RS2 and LS2, the light-shielded pixels 21AR and 21AL corresponding to pupil distance S3 are designated as light-shielded pixels RS3 and LS3, the light-shielded pixels 21AR and 21AL corresponding to pupil distance S4 are designated as light-shielded pixels RS4 and LS4, the light-shielded pixels 21AR and 21AL corresponding to pupil distance S5 are designated as light-shielded pixels RS5 and LS5, the light-shielded pixels 21AR and 21AL corresponding to pupil distance S6 are designated as light-shielded pixels RS6 and LS6, and the light-shielded pixels 21AR and 21AL corresponding to pupil distance S7 are designated as light-shielded pixels RS7 and LS7.

[0108] In addition, the first pixel row in which light-shielding pixels RS0, LS0 are arranged is referred to as first pixel row 22A0, the first pixel row in which light-shielding pixels RS1, LS1 are arranged is referred to as first pixel row 22A1, the first pixel row in which light-shielding pixels RS2, LS2 are arranged is referred to as first pixel row 22A2, the first pixel row in which light-shielding pixels RS3, LS3 are arranged is referred to as first pixel row 22A3, the first pixel row in which light-shielding pixels RS4, LS4 are arranged is referred to as first pixel row 22A4, the first pixel row in which light-shielding pixels RS5, LS5 are arranged is referred to as first pixel row 22A5, the first pixel row in which light-shielding pixels RS6, LS6 are arranged is referred to as first pixel row 22A6, and the first pixel row in which light-shielding pixels RS7, LS7 are arranged is referred to as first pixel row 22A7.

[0109] FIG. 20 shows an example of the arrangement of first pixel rows 22A0 to 22A7. 20 is an enlarged view of a portion of the image sensor 7. As shown in the figure, a first pixel row 22A0 in which light-shielded pixels RS0, LS0 are arranged to a first pixel row 22A7 in which light-shielded pixels RS7, LS7 are arranged are vertically spaced a certain distance apart, and each pixel row from the first pixel row 22A0 to the first pixel row 22A7 is periodically arranged in the vertical direction.

[0110] Each of the first pixel rows 22A0 to 22A7 outputs phase difference signals SG0 to SG7 for calculating the defocus amount, and the subsequent defocus amount calculation unit 8a selects an appropriate phase difference signal SG according to the pupil distance of the exit pupil EP to calculate the defocus amount. For example, when the pupil distance of the exit pupil EP is pupil distance S3, the defocus amount is calculated using the phase difference signal SG3 output from the first pixel row 22A3.

[0111] Next, the relationship between the pupil distance and the PD split pixel 21B as the pixel 21M arranged in the off-axis region ArM will be described. In the PD split pixel 21B arranged in the off-axis region ArM, the condition that only the light that has passed through the left pupil region EPL is incident on the right PD 40R, and only the light that has passed through the right pupil region EPR is incident on the left PD 40L, occurs only when an optical system 16 having a pupil distance as designed for the image sensor 7 is used. In this case, the light that has passed through the center of the exit pupil EP is incident on the boundary between the two split pixels, so that the pupil division function works properly.

[0112] However, when an optical system 16 having a pupil distance different from the pupil distance designed for the image sensor 7 is used, the light passing through the center of the exit pupil EP is biased toward one of the divided pixels. In order to make the pupil division function function properly in this state, it is possible to change the ratio of the sizes of the two divided pixels so that the light passing through the center of the exit pupil EP is incident on the boundary between the two divided pixels. In other words, just as multiple types of light-shielded pixels 21A are provided on the image sensor 7, it is necessary to provide PD divided pixels 21B on the image sensor 7 corresponding to each of the pupil distances S0 to S7.

[0113] However, changing the area ratio between the two split pixels for each PD split pixel 21B is technically difficult and leads to an increase in manufacturing costs. Therefore, in the imaging device 1 according to the present embodiment, only the PD divided pixels 21B corresponding to a specific pupil distance (for example, pupil distance S3) are provided on the imaging element . Although an example in which eight types of light-shielding pixels 21AR and 21AL are provided according to pupil distance has been described here, other types may be used as well. That is, only one type of light-shielding pixels 21AR and 21AL may be provided regardless of pupil distance, or multiple types may be provided according to pupil distance.

[0114] <6. Differences in AF control due to differences in phase detection pixels> It has been explained that the AF control performed based on the phase difference signal output from the imaging element 7 includes a method using the light-shielded pixels 21A and a method using the PD divided pixels 21B. Here, the differences will be described with reference to FIG.

[0115] First, regarding the use of each pixel, the light-shielding pixel 21A is a pixel dedicated to phase difference detection. That is, it is a pixel that is not used to generate a normal image signal. A normal image signal is a signal for generating an image of a subject. Since the light-shielding pixel 21A blocks a part of the light that enters the pixel area by the light-shielding portion 31, the output signal of the light-shielding pixel 21A is not used to generate a normal image signal. On the other hand, the PD split pixel 21B is a pixel that is used for detecting a phase difference and also for generating a normal image signal. The PD split pixel 21B can be treated as a normal pixel by adding together the outputs of the left PD 40L and the right PD 40R.

[0116] As for the number of arrangements, as shown in FIG. 5, the number of light-shielded pixels 21A is reduced, and the number of PD divided pixels 21B is increased. The light-shielded pixels 21A cannot be used as normal pixels (i.e., pixels for generating an image of a subject), and therefore cannot be arranged in large numbers. In addition, since the light-shielded pixels 21A cannot be arranged in large numbers, it is not possible to include many light-shielded pixels having the same light-shielded region, and output signals cannot be added. Therefore, when AF control is performed using the light-shielded pixels 21A, the low-illumination performance is degraded. On the other hand, the PD divided pixels 21B can be used as normal pixels, so they can be arranged in large numbers, and since the left PD 40L and the right PD 40R are the same size, multiple output signals can be added together. This improves the S / N ratio and provides high low-illumination performance.

[0117] The degree of freedom in pupil correction design will be described. It is relatively easy to create different light-shielding portions 31 with different light-shielding areas for the light-shielding pixels 21A. Therefore, it is easy to provide a plurality of types of light-shielding pixels 21A according to a plurality of pupil distances (light-shielding pixels RS0 to RS7, LS0 to LS7), so that a highly reliable phase difference signal can be output even if the pupil distance differs. Therefore, the light-shielding pixels 21A have a degree of freedom in pupil correction design. On the other hand, since it is difficult to manufacture the PD split pixel 21B by changing the size ratio of the left PD 40L and the right PD 40R, it is difficult to provide a plurality of types of PD split pixels 21B according to a plurality of pupil distances. Therefore, in the present embodiment, only the PD split pixel 21B corresponding to a specific pupil distance is provided on the image sensor 7. Therefore, if the pupil distance differs from the design, the reliability of the phase difference signal decreases. In other words, the PD split pixel 21B does not have freedom of pupil correction design.

[0118] The off-axis performance depends on the degree of freedom in pupil correction design. The off-axis performance is the accuracy of AF control for a subject exposed in an off-axis area ArM other than the on-axis area ArC, which is a rectangular area set near the center of the image sensor 7. Since the light-shielded pixel 21A has a degree of freedom in pupil correction design, even if light that has passed through the exit pupil EP is obliquely incident on the light-shielded pixel 21A located in the off-axis region ArM, an appropriate phase difference signal is output from either the light-shielded pixels RS0 to RS7 or the light-shielded pixels LS0 to LS7. Therefore, the off-axis performance of the light-shielded pixel 21A is high. On the other hand, since the PD split pixel 21B does not have freedom of pupil correction design, when the pupil distance differs from the design, the pupil division function does not work well when light that has passed through the exit pupil EP is obliquely incident on the PD split pixel 21B located in the off-axis region ArM, and the reliability of the output phase difference signal becomes low. Therefore, the off-axis performance when the PD split pixel 21B is used is lower than that when the light-shielded pixel 21A is used.

[0119] <7. Phase difference signal selection> In the present embodiment, which of the phase difference signals output from the light-shielded pixel 21A and the PD division pixel 21B is selected when calculating the defocus amount will be described with reference to FIG.

[0120] The graph shown in the upper part of Fig. 22 shows the change in AF error in the on-axis area ArC with respect to the amount of exposure. The solid line in the graph is for the case where AF control is performed based on the phase difference signal output from the light-shielded pixel 21A. The dashed line in the graph is for the case where AF control is performed based on the phase difference signal output from the PD division pixel 21B.

[0121] Below the graph in FIG. 22, phase difference signals selected for each combination of exposure amount and target area of ​​AF control are shown.

[0122] First, it is possible to select either the phase difference signal output from the light-shielding pixel 21A or the PD divided pixel 21B depending on the amount of exposure. For example, when the amount of exposure is equal to or greater than a threshold Th, the phase difference signal output from the light-shielding pixel 21A is selected. This is because the phase difference signal output from the light-shielding pixel 21A has a short readout time (described later in detail), and it is desirable to select the phase difference signal output from the light-shielding pixel 21A if there is no problem in calculating the defocus amount. On the other hand, when the exposure amount is less than the threshold value Th, if the phase difference signal output from the light-shielded pixel 21A is selected, the reliability of the calculated defocus amount is reduced, so it is possible to select the phase difference signal output from the PD division pixel 21B. Note that, in this example, when the exposure amount is less than the threshold value Th, either one of the phase difference signals output from the light-shielded pixel 21A and the PD division pixel 21B, or the contrast method is selected in consideration of other conditions.

[0123] Let me explain in detail. When the exposure amount is equal to or greater than the threshold Th, the phase difference signal output from the light-shielded pixel 21A is selected regardless of whether it is the on-axis region ArC or the off-axis region ArM. When the exposure amount is equal to or greater than the threshold Th, the readout of the charge obtained by photoelectrically converting the light received by the light-shielded pixel 21A and the PD division pixel 21B will be described with reference to FIGS. 23 and 24.

[0124] The signals output from the light-shielded pixels 21A are read out as phase difference signals SGn for each pixel row. The phase difference signals SGn are any of the phase difference signals SG0 to SG7, and an appropriate phase difference signal SGn is selected according to the pupil distance.

[0125] The signal output from the PD divided pixel 21B is used to generate a normal image signal, so the sum of the outputs of the divided pixels is read out. Figure 24 shows the readout timing of the signals from the light-shielded pixel 21A and the PD divided pixel 21B.

[0126] As shown in the figure, in synchronization with the vertical synchronization signal Vsync, readout of the first pixel row 22A including the light-shielded pixels 21A from which the phase difference signals SGn are acquired is performed sequentially from the top of the image sensor 7. A readout time required to read out all of the phase difference signals SGn output from the image sensor 7 is set to time T1.

[0127] The signals output from the PD split pixel 21B include a signal SGR output from the second pixel row 22B to which the PD split pixels 21Br and 21Bg belong, and a signal SGB output from the second pixel row 22B to which the PD split pixels 21Bb and 21Bg belong. The signals SGR and SGB are read out in order from the top of the image sensor 7. The readout time required to read out all of the signals SGR and SGB output from the image sensor 7 is set to time T2.

[0128] Since the number of the PD divided pixels 21B is greater than the number of the light-shielded pixels 21A, the time T2 is longer than the time T1.

[0129] When the exposure amount is equal to or greater than the threshold Th, rapid AF control can be achieved by using the phase difference signal from the light-shielded pixel 21A, which has a short readout time. In the off-axis area ArM, the reliability of the phase difference signal output from the PD divided pixel 21B decreases as the pupil distance becomes farther from the designed distance, but this can be avoided by selecting the phase difference signal output from the light-shielded pixel 21A in the off-axis area ArM when the exposure amount is equal to or greater than the threshold Th. Furthermore, since the signal readout time of the light-shielded pixel 21A is shorter, it is possible to shorten the time required for focusing by AF control.

[0130] Next, the case where the exposure amount is less than the threshold value Th will be described. In the on-axis region ArC, the phase difference signal output from the PD division pixel 21B is selected. Readout of the electric charge obtained by photoelectrically converting the light received by the light-shielded pixel 21A and the PD division pixel 21B will be described with reference to FIGS. 25 and 26.

[0131] The output signal from the first pixel row 22A in which the light-shielded pixels 21A are arranged is used to generate a normal image signal. Specifically, only the output signal from each divided pixel as the PD divided pixel 21Bg arranged between the light-shielded pixels 21A is read out.

[0132] Since the output signal from the second pixel row 22B in which the PD divided pixel 21B is arranged is used as a phase difference signal, the output of each divided pixel is read out (see FIG. 25). Also, since the exposure amount is less than the threshold Th, in order to improve the S / N ratio, the necessary number of outputs from adjacent second pixel rows 22B are added to improve the S / N ratio of the phase difference signal. In FIG. 25, two second pixel rows 22B are added, but if the exposure amount further decreases, three or more second pixel rows 22B are added. This makes it possible to suppress deterioration of the S / N ratio due to a decrease in the exposure amount, and to perform appropriate AF control.

[0133] The phase difference signals from the second pixel row 22B include a phase difference signal SGr output from the second pixel row 22B including the PD divided pixel 21Br, and a phase difference signal SGb output from the second pixel row 22B including the PD divided pixel 21Bb. When adding the phase difference signals, the phase difference signals SGr are added together, and the phase difference signals SGb are added together.

[0134] The timing of reading out the signal from the PD divided pixel 21B is shown in Fig. 26. Note that the signal from the light-shielded pixel 21A is not used for AF control or image signal generation, and is therefore not read out.

[0135] The phase difference signals SGr and SGb output from the PD segment pixels 21B are read out in sequence from the top of the imaging element . The readout time required to read out all of the phase difference signals SGr and SGb output from the image sensor 7 is set to time T3. Time T3 is set to be approximately twice the time T2 in Fig. 24. This is based on the difference between whether readout is performed in pixel units or in divided pixel units, as shown in Figs. 23 and 25.

[0136] Next, a phase difference signal when the exposure amount is less than the threshold value Th and the off-axis area ArM is described. In this case, as shown in FIG. 22, the phase difference signal output from the light-shielded pixel 21A is selected. In this case, as shown in Figures 23 and 24, the phase difference signal SGn output from the first pixel row 22A is read out at time T1, and the signals SGR and SGB for generating the normal image signal output from the second pixel row 22B are read out at time T2.

[0137] However, when the optical system 16 has an ideal pupil distance set for the PD divided pixel 21B, it is possible to select the phase difference signal output from the PD divided pixel 21B even in AF control of the off-axis area ArM. In this case, as shown in Figures 25 and 26, the phase difference signal SGn output from the light-shielded pixel 21A in the first pixel row 22A is not read out, and the outputs from the PD divided pixels 21B arranged in the first pixel row 22A and the second pixel row 22B are read out at time T3.

[0138] In addition, in the off-axis region ArM when the optical system 16 has an ideal pupil distance set for the PD split pixel 21B, it may not be clear which of the phase difference signal output from the light-shielded pixel 21A and the phase difference signal output from the PD split pixel 21B has higher reliability. In that case, both the phase difference signal of the light-shielded pixel 21A and the phase difference signal of the PD split pixel 21B may be acquired and AF control may be performed using both of them.

[0139] A case in which both the phase difference signal from the light-shielded pixel 21A and the phase difference signal from the PD division pixel 21B are acquired is shown in FIGS. 27 and 28. FIG. As shown in FIG. 27, a phase difference signal SGn output from a first pixel row 22A in which the light-shielded pixels 21A are arranged, and phase difference signals SGr, SGb output from a second pixel row 22B in which the PD divided pixels 21B are arranged are read out.

[0140] The readout time of the phase difference signal SGn is set to time T1, and the readout time of the phase difference signal SGr and the phase difference signal SGb is set to time T3. The total readout time obtained by adding the time T1 and the time T3 is set to be shorter than one period of the vertical synchronization signal Vsync.

[0141] 22, in the off-axis region ArM where the exposure amount is less than the threshold value Th, the phase difference signal output from the light-shielding pixel 21A is used. However, there are cases where sufficient AF control cannot be performed even if the phase difference signal output from the light-shielding pixel 21A is used. For example, there are cases where the exposure amount is significantly lower than the threshold value Th and further the pupil distance of the exit pupil is significantly different from the design of the PD divided pixel 21B. In such a case, the AF control may be performed using the contrast method described above, which allows appropriate AF control to be performed under a wide range of conditions.

[0142] <8. Control of exposure amount> Selection of a phase difference signal based on the amount of exposure has been described above. Here, calculation of the amount of exposure will be described with reference to FIG. The image sensor 7 outputs a normal pixel output for generating a normal image signal, and a phase difference pixel output as a phase difference signal for AF control.

[0143] The normal pixel output and the phase difference pixel output are each input to an output level detection circuit provided in the camera signal processing unit 8. The output level detection circuit calculates an output average value in an exposure calculation target area on the pixel based on the input normal pixel output and phase difference pixel output, and each output average value is output from the camera signal processing unit 8 and input to the camera control unit 14. The exposure calculation target area is an area determined according to the metering mode, and is the area in the image that is the subject of exposure calculation. For example, when "center-weighted metering mode" is selected, the area in the center of the image is the subject of exposure calculation. When "spot metering mode" is selected, a specific, narrow area is designated as the subject of exposure calculation. When "full-screen average metering mode" is selected, the entire image is the subject of exposure calculation.

[0144] The camera control unit 14 performs exposure calculation according to the detection result output from the camera signal processing unit 8, and determines the shutter speed (or parameters capable of adjusting the exposure, such as the F-number and gain). The camera control unit 14 performs processing to set the determined shutter speed in the image sensor 7. The shutter speed is set, for example, by setting the travel timing of the electron back film.

[0145] The exposure amount calculation performed by the camera control unit 14 may be performed based on only the normal pixel output, or may be performed based on only the phase difference pixel output. Also, it may be performed based on both the normal pixel output and the phase difference pixel output, in which case the exposure control of the phase difference pixels is performed based on the phase difference pixel output, and the exposure control of the normal pixels is performed based on the normal pixel output.

[0146] <9. Processing flow> <9-1. First Example> A first example of the processing flow of the AF control is shown in FIGS. 30, 31, and 32 are processes executed by the camera control unit 14 and the camera signal processing unit 8 of the imaging device 1. Here, the camera control unit 14 and the camera signal processing unit 8 are collectively referred to as the "processing unit 60."

[0147] When the processing unit 60 (camera control unit 14 and camera signal processing unit 8) of the imaging device 1 detects that the shutter button 6S, which is one of the controls 6, is half-pressed, the processing unit 60 performs an exposure amount calculation in step S101. In the exposure amount calculation, as described with reference to Fig. 29, detection processing is performed on the output levels of either or both of the normal pixel output and the phase difference pixel output, and the exposure amount is calculated based on the detection result.

[0148] The processing unit 60 determines in step S102 whether the exposure amount is equal to or greater than a threshold value Th. If the exposure amount is equal to or greater than the threshold Th, the processing unit 60 performs additive readout setting on the PD split pixel 21B in step S103. The additive readout setting is a setting for adding up the charges accumulated in the left PD 40L and the right PD 40R of the PD split pixel 21B and reading them out at once. In other words, the PD split pixel 21B is treated as one pixel.

[0149] Next, the processing unit 60 reads out the light-shielded pixels 21A in step S104, and then performs additive reading out of the PD division pixels 21B in step S105. As a result, the phase difference signal read out from the light-shielded pixels 21A and the normal pixel signal read out from the PD division pixels 21B are read out.

[0150] In step S107, the processing unit 60 calculates the defocus amount based on the phase difference signal output from the light-shielding pixel 21A. At this time, the processing unit 60 calculates the defocus amount using the output from the first pixel row 22A0-22A7 including the light-shielding pixel 21A appropriate for the pupil distance of the exit pupil EP. Note that the pupil distance of the exit pupil EP is obtained by the camera control unit 14 from the lens barrel control unit 18 when the lens barrel 3 is attached to the imaging device 1, for example, and stored in the memory unit 15.

[0151] In step S108, the processing unit 60 generates display image data for a through image based on the pixel signal from the PD divided pixel 21B. The display image data generated here is displayed as a through image on the EVF monitor 5a, the rear monitor 4, etc.

[0152] Processing unit 60 drives the lens based on the defocus amount in step S109 in Fig. 31, and stops the lens drive in step S110. As a result, the focus lens arranged inside lens barrel 3 is driven via driver unit 17 and focused.

[0153] In step S111, the processing unit 60 determines whether or not the half-pressed state of the shutter button 6S is being detected. If the half-pressed state of the shutter button 6S continues, the processing unit 60 returns to the process of step S101 in FIG. 30 and performs focus control according to the exposure amount. This allows the focus state to be continuously maintained as long as the shutter button 6S remains half-pressed.

[0154] If it is determined in step S102 that the exposure amount is less than the threshold Th, the processing unit 60 performs non-additive readout setting for the PD split pixel 21B in step S112. The non-additive readout setting is a setting for reading out the charges accumulated in the left PD 40L and the right PD 40R of the PD split pixel 21B so as to be distinguishable from each other. That is, each split pixel is treated independently.

[0155] The processing unit 60 performs reading of the light-shielded pixel 21A in step S113, and performs non-additive reading of the PD divided pixels in step S114.

[0156] In step S115, the processing unit 60 calculates a defocus amount based on the phase difference signal output from the light-shielded pixel 21A. In addition, in step S116, the processing unit 60 calculates a defocus amount based on the phase difference signal output from the PD divided pixel 21B.

[0157] By executing the processes from step S113 to step S116, a state is created in which focusing control based on the phase difference signals of the light-shielded pixel 21A and the PD divided pixel 21B can be performed.

[0158] In step S117 of FIG. 32, the processing unit 60 performs addition processing to combine the non-added signals output from each of the divided pixels of the PD divided pixel 21B for each PD divided pixel 21B, and performs processing to generate display image data for a through image.

[0159] In step S118, the processing unit 60 compares the difference integral value at the time of calculating the defocus amount between the light-shielded pixel 21A and the PD divided pixel 21B. The difference integral value represents the magnitude of the phase difference, as described with reference to each of Figs. 8 to 12. In calculating the defocus amount, the shift amount of the waveform is determined so that the difference integral value becomes small, and it can be said that the smaller the difference integral value at the time of finally calculating the defocus amount, the higher the reliability of the calculated defocus amount. Step S118 is a comparison process for determining which of the defocus amount calculated based on the output from the light-shielded pixel 21A and the defocus amount calculated based on the output from the PD divided pixel 21B has a higher reliability.

[0160] In step S119, the processing unit 60 determines whether the difference integral value in the light-shielded pixel 21A is smaller. If it is determined that the difference integral value in the light-shielded pixel 21A is smaller, that is, if it is determined that the defocus amount calculated based on the output of the light-shielded pixel 21A is more reliable, the processing unit 60 performs lens driving based on the defocus amount calculated based on the output of the light-shielded pixel 21A in step S120.

[0161] On the other hand, if it is determined in step S119 that the difference integral value in the PD split pixel 21B is smaller, that is, if it is determined that the defocus amount calculated based on the output of the PD split pixel 21B is more reliable, the processing unit 60 performs lens driving based on the defocus amount calculated based on the output of the PD split pixel 21B in step S121.

[0162] After executing the process of step S120 or step S121, the processing unit 60 stops the lens driving in step S122.

[0163] In step S123, the processing unit 60 determines whether or not the half-pressed state of the shutter button 6S is being detected. If the half-pressed state of the shutter button 6S continues, the processing unit 60 returns to the process of step S101 in Fig. 30 and performs focus control according to the exposure amount. This allows the focus state to be continuously maintained as long as the shutter button 6S remains half-pressed.

[0164] In addition, each process described in steps S103 to S111 in Figures 30 and 31 treats the light-shielded pixel 21A as a pixel that outputs a phase difference signal for focus control, and treats the PD divided pixel 21B as a pixel that outputs a normal pixel signal, as shown in Figures 23 and 24.

[0165] In addition, the processes described in steps S112 to S123 in Figures 30 and 31 treat each of the light-shielded pixel 21A and the PD divided pixel 21B as a pixel that outputs a phase difference signal, as shown in Figures 27 and 28, and treat the PD divided pixel 21B as a pixel that outputs a normal pixel signal.

[0166] As shown in Figures 25 and 26, when the exposure amount is less than the threshold Th, the light-shielded pixel 21A is not used, and the PD divided pixel 21B is treated as a pixel that outputs a phase difference signal and a pixel that outputs a normal pixel signal, this can be achieved by not executing the processes of steps S113 and S115 in Figure 30 and the processes of steps S118, S119, and S120 in Figure 32.

[0167] <9-2. Second Example> A second example of the processing flow of AF control will be described with reference to Fig. 30, Fig. 31, and Fig. 33. Note that the same processes as in the first example are given the same reference numerals and the description will be omitted as appropriate.

[0168] When the processing unit 60 (camera control unit 14 and camera signal processing unit 8) of the imaging device 1 detects that the shutter button 6S is half-pressed, it performs exposure amount calculation in step S101 of FIG. Next, in step S102, the processing unit 60 determines whether the exposure amount is equal to or greater than a threshold value Th. When the exposure amount is equal to or greater than the threshold Th, AF control is performed using the phase difference signal output from the light-shielded pixel 21A. The process in this case is the same as that in the first example from step S103 in FIG. 30 to step S111 in FIG.

[0169] On the other hand, if it is determined that the exposure amount is less than the threshold value Th, the processing unit 60 executes the processes of steps S112 to S116 in FIG. 30, and then executes step S117 in FIG. Thereby, a phase difference signal from the light-shielded pixel 21A and a phase difference signal from the PD division pixel 21B are acquired, and display image data for displaying a through image is generated based on the signal from the PD division pixel 21B.

[0170] Next, in step S131 of FIG. 33, the processing unit 60 determines whether the deviation between the pupil distance of the exit pupil EP for the PD split pixel 21B to output an appropriate phase difference signal and the pupil distance of the exit pupil EP of the optical system 16 is a predetermined value or more, that is, whether the pupil distance of the exit pupil EP of the optical system 16 is closer, farther, or the same as the information on the pupil distance of the exit pupil EP acquired from the lens barrel 3. As described above, the angle of incidence of light to pixel 21 varies depending on the pupil distance of the exit pupil EP. Therefore, if the pupil distance deviates from the design value, it is highly likely that a reliable phase difference signal cannot be obtained from the PD split pixel 21B. If it is determined that the deviation is a predetermined value or more, that is, if it is determined that the pupil distance of the exit pupil EP is closer than designed in the optical system 16 or is farther away, the phase difference signal output from the PD split pixel 21B arranged in the off-axis region ArM has low reliability, so in step S132, the processing unit 60 calculates the defocus amount using the phase difference signal output from the PD split pixel 21B in the on-axis region ArC and the phase difference signal output from the light-shielded pixel 21A in the off-axis region ArM.

[0171] Next, the processing unit 60 drives the lens based on the calculated defocus amount in step S133.

[0172] On the other hand, if it is determined in step S131 that the deviation in pupil distance is less than a predetermined value, that is, if it is determined that the pupil distance of the exit pupil EP is as designed in the optical system 16 or is close to the design, the reliability of the phase difference signal output from the PD divided pixel 21B arranged in the off-axis region ArM is high, so the processing unit 60 calculates the defocus amount using the phase difference signal output from the PD divided pixel in step S134. Next, in step S135, the processing unit 60 drives the lens based on the defocus amount.

[0173] After executing the process of step S133 or step S135, the processing unit 60 stops the lens driving in step S136.

[0174] In step S137, the processing unit 60 determines whether or not the half-pressed state of the shutter button 6S is being detected. If the half-pressed state of the shutter button 6S continues, the processing unit 60 returns to the process of step S101 in Fig. 30 and performs focus control according to the exposure amount. This allows the focus state to be continuously maintained as long as the shutter button 6S remains half-pressed.

[0175] In the above example, focusing is performed on both the on-axis area ArC and the off-axis area ArM, i.e., the entire surface of the image sensor 7. In addition, there are cases where focusing is performed on a specific area. For example, when reflected light from a person to be focused is received in the on-axis area ArC (i.e., when the person to be focused is captured in the on-axis area ArC), focusing can be performed on the on-axis area ArC (or the area within that where the person to be focused is captured). In that case, the determination process in step S131 is not executed, and the defocus amount is calculated in step S132 using the phase difference signal output from the PD divided pixel 21B in the on-axis area ArC. Alternatively, if the person to be focused is in the off-axis area ArM, focusing may be performed on the off-axis area ArM. Specifically, if the result of step S131 is “Yes”, the defocus amount is calculated using the phase difference signal output from the light-shielded pixel 21A in the off-axis area ArM (step S132), and if the result of step S131 is “No”, the defocus amount is calculated using the phase difference signal output from the PD divided pixel 21B in the off-axis area ArM (step S134). In addition, there may be cases where the person to be focused straddles the on-axis area ArC and the off-axis area ArM. In such cases, the defocus amount may be calculated by performing the processes in steps S131 to S135 as described above.

[0176] Alternatively, when the person or object to be focused has not been decided, such as when there are no instructions from the user, the defocus amount is calculated (steps S132, S134) for each on-axis area ArC or off-axis area ArM, or for each subject (each area in which a single subject is captured), and then a determination process for determining the focus target or a process for having the user select the focus target or ROI (Region of Interest) is performed before driving the lens in steps S133 and S135, and the lens is driven according to the results of the determination process or selection process.

[0177] <10. Variations> In the above-described example, AF control is performed based on the calculated defocus amount. In other examples, notification control and display control for the user may be performed based on the defocus amount, and the focus operation may be performed manually. For example, the UI control unit 14a of the camera control unit 14 may display information related to focusing on the EVF monitor 5a or the rear monitor 4, thereby assisting the user in performing manual focus operations.

[0178] In addition, although the above-mentioned light-shielded pixel 21A has been described as having one PD, it may have multiple PDs as split pixels. For example, all pixels of the image sensor 7 may be PD split pixels consisting of two split pixels. That is, each pixel may have a left PD 40L and a right PF 40R. Even in such a configuration, the light-shielded pixel 21A can handle the two split pixels as one PD by adding and reading out the outputs from the two split pixels. Since all the pixels on the imaging element 7 have the same configuration, the imaging element 7 can be manufactured easily and the manufacturing costs can be reduced.

[0179] <11. Summary> As described in the above examples, the imaging device 1 in the present technology is equipped with an imaging element 7 that includes a light-shielding pixel 21A having a pupil division function by being equipped with a light-shielding portion 31 that blocks one of a pair of light beams that pass through a pair of partial regions (left pupil region EPL and right pupil region EPR) that are biased in opposite directions in a predetermined direction (e.g., the left-right direction) in the exit pupil EP and a light-receiving element (PD30) that receives the other light beam, and a photodiode split pixel (PD split pixel 21B) having a pupil division function by being equipped with split pixels (left PD40L and right PD40R) that receive each of the pair of light beams that pass through the pair of partial regions. The light-shielded pixel 21A is composed of, for example, a light-shielded pixel 21AR which is a pixel into which only light that has passed through a left pupil region EPL which is defined by the light-shielding portion 31 as the left half of the exit pupil EP is incident, and a light-shielded pixel 21AL which is a pixel into which only light that has passed through a right pupil region EPR which is defined by the light-shielding portion 31 as the right half of the exit pupil is incident. The PD split pixel 21B is a so-called PD split pixel in which a plurality of split pixels (left PD 40L and right PD 40R) are arranged in an area equivalent to one pixel. Each pixel is, for example, a color pixel in a Bayer array, and includes one color filter 33 and a light receiving element (PD 30, left PD 40L, right PD 40R) that receives incident light passing through the color filter 33. It also includes a defocus amount calculation unit 8a that calculates the defocus amount using at least one of the signal (phase difference signal SG) output from the light-shielded pixel 21A and the signal (phase difference signals SGr, SGb) output from the PD division pixel 21B based on the exposure amount. Both the light-shielded pixel 21A and the PD divided pixel 21B output a phase difference signal from which a defocus amount can be calculated. Therefore, an appropriate defocus amount can be calculated using an appropriate phase difference signal based on the exposure amount among the phase difference signals output from the light-shielded pixel 21A and the PD divided pixel 21B.

[0180] Furthermore, each pixel included in the imaging element 7 may be either a light-shielded pixel 21A or a PD divided pixel 21B. That is, all pixels of the imaging element 7 are either the light-shielded pixels 21A or the PD divided pixels 21B. Therefore, the manufacturing process of the imaging element 7 does not need to be complicated, and the manufacturing time and cost can be reduced.

[0181] As described with reference to FIG. 22 in the selection of the phase difference signal, the defocus amount calculation unit 8a may calculate the defocus amount using the output signal (phase difference signal SG) of the light-shielded pixel 21A when the exposure amount is equal to or greater than the threshold value Th. Since the light-shielded pixel 21A includes the light-shielding portion 31, the output signal level is made smaller than that of the PD divided pixel 21B. In addition, the number of light-shielding pixels 21A is smaller than that of the PD division pixels 21B, and the light-shielding pixels 21A are arranged more discretely. Therefore, when the amount of exposure is small, the signals output from the plurality of pixels cannot be added to increase the signal level, and it is difficult to improve the S / N ratio. This may reduce the reliability of the calculated defocus amount. However, if the amount of exposure is equal to or greater than a threshold value Th, the output level of the phase difference signal from the light-shielding pixels 21A is set to a certain level or higher, so that it is possible to increase the reliability of the defocus amount calculated using the output signal from the light-shielding pixels 21A.

[0182] As explained with reference to Figure 22 regarding the selection of the phase difference signal, when the exposure amount is less than the threshold value Th, the defocus amount calculation unit 8a may calculate the defocus amount using at least one of the output signal of the light-shielding pixel 21A and the output signal of the PD division pixel 21B, depending on whether the area is an on-axis area ArC that is an area including the central portion of the image sensor 7, or an off-axis area ArM that is an area other than the on-axis area ArC of the image sensor 7. When the exposure amount is less than the threshold value Th, there are cases where it is more preferable to calculate the defocus amount using the output signal of the PD divided pixel 21B, which has high low-illumination performance, and cases where it is more preferable to calculate the defocus amount using the output signal of the light-shielding pixel 21A, which can select an appropriate output according to the pupil distance of the exit pupil EP. According to this configuration, it is possible to select an appropriate output signal of the image surface phase difference pixel, thereby making it possible to calculate a defocus amount with high reliability.

[0183] As described with reference to FIG. 22 in the selection of the phase difference signal, the defocus amount calculation unit 8a may calculate the defocus amount using the output signal (phase difference signals SGr, SGb) of the PD divided pixel 21B in the on-axis region ArC. Since the PD divided pixel 21B does not include the light-shielding portion 31, the output signal level is made higher than that of the light-shielded pixel 21A. In addition, the number of PD divided pixels 21B is greater than that of the light-shielded pixels 21A. Therefore, even when the exposure amount is low, the S / N ratio of the phase difference signal can be improved by adding the outputs of multiple pixel rows, and the output level of the phase difference signal required to calculate the defocus amount can be easily ensured. This makes it possible to calculate the focus position information and the lens driving amount with high accuracy.

[0184] As described with reference to FIG. 22 in the selection of the phase difference signal, the defocus amount calculation section 8a may calculate the defocus amount using the output signal of the light-shielded pixel 21A in the off-axis region ArM. This allows a highly reliable defocus amount to be calculated, improving the focusing accuracy.

[0185] As described with reference to FIG. 22 in the selection of the phase difference signal, the defocus amount calculation section 8a may calculate the defocus amount using the output signal of the photodiode divided pixel (PD divided pixel 21B) in the off-axis region ArM. This allows a highly reliable defocus amount to be calculated, improving the focusing accuracy.

[0186] As explained with reference to FIG. 22 regarding the selection of the phase difference signal, the defocus amount calculation unit 8a may calculate the defocus amount using the output signal having a higher reliability between the output signal of the light-shielded pixel 21A and the output signal of the photodiode split pixel (PD split pixel 21B). This allows a highly reliable defocus amount to be calculated, improving the focusing accuracy. Each divided pixel (left PD 40L, right PD 40R) of the PD divided pixel 21B has, for example, the same light receiving area. In this case, in order for the light passing through the left pupil region EPL of the exit pupil EP to be incident on one divided pixel (right PD 40R) and the light passing through the right pupil region EPR to be incident on the other divided pixel (left PD 40L), it is necessary for the optical system 16 to have a specific pupil distance according to the arrangement of the on-chip microlens 34 and the like provided in the PD divided pixel 21B. On the other hand, it is possible to provide a plurality of types of light-shielding pixels 21A (light-shielding pixels LS0 to LS7, light-shielding pixels RS0 to RS7) according to the pupil distance. Therefore, in the off-axis region ArM of the imaging element 7, the light-shielding pixels 21A may have a reduced signal level of the output signal due to a lack of light reception, and the PD divided pixel 21B may have a reduced reliability of the output signal due to an inappropriate pupil distance. Therefore, there is a risk of low reliability when the defocus amount is always calculated based on one of the phase difference signals. In the above example, when the exposure amount is less than the threshold value Th and in the off-axis region ArM, both the output signal of the light-shielded pixel 21A and the output signal of the PD divided pixel 21B are acquired, and the defocus amount is calculated using the phase difference signal with higher reliability. Therefore, a highly reliable defocus amount can be calculated.

[0187] As explained with reference to Figure 22 in the selection of the phase difference signal, when the exposure amount is less than the threshold value Th, the defocus amount calculation unit 8a may perform autofocus control based on the contrast method in the off-axis area ArM, which is an area other than the on-axis area ArC, which is an area including the central part of the image sensor 7. In the off-axis region ArM when the exposure amount is insufficient, the reliability of the phase difference information based on the output signal (phase difference signal SG) of the light-shielded pixel 21A and the output signal (phase difference signals SGr, SGb) of the PD division pixel 21B may both be low. In such a case, by adopting the contrast method, appropriate autofocus control can be performed.

[0188] As described with reference to FIG. 20 regarding the configuration of the image sensor 7, the image sensor 7 may have a plurality of types of light-shielded pixels 21A (light-shielded pixels LS0 to LS7, light-shielded pixels RS0 to RS7) according to the pupil distance of the exit pupil EP. As a result, even if the pupil position of the exit pupil EP changes as the optical system 16 is driven, an appropriate phase difference signal can be obtained from any one of the multiple types of light-shielded pixels 21A. Therefore, even if the pupil distance of the exit pupil EP varies over a wide range due to the use of an interchangeable lens barrel 3, for example, it is possible to calculate a highly reliable defocus amount.

[0189] As described with reference to FIGS. 17 and 19 in the configuration of the imaging element 7, the light-shielded pixels 21A (light-shielded pixels LS0 to LS7, light-shielded pixels RS0 to RS7) may have light-shielded portions 31 with different light-shielded regions. For example, the closer the pupil distance is, the larger the light blocking area of ​​the light blocking portion 31 is made. As a result, in each pixel included in the light-shielded pixel 21A, a light-shielding portion 31 is formed so that only light that passes through one side region of the exit pupil EP depending on the pupil distance is received, and an appropriate output signal (phase difference signal SG) of the light-shielded pixel 21A can be output.

[0190] As described with reference to FIG. 30 regarding the first example of the processing flow of AF control, the defocus amount calculation unit 8a may calculate the defocus amount using the output signal (phase difference signal SG) of the light-shielded pixel 21A selected according to the pupil distance of the exit pupil EP. As a result, the light-shielded pixel 21A that receives the light passing through one side area of ​​the exit pupil EP is selected. Therefore, it is possible to calculate the defocus amount with high reliability by using the output signal of the appropriate light-shielded pixel 21A.

[0191] As explained with reference to FIG. 30 regarding the first example of the processing flow of AF control, a camera control unit 14 that acquires the pupil distance from a barrel control unit 18 that the lens barrel 3 has may be provided. For example, in an imaging device 1 equipped with an interchangeable lens barrel 3, a light-shielded pixel 21A (light-shielded pixels LS0 to LS7 and a part of light-shielded pixels RS0 to RS7) is selected according to the pupil distance.

[0192] As described with reference to FIG. 20 in the configuration of the image sensor 7, the light-shielded pixels 21A arranged in the same row (same pixel row) on the image sensor 7 may correspond to the same pupil distance. This prevents output signals from a plurality of types of light-shielded pixels 21A from being mixed into pixel signals read out for each pixel row. Therefore, it is easy to process the output signal of the light-shielded pixel 21A selected in accordance with the pupil distance, and the processing load is reduced.

[0193] As described with reference to Figures 31 and 33 for the first and second examples of the processing flow of AF control, the camera may be provided with a camera control unit 14 that issues driving instructions to the focus lens of the imaging optical system (optical system 16) based on the defocus amount. In this way, the focus lens is controlled by a phase difference signal based on the amount of exposure. Therefore, appropriate focus control based on the amount of exposure can be performed.

[0194] As described in the modified example, a user interface control unit (UI control unit 14a) that performs display control based on the defocus amount may be provided. This makes it possible to provide information to inform the photographer of the focus state according to the current lens position, for example. Therefore, the photographer can achieve focus by performing a focus operation based on the notification information.

[0195] As described with reference to FIG. 23 regarding the selection of the phase difference signal, when the output signal (phase difference signal SG) of the light-shielded pixel 21A is used to calculate the defocus amount, the image sensor 7 may add and output the output signals of the PD division pixel 21B (left PD 40L, right PD 40R). This makes it possible to reduce the number of times the output from each divided pixel is read out, compared to reading the output from each divided pixel separately. Therefore, the time required to read out pixel signals can be reduced.

[0196] As described with reference to FIG. 25 regarding the selection of the phase difference signal, when the defocus amount is calculated using the output signals (phase difference signals SGr, SGb) of the PD division pixel 21B, the image sensor 7 may output each of the output signals of the PD division pixel 21B (left PD40L, right PD40R). This allows the output signals of the divided pixels to be acquired without being added. Therefore, the output signal of the PD split pixel 21B can be acquired without losing phase difference information, and the defocus amount can be calculated.

[0197] As explained with reference to FIG. 25 regarding the selection of the phase difference signal, a signal processing unit (camera signal processing unit 8) is provided that performs signal processing on the image signal output from the imaging element 7, and the signal processing unit may add output signals of the PD split pixels 21B in the column direction when the exposure amount is less than a predetermined amount (for example, when the exposure amount is such that the output level is less than a predetermined amount). This makes it possible to keep the output level of the signal (phase difference signals SGr, SGb) output from the PD divided pixel 21B at a predetermined level or higher. Therefore, the S / N ratio of the output signal of the PD divided pixel 21B can be improved, and the reliability of the calculated defocus amount can be increased.

[0198] The program of the embodiment is a program that causes, for example, a CPU, a DSP, or a device including these to execute each of the processes shown in Figs. In other words, the program of the embodiment is a program that causes an imaging device or the like to execute a process of selecting one of the phase difference signals based on the exposure amount and calculating the defocus amount from an output signal of a light-shielding pixel having a pupil division function by having a light-shielding part that blocks one of a pair of light beams that have passed through a pair of partial regions that are biased in opposite directions to each other in a predetermined direction in the exit pupil and a light-receiving element that receives the other light beam, and an output signal of a PD split pixel having a pupil division function by having a split pixel that receives each of the pair of light beams that have passed through the pair of partial regions. The imaging device 1 described above can be realized by such a program.

[0199] A program for realizing such an imaging device 1 can be recorded in advance in a HDD serving as a recording medium built into a device such as the imaging device 1, or in a ROM or the like in a microcomputer having a CPU. Alternatively, the software may be temporarily or permanently stored (recorded) on a removable recording medium such as a flexible disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magnet optical) disk, a DVD (Digital Versatile Disc), a Blu-ray Disc (registered trademark), a magnetic disk, a semiconductor memory, a memory card, etc. Such removable recording media may be provided as a so-called package software. Furthermore, such a program can be installed in a personal computer or the like from a removable recording medium, or can be downloaded from a download site via a network such as a LAN (Local Area Network) or the Internet.

[0200] Furthermore, such a program is suitable for widespread provision of the imaging device 1 of the embodiment. For example, by downloading the program to a mobile terminal device such as a smartphone or tablet equipped with a camera function, a mobile phone, a personal computer, a game device, a video device, a PDA (Personal Digital Assistant), or the like, these devices can function as the imaging device 1 of the present disclosure.

[0201] It should be noted that the effects described in this specification are merely examples and are not limiting, and other effects may also be obtained.

[0202] <12. This Technology> The present technology can also be configured as follows. (1) An imaging element including a light-shielding pixel and a photodiode divided pixel; a defocus amount calculation unit that calculates a defocus amount using at least one of the output signals of the light-shielding pixels and the output signals of the photodiode division pixels based on an exposure amount. Imaging device. (2) The light-shielding pixel has a pupil division function by including a light-shielding portion that shields one of a pair of light beams that have passed through a pair of partial regions that are biased in opposite directions in a predetermined direction in the exit pupil, and a light-receiving element that receives the other light beam. The imaging device according to (1) above. (3) The photodiode segment pixel has a pupil division function by including segment pixels that receive each of a pair of light beams that have passed through the pair of partial regions. The imaging device according to (2) above. (4) Each pixel of the imaging element is either one of the light-shielding pixels and the photodiode divided pixels. The imaging device according to any one of (1) to (3) above. (5) The defocus amount calculation unit calculates the defocus amount using the output signals of the light-shielding pixels when the exposure amount is equal to or greater than a threshold value. The imaging device according to any one of (1) to (4) above. (6) When the exposure amount is less than a threshold, the defocus amount calculation unit calculates the defocus amount using at least one of the output signals of the light-shielding pixels and the output signals of the photodiode divided pixels, depending on whether the area is an on-axis area that is an area including a central portion of the image sensor or an off-axis area that is an area other than the on-axis area of ​​the image sensor. An imaging device according to any one of (1) to (5) above. (7) The defocus amount calculation unit calculates the defocus amount using an output signal of the photodiode divided pixel in the on-axis region. The imaging device according to (6) above. (8) The defocus amount calculation unit calculates the defocus amount using output signals of the light-shielded pixels in the off-axis region. The imaging device according to any one of (6) to (7) above. (9) The defocus amount calculation unit calculates the defocus amount using output signals of the photodiode divided pixels in the off-axis region. The imaging device according to any one of (6) to (7) above. (10) The defocus amount calculation unit calculates the defocus amount using one of the output signals of the light-shielded pixels and the output signals of the photodiode divided pixels, whichever has a higher reliability. The imaging device according to any one of (1) to (9) above. (11) When the exposure amount is less than a threshold, the defocus amount calculation unit performs autofocus control based on a contrast method in an off-axis area other than an on-axis area including a central portion of the image sensor. The imaging device according to any one of (1) to (10) above. (12) The imaging element has a plurality of types of light-shielding pixels corresponding to the pupil distance of the exit pupil. The imaging device according to (2) above. (13) The plurality of types of light-shielding pixels have different light-shielding regions of the light-shielding portion. The imaging device according to (12) above. (14) The defocus amount calculation unit calculates the defocus amount using output signals of the light-shielded pixels selected according to a pupil distance of an exit pupil. The imaging device according to any one of (12) to (13) above. (15) A camera control unit that acquires the pupil distance from a lens barrel control unit provided in the lens barrel. The imaging device according to any one of (12) to (14) above. (16) The light-shielding pixels arranged in the same row on the image sensor correspond to the same pupil distance. The imaging device according to any one of (12) to (15) above. (17) a camera control unit that issues a drive instruction to a focus lens of an imaging optical system based on the defocus amount; The imaging device according to any one of (1) to (16) above. (18) a user interface control unit that performs display control based on the defocus amount; An imaging device according to any one of (1) to (17) above. (19) When calculating the defocus amount using the output signals of the light-shielded pixels, the image sensor adds and outputs the output signals of the photodiode division pixels. An imaging device according to any one of (1) to (18) above. (20) When the defocus amount is calculated using the output signals of the photodiode divided pixels, the image sensor outputs each of the output signals of the photodiode divided pixels. The imaging device according to any one of (1) to (19) above. (twenty one) a signal processing unit that performs signal processing on the image signal output from the imaging element; The signal processing unit adds output signals of the photodiode divided pixels in a column direction when the amount of exposure is less than a predetermined amount. The imaging device according to any one of (1) to (20) above. (twenty two) At least one of the phase difference signals is selected from the output signals of the light-shielded pixels and the output signals of the photodiode division pixels based on the amount of exposure, and the amount of defocus is calculated. A method for calculating a defocus amount. [Explanation of symbols]

[0203] 1. Imaging device 3 Lens barrel 7. Image sensor 7a Image plane phase difference pixel 8 Camera signal processing section 8a Defocus amount calculation unit 10 Display 14 Camera control section 14a UI control section 15 Memory section 16 Optical system 18. Telescope control section 21 pixels 21A Light-shielding pixel 21B PD split pixel 30PD 31 Light shielding section 40L Left PD 40R Right PD EP exit pupil EPL left pupil area EPR right pupil area RS0, RS1, RS2, RS3, RS4, RS5, RS6, RS7 Light-shielding pixels LS0, LS1, LS2, LS3, LS4, LS5, LS6, LS7 light-shielding pixels SG,SG0,SG1,SG2,SG3,SG4,SG5,SG6,SG7 Phase difference signal SGr and SGb phase difference signals S0, S1, S2, S3, S4, S5, S6, S7 interpupillary distances Th threshold value

Claims

1. an image sensor including a light-shielding pixel and a photodiode divided pixel that receives incident light through a color filter, the image sensor including a central portion being defined as an on-axis region and a region other than the on-axis region being defined as an off-axis region; a signal processing unit that performs signal processing on an image signal output from the imaging element; a defocus amount calculation unit that calculates a defocus amount using an output signal selected based on an exposure amount from among the output signals of the light-shielding pixels and the output signals of the photodiode division pixels, a plurality of the light-shielded pixels and a plurality of the photodiode divided pixels are provided in each of the on-axis region and the off-axis region; when the exposure amount is less than a threshold value, the signal processing unit adds output signals of the photodiode split pixels in which the color filters of the same color are arranged, among the photodiode split pixels, in an arrangement direction of the photodiode split pixels; The defocus amount calculation unit When the exposure amount is less than the threshold value, the defocus amount is calculated using the added output signal; When the exposure amount is equal to or greater than the threshold value, the defocus amount is calculated using the output signals of the light-shielded pixels for at least the on-axis region out of the on-axis region and the off-axis region. Imaging device.

2. The light-shielding pixel has a pupil division function by including a light-shielding portion that shields one of a pair of light beams that have passed through a pair of partial regions that are biased in opposite directions in a predetermined direction in the exit pupil, and a light-receiving element that receives the other light beam. The imaging device according to claim 1 .

3. The photodiode segment pixel has a pupil division function by including segment pixels that receive each of a pair of light beams that have passed through the pair of partial regions. The imaging device according to claim 2 .

4. Each pixel of the imaging element is either one of the light-shielding pixels and the photodiode divided pixels. The imaging device according to claim 1 .

5. When the exposure amount is equal to or greater than the threshold value, the defocus amount calculation unit calculates the defocus amount for the off-axis region using the output signals of the light-shielded pixels. The imaging device according to claim 1 .

6. When the exposure amount is less than the threshold value, the defocus amount calculation unit calculates the defocus amount using at least one of the output signals of the light-shielded pixels and the output signals of the photodiode divided pixels depending on whether the region is the on-axis region or the off-axis region. The imaging device according to claim 1 .

7. When the exposure amount is less than the threshold value, the defocus amount calculation unit calculates the defocus amount using the output signal of the photodiode divided pixel in the on-axis region. The imaging device according to claim 6.

8. When the exposure amount is less than the threshold value, the defocus amount calculation unit calculates the defocus amount in the off-axis region by using the output signal of the photodiode divided pixel. The imaging device according to claim 6.

9. The defocus amount calculation unit calculates the defocus amount using one of the output signals of the light-shielded pixels and the output signals of the photodiode divided pixels, whichever has a higher reliability. The imaging device according to claim 1 .

10. The imaging element has a plurality of types of light-shielding pixels corresponding to the pupil distance of the exit pupil. The imaging device according to claim 2 .

11. The plurality of types of light-shielding pixels have different light-shielding regions of the light-shielding portion. The imaging device according to claim 10.

12. The defocus amount calculation unit calculates the defocus amount using output signals of the light-shielded pixels selected according to a pupil distance of an exit pupil. The imaging device according to claim 10.

13. A camera control unit that acquires the pupil distance from a lens barrel control unit provided in the lens barrel. The imaging device according to claim 10.

14. The light-shielding pixels arranged in the same row on the image sensor correspond to the same pupil distance. The imaging device according to claim 10.

15. a camera control unit that issues a drive instruction to a focus lens of an imaging optical system based on the defocus amount; The imaging device according to claim 1 .

16. a user interface control unit that performs display control based on the defocus amount; The imaging device according to claim 1 .

17. When calculating the defocus amount using the output signals of the light-shielded pixels, the image sensor adds and outputs the output signals of the photodiode division pixels. The imaging device according to claim 1 .

18. When the defocus amount is calculated using the output signals of the photodiode divided pixels, the image sensor outputs each of the output signals of the photodiode divided pixels. The imaging device according to claim 1 .

19. At least one of the phase difference signals is selected based on the exposure amount from output signals of light-shielding pixels arranged in a plurality of on-axis regions, which are regions including a central portion of an image sensor, and off-axis regions, which are regions other than the on-axis regions, and output signals of photodiode divided pixels arranged in a plurality of the on-axis regions and the off-axis regions, and a defocus amount is calculated; In the calculation when the exposure amount is less than a threshold value, output signals of the photodiode split pixels having the same color filter arranged thereon among the photodiode split pixels having a plurality of types of color filters arranged thereon are added in an arrangement direction of the photodiode split pixels to calculate the defocus amount, In the calculation when the exposure amount is equal to or greater than the threshold value, the defocus amount is calculated using the output signal of the light-shielded pixel for at least the on-axis region out of the on-axis region and the off-axis region. A method for calculating a defocus amount.

Images