Imaging device and defocus amount calculation method

Abstract

The imaging apparatus according to the present technology includes: an imaging element having a light-shielding pixel and a photodiode segmented pixel; and a defocus calculation unit that calculates the defocus amount based on the exposure using the output signal of the light-shielding pixel and / or the output signal of the photodiode segmented pixel.

Description

Technical Field

[0001] This technology relates to an imaging device comprising an imaging element having a group of pixels with an output phase difference signal and a method for calculating defocus. Background Technology

[0002] Some imaging devices have the function of acquiring focus information of a subject for automatic focus control. Among these devices, there is a device that provides pixels for detecting the focus. For example, Patent Document 1 discloses a configuration in which pixels of a photodiode (PD) segmentation method (photodiode segmentation pixels) and pixels of a light-blocking pixel method (light-blocking pixels) are provided as pixels for detecting the focus.

[0003] Citation List

[0004] Patent documents

[0005] Patent Document 1: WO2017 / 212909 Summary of the Invention

[0006] The problem to be solved by the present invention

[0007] Among these pixels used for focus detection, a signal for detecting phase difference is output. However, a problem exists: in dark environments where light levels tend to be insufficient, the S / N ratio of the phase difference signal deteriorates.

[0008] Therefore, the purpose of this technology is to obtain the amount of defocus on the subject with higher accuracy, even in dark environments where light levels are low.

[0009] Solution to the problem

[0010] An imaging apparatus according to the present technology includes: an imaging element including a light-shielding pixel and a photodiode segmentation pixel; and a defocus calculation unit that calculates the defocus amount based on the exposure amount using at least one of the output signal of the light-shielding pixel and the output signal of the photodiode segmentation pixel.

[0011] Therefore, the amount of defocus can be calculated by appropriately selecting one of the output signals of the light-blocking pixels in the imaging device and the output signals of the photodiode-segmented pixels.

[0012] In the above imaging device, the light-shielding pixel can have a pupil splitting function by including a light-shielding part and a light-receiving element, wherein the light-shielding part blocks one of a pair of light beams that have been deviated in opposite directions in a predetermined direction in the exit pupil, and the light-receiving element receives the other light beam.

[0013] Therefore, a light-blocking pixel is, for example, one of the following: a pixel that receives light only through the left region of the left half of the exit pupil caused by the light-blocking portion, and a pixel that receives light only through the right region of the right half of the exit pupil caused by the light-blocking portion.

[0014] In the aforementioned imaging device, the photodiode segmentation pixel can have a pupil segmentation function by including a segmentation pixel that receives each of a pair of light beams that have passed through the pair of partial regions.

[0015] Therefore, the segmented pixels included in the photodiode segmented pixels are segmented pixels that are only incident on by light through the left region of the left half of the exit pupil and segmented pixels that are only incident on by light through the right region of the right half of the exit pupil formed by the light-blocking part.

[0016] In the above-mentioned imaging device, each pixel included in the imaging element can be one of a light-shielding pixel and a photodiode-segmented pixel.

[0017] That is, in the imaging element, all pixels are one of the light-blocking pixels and photodiode-segmented pixels.

[0018] When the exposure is equal to or greater than the threshold, the defocus calculation unit in the above imaging device can use the output signal of the light-blocking pixel to calculate the defocus.

[0019] Since the light-blocking pixel includes the light-blocking part, its output signal level is lower than that of the PD segmentation pixel.

[0020] When the exposure is less than a threshold, the defocus calculation unit in the above imaging device can calculate the defocus amount by using at least one of the output signal of the light-blocking pixel and the output signal of the photodiode segmentation pixel, depending on whether it is an on-axis region or an off-axis region. The on-axis region is the region including the central part of the imaging element, and the off-axis region is the region other than the on-axis region of the imaging element.

[0021] Therefore, focus control is performed even when the exposure is less than the threshold.

[0022] The defocus calculation unit in the above imaging device can use the output signal of the pixel segmentation by the photodiode in the on-axis region to calculate the defocus amount.

[0023] Since the pixel segmentation of a photodiode does not include the light-blocking portion, its output signal level is higher than that of the light-blocking pixel.

[0024] The defocus calculation unit in the above imaging device can use the output signal of the light-blocking pixel in the off-axis region to calculate the defocus amount.

[0025] Therefore, the defocusing amount is calculated with high reliability.

[0026] The defocus calculation unit in the above imaging device can use the output signal of the pixel segmentation by the photodiode in the off-axis region to calculate the defocus amount.

[0027] Therefore, the defocusing amount is calculated with high reliability.

[0028] The defocus calculation unit in the above imaging device can use the output signal with higher reliability from the output signal of the light-blocking pixel and the output signal of the photodiode segmentation pixel to calculate the defocus amount.

[0029] Therefore, the defocusing amount is calculated with high reliability.

[0030] When the exposure is less than a threshold, the defocus calculation unit in the above imaging device can perform automatic focus control based on the contrast method in the off-axis region. The on-axis region is the region including the central part of the imaging element, and the off-axis region is the region other than the on-axis region of the imaging element.

[0031] In cases of insufficient exposure, the accuracy of the phase difference information between the output signals of the light-blocking pixels and the output signals of the photodiode-segmented pixels may be low in the off-axis region.

[0032] The imaging elements in the above-mentioned imaging device may include various types of light-blocking pixels corresponding to the pupil distance of the exit pupil.

[0033] Therefore, even if the position of the exit pupil changes as the imaging optics system is driven, a suitable phase difference signal can be obtained from any of the various types of light-blocking pixels.

[0034] The light-shielding areas of the various types of light-shielding pixels in the aforementioned imaging devices are different.

[0035] For example, as the pupil distance decreases, the area that blocks light can become larger.

[0036] The defocus calculation unit in the imaging device can use the output signal of the light-blocking pixel selected according to the pupil distance of the exit pupil to calculate the defocus amount.

[0037] Therefore, the light-blocking pixels that receive light passing through one side of the exit pupil are selected.

[0038] In the aforementioned imaging device, a camera control unit may be provided, which obtains the pupil distance from a lens barrel control unit included in the lens barrel.

[0039] For example, in an imaging device that includes interchangeable lens barrels, a light-blocking pixel corresponding to the pupil distance is selected.

[0040] In the above imaging device, the light-blocking pixels arranged in the same row on the imaging element can correspond to the same pupil distance.

[0041] Therefore, the output signals of various types of light-blocking pixels are not mixed with the pixel signals read for each pixel row.

[0042] In the aforementioned imaging apparatus, a camera control unit may be provided, which executes driving commands for the focusing lens included in the imaging optical system based on the defocus amount.

[0043] Therefore, the focusing lens is controlled by a phase difference signal based on the exposure.

[0044] In the aforementioned imaging device, a user interface control unit can be provided, which performs display control based on the amount of defocus.

[0045] Therefore, for example, information can be provided to inform the photographer of the focus status based on the current lens position.

[0046] In the above imaging device, when calculating the defocus amount using the output signal of the light-blocking pixel, the imaging element can add the output signals of the photodiode-divided pixels and output them.

[0047] Therefore, the number of reads can be reduced compared to reading the output from each segmented pixel separately.

[0048] In the above imaging device, when the defocus amount is calculated using the output signal of the photodiode segmented pixel, the imaging element can output each of the output signals of the photodiode segmented pixel.

[0049] Therefore, the output signal of the segmented pixels is obtained without addition.

[0050] In the above-described imaging apparatus, a signal processing unit may be provided, which performs signal processing on the image signal output from the imaging element. In the case where the exposure is less than a predetermined amount, the signal processing unit may add the output signals of the photodiode-divided pixels in the column direction.

[0051] Therefore, the output level of the signal from the segmented pixel of the photodiode can be maintained at a predetermined level or higher.

[0052] The defocusing amount calculation method according to this technology includes: selecting at least one phase difference signal from the output signal of the light-blocking pixel and the output signal of the photodiode segmented pixel, based on the exposure amount, and calculating the defocusing amount. Attached Figure Description

[0053] Figure 1This is a perspective view of an imaging apparatus according to an embodiment of the present technology.

[0054] Figure 2 This is a rear view of the imaging device.

[0055] Figure 3 This is a block diagram of the imaging device.

[0056] Figure 4 This is a schematic diagram of the imaging device.

[0057] Figure 5 This is an example of the configuration of the imaging element.

[0058] Figure 6 This is a construction example of light-blocking pixels arranged in an on-axis region.

[0059] Figure 7 This is a construction example of PD segmented pixels arranged in an on-axis region.

[0060] Figure 8 This is an example of the output signal for the first pixel row.

[0061] Figure 9 It is a graph used to show the difference between the pixel outputs of the left opening and the pixel outputs of the right opening.

[0062] Figure 10 It is a graph used to show the difference values ​​of the left-opening pixel output and the right-opening pixel output after shifting.

[0063] Figure 11 It is a graph used to show the difference values ​​of the left-opening pixel output and the right-opening pixel output after shifting.

[0064] Figure 12 It is a graph used to show the difference values ​​of the left-opening pixel output and the right-opening pixel output after shifting.

[0065] Figure 13 It is a graph showing the relationship between the differential integral value and the shift amount.

[0066] Figure 14 It is a graph showing the relationship between the amount of displacement and the amount of defocus.

[0067] Figure 15 It is a diagram used to show the on-axis region and the off-axis region.

[0068] Figure 16 This is an illustration of the position of pixels on the imaging element.

[0069] Figure 17 This is an example of a configuration of light-shielding pixels arranged in an off-axis region.

[0070] Figure 18This is a configuration example of PD segmented pixels arranged in the off-axis region.

[0071] Figure 19 This is an example of a configuration of light-shielding pixels arranged in an off-axis region.

[0072] Figure 20 This is an example of the arrangement of light-blocking pixels based on the pupil distance.

[0073] Figure 21 This is a graph showing the performance difference between light-blocking pixels and PD-segmented pixels.

[0074] Figure 22 It is a graph used to illustrate exposure and AF error, as well as the phase difference signal selected under various conditions.

[0075] Figure 23 This is a diagram illustrating a method for reading signals from each pixel when using a phase difference signal output from a light-blocking pixel.

[0076] Figure 24 It is a graph used to show the readout time of the signal from each pixel when using the phase difference signal output from the light-blocking pixel.

[0077] Figure 25 This is a diagram illustrating a method for reading signals from individual pixels when using phase difference signals output from pixel segmentation from a PD.

[0078] Figure 26 It is a graph used to show the readout time of the signal from each pixel when using the phase difference signal output from the PD segmented pixel.

[0079] Figure 27 This is a diagram illustrating a method for reading signals from each pixel when using a phase difference signal output from a light-blocking pixel and a phase difference signal output from a PD segmentation pixel.

[0080] Figure 28 It is a graph used to show the readout time of the signal from each pixel when using the phase difference signal output from the shading pixel and the phase difference signal output from the PD segmentation pixel.

[0081] Figure 29 It is a flowchart related to the calculation of exposure.

[0082] Figure 30 This is a flowchart executed when the shutter button is detected to be half-pressed.

[0083] Figure 31 This is a flowchart executed when the shutter button is detected to be half-pressed.

[0084] Figure 32This is a flowchart executed when the shutter button is detected to be half-pressed.

[0085] Figure 33 This is a flowchart executed when the shutter button is detected to be half-pressed. Detailed Implementation

[0086] Hereinafter, embodiments will be described in the following order with reference to the accompanying drawings.

[0087] <1. Configuration of Imaging Device>

[0088] <2. Configuration of Imaging Elements>

[0089] <3. AF Control>

[0090] <4. On-axis and Off-axis Regions>

[0091] <5. Pupil Distance and Pixel Structure>

[0092] <6. Differences in AF control due to phase difference pixels>

[0093] <7. Selection of Phase Difference Signal>

[0094] <8. Exposure Control>

[0095] <9. Processing Flow>

[0096] <9-1. First Example>

[0097] <9-2. Second Example>

[0098] <10. Examples of Deformation>

[0099] <11. Summary>

[0100] <12. This technology>

[0101] <1. Configuration of Imaging Device>

[0102] Figure 1 The appearance of the imaging device 1 according to this embodiment is shown.

[0103] Note that in each of the following examples, the subject side will be described as the front and the photographer side as the back, but these directions are for ease of description and the implementation of this technology is not limited to these directions.

[0104] like Figure 1 and Figure 2 As shown, the imaging device 1 includes a camera housing 2 in which necessary units are arranged inside and outside, and a lens barrel 3 attached to the front surface portion 2a of the camera housing 2.

[0105] The rear monitor 4 is mounted on the rear surface portion 2b of the camera housing 2. Through-view images, recorded images, etc., are displayed on the rear monitor 4.

[0106] The rear monitor 4 is, for example, a display device, such as a liquid crystal display (LCD), an organic electroluminescent display (EL), etc.

[0107] The rear monitor 4 can rotate relative to the camera housing 2. For example, the upper part of the rear monitor 4 is set as the axis of rotation, and the lower part of the rear monitor 4 can rotate to move backward.

[0108] Note that the right or left end of the rear monitor 4 can be used as a rotation axis. Furthermore, it can rotate around multiple axes.

[0109] An electric viewfinder (EVF) 5 is mounted on the upper surface portion 2c of the camera housing 2. The EVF 5 includes an EVF monitor 5a and a frame-shaped housing 5b that protrudes rearward to surround the EVF monitor 5a and the left and right sides of the EVF monitor 5a.

[0110] The EVF monitor 5a is formed using an LCD, OLED display, or similar device. Note that an optical viewfinder (OVF) can be provided instead of the EVF monitor 5a.

[0111] Various actuators 6 are arranged on the rear surface portion 2b and the upper surface portion 2c. Examples of actuators 6 include a playback menu activation button, an input button, a cross key, a cancel button, a zoom key, a slide key, a shutter button 6S (release button), etc.

[0112] The various operators 6 include various modes, such as buttons, dials, and pressable and rotatable combination operators. Using the various modes of operators 6, for example, menu operations, playback operations, mode selection / switching operations, focusing operations, zoom operations, and parameter selection / setting such as shutter speed and F-number can be performed.

[0113] Figure 3 This is a block diagram of imaging device 1.

[0114] Imaging elements 7, camera signal processing unit 8, recording unit 9, display unit 10, output unit 11, operation unit 12, power supply unit 13, camera control unit 14, memory unit 15, etc. are provided inside and outside the camera housing 2 of the imaging device 1.

[0115] The lens barrel 3 includes an optical system 16, a driver unit 17, a lens barrel control unit 18, an operation unit 19, a memory unit 20, etc.

[0116] like Figure 3 and Figure 4As shown, the optical system 16 includes various lenses (e.g., an incident lens, a zoom lens, a focusing lens, and a condenser lens), an aperture mechanism, and a shutter unit (e.g., a focal plane shutter). The aperture mechanism performs exposure control by adjusting the aperture amount, such as that of a lens or an aperture stop, so that sensing is performed in a state where the signal charge is not saturated and within the dynamic range.

[0117] Note that a portion of each unit constituting the optical system 16 may be arranged in the camera housing 2.

[0118] The imaging element 7 is, for example, a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) type, and controls the exposure of light incident from the subject via the optical system 16.

[0119] The sensor surface of the imaging element 7 includes a sensing element in which a plurality of pixels are arranged in two dimensions. Some of the pixels in the imaging element 7 are pixels that output signals for calculating the amount of defocus. In the following description, the pixel that outputs the signal for calculating the amount of defocus is referred to as "image plane phase difference pixel 7a".

[0120] In the imaging element 7 of this embodiment, all pixels are image plane phase difference pixels 7a used to detect the phase difference of the optical image of the subject. However, the imaging element 7 may include pixels other than the image plane phase difference pixels 7a. A detailed configuration example of the imaging element 7 will be described later.

[0121] Imaging element 7 includes a processing unit that performs processes such as correlated double sampling (CDS), automatic gain control (AGC), and analog-to-digital (A / D) conversion on the electrical signals converted from pixel photoelectric signals. Therefore, imaging element 7 outputs the captured image signal as digital data to camera signal processing unit 8 and camera control unit 14.

[0122] The image plane phase difference pixel 7a outputs a signal used to calculate the amount of defocus. In this embodiment, the imaging element 7 includes both a light-blocking pixel obtained by a photodiode (PD) light-blocking method and a PD segmentation pixel obtained by a PD segmentation method as the image plane phase difference pixel 7a. The signal output from the image plane phase difference pixel 7a is a signal obtained through photoelectric conversion, but it is a signal that can detect phase difference information through subsequent processing. Therefore, in the following description, the signal output from the image plane phase difference pixel 7a that can detect phase difference information can be referred to as a "phase difference signal".

[0123] Imaging element 7 outputs the phase difference signal obtained from the phase difference pixel 7a of the image plane to the camera signal processing unit 8 and camera control unit 14 in subsequent stages. The phase difference signal is used for related calculations to calculate the amount of defocus.

[0124] The camera signal processing unit 8 includes, for example, a microprocessor dedicated to digital signal processing, such as a digital signal processor (DSP), a microcomputer, etc.

[0125] The camera signal processing unit 8 includes units for performing various types of signal processing on digital signals (captured image signals) sent from the imaging element 7.

[0126] Specifically, it performs processing such as correction between R, G and B color channels, white balance correction, aberration correction, and shadow correction.

[0127] In addition, the camera signal processing unit 8 performs YC generation processing to generate (separate) luminance (Y) signals and color (C) signals from image data of R, G and B, processing to adjust luminance and color, and processing such as inflection point correction and gamma correction.

[0128] Furthermore, the camera signal processing unit 8 performs conversion to the final output format by performing resolution conversion processing, encoding / decoding processing for encoding for recording or communication, etc. The image data converted to the final output format is stored in the storage unit 15. Additionally, when the image data is output to the display unit 10, the image is displayed on the rear monitor 4 or the EVF monitor 5a. Furthermore, data can be displayed on a device such as a monitor located outside the imaging device 1 by outputting data from an external output terminal.

[0129] The camera signal processing unit 8 includes a defocus calculation unit 8a.

[0130] The defocus calculation unit 8a detects phase difference information from the output signal of the image plane phase difference pixel 7A. Specifically, it detects phase difference information based on signal groups output from multiple image plane phase difference pixels 7A and signal groups output from other multiple image plane phase difference pixels 7A. The camera signal processing unit 8 (or camera control unit 14) has information about which signal groups are compared to detect phase difference information. This will be described in detail in paragraph

[0079] .

[0131] The defocus calculation unit 8a calculates the defocus amount based on the detected phase difference information. The calculated defocus amount can be used for autofocus (AF) by driving the focusing lens included in the optical system 16 via the lens barrel control unit 18. Furthermore, the defocus amount can be used to present information to the user regarding the degree of focus of the subject.

[0132] The recording unit 9 includes, for example, a non-volatile memory and is used as a storage unit for storing image files (content files) such as still image data and moving image data, attribute information of the image files, thumbnails, etc.

[0133] Image files are stored in formats such as Joint Image Experts Group (JPEG), Tagged Image File Format (TIFF), and Graphics Interchange Format (GIF).

[0134] Various practical forms of the recording unit 9 can be considered. For example, the recording unit 9 can be configured as a flash memory built into the imaging device 1, or it can be configured as an access unit that can be attached to and removed from the imaging device 1 by a memory card (e.g., a portable flash memory) and access the memory card for storage and retrieval. Alternatively, as a form included in the imaging device 1, the recording unit can be implemented as a hard disk drive (HDD) or the like.

[0135] Display unit 10 performs processing for various displays to the camera. Display unit 10 is, for example, a rear monitor 4 or an EVF monitor 5a. Display unit 10 performs processing to display image data input from camera signal processing unit 8 that has been converted to an appropriate resolution. Therefore, a so-called pass-through image is displayed, which is an image captured during the released standby period.

[0136] In addition, the display unit 10 displays various operation menus, icons, messages, etc. on the screen as a graphical user interface (GUI) based on instructions from the camera control unit 14.

[0137] In addition, the display unit 10 can display a reproduced image of the image data read from the recording medium in the recording unit 9.

[0138] Note that in this example, both EVF monitor 5a and rear monitor 4 are configured, but embodiments of this technology are not limited to this configuration, and only one of EVF monitor 5a and rear monitor 4 may be configured, or one or both of EVF monitor 5a and rear monitor 4 may be configured to be detachable.

[0139] The output unit 11 performs data communication and network communication with external devices via wired or wireless means. For example, it sends captured image data (still image files or moving image files) to external display devices, recording devices, or playback devices.

[0140] Furthermore, the output unit 11 can be used as a network communication unit. For example, communication can be performed through various networks such as the Internet, home networks, and local area networks (LANs), and various data can be sent to and received from servers, terminals, etc. on the network.

[0141] The operation unit 12 arranged in the camera housing 2 includes not only the various operators 6 mentioned above, but also a rear monitor 4 with a touch panel system, and outputs operation information corresponding to various operations such as tapping and sliding operations by the imaging personnel to the camera control unit 14.

[0142] Note that the operation unit 12 can be used as a receiving unit for an external operation device, such as a remote control, that is separate from the imaging device 1.

[0143] The power supply unit 13 generates the power supply voltage (Vcc) required for each unit from, for example, a battery filled inside, and provides the generated power supply voltage as the operating voltage.

[0144] With the lens barrel 3 installed in the imaging device 1, the power supply voltage Vcc generated by the power supply unit 13 is also supplied to the circuit in the lens barrel 3.

[0145] Note that the power supply unit 13 can be formed by a circuit that charges the battery or by a circuit that uses the DC voltage converted and input by an AC adapter connected to a commercial AC power supply to generate the power supply voltage Vcc.

[0146] The camera control unit 14 includes a microcomputer (arithmetic processing device) which includes a central processing unit (CPU) and performs overall control of the imaging device 1. For example, it executes shutter speed control according to the operator's actions, instructions regarding various types of signal processing in the camera signal processing unit 8, imaging and recording operations according to the user's actions, and reproduction operations of recorded image files.

[0147] The camera control unit 14 switches between various imaging modes. Examples of various imaging modes include still image imaging mode, video imaging mode, and continuous imaging mode for continuously acquiring still images.

[0148] The camera control unit 14 includes a user interface control unit (UI control unit) 14a, which enables the user to operate these functions. The UI control unit 14a performs processing such as detecting operations on each of the actuators 6 provided in the imaging device 1, display processing on the rear monitor 4, and operation detection processing.

[0149] The UI control unit 14a performs display control to notify the user of the amount of defocus. By recognizing the notification regarding the amount of defocus, the user can manually perform a focusing operation or set any defocus state. For example, the notification can be executed by the display unit 10. Specifically, it can display an icon image or characters, etc.

[0150] In addition, the camera control unit 14 sends commands to the lens barrel control unit 18 to control the various lenses included in the optical system 16.

[0151] For example, executing a specified aperture value to ensure the amount of light required for AF control, and operating instructions for the aperture mechanism based on the aperture value.

[0152] The camera control unit 14 can obtain information about the various lenses included in the optical system 16 via the lens barrel control unit 18. This lens information includes, for example, information about the lens type, the position of the zoom lens, the F-number, and the position of the exit pupil. Furthermore, the camera control unit 14 can obtain the aperture value of the aperture mechanism included in the optical system 16.

[0153] The storage unit 15 stores information and the like used in the processing performed by the camera control unit 14. Examples of the storage units 15 shown include, for instance, read-only memory (ROM), random access memory (RAM), and flash memory.

[0154] The storage unit 15 can be a storage area built into the microcomputer chip as a camera control unit 14, or it can be composed of a separate storage chip.

[0155] The programs used by the camera control unit 14 are stored in the ROM, flash memory, etc. of the storage unit 15. The ROM, flash memory, etc. store the operating system (OS) used by the CPU to control each unit, content files such as image files, and applications, firmware, etc. for various operations.

[0156] The camera control unit 14 executes a program to control the entire imaging device 1 and the lens barrel 3.

[0157] The RAM of storage unit 15 is used as the working area of ​​camera control unit 14 by temporarily storing data, programs, etc. used in various data processing executed by the CPU of camera control unit 14.

[0158] The lens barrel control unit 18 of the lens barrel 3 includes, for example, a microcomputer, and outputs control signals to the driver unit 17 based on instructions from the camera control unit 14 to actually drive the various lenses of the optical system 16.

[0159] Note that information communication between the camera control unit 14 and the lens barrel control unit 18 can be enabled only when the lens barrel 3 is installed in the camera housing 2, or information communication between the camera control unit 14 and the lens barrel control unit 18 can be enabled wirelessly when the lens barrel 3 is not installed in the camera housing 2.

[0160] The lens barrel control unit 18 transmits information about the exit pupil position and pupil distance to the camera control unit 14 based on the types and drive positions of the various lenses included in the optical system 16. Specifically, it retrieves information about the pupil distance from information stored in the ROM, which is a memory unit 20, and transmits this information to the camera control unit 14.

[0161] The drive unit 17 may include, for example, a motor driver for a zoom lens drive motor, a motor driver for a focusing lens drive motor, and an aperture mechanism driver for a motor that drives the aperture mechanism.

[0162] Each driver provides drive current to the corresponding drive motor according to the instructions from the lens barrel control unit 18.

[0163] The operation unit 19 of the lens barrel 3 refers to the operator arranged on the side of the lens barrel 3. The operation information of the operation unit 19 is provided to the lens barrel control unit 18, and is notified to the camera control unit 14 via the lens barrel control unit 18.

[0164] Based on the operation of the operation unit 19, the lens barrel control unit 18 performs operation control of the optical system 16 and various settings and operation control of the camera control unit 14.

[0165] The operation unit 19 can be used as a receiving unit for an external operation device, such as a remote control, that is separate from the lens barrel 3.

[0166] Storage unit 20 includes ROM, flash memory, etc., and stores programs, data, etc. used by the lens control unit 18. Storage unit 20 stores the operating system (OS) used by the CPU to control each unit, application programs, firmware, etc. for various operations, etc.

[0167] In addition, the information stored in the storage unit 20 includes information such as the pupil distance of the exit pupil of the optical system 16.

[0168] <2. Configuration of Imaging Elements>

[0169] Reference Figure 5 Describe the configuration of imaging element 7.

[0170] Imaging element 7 is formed by arranging pixels in a matrix in the row and column directions. Each pixel 21 is one of a light-shielding pixel 21A which has a light-shielding portion that blocks a portion of the light incident on a pixel, and a PD segmentation pixel 21B which is composed of two segmented pixels.

[0171] The imaging element 7 includes a first pixel row 22A and a second pixel row 22B. The first pixel row includes light-blocking pixels 21A to output the phase difference signal of the PD light-blocking method, and the second pixel row includes only PD segmentation pixels 21B to output the phase difference signal of the PD segmentation method.

[0172] The first pixel rows 22A are discretely arranged in the vertical direction, and multiple rows of second pixel rows 22B are arranged between the first pixel rows 22A and the first pixel rows 22A. The first pixel rows 22A can be arranged regularly or irregularly. However, when the imaging element 7 is arranged regularly, the design and manufacturing costs associated with the manufacturing of the imaging element 7 can be suppressed.

[0173] Each of the PD segmentation pixels 21B included in the second pixel row 22B is covered by a color filter of the Bayer array, and depending on the type of color filter, is one of a PD segmentation pixel with red (R) spectral sensitivity, a PD segmentation pixel with green (G) spectral sensitivity, and a PD segmentation pixel with blue (B) spectral sensitivity.

[0174] Reference Figure 6 The schematic diagram illustrates the configuration of the light-shielding pixel 21A.

[0175] The light-shielding pixel 21A includes a PD 30, a light-shielding portion 31 arranged in front of the PD 30 (on the subject side), an inner lens 32 arranged in front of the light-shielding portion 31, a color filter (cyan) 33 arranged in front of the inner lens 32, and an on-chip microlens 34 arranged in front of the color filter 33.

[0176] Note that the inner lens 32 and color filter 33 may not be included in the light-shielding pixel 21A.

[0177] PD 30 is a light receiving element that receives a portion of the light through the exit pupil EP, but by means of the light-shielding part 31 arranged in front, light can only be received in a portion of the light receiving area of ​​PD 30.

[0178] In other words, the light-shielding portion 31 is formed to cover the left half of the PD 30. The right opening 35R is formed in the light-shielding portion 31.

[0179] The inner lens 32 and the on-chip microlens 34 are optical components provided to effectively converge light passing through the exit pupil EP and entering a pixel onto the PD 30.

[0180] Color filter 33 is, for example, a filter with cyan (Cy) spectral sensitivity.

[0181] like Figure 6 As shown, PD 30 is configured to receive only light passing through the left region (hereinafter referred to as the "left pupil region EPL"), which is the left half of the exit pupil EP. That is, light passing through the right region (hereinafter referred to as the "right pupil region EPR"), which is the right half of the exit pupil EP, is blocked by the light-blocking portion 31 and does not reach PD 30. Thus, pupil segmentation function is achieved.

[0182] like Figure 6 The light-shielding pixel 21A shown is configured to receive light passing through the left pupil region EPL and is referred to as light-shielding pixel 21AR because it receives light in a region offset to the right on the light-receiving surface. That is, a right opening 35R is formed in the light-shielding pixel 21AR.

[0183] In addition, relative to Figure 6 The illustrated configuration has a mirror-symmetrical light-shielding pixel 21A configured to receive light passing through the right pupil region EPR. This pixel receives light in a region offset to the left on the light-receiving surface and is therefore referred to as light-shielding pixel 21AL. A left opening 35L is formed in the light-shielding portion 31 included in the light-shielding pixel 21AL.

[0184] like Figure 5 As shown, the distance between the light-shielding pixel 21AR and the light-shielding pixel 21AL is, for example, the distance between two pixels, and they are set alternately.

[0185] The signals output from the light-blocking pixel 21AR and the light-blocking pixel 21AL are processed as a pair of phase difference signals by the camera signal processing unit 8 (or camera control unit 14). That is, the defocus calculation unit 8a of the camera signal processing unit 8 in the subsequent stage uses the phase difference between the signals output from the light-blocking pixel 21AR and the light-blocking pixel 21AL to calculate the defocus amount.

[0186] Next, the PD segmentation pixel 21B will be described.

[0187] Figure 7 This diagram schematically illustrates the configuration of PD segmentation pixels 21B, which are pixels 21 in the imaging element 7.

[0188] The PD segmentation pixel 21B includes two segmentation pixels, namely, a left PD 40L as the left segmentation pixel and a right PD 40R as the right segmentation pixel, a pixel boundary metal 41 arranged in front of the segmentation pixel, an inner lens 32, a color filter 33, and an on-chip microlens 34. The color filter 33 is any one of a color filter 33R with red (R) spectral sensitivity, a color filter 33G with green (G) spectral sensitivity, and a color filter 33B with blue (B) spectral sensitivity.

[0189] Note that inner lens 32 and other elements may not be set in PD segmented pixel 21B.

[0190] As shown in the figure, the left PD 40L receives light that has passed through the exit pupil EP and the right pupil region EPR. The right PD 40R receives light that has passed through the left pupil region EPL. Therefore, pupil segmentation is achieved.

[0191] <3. AF Control>

[0192] The AF control method using each of the above-mentioned pixels 21 for focusing will be described.

[0193] First, the AF control using the phase difference signal output from the light-blocking pixel 21A or the PD segmentation pixel 21B will be described.

[0194] The phase difference signal will be described using the light-shielding pixel 21A as an example.

[0195] Figure 8 It is a graph showing the relationship between the position of the light-blocking pixel 21A for a specific first pixel row 22A on the imaging element 7 and the output.

[0196] The signal output from the light-blocking pixel 21AL, which forms the left opening 35L, is represented by a solid line. Furthermore, the signal output from the light-blocking pixel 21AR, which forms the right opening 35R, is represented by a dashed line. The curve of the solid line is defined as the left opening pixel output 50L, and the curve of the dashed line is defined as the right opening pixel output 50R.

[0197] Figure 9 The shaded area represents the difference between the waveform of the left-opening pixel output 50L and the waveform of the right-opening pixel output 50R.

[0198] Next, the waveform obtained by shifting the waveform of the left-opening pixel output 50L by a certain distance in the right direction of the curve is... Figure 10 The waveform shown in the figure is 50L1. Figure 10 The shaded area represents the difference between waveform 50L1 and the right-opening pixel output 50R.

[0199] The waveform obtained by shifting waveform 50L1 a certain distance to the right is... Figure 11 The waveform shown in the figure is 50L2. Figure 11 The shaded area represents the difference between waveform 50L2 and the right-opening pixel output 50R.

[0200] The waveform obtained by further shifting waveform 50L2 a certain distance to the right is... Figure 12 The waveform shown in the figure is 50L3. Figure 12 The shaded area represents the difference between waveform 50L3 and the right-opening pixel output 50R.

[0201] Figure 13 It shows Figure 9 , Figure 10 , Figure 11 and Figure 12The graph shows the difference integral values ​​as indicated by the shaded area.

[0202] As shown in the figure, the difference integral value decreases as the shift amount increases, and when the predetermined shift amount is exceeded, the difference integral value increases again as the shift amount increases.

[0203] The shift amount with the smallest differential integral value is the phase difference. That is, by moving the focusing lens, the outputs of the light-blocking pixels 21AL and 21AR are shifted by phase difference, and the waveforms of the left-opening pixel output 50L and the right-opening pixel output 50R are substantially overlapped, thus enabling appropriate AF control.

[0204] Note that the so-called front pin and rear pin can be distinguished based on the direction of the waveform shift of the 50L output from the left-opening pixel. That is, in Figure 9 In this state, by shifting the waveform of the left-opening pixel output 50L to the right, the differential integral value can be minimized. This state is known as the front pin state.

[0205] On the other hand, the so-called back pin state is set when the differential integral value can be minimized by shifting the left opening pixel output waveform by 50L to the left.

[0206] Note that, in detail, since the light-blocking pixels 21AL and 21AR are separated by two pixels on the imaging element 7, the optimal focus can be created by moving the focusing lens to shift the waveforms of the left aperture pixel output 50L and the right aperture pixel output 50R by two pixels.

[0207] Figure 14 This illustrates the relationship between displacement and defocus. Displacement is where... Figure 13 The difference integral value shown initially represents the decrease in shift, and can be redescribed as a phase difference. The relationship between the shift and the amount of defocus is represented by a linear function. The larger the shift, the larger the defocus, and a large shift indicates a state where focus is not achieved. The amount of defocus can be calculated based on the shift.

[0208] Although AF control based on the phase difference signal output from the light-blocking pixel 21A has been described so far, AF control based on the phase difference signal output from the PD segmentation pixel 21B can be performed similarly.

[0209] Specifically, the amount of defocus can be obtained by comparing the waveform of the signal output from the left PD 40L of the PD segmentation pixel 21B (corresponding to the left opening pixel output 50L in each figure) with the waveform of the signal output from the right PD 40R (corresponding to the right opening pixel output 50R in each figure) and calculating the amount of shift used to substantially match these waveforms.

[0210] Note that when it is not possible to calculate the amount of defocus with high reliability based on the signal output from the light-blocking pixel 21A or the PD segmentation pixel 21B, it is conceivable to use the contrast method to calculate the amount of defocus.

[0211] For example, contrast is detected based on the brightness signal generated in the camera signal processing unit 8, and focus control is performed.

[0212] Note that the contrast method described here is an example, and various known methods can be used to calculate the amount of defocus using the contrast method.

[0213] <4. On-axis and Off-axis Regions>

[0214] In imaging element 7, the configuration of pixel 21 can differ between the region near the center and other regions. Specifically, the region near the center of imaging element 7 is described as the on-axis region ArC, and the other regions are described as the off-axis region ArM.

[0215] Notice, Figure 15 Examples of on-axis region ArC and off-axis region ArM are shown, but the on-axis region ArC can be a substantially circular region or can have other shapes. Furthermore, whether each pixel belongs to the on-axis region ArC or the off-axis region ArM is pre-stored in the memory unit 15. Therefore, the camera control unit 14 can refer to the memory unit 15 to determine whether a pixel or region belongs to the on-axis region ArC or the off-axis region ArM.

[0216] like Figure 16 As shown, pixel 21C, arranged in the on-axis region ArC, receives light incident from a direction substantially perpendicular to the light-receiving surface. On the other hand, pixel 21M, arranged in the off-axis region ArM, receives light incident on the light-receiving surface from an inclined direction as shown in the figure.

[0217] Each pixel 21 of the imaging element 7 is configured such that each portion is laterally shifted toward the central portion as it is positioned at a greater distance from the central portion.

[0218] Figure 6 The light-shielding pixel 21AR shown is an example of pixel 21C, which is arranged substantially at the center of the on-axis region ArC.

[0219] Reference Figure 17 Describes the light-shielding pixel 21AR as pixel 21M arranged in the off-axis region Arm.

[0220] As shown in the figure, the inner lens 32 and the color filter 33 are offset relative to the PD 30 and the light-blocking portion 31 toward the exit pupil EP side. Furthermore, the on-chip microlens 34 is offset relative to the inner lens 32 and the color filter 33 toward the exit pupil EP side.

[0221] With this configuration, light incident at an angle from the exit pupil EP can be effectively incident on the PD 30.

[0222] Next, in Figure 18 The image shows a PD segmentation pixel 21B, which is a pixel 21M arranged in the off-axis region Arm.

[0223] As shown in the figure, pixel boundary metal 41 is arranged in front of the left PD 40L and the right PD 40R. The inner lens 32 and the color filter 33 are arranged in front of the pixel boundary metal 41, offset relative to the pixel boundary metal 41 towards the exit pupil EP side. In addition, the on-chip microlens 34 is configured to be offset relative to the inner lens 32 and the color filter 33 towards the exit pupil EP side.

[0224] Regardless of whether pixel 21 is a light-blocking pixel 21A or a PD-segmented pixel 21B, the offset of each unit, such as the inner lens 32 and the color filter 33, is determined based on the arrangement position of pixel 21. Therefore, it becomes easy to form each part on the imaging element 7.

[0225] Furthermore, because each part is offset, light incident obliquely from the exit pupil EP can be effectively incident on the left PD 40L and the right PD 40R, and light passing through the left pupil region EPL is incident on the right PD 40R, while light passing through the right pupil region EPR is incident on the left PD 40L. In other words, pupil segmentation is achieved.

[0226] <5. Pupil Distance and Pixel Structure>

[0227] The angle of light incident on pixel 21 is also affected by the distance between the exit pupil and the imaging element 7. The shorter the distance between the exit pupil and the imaging element 7, the larger the incident angle on pixel 21. When the incident angle changes, the light receiving area of ​​PD 30, which receives light that has passed through the left pupil region EPL, changes.

[0228] For example, Figure 17 The configuration is shown where the pupil distance of the exit pupil EP is a predetermined distance and the right opening 35R is greater than half the surface area of ​​PD 30.

[0229] When the lens barrel 3 is an interchangeable lens, the pupil distance of the exit pupil EP changes depending on the type of optical system 16. If the exit pupil EP is closer than the predetermined distance, Figure 17The light-blocking pixel 21AR shown not only receives light passing through the left pupil region EPL of the exit pupil EP, but also receives light passing through the right pupil region EPR.

[0230] Therefore, in this embodiment, various types of light-shielding pixels 21AR and light-shielding pixels 21AL are provided according to the pupil distance of the exit pupil EP.

[0231] This will refer to Figure 17 and Figure 19 Detailed description. Figure 17 In the optical system 16, the pupil distance for the exit pupil EP is set to a predetermined distance, forming a right opening 35R of the light-shielding portion 31, so that light passing through the left pupil region EPL is received by the PD30.

[0232] On the other hand, Figure 19 In the optical system 16 where the pupil distance of the exit pupil EP is shorter than a predetermined distance, the right opening 35R of the light-blocking portion 31 is formed such that light passing through the left pupil region EPL is received by the PD 30.

[0233] As can be understood from these two figures, in order for PD 30 to receive light that has passed through the left pupil region EPL in the optical system 16 with a short pupil distance, it is necessary to process the incident light that is incident at a more oblique angle.

[0234] That is, in Figure 19 In the structure shown, the right opening 35R is narrowed, and the opening end of the right opening 35R on the exit pupil EP side is located further away from the exit pupil EP.

[0235] By adopting this configuration, the light-shielding pixels 21AR and 21AL can correspond to the short pupil distance of the outgoing pupil EP, and the pupil segmentation function can be made to work.

[0236] Note that in Figure 17 and Figure 19 In the light-shielding pixel 21AR shown, the offsets of the inner lens 32, the color filter 33, and the on-chip microlens 34 are the same.

[0237] In other words, only the shape of the right opening 35R of the light-shielding part 31 is different.

[0238] Provided on imaging element 7 Figure 17 The multiple light-shielding pixels 21AR and 21AL shown in the figure Figure 19 The multiple light-shielding pixels 21AR and 21AL are shown.

[0239] In this example, eight types of light-blocking pixels 21AR and 21AL, corresponding to the pupil distance, are arranged on the imaging element 7. That is, among the eight types of light-blocking pixels 21AR and 21AL, the opening areas of the right opening 35R and the left opening 35L are different in eight stages.

[0240] In the following description, the eight pupil distances are referred to as pupil distance S0 to pupil distance S7, starting from the nearest pupil distance.

[0241] Then, the light-blocking pixels 21AR and 21AL corresponding to the pupil distance S0 are set as light-blocking pixels RS0 and LS0. Similarly, the light-blocking pixels 21AR and 21AL corresponding to the pupil distance S1 are set as light-blocking pixels RS1 and LS1, the light-blocking pixels 21AR and 21AL corresponding to the pupil distance S2 are set as light-blocking pixels RS2 and LS2, the light-blocking pixels 21AR and 21AL corresponding to the pupil distance S3 are set as light-blocking pixels RS3 and LS3, the light-blocking pixels 21AR and 21AL corresponding to the pupil distance S4 are set as light-blocking pixels RS4 and LS4, the light-blocking pixels 21AR and 21AL corresponding to the pupil distance S5 are set as light-blocking pixels RS5 and LS5, the light-blocking pixels 21AR and 21AL corresponding to the pupil distance S6 are set as light-blocking pixels RS6 and LS6, and the light-blocking pixels 21AR and 21AL corresponding to the pupil distance S7 are set as light-blocking pixels RS7 and LS7.

[0242] Furthermore, the first pixel row of the light-blocking pixels RS0 and LS0 is set as first pixel row 22A0, the first pixel row of the light-blocking pixels RS1 and LS1 is set as first pixel row 22A1, the first pixel row of the light-blocking pixels RS2 and LS2 is set as first pixel row 22A2, the first pixel row of the light-blocking pixels RS3 and LS3 is set as first pixel row 22A3, the first pixel row of the light-blocking pixels RS4 and LS4 is set as first pixel row 22A4, the first pixel row of the light-blocking pixels RS5 and LS5 is set as first pixel row 22A5, the first pixel row of the light-blocking pixels RS6 and LS6 is set as first pixel row 22A6, and the first pixel row of the light-blocking pixels RS7 and LS7 is set as first pixel row 22A7.

[0243] Figure 20 An example of the arrangement pattern of the first pixel rows 22A0 to 22A7 is shown.

[0244] Figure 20This is an enlarged view of a portion of the imaging element 7. As shown, the first pixel row 22A0, in which light-shielding pixels RS0 and LS0 are provided, and the first pixel row 22A7, in which light-shielding pixels RS7 and LS7 are provided, are set to be perpendicularly separated from each other by a specific distance, and the pixel rows from the first pixel row 22A0 to the first pixel row 22A7 are periodically arranged in the vertical direction.

[0245] Each pixel row from the first pixel row 22A0 to the first pixel row 22A7 outputs phase difference signals SG0 to SG7 to calculate the defocus amount, and the defocus amount calculation unit 8a in the subsequent stage selects an appropriate phase difference signal SG according to the pupil distance of the exit pupil EP and calculates the defocus amount. For example, when the pupil distance of the exit pupil EP is the pupil distance S3, the phase difference signal SG3 output from the first pixel row 22A3 is used to calculate the defocus amount.

[0246] Next, the relationship between the PD segmentation pixel 21B, which is a pixel 21M arranged in the off-axis region Arm, and the pupil distance will be described. The condition in which only light that has passed through the left pupil region EPL is incident on the right PD 40R and only light that has passed through the right pupil region EPR is incident on the left PD 40L only occurs when using the optical system 16 with the pupil distance designed in the imaging element 7. In this case, since the light passing through the center of the exit pupil EP enters the boundary between the two segmentation pixels, the pupil segmentation function operates appropriately.

[0247] However, when using an optical system 16 with a pupil distance different from that designed by the imaging element 7, light passing through the center of the exit pupil EP is unevenly incident on any of the segmented pixels. To make the pupil segmentation function function properly in this state, it is conceivable to change the size ratio of the two segmented pixels so that light passing through the center of the exit pupil EP enters the boundary between the two segmented pixels. That is, similar to setting various types of light-blocking pixels 21A on the imaging element 7, it is necessary to set PD segmentation pixels 21B in the imaging element 7, each corresponding to a pupil distance S0 to S7.

[0248] However, changing the area ratio of the two segmented pixels in each PD segmented pixel 21B is technically challenging and leads to increased manufacturing costs.

[0249] Therefore, in the imaging apparatus 1 according to this embodiment, only the imaging element 7 is provided with PD segmentation pixels 21B corresponding to a specific pupil distance (e.g., pupil distance S3).

[0250] Note that an example of setting eight types of light-shielding pixels 21AR and 21AL based on the pupil distance has been described here, but other types can be set. That is, regardless of the pupil distance, only one type of light-shielding pixel 21AR and 21AL can be provided, or multiple types can be provided based on the pupil distance.

[0251] <6. Differences in AF control due to phase difference pixels>

[0252] AF control based on the phase difference signal output from the imaging element 7 has been described, including methods using the light-blocking pixel 21A and methods using the PD-segmented pixel 21B.

[0253] Here, we will refer to Figure 21 Describe the differences between them.

[0254] First, regarding the application of each pixel, the light-blocking pixel 21A is a pixel dedicated to phase difference detection. That is, these pixels are not used to generate the normal image signal. The normal image signal is the signal used to generate an image of the subject. Because the light-blocking pixel 21A blocks a portion of the light incident on the pixel area through the light-blocking portion 31, the output signal of the light-blocking pixel 21A is not used to generate the normal image signal.

[0255] On the other hand, PD segmentation pixel 21B is a pixel used for phase difference detection and also for generating normal image signals. By adding the outputs of the left PD 40L and the right PD 40R, PD segmentation pixel 21B can be regarded as a normal pixel.

[0256] As for the number of permutations, such as Figure 5 As shown, the number of light-blocking pixels 21A decreases, while the number of PD segmentation pixels 21B increases.

[0257] Since the light-shielding pixel 21A cannot be used as a regular pixel (i.e., a pixel used to generate an image of the subject), a large number of light-shielding pixels cannot be set. Furthermore, because a large number of light-shielding pixels cannot be set, a large number of light-shielding pixels with the same light-shielding area cannot be included, and the output signals cannot be summed. Therefore, when performing AF control using the light-shielding pixel 21A, low-light performance is reduced.

[0258] On the other hand, since the PD segmentation pixel 21B can be used as a normal pixel, a large number of PD segmentation pixels can be set, and since the left PD 40L and right PD 40R have the same size, multiple output signals can be added together. Therefore, the signal-to-noise ratio can be improved, and low-light performance is high.

[0259] This will describe the design flexibility of pupil correction.

[0260] In the light-shielding pixel 21A, light-shielding portions 31 with different light-shielding areas can be created relatively easily. Therefore, since various types of light-shielding pixels 21A (light-shielding pixels RS0 to RS7 and LS0 to LS7) can be easily set according to multiple pupil distances, phase difference signals can be output with high reliability even if the pupil distances are different. Therefore, the light-shielding pixel 21A has pupil correction design flexibility.

[0261] On the other hand, since it is difficult to manufacture the PD segmentation pixel 21B by changing the size ratio of the left PD 40L and the right PD 40R, it is difficult to provide multiple types of PD segmentation pixels 21B according to multiple pupil distances. Therefore, in this embodiment, only PD segmentation pixels 21B corresponding to a specific pupil distance are provided on the imaging element 7. Therefore, if the pupil distance is different from the design, the reliability of the phase difference signal is reduced. That is to say, the PD segmentation pixel 21B does not have pupil correction design flexibility.

[0262] Off-axis performance corresponds to the flexibility of pupil correction design. Off-axis performance is the accuracy of AF control of a subject exposed in the off-axis region Arm, which is the area other than the on-axis region ArmC, a rectangular area located near the center of the imaging element 7.

[0263] Because the light-shielding pixel 21A has flexible pupil correction design, even if light passing through the exit pupil EP is incident obliquely on the light-shielding pixel 21A located in the off-axis region Arm, an appropriate phase difference signal is output in any of the light-shielding pixels RS0 to RS7 and the light-shielding pixels LS0 to LS7. Therefore, the off-axis performance of the light-shielding pixel 21A is high.

[0264] On the other hand, since the PD segmentation pixel 21B lacks the flexibility of pupil correction design, when the pupil distance differs from the design, and light passing through the exit pupil EP is incident obliquely onto the PD segmentation pixel 21B located in the off-axis region Arm, the pupil segmentation function does not function well, and the reliability of the output phase difference signal decreases. Therefore, the off-axis performance when using the PD segmentation pixel 21B is lower than the off-axis performance when using the light-shielding pixel 21A.

[0265] <7. Selection of Phase Difference Signal>

[0266] In this embodiment, reference will be made to Figure 22 Describe which phase difference signal to select from the phase difference signals output by the light-blocking pixel 21A and the PD segmentation pixel 21B when calculating the amount of defocus.

[0267] Figure 22The upper part of the graph shows the variation of AF error relative to exposure in the on-axis region ArC. The solid line in the graph represents the case where AF control is performed based on the phase difference signal output from the light-blocking pixel 21A. Conversely, the dashed line in the graph represents the case where AF control is performed based on the phase difference signal output from the PD segmentation pixel 21B.

[0268] Figure 22 The lower part of the graph shows the phase difference signal selected for each combination of exposure and AF control target area.

[0269] First, it is conceivable to select the phase difference signal output from either the light-shielding pixel 21A or the PD segmentation pixel 21B based on the exposure. For example, when the exposure is equal to or greater than a threshold Th, the phase difference signal output from the light-shielding pixel 21A is selected. This is because the phase difference signal output from the light-shielding pixel 21A has a short readout time (specifically, described later), and if there are no problems in the calculation of the defocus amount, it is desirable to select the phase difference signal output from the light-shielding pixel 21A.

[0270] On the other hand, when the exposure is less than the threshold Th, the reliability of the calculated defocus amount decreases if the phase difference signal output from the light-shielding pixel 21A is selected. Therefore, it is conceivable to select the phase difference signal output from the PD segmentation pixel 21B. Note that in this example, when the exposure is less than the threshold Th, other conditions are considered when selecting one of the phase difference signals or the contrast method output from the light-shielding pixel 21A and the PD segmentation pixel 21B.

[0271] A detailed description will be provided.

[0272] When the exposure is equal to or greater than the threshold Th, the phase difference signal output from the light-blocking pixel 21A is selected, regardless of whether the region is an on-axis region ArC or an off-axis region ArM. (Refer to...) Figure 23 and Figure 24 This describes the reading of charge obtained by photoelectric conversion of light received by the light-shielding pixel 21A and the PD segmentation pixel 21B when the exposure is equal to or greater than the threshold Th.

[0273] The signal output from the light-blocking pixel 21A is read out as a phase difference signal SGn for each pixel row. The phase difference signal SGn is one of the phase difference signals SG0 to SG7, and the appropriate phase difference signal SGn is selected according to the pupil distance.

[0274] Since the signal output from the PD segmentation pixel 21B is used to generate the normal image signal, the signal obtained by adding the outputs of the segmentation pixels is read. Figure 24 The reading timing of the signals from the light-shielding pixel 21A and the PD segmentation pixel 21B is shown.

[0275] As shown in the figure, based on the vertical synchronization signal Vsync, starting from the top of the imaging element 7, the first pixel row 22A to which the phase difference signal SGn of the light-shielding pixel 21A belongs is read out sequentially. The readout time required to read out all the phase difference signals SGn output from the imaging element 7 is set as time T1.

[0276] The signals output from PD segmentation pixel 21B include signal SGR output from the second pixel row 22B to which PD segmentation pixels 21Br and PD segmentation pixels 21Bg belong, and signal SGB output from the second pixel row 22B to which PD segmentation pixels 21Bb and PD segmentation pixels 21Bg belong. Signals SGR and SGB are read sequentially from the top of imaging element 7.

[0277] Set the readout time required to read out all signals SGR and SGB output from imaging element 7 to time T2.

[0278] Since the number of PD segmentation pixels 21B is greater than the number of light-blocking pixels 21A, time T2 is set to be longer than time T1.

[0279] When the exposure is equal to or greater than the threshold Th, fast AF control can be achieved by using the phase difference signal from the light-blocking pixel 21A, which has a short readout time. Furthermore, in the off-axis region Arm, the reliability of the phase difference signal output from the PD segmentation pixel 21B decreases as the pupil distance moves further away from the design distance; however, when the exposure is equal to or greater than the threshold Th, this can be avoided by selecting the phase difference signal output from the light-blocking pixel 21A in the off-axis region Arm.

[0280] Furthermore, since the signal readout time in the light-shielding pixel 21A is short, the time until focusing is completed by AF control can be shortened.

[0281] Next, we will describe the case where the exposure is less than the threshold Th.

[0282] In the on-axis region ArC, the phase difference signal output from PD segmentation pixel 21B is selected. The reference... Figure 25 and Figure 26 The description describes the reading of charge obtained by photoelectric conversion of light received by light-shielding pixel 21A and PD segmentation pixel 21B.

[0283] The output signal from the first pixel row 22A, where the light-shielding pixels 21A are disposed, is used to generate a normal image signal. Specifically, only the output signal from each segmented pixel, which is a PD segmentation pixel 21Bg arranged between the light-shielding pixels 21A, is read out.

[0284] Since the output signal from the second pixel row 22B, where the PD segmentation pixel 21B is set, is used as the phase difference signal, the output of each segmentation pixel is read out (see...). Figure 25 Furthermore, since the exposure is less than the threshold Th, in order to improve the S / N ratio, the required output counts of adjacent second pixel rows 22B are summed to improve the S / N ratio of the phase difference signal. Figure 25 In the initial step, two second pixel rows 22B are added together, but with a further reduction in exposure, three or more second pixel rows 22B are added together. Therefore, the degradation of the S / N ratio can be suppressed relative to the reduction in exposure, and appropriate AF control can be performed.

[0285] Note that the phase difference signal from the second pixel row 22B includes the phase difference signal SGr output from the second pixel row 22B including the PD segmentation pixel 21Br and the phase difference signal SGb output from the second pixel row 22B including the PD segmentation pixel 21Bb. In the addition of the phase difference signals, the phase difference signals SGr are added and the phase difference signals SGb are added.

[0286] Figure 26 The timing for reading the signal from PD segmentation pixel 21B is shown. Note that the signal from the light-blocking pixel 21A is not used for AF control or image signal generation, and therefore is not read.

[0287] The phase difference signals SGr and SGb output from the PD segmented pixel 21B are read out sequentially from the top of the imaging element 7.

[0288] The readout time required to read out all the phase difference signals SGr and SGb output from imaging element 7 is defined as time T3. Time T3 is essentially... Figure 24 Twice the time T2. For example... Figure 23 and Figure 25 As shown, this is based on the difference between performing reads in units of pixels or in units of segmented pixels.

[0289] Next, the phase difference signal will be described when the exposure is less than the threshold Th and the signal is in the off-axis region ArM. In this case, as... Figure 22 As shown, the phase difference signal output from the light-blocking pixel 21A is selected.

[0290] In this case, such as Figure 23 and Figure 24 As shown, the phase difference signal SGn output from the first pixel row 22A is read at time T1, and the signals SGR and SGB output from the second pixel row 22B for generating the normal image signal are read at time T2.

[0291] However, when the optical system 16 includes an ideal pupil distance set for the PD segmented pixel 21B, the phase difference signal output from the PD segmented pixel 21B can be selected even in the AF control of the off-axis region Arm. In this case, as Figure 25 and Figure 26 As shown, the phase difference signal SGn output from the light-blocking pixel 21A of the first pixel row 22A is not read, but the output from the PD segmentation pixel 21B arranged in the first pixel row 22A and the second pixel row 22B is read at time T3.

[0292] Note that in the off-axis region Arm where the optical system 16 includes an ideal pupil distance set for the PD segmentation pixel 21B, it may be unclear which of the phase difference signals output from the light-shielding pixel 21A and the PD segmentation pixel 21B has higher reliability. In this case, both the phase difference signal from the light-shielding pixel 21A and the phase difference signal from the PD segmentation pixel 21B are acquired, and both can be used to perform AF control.

[0293] Figure 27 and Figure 28 The diagram illustrates the acquisition of the phase difference signal from the light-blocking pixel 21A and the phase difference signal from the PD segmentation pixel 21B.

[0294] like Figure 27 As shown, the phase difference signal SGn output from the first pixel row 22A where the light-shielding pixel 21A is arranged, and the phase difference signals SGr and SGb output from the second pixel row 22B where the PD segmentation pixel 21B is arranged are read out.

[0295] The readout time for phase difference signal SGn is set to time T1, and the readout time for phase difference signals SGr and SGb is set to time T3. The total readout time obtained by adding time T1 and time T3 is shorter than one cycle of the vertical synchronization signal Vsync.

[0296] Note that in Figure 22 Previously, it was described that the phase difference signal output from the light-shielding pixel 21A was used in the off-axis region Arm where the exposure was less than the threshold Th. However, there are cases where sufficient AF control cannot be performed even when using the phase difference signal output from the light-shielding pixel 21A. For example, there are cases where the exposure is significantly lower than the threshold Th, and the pupil distance of the exit pupil is significantly different from the design of the PD segmentation pixel 21B.

[0297] In this case, AF control using the contrast method described above can be performed. Therefore, appropriate AF control can be performed under a wide range of conditions.

[0298] <8. Exposure Control>

[0299] Although the selection of phase difference signals based on exposure has already been described, here we will refer to... Figure 29 This describes the calculation of exposure.

[0300] Imaging element 7 outputs normal pixel output for generating normal image signals and phase difference pixel output as phase difference signals for AF control.

[0301] Normal pixel output and phase difference pixel output are input to the output level detection circuit included in the camera signal processing unit 8. The output level detection circuit calculates the average output value in the exposure calculation target area on the pixel based on the input normal pixel output and phase difference pixel output, and each average output value is output from the camera signal processing unit 8 and input to the camera control unit 14.

[0302] Note that the exposure calculation target area is determined based on the metering mode and is the area in the image where the exposure calculation will be performed. For example, when "Center-Weighted Metering" is selected, the exposure calculation is performed on the central portion of the image. Furthermore, when "Spot Metering" is selected, a specified narrow area is set as the target for the exposure calculation. Additionally, when "Whole Image Average Metering" is selected, the exposure calculation is performed on the entire image area.

[0303] The camera control unit 14 calculates the exposure based on the detection results output from the camera signal processing unit 8, and determines the shutter speed (or parameters that can adjust the exposure, such as F-number or gain). The camera control unit 14 processes the shutter speed determined by the imaging element 7.

[0304] Shutter speed can be set, for example, by setting the travel time of the electronic backing film.

[0305] The exposure calculation performed by the camera control unit 14 can be performed based solely on the normal pixel output, or solely on the phase-difference pixel output. Alternatively, exposure control can be performed based on both the normal pixel output and the phase-difference pixel output. In this case, phase-difference pixel exposure control is performed based on the phase-difference pixel output, and normal pixel exposure control is performed based on the normal pixel output.

[0306] <9. Processing Flow>

[0307] <9-1. First Example>

[0308] The first example of the AF control processing flow is in Figure 30 , Figure 31 and Figure 32 As shown in the image.

[0309] Notice, Figure 30 , Figure 31 and Figure 32 Each of the processes shown is performed by the camera control unit 14 and the camera signal processing unit 8 of the imaging device 1. Here, the camera control unit 14 and the camera signal processing unit 8 are collectively referred to as "processing unit 60".

[0310] When a half-pressed state is detected in the shutter button 6S, which is one of the operators 6, in step S101, the processing unit 60 (camera control unit 14 or camera signal processing unit 8) of the imaging device 1 performs an exposure calculation. In the exposure calculation, as referenced... Figure 29 The output level of one or both of the normal pixel output and the phase difference pixel output is detected and processed, and the exposure is calculated based on the detection result.

[0311] In step S102, the processing unit 60 determines whether the exposure amount is equal to or greater than the threshold Th.

[0312] When the exposure is equal to or greater than the threshold Th, in step S103, the processing unit 60 performs an addition readout setting on the PD segmentation pixel 21B. The addition readout setting is a setting used to add and simultaneously read out the charges accumulated in the left PD 40L and right PD 40R included in the PD segmentation pixel 21B. That is, the PD segmentation pixel 21B is treated as a single pixel.

[0313] Next, the processing unit 60 reads out the light-shielding pixel 21A in step S104 and performs the summation reading of the PD segmentation pixel 21B in step S105. Therefore, the phase difference signal read from the light-shielding pixel 21A and the normal pixel signal read from the PD segmentation pixel 21B are read out.

[0314] In step S107, the processing unit 60 calculates the defocus amount based on the phase difference signal output from the light-blocking pixel 21A. At this time, the processing unit 60 calculates the defocus amount using the output from the first pixel row 22A0 to 22A7, which includes the appropriate light-blocking pixel 21A, according to the pupil distance of the exit pupil EP. Note that, for example, when the lens barrel 3 is mounted to the imaging device 1, the camera control unit 14 obtains the pupil distance of the exit pupil EP from the lens barrel control unit 18 and stores the pupil distance in the storage unit 15.

[0315] In step S108, the processing unit 60 generates display image data of the through image based on the pixel signals from the PD segmentation pixel 21B. The generated display image data is then displayed as a through image on the EVF monitor 5a or the rear monitor 4, etc.

[0316] Processing unit 60 in Figure 31In step S109, lens driving is performed based on the defocus amount, and in step S110, lens driving is stopped. As a result, the focusing lens arranged in the lens barrel 3 is driven and focused via the driver unit 17.

[0317] In step S111, the processing unit 60 determines whether a half-pressed state of the shutter button 6S is being detected. If the half-pressed state of the shutter button 6S continues, the processing unit 60 returns to... Figure 30 The process in step S101 is performed, and focus control is executed based on the exposure amount.

[0318] Therefore, as long as the shutter button is held in a half-pressed state for 6 seconds, the focus will be maintained continuously.

[0319] If, in step S102, it is determined that the exposure amount is less than the threshold Th, in step S112, the processing unit 60 performs a non-additive readout setting on the PD segmentation pixel 21B. The non-additive readout setting is a setting used for readout that allows differentiation of the accumulated charge in each of the left PD 40L and right PD 40R included in the PD segmentation pixel 21B at one time. That is, each segmentation pixel is processed independently.

[0320] In step S113, the processing unit 60 reads out the light-blocking pixel 21A and in step S114 performs non-additive readout of the PD segmented pixels.

[0321] In step S115, the processing unit 60 calculates the defocus amount based on the phase difference signal output from the light-blocking pixel 21A. Furthermore, in step S116, the processing unit 60 calculates the defocus amount based on the phase difference signal output from the PD segmentation pixel 21B.

[0322] By executing each process from step S113 to step S116, focus control can be performed based on the phase difference signals of the light-shielding pixel 21A and the PD segmentation pixel 21B.

[0323] exist Figure 32 In step S117, the processing unit 60 performs addition processing for each PD segmentation pixel 21B to combine the non-additive signals output from the segmentation pixels of the PD segmentation pixel 21B, and performs processing for generating display image data for passing through the image.

[0324] In step S118, the processing unit 60 compares the differential integral values ​​when calculating the defocus amount of each of the light-shielding pixel 21A and the PD segmentation pixel 21B. The differential integral values ​​are as follows: Figures 8 to 12 Each of these terms represents the magnitude of the phase difference. In the calculation of defocus, the amount of waveform shift is determined to reduce the difference integral value. However, it can be considered that the smaller the difference integral value is when calculating the final defocus, the higher the reliability of the calculated defocus.

[0325] Step S118 is a comparison process used to determine which of the defocus amounts calculated based on the output of the light-blocking pixel 21A and the defocus amounts calculated based on the output of the PD segmentation pixel 21B is highly reliable.

[0326] In step S119, the processing unit 60 determines whether the differential integral value in the light-shielding pixel 21A is small. If it is determined that the differential integral value in the light-shielding pixel 21A is small, that is, if it is determined that the defocus amount calculated based on the output of the light-shielding pixel 21A has high reliability, in step S120, the processing unit 60 performs lens driving based on the defocus amount calculated from the output of the light-shielding pixel 21A.

[0327] On the other hand, if it is determined in step S119 that the differential value in the PD segmentation pixel 21B is small, that is, if it is determined that the defocus amount calculated based on the output of the PD segmentation pixel 21B has high reliability, the processing unit 60 performs lens driving based on the defocus amount calculated based on the output of the PD segmentation pixel 21B in step S121.

[0328] After performing step S120 or step S121, the processing unit 60 stops the lens drive in step S122.

[0329] In step S123, the processing unit 60 determines whether a half-pressed state of the shutter button 6S is being detected. If the half-pressed state of the shutter button 6S continues, the processing unit 60 returns to... Figure 30 The process in step S101 is performed, and focus control is executed based on the exposure amount.

[0330] Therefore, as long as the shutter button is held in a half-pressed state for 6 seconds, the focus will be maintained continuously.

[0331] Note that in Figure 30 and Figure 31 In each of the processes described in steps S103 to S111, such as Figure 23 and Figure 24 As shown, the light-blocking pixel 21A is considered as the pixel that outputs the phase difference signal for focus control, and the PD segmentation pixel 21B is considered as the pixel that outputs the normal pixel signal.

[0332] In addition, Figure 30 and Figure 31 In each of the processes described in steps S112 to S123, such as Figure 27 and Figure 28 As shown, each pixel of the light-blocking pixel 21A and the PD segmentation pixel 21B is regarded as a pixel that outputs a phase difference signal, and the PD segmentation pixel 21B is also regarded as a pixel that outputs a normal pixel signal.

[0333] like Figure 25 and Figure 26 As shown, when the exposure is set to be less than the threshold Th, PD segmentation pixel 21B is treated as a pixel that outputs a phase difference signal and a pixel that outputs a normal pixel signal, instead of using the light-blocking pixel 21A. This can be achieved by not performing... Figure 30 The processing of steps S113 and S115 and Figure 32 This is achieved through the processing of steps S118, S119, and S120.

[0334] <9-2. Second Example>

[0335] Reference Figure 30 , Figure 31 and Figure 33 A second example describing the processing flow of AF control. Note that processes similar to those in the first example are denoted by the same reference numerals, and their descriptions will be omitted appropriately.

[0336] When the shutter button is detected to be half-pressed for 6 seconds, the processing unit 60 of the imaging device 1 (camera control unit 14 or camera signal processing unit 8) in... Figure 30 In step S101, the exposure calculation is performed.

[0337] Next, in step S102, the processing unit 60 determines whether the exposure amount is equal to or greater than the threshold Th.

[0338] When the exposure is equal to or greater than the threshold Th, AF control is performed using the phase difference signal output from the light-blocking pixel 21A. The processing in this case is similar to that in the first example... Figure 30 Step S103 to Figure 31 The processing of step S111 in the process.

[0339] On the other hand, if it is determined that the exposure is less than the threshold Th, the processing unit 60 performs... Figure 30 Each process from step S112 to step S116 in the process is then executed. Figure 33 Step S117 in the process.

[0340] Therefore, the phase difference signal from the light-blocking pixel 21A and the phase difference signal from the PD segmentation pixel 21B are acquired, and display image data for through image display is generated based on the signal from the PD segmentation pixel 21B.

[0341] Next, in Figure 33In step S131, the processing unit 60 determines whether the deviation between the pupil distance of the exit pupil EP of the PD segmentation pixel 21B outputting an appropriate phase difference signal and the pupil distance of the exit pupil EP of the optical system 16 is greater than a predetermined value, that is, whether the pupil distance of the exit pupil EP of the optical system 16 is closer, farther away, or consistent with the pupil distance information of the exit pupil EP obtained from the lens barrel 3.

[0342] As described above, since the incident angle of light on pixel 21 varies depending on the pupil distance of the exit pupil EP, if the pupil distance deviates from the design, it is very likely that a phase difference signal with high reliability cannot be obtained from the PD segmented pixel 21B.

[0343] When the deviation is determined to be equal to or greater than a predetermined value—that is, when the pupil distance of the exit pupil EP is determined to be closer than designed for the optical system 16, or when the pupil distance is determined to be farther than designed for the optical system 16—the phase difference signal output from the PD segmentation pixel 21B arranged in the off-axis region Arm has low reliability. Therefore, in step S132, the processing unit 60 uses the phase difference signal output from the PD segmentation pixel 21B in the on-axis region Arm and the phase difference signal output from the light-blocking pixel 21A in the off-axis region Arm to calculate the defocus amount.

[0344] Subsequently, in step S133, the processing unit 60 performs lens driving based on the calculated defocus amount.

[0345] On the other hand, if the deviation of the pupil distance is determined to be less than a predetermined value in step S131, that is, if the pupil distance of the exit pupil EP is determined to be the designed optical system 16 or if the exit pupil EP is determined to be close to the designed optical system 16, the processing unit 60 uses the phase difference signal output from the PD segmented pixel 21B arranged in the off-axis region Arm to calculate the defocus amount in step S134 because the phase difference signal is highly reliable.

[0346] Next, in step S135, the processing unit 60 performs lens driving based on the defocus amount.

[0347] After performing step S133 or step S135, the processing unit 60 stops the lens drive in step S136.

[0348] In step S137, the processing unit 60 determines whether a half-pressed state of the shutter button 6S is being detected. If the half-pressed state of the shutter button 6S continues, the processing unit 60 returns to... Figure 30 The process in step S101 is performed, and focus control is executed based on the exposure amount.

[0349] Therefore, as long as the shutter button is held in a half-pressed state for 6 seconds, the focus will be maintained continuously.

[0350] In addition, in the above example, focusing is performed on both the on-axis region ArC and the off-axis region ArM (i.e., the entire surface of the imaging element 7). However, there are also cases where focusing is performed on a specific area. For example, if the on-axis region ArC receives reflected light from the person to be focused (i.e., if the person to be focused is captured in the on-axis region ArC), focusing only needs to be performed on the on-axis region ArC (or, specifically, the area in which the person to be focused is captured).

[0351] In this case, without performing the judgment process in step S131, the defocus amount is calculated in step S132 using the phase difference signal output from the PD segmentation pixel 21B in the on-axis region ArC.

[0352] Alternatively, if the person to be focused is shown in the off-axis region Arm, focusing only needs to be performed in the off-axis region Arm. Specifically, if it is determined to be "yes" in step S131, the amount of defocus is calculated using the phase difference signal output from the light-blocking pixel 21A of the off-axis region Arm (step S132), and if it is determined to be "no" in step S131, the amount of defocus is calculated using the phase difference signal output from the PD segmentation pixel 21B of the off-axis region Arm (step S134).

[0353] Furthermore, there may be situations where the person to be focused on could be located in both the on-axis region ArC and the off-axis region ArM. In such cases, as described above, the amount of defocus can be calculated by performing each of steps S131 to S135.

[0354] Alternatively, in the absence of a person or object to be focused, for example, without instructions from the user, after calculating the defocus amount for each on-axis region ArC or off-axis region ArM or for each subject (for each area of ​​a subject being photographed) (steps S132 and S134), the determination process for determining the focus target, the process for allowing the user to select the focus target or region of interest (ROI), etc., can be performed before the lens drive is performed in steps S133 and S135, and the lens drive can be performed based on the result of the determination process or the selection process.

[0355] <10. Examples of Deformation>

[0356] In the example above, AF control is performed based on the calculated amount of defocus. As another example, it is conceivable to use the amount of defocus to provide notification or display control to the user, and to allow for manual focusing.

[0357] For example, the UI control unit 14a of the camera control unit 14 can assist the user's manual focusing operation by displaying information about focus on the EVF monitor 5a or the rear monitor 4.

[0358] Furthermore, while an example of a single PD being included in the aforementioned light-shielding pixel 21A has been described, it can also include multiple PDs as segmented pixels. For example, all pixels of the imaging element 7 can be PD segmented pixels comprising two segmented pixels. That is, each pixel can include a left PD 40L and a right PF 40R. Even with this configuration, in the light-shielding pixel 21A, by adding and reading the outputs from the two segmented pixels, they can be treated as a single PD.

[0359] Since all pixels on imaging element 7 have the same configuration, manufacturing imaging element 7 is convenient. Furthermore, manufacturing costs can be reduced.

[0360] <11. Summary>

[0361] As described in each of the above examples, the imaging device 1 according to the present technology includes an imaging element 7, which includes a light-shielding pixel 21A that has a pupil segmentation function by including a light-shielding portion 31 and a light-receiving element (PD 30), the light-shielding portion blocking one of a pair of partial regions (left pupil region EPL and right pupil region EPR) that have deviated in opposite directions in a predetermined direction (e.g., left-right direction) in the exit pupil EP, the light-receiving element receiving the other light beam; and a photodiode segmentation pixel (PD segmentation pixel 21B) that has a pupil segmentation function by including segmentation pixels (left PD 40L and right PD 40R) that respectively receive the pair of light beams passing through the pair of partial regions.

[0362] The light-shielding pixel 21A includes, for example, light-shielding pixel 21AR and light-shielding pixel 21AL. Light-shielding pixel 21AR is a pixel on which only light passing through the left pupil region EPL is incident, and the left pupil region EPL is the left half of the exit pupil of the light-shielding part 31. Light-shielding pixel 21AL is a pixel on which only light passing through the right pupil region EPR is incident, and the right pupil region EPR is the right half of the exit pupil of the light-shielding part 31.

[0363] Furthermore, PD segmentation pixel 21B is a so-called PD segmentation pixel, in which multiple segmentation pixels (left PD 40L and right PD 40R) are arranged in the region of a single pixel. Each pixel is, for example, a color pixel of a Bayer array and includes a color filter 33 and a light receiving element (PD 30, left PD 40L and right PD 40R) that receives incident light passing through the color filter 33.

[0364] Furthermore, the defocus calculation unit 8a calculates the defocus amount based on the exposure amount using at least one of the signal (phase difference signal SG) output from the light-blocking pixel 21A and the signal (phase difference signals SGr and SGb) output from the PD segmentation pixel 21B.

[0365] The light-blocking pixel 21A and the PD segmentation pixel 21B each output a phase difference signal capable of calculating the amount of defocus. Therefore, by using an appropriate phase difference signal based on the exposure from the phase difference signals output from the light-blocking pixel 21A and the PD segmentation pixel 21B, an appropriate amount of defocus can be calculated.

[0366] Furthermore, each pixel included in the imaging element 7 can be one of the light-shielding pixel 21A and the PD segmentation pixel 21B.

[0367] That is, in the imaging element 7, all pixels are one of the light-blocking pixels 21A and PD segmentation pixels 21B.

[0368] Therefore, it is not necessary to complicate the manufacturing of the imaging element 7, and the manufacturing time can be shortened and the manufacturing cost reduced.

[0369] For reference Figure 22 As described in the selection of the phase difference signal, the defocus calculation unit 8a can use the output signal (phase difference signal SG) of the light-blocking pixel 21A to calculate the defocus amount when the exposure is equal to or greater than the threshold Th.

[0370] Since the light-shielding pixel 21A includes the light-shielding portion 31, its output signal level is lower than that of the PD segmentation pixel 21B.

[0371] Furthermore, the number of light-shielding pixels 21A is less than the number of PD segmentation pixels 21B, and the light-shielding pixels are discretely arranged. Therefore, when the exposure is low, it is not possible to increase the signal level by adding the signals output from multiple pixels, and it is difficult to improve the S / N ratio. As a result, there is a possibility that the reliability of the calculated defocus amount is reduced. However, when the exposure is equal to or greater than the threshold Th, the output level of the phase difference signal from the light-shielding pixels 21A is equal to or greater than a certain level, thereby improving the reliability of the defocus amount calculated using the output signal of the light-shielding pixels 21A.

[0372] For reference Figure 22 In the selection of the phase difference signal, when the exposure is less than the threshold Th, the defocus calculation unit 8a can calculate the defocus amount using at least one of the output signal of the light-blocking pixel 21A and the output signal of the PD segmentation pixel 21B, depending on whether it is the on-axis region ArC or the off-axis region ArM. The on-axis region ArC is the region including the central part of the imaging element 7, and the off-axis region ArM is the region of the imaging element 7 other than the on-axis region ArC.

[0373] When the exposure is less than the threshold Th, there are two scenarios: one is to preferably use the output signal of the PD segmentation pixel 21B, which has high low-light performance, to calculate the defocus amount; the other is to preferably use the output signal of the light-shielding pixel 21A, which can select an appropriate output based on the pupil distance of the exit pupil EP, to calculate the defocus amount. According to this structure, the output signal of an appropriate image plane phase difference pixel can be selected, enabling highly reliable calculation of the defocus amount.

[0374] For reference Figure 22 As described in the selection of phase difference signals, the defocus calculation unit 8a can use the output signals (phase difference signals SGr and SGb) of the PD segmentation pixel 21B in the on-axis region ArC to calculate the defocus amount.

[0375] Since the PD segmentation pixel 21B does not include the light-blocking portion 31, its output signal level is higher than that of the light-blocking pixel 21A.

[0376] Furthermore, the number of PD segmentation pixels 21B is greater than the number of light-blocking pixels 21A. Therefore, even under low exposure conditions, the S / N ratio of the phase difference signal can be improved by summing the outputs of multiple pixel rows, and the output level of the phase difference signal required for calculating the amount of defocus can be easily ensured. Thus, focus position information and lens drive amount can be calculated with high precision.

[0377] For reference Figure 22 As described in the selection of the phase difference signal, the defocus calculation unit 8a can use the output signal of the light-blocking pixel 21A in the off-axis region Arm to calculate the defocus amount.

[0378] Therefore, calculating the defocus amount with high reliability can improve focusing accuracy.

[0379] For reference Figure 22 As described in the selection of the phase difference signal, the defocus calculation unit 8a can use the output signal of the photodiode segmented pixel (PD segmented pixel 21B) in the off-axis region Arm to calculate the defocus amount.

[0380] Therefore, calculating the defocus amount with high reliability can improve focusing accuracy.

[0381] For reference Figure 22 As described in the selection of the phase difference signal, the defocus calculation unit 8a can use the output signal with higher reliability from the output signal of the light-blocking pixel 21A and the output signal of the photodiode segmentation pixel (PD segmentation pixel 21B) to calculate the defocus amount.

[0382] Therefore, a defocusing amount with high reliability was calculated, which can improve focusing accuracy.

[0383] Each segmented pixel (left PD 40L and right PD 40R) of PD segmented pixel 21B has, for example, the same light receiving area.

[0384] In this configuration, to direct light passing through the left pupil region EPL of the exit pupil EP onto one segmented pixel (right PD 40R) and light passing through the right pupil region EPR onto another segmented pixel (left PD 40L), the optical system 16 requires a specific pupil distance based on the arrangement of on-chip microlenses 34, etc., included in the PD segmented pixels 21B. On the other hand, various types of light-shielding pixels 21A (light-shielding pixels LS0 to LS7 and light-shielding pixels RS0 to RS7) can be provided depending on the pupil distance. Therefore, in the off-axis region Arm of the imaging element 7, the light-shielding pixels 21A may have a reduced output signal level due to insufficient light reception, and the PD segmented pixels 21B may have reduced output signal reliability due to inappropriate pupil distances. Therefore, if the defocusing amount is always calculated based on either of the phase difference signals, the reliability may be low.

[0385] In the example above, in the off-axis region ArM where the exposure is less than the threshold Th, both the output signal of the light-blocking pixel 21A and the output signal of the PD segmentation pixel 21B are acquired, and the defocus amount is calculated using the phase difference signal, which has higher reliability. Therefore, a highly reliable defocus amount can be calculated.

[0386] For reference Figure 22 As described in the selection of the phase difference signal, when the exposure is less than the threshold Th, the defocusing amount calculation unit 8a can perform autofocus control based on the contrast method in the off-axis region ArM. The on-axis region ArC is the region including the central part of the imaging element 7, and the off-axis region ArM is the region other than the on-axis region ArC.

[0387] For the off-axis region ArM under insufficient exposure, there is a situation where the reliability of the phase difference information based on the output signal (phase difference signal SG) of the light-blocking pixel 21A and the output signal (phase difference signals SGr and SGb) of the PD segmentation pixel 21B is low.

[0388] In this case, appropriate autofocus control can be performed by employing a contrast method.

[0389] For reference Figure 20 As described above, in the configuration of the imaging element 7, the imaging element 7 can include various types of light-blocking pixels 21A (light-blocking pixels LS0 to LS7 and light-blocking pixels RS0 to RS7) according to the pupil distance of the exit pupil EP.

[0390] Therefore, even if the pupil position of the exit pupil EP changes when the driving optical system 16 is driven, a suitable phase difference signal can be obtained from any of the various types of light-shielding pixels 21A.

[0391] Therefore, for example, even if the pupil distance of the exit pupil EP fluctuates within a wide range by using the interchangeable lens barrel 3, the amount of defocus can be calculated with high reliability.

[0392] For reference Figure 17 and Figure 19 In the structure of the imaging element 7, the light-shielding areas of the light-shielding portions 31 of the various types of light-shielding pixels 21A (light-shielding pixels LS0 to LS7 and light-shielding pixels RS0 to RS7) may be different.

[0393] For example, the smaller the pupil distance, the larger the light-blocking area of ​​the light-blocking part 31.

[0394] Therefore, in each pixel included in the light-shielding pixel 21A, a light-shielding portion 31 is formed, such that light passing through only one side region of the exit pupil EP is received according to the pupil distance, and an appropriate output signal (phase difference signal SG) of the light-shielding pixel 21A can be output.

[0395] For reference Figure 30 As described in the first example of the AF control processing flow, the defocus calculation unit 8a can use the output signal (phase difference signal SG) of the light-blocking pixel 21A selected according to the pupil distance of the exit pupil EP to calculate the defocus amount.

[0396] Therefore, a light-blocking pixel 21A is selected to receive light passing through one side region of the exit pupil EP.

[0397] Therefore, the amount of defocus can be calculated with high reliability using the appropriate output signal of the light-shielding pixel 21A.

[0398] For reference Figure 30 The first example of the AF control process described may include a camera control unit 14 that obtains the pupil distance from a lens barrel control unit 18 included in the lens barrel 3.

[0399] For example, in an imaging device 1 that includes an interchangeable lens barrel 3, a portion of the light-blocking pixels 21A (light-blocking pixels LS0 to LS7 and light-blocking pixels RS0 to RS7) are selected based on the pupil distance.

[0400] For reference Figure 20 In the configuration of the imaging element 7, the light-blocking pixels 21A arranged in the same row (same pixel row) on the imaging element 7 can correspond to the same pupil distance.

[0401] Therefore, the output signals of the various types of light-blocking pixels 21A are not mixed with the pixel signals read from each pixel row.

[0402] Therefore, the output signal of the light-blocking pixel 21A selected according to the pupil distance is easy to process, and the processing load can be reduced.

[0403] For reference Figure 31 and Figure 33 The first and second examples of the processing flow for AF control may include a camera control unit 14 that gives drive commands to the focusing lens included in the imaging optical system (optical system 16) based on the amount of defocus.

[0404] Therefore, the focusing lens is controlled by a phase difference signal based on the exposure.

[0405] Therefore, appropriate focus control based on exposure can be performed.

[0406] As described in the modified example, a user interface control unit (UI control unit 14a) can be provided to perform display control based on the amount of defocus.

[0407] Therefore, for example, information can be provided to inform the photographer of the focus status based on the current lens position.

[0408] Therefore, photographers can perform focusing operations based on the notification information to achieve focus.

[0409] For the selection of the phase difference signal, refer to... Figure 23 As described above, when calculating the defocus amount using the output signal (phase difference signal SG) of the light-shielding pixel 21A, the imaging element 7 can add the output signals of the PD segmentation pixel 21B (left PD 40L and right PD 40R) and output them.

[0410] Therefore, the number of reads can be reduced compared to reading the output from each segmented pixel separately.

[0411] Therefore, the time required to read out pixel signals can be reduced.

[0412] For reference Figure 25 Regarding the selection of phase difference signals, when calculating the defocus amount using the output signals (phase difference signals SGr and SGb) of the PD segmentation pixel 21B, the imaging element 7 can output each of the output signals of the PD segmentation pixel 21B (left PD 40L and right PD 40R).

[0413] Therefore, the output signal of the segmented pixels is obtained without addition.

[0414] Therefore, the output signal of PD segmented pixel 21B can be obtained without losing phase difference information, and the defocus amount can be calculated.

[0415] For the selection of the phase difference signal, refer to... Figure 25 The aforementioned signal processing unit (camera signal processing unit 8) may include a signal processing unit that performs signal processing on the image signal output from the imaging element 7, and when the exposure is less than a predetermined value (for example, when the exposure is an exposure where the output level is less than a predetermined value), the signal processing unit may add the output signal of the PD segmented pixel 21B in the column direction.

[0416] Therefore, the output level of the signals (phase difference signals SGr and SGb) output from the PD segmentation pixel 21B can be maintained at a predetermined level or higher.

[0417] Therefore, the S / N ratio of the output signal of the PD segmented pixel 21B can be improved, and the reliability of the calculated defocus amount can be improved.

[0418] The program according to this embodiment is used to cause, for example, a CPU, DSP, or a device including a CPU and DSP, to execute... Figures 30 to 33 The procedure for each process shown.

[0419] That is, the program of this embodiment is a program for causing an imaging device or the like to perform the following process, wherein the process is used to select at least one phase difference signal from the output signal of the light-shielding pixel and the output signal of the PD segmentation pixel based on the exposure amount, and to calculate the defocus amount. The light-shielding pixel has a pupil segmentation function by including a light-shielding portion and a light receiving element. The light-shielding portion blocks one of a pair of beams that are deviated in opposite directions in the exit pupil. The light receiving element receives the other beam. The PD segmentation pixel has a pupil segmentation function by including a segmentation pixel that receives each beam of the beams that are in the pair of beams that are in the pair of beams.

[0420] Using such a procedure, the aforementioned imaging device 1 can be realized.

[0421] The program for implementing this imaging device 1 can be pre-recorded in an HDD, a ROM in a microcomputer with a CPU, or other recording medium built into a device such as the imaging device 1.

[0422] Alternatively, the program can be temporarily or permanently stored (recorded) on a removable recording medium, such as a floppy disk, CD-ROM, magneto-optical (MO) disk, DVD, Blu-ray disc (registered trademark), magnetic disk, semiconductor memory, or memory card. This removable recording medium can be provided as a so-called software package.

[0423] Furthermore, such programs can be installed on personal computers from removable recording media, or downloaded from download sites via networks such as local area networks (LANs) or the Internet.

[0424] Furthermore, such a program is suitable for providing the imaging device 1 of this embodiment in a wide range of applications. For example, by downloading the program to mobile terminal devices such as smartphones or tablets, mobile phones, personal computers, gaming devices, video devices, personal digital assistants (PDAs), etc., these devices can be used as the imaging device 1 of this disclosure.

[0425] Note that the effects described in this manual are merely examples and not limitations, and other effects may be provided.

[0426] <12. This technology>

[0427] This technology can also be configured as follows. (1)

[0429] An imaging device, comprising:

[0430] Imaging elements, including light-blocking pixels and photodiode-segmented pixels; and

[0431] The defocus calculation unit calculates the defocus amount based on the exposure amount using at least one of the output signal of the light-blocking pixel and the output signal of the photodiode segmentation pixel. (2)

[0433] According to the imaging device described in (1) above,

[0434] The light-shielding pixel has a pupil segmentation function by including a light-shielding portion and a light-receiving element. The light-shielding portion blocks one of a pair of light beams that have been deviated in opposite directions in a predetermined direction in the exit pupil, and the light-receiving element receives the other light beam. (3)

[0436] According to the imaging device described in (2) above,

[0437] The photodiode segmentation pixel has a pupil segmentation function by including a segmentation pixel that receives each of a pair of light beams that have passed through the pair of partial regions. (4)

[0439] According to any one of the imaging devices described in (1) to (3) above,

[0440] Each pixel in the imaging element is one of the light-blocking pixel and the photodiode segmentation pixel. (5)

[0442] According to any one of the imaging devices described above (1) to (4),

[0443] Specifically, when the exposure is equal to or greater than the threshold, the defocus calculation unit uses the output signal of the light-blocking pixel to calculate the defocus amount. (6)

[0445] According to any one of the imaging devices described above (1) to (5),

[0446] When the exposure amount is less than a threshold, the defocus calculation unit calculates the defocus amount by using at least one of the output signal of the light-blocking pixel and the output signal of the photodiode segmentation pixel, depending on whether it is an on-axis region or an off-axis region. The on-axis region is the region including the central portion of the imaging element, and the off-axis region is the region other than the on-axis region of the imaging element. (7)

[0448] According to the imaging device described above (6),

[0449] The defocus calculation unit uses the output signal of the photodiode segmenting the pixel in the on-axis region to calculate the defocus amount. (8)

[0451] According to the imaging apparatus described above (6) and (7),

[0452] The defocus calculation unit uses the output signal of the light-blocking pixel in the off-axis region to calculate the defocus amount. (9)

[0454] According to the imaging apparatus described above (6) and (7),

[0455] The defocus calculation unit uses the output signal of the photodiode segmenting the pixel in the off-axis region to calculate the defocus amount. (10)

[0457] According to any one of the imaging devices described above (1) to (9),

[0458] The defocus calculation unit uses the output signal with higher reliability from the output signal of the light-blocking pixel and the output signal of the photodiode segmentation pixel to calculate the defocus amount. (11)

[0460] According to any one of the imaging devices described above (1) to (10),

[0461] Wherein, when the exposure amount is less than a threshold, the defocusing amount calculation unit performs automatic focus control in the off-axis region based on a contrast method. The on-axis region is the region including the central part of the imaging element, and the off-axis region is the region other than the on-axis region of the imaging element. (12)

[0463] According to the imaging device described in (2) above,

[0464] The imaging element includes various types of light-blocking pixels corresponding to the pupil distance of the exit pupil. (13)

[0466] According to the imaging device described above (12),

[0467] The light-shielding areas of the light-shielding portions of the various types of light-shielding pixels are different. (14)

[0469] According to the imaging apparatus described above (12) and (13),

[0470] The defocus calculation unit uses the output signal of the light-blocking pixel selected according to the pupil distance of the exit pupil to calculate the defocus amount. (15)

[0472] The imaging apparatus according to any one of (12) to (14) above further includes:

[0473] The camera control unit obtains the pupil distance from the lens barrel control unit included in the lens barrel. (16)

[0475] According to any one of the imaging devices described above (12) to (15),

[0476] In this context, the light-blocking pixels arranged in the same row on the imaging element correspond to the same pupil distance. (17)

[0478] The imaging apparatus according to any one of (1) to (16) above further includes:

[0479] The camera control unit executes drive commands for the focusing lenses included in the imaging optical system based on the defocus amount. (18)

[0481] The imaging apparatus according to any one of (1) to (17) above further includes:

[0482] The user interface control unit performs display control based on the amount of defocus. (19)

[0484] According to any one of the imaging devices described above (1) to (18),

[0485] In the case of calculating the defocus amount using the output signal of the light-blocking pixel, the imaging element adds the output signals of the photodiode-divided pixels and outputs them. (20)

[0487] According to any one of the imaging devices described above (1) to (19),

[0488] When calculating the defocus amount using the output signal of the photodiode segmented pixel, the imaging element outputs each of the output signals of the photodiode segmented pixel. (twenty one)

[0490] The imaging apparatus according to any one of (1) to (20) above further includes:

[0491] The signal processing unit performs signal processing on the image signal output from the imaging element.

[0492] When the exposure amount is less than a predetermined amount, the signal processing unit adds the output signals of the photodiode segmented pixels in the column direction. (twenty two)

[0494] A method for calculating defocusing amount includes:

[0495] Based on the exposure, select the phase difference signal of at least one of the output signals of the light-blocking pixel and the output signals of the photodiode segmented pixel, and calculate the defocus amount.

[0496] List of reference numerals

[0497] 1 Imaging device

[0498] 3. Lens tube

[0499] 7 Imaging elements

[0500] 7a Image plane phase difference pixels

[0501] 8 Camera signal processing unit

[0502] 8a Defocus Calculation Unit

[0503] 10 display units

[0504] 14 Camera control unit

[0505] 14a UI control unit

[0506] 15 storage units

[0507] 16 Optical System

[0508] 18 Lens tube control unit

[0509] 21 pixels

[0510] 21A Light-shielding Pixels

[0511] 21B PD segmented pixels

[0512] 30 PD

[0513] 31. Light-blocking section

[0514] 40L Left PD

[0515] 40R Right PD

[0516] EP Exit Pupil

[0517] EPL Left pupil area

[0518] EPR Right Pupil Area

[0519] RS0, RS1, RS2, RS3, RS4, RS5, RS6, RS7 Light-blocking pixels

[0520] LS0, LS1, LS2, LS3, LS4, LS5, LS6, LS7 Light-blocking pixels

[0521] SG, SG0, SG1, SG2, SG3, SG4, SG5, SG6, SG7 Phase difference signals

[0522] SGr, SGb phase difference signals

[0523] S0, S1, S2, S3, S4, S5, S6, S7 Pupil distance

[0524] Th threshold

Claims

1. An imaging device, comprising: Light-blocking pixels and photodiodes that receive incident light via color filters divide the pixels; An imaging element, wherein the imaging element includes a region comprising a central portion and an on-axis region and a region outside the on-axis region, which is configured as an off-axis region; A signal processing unit that performs signal processing on the image signal output from the imaging element; as well as The defocus calculation unit calculates the defocus amount using at least one of the output signal of the light-blocking pixel selected based on the exposure amount and the output signal of the photodiode segmentation pixel. Specifically, multiple light-shielding pixels and photodiode segmentation pixels are respectively arranged in the on-axis region and the off-axis region. When the exposure is less than a threshold, the signal processing unit adds the output signals of the photodiode segmented pixels with color filters of the same color arranged in the arrangement direction of the photodiode segmented pixels. When the exposure is less than the threshold, the defocus calculation unit uses the summed output signals to calculate the defocus amount; when the exposure is greater than or equal to the threshold, for at least the on-axis region of the on-axis region and the off-axis region, the output signal of the light-blocking pixel is used to calculate the defocus amount.

2. The imaging device according to claim 1, in, The light-shielding pixel has a pupil-splitting function by including a light-shielding portion and a light-receiving element, wherein the light-shielding portion blocks one of a pair of light beams that have been deviated in opposite directions in a predetermined direction in the exit pupil, and the light-receiving element receives the other light beam.

3. The imaging device according to claim 2, in, The photodiode segmentation pixel has a pupil segmentation function by including a segmentation pixel that receives each of a pair of light beams that have passed through the pair of partial regions.

4. The imaging device according to claim 1, in, Each pixel in the imaging element is one of the light-blocking pixel and the photodiode segmentation pixel.

5. The imaging device according to claim 1, in, When the exposure is equal to or greater than the threshold, the defocus calculation unit also uses the output signal of the light-blocking pixel to calculate the defocus amount of the off-axis region.

6. The imaging device according to claim 1, in, When the exposure is less than a threshold, the defocus calculation unit calculates the defocus amount by using at least one of the output signal of the light-blocking pixel and the output signal of the photodiode segmentation pixel, depending on whether it is an on-axis region or an off-axis region.

7. The imaging apparatus according to claim 6, in, When the exposure is less than a threshold, the defocus calculation unit uses the output signal of the photodiode segmenting the pixel in the on-axis region to calculate the defocus amount.

8. The imaging apparatus according to claim 6, in, When the exposure is less than a threshold, the defocus calculation unit uses the output signal of the light-blocking pixel in the off-axis region to calculate the defocus amount.

9. The imaging apparatus according to claim 1, in, The defocus calculation unit uses the output signal with higher reliability from the output signal of the light-blocking pixel and the output signal of the photodiode segmentation pixel to calculate the defocus amount.

10. The imaging apparatus according to claim 2, in, The imaging element includes various types of light-blocking pixels corresponding to the pupil distance of the exit pupil.

11. The imaging apparatus according to claim 10, in, The light-shielding areas of the various types of light-shielding pixels are different.

12. The imaging apparatus according to claim 10, in, The defocus calculation unit uses the output signal of the light-blocking pixel selected according to the pupil distance of the exit pupil to calculate the defocus amount.

13. The imaging apparatus according to claim 10, further comprising: The camera control unit obtains the pupil distance from the lens barrel control unit included in the lens barrel.

14. The imaging apparatus according to claim 10, in, The light-blocking pixels arranged in the same row on the imaging element correspond to the same pupil distance.

15. The imaging apparatus according to claim 1, further comprising: The camera control unit executes drive commands for the focusing lenses included in the imaging optical system based on the defocus amount.

16. The imaging apparatus according to claim 1, further comprising: The user interface control unit performs display control based on the amount of defocus.

17. The imaging apparatus according to claim 1, in, When calculating the defocus amount using the output signal of the light-blocking pixel, the imaging element adds the output signals of the photodiode-divided pixels and outputs them.

18. The imaging apparatus according to claim 1, in, When calculating the defocus amount using the output signal of the photodiode segmented pixel, the imaging element outputs each of the output signals of the photodiode segmented pixel.

19. A method for calculating defocusing amount, comprising: Based on the exposure, at least one of the phase difference signals is selected from the output signals of multiple light-blocking pixels in each region of the imaging element, including the on-axis region containing the central portion and the off-axis region outside the on-axis region, and the output signals of multiple photodiode segmented pixels in each region of the on-axis region and the off-axis region, and the defocus amount is calculated. When the exposure is less than a threshold, the output signals of the photodiode segmented pixels with color filters of the same color are added together in the arrangement direction of the photodiode segmented pixels to calculate the defocus amount. as well as When the exposure is greater than or equal to the threshold, the defocus amount is calculated using the output signal of the light-blocking pixel for at least the on-axis region and the off-axis region.

Images