Imaging device and imaging method

Abstract

This imaging device is provided with a control unit and an imaging element provided with photodiode split pixels. The control unit carries out a first reading, in which the sums of first pixels and second pixels that configure photodiode split pixels are read out as pixel values configuring an image and a second reading, in which the values of the first pixels and the values of the second pixels used in phase difference detection can be acquired from the photodiode split pixels that are not read out in the first reading. In this case, in 1 vertical period, the first reading is performed after the second reading has been performed.

Description

[Technical field]

[0001] The present technology relates to an imaging device including an imaging element having a pixel group that outputs a phase difference signal, and an imaging method. [Background technology]

[0002] 2. Description of the Related Art Some imaging devices have a function of acquiring focus information about a subject in order to perform autofocus control. Among such imaging devices, there are known ones that have pixels for detecting the focus, for example. Patent document 1 discloses a technology for performing focus detection by reading out multiple pupil-divided image signals at an image sensor, and for changing the readout row of the focus detection pixel in accordance with the accumulation time or sensitivity setting of the image sensor by controlling the readout row that acquires the multiple image signals. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2018-81224 A Summary of the Invention [Problem to be solved by the invention]

[0004] By the way, a pixel using a PD (Photodiode) division method can read out a plurality of pupil-divided image signals (pixel values), but for this reading, it is known that while the sum of two divided pixels of a PD division pixel is being read out as a normal pixel readout for generating an image signal, for some PD division pixels, one of the divided pixels is also read out. By reading out the sum and one of the pixels, the other pixel value can also be calculated, and a phase difference detection is possible using the pixel values ​​of one pixel and the other pixel. If such readout is performed within one vertical period (also written as "1V") and AF control is performed using phase difference detection, a response delay may occur that prevents the AF operation from being reflected in the next vertical period. Therefore, in this technology, we devise a readout operation for an image sensor that has PD split pixels so as not to reduce the responsiveness of the AF operation. [Means for solving the problem]

[0005] The imaging device according to the present technology includes an imaging element having photodiode split pixels, a first readout for reading out the sum of values ​​of a first pixel and a second pixel constituting the photodiode split pixel as a pixel value constituting an image, and a second readout for performing a readout capable of obtaining the value of the first pixel and the value of the second pixel used for phase difference detection from photodiode split pixels not targeted for readout in the first readout, and a control unit that ensures that the first readout is performed after the second readout in one vertical period. A photodiode split pixel (PD split pixel) becomes a pixel used for image generation by reading out the sum of the first pixel and the second pixel. For phase difference detection, a readout is performed to obtain the values ​​of the first pixel and the second pixel. In this case, a second readout for phase difference detection is performed prior to the first readout to read out the sum. The vertical period is a period defined by a vertical synchronization signal, and is one frame period of an image.

[0006] In the imaging device of the present technology described above, when it is determined that the environment is a bright environment by light / dark determination, it is considered that the control unit performs the second readout and the first readout as the readout for the one vertical period. By automatic exposure control including shutter speed control, the frame rate is increased when capturing images in a bright environment and the frame rate is decreased when capturing images in a dark environment. In the state where the frame rate is high, the second readout is performed prior to the first readout.

[0007] In the imaging device of the present technology described above, when the control unit determines that the environment is dark based on the light / dark determination, it is possible to perform a third readout for the one vertical period, by reading out the added value of the first pixel and the second pixel for the pixel to be generated as well as reading out the value of one of the first pixel and the second pixel, thereby obtaining the pixel values ​​that constitute the image and the value of the first pixel and the value of the second pixel that are used for phase difference detection. When capturing images in a dark environment where the frame rate is low, a read operation is performed in which both the pixel value and the value for phase difference detection are read as a third read, rather than a second read being performed prior to a first read.

[0008] In the imaging device of the present technology described above, in the third readout, it is considered that the value of one of the first pixel and the second pixel, and the added value are read out by adding up the values ​​of multiple photodiode divided pixels in the vertical direction. In a dark environment, the amount of exposure decreases, resulting in a lower pixel value. In response to this, pixel values ​​are read out as vertically added values.

[0009] In the imaging device of the present technology described above, in the first readout, the added values ​​of the first pixel and the second pixel of a PD divided pixel are vertically added and read out on (N-1) or less horizontal lines in a period of N horizontal lines (N is a natural number equal to or greater than 3), and in the second readout, a readout is performed that can obtain the value of the first pixel and the value of the second pixel from the PD divided pixel of a horizontal line that is not read out in the first readout. In the first readout, all horizontal lines are not read out, but (N-1) of the horizontal lines are read out in a period of N horizontal lines. In other words, thinning-out readout is performed.

[0010] In the imaging device of the present technology described above, it is considered that the value of N lines as the period of the horizontal lines is changed according to the brightness. In the first and second readout, image generation and phase difference detection are performed from the values ​​of pixels that are thinned out and read out, but the thinning rate can be changed according to the brightness conditions.

[0011] In the imaging device of the present technology described above, the second readout may involve reading out an added value of the first pixel and the second pixel, and reading out a value of one of the first pixel and the second pixel. By reading out the sum of the first and second pixels and the value of one of the pixels, the value of the other pixel can be calculated, and the value of the first pixel and the value of the second pixel can be determined.

[0012] In the imaging device of the present technology described above, it is considered that the added value obtained by the first readout is used to generate a live view image. The pixel value read out as the sum of the first pixel and the second pixel is used for a live view image (also called a through image, an image for an imaging monitor), which is an image with a relatively low resolution.

[0013] In the imaging device of the present technology described above, the imaging element has a light-shielding pixel having a pupil division function by being equipped with a light-shielding section that blocks one of a pair of light beams that pass through a pair of partial regions that are biased in opposite directions in a predetermined direction at the exit pupil, and a light-receiving element that receives the other light beam, and the control unit controls to perform a fourth readout that reads out the light-shielding pixel, and it is considered that a phase difference detection process is performed using the value of the light-shielding pixel obtained in the fourth readout. A light-shielded pixel is, for example, either a pixel into which only light that has passed through a left region defined by the light-shielding portion as the left half of the exit pupil is incident, or a pixel into which only light that has passed through a right region defined by the light-shielding portion as the right half of the exit pupil is incident.

[0014] In the imaging device of the present technology described above, it is conceivable that, as the readout during one vertical period, the fourth readout is performed prior to the second readout. The readout of the light-shielded pixels as the fourth readout and the second readout are performed before and after each other in a time-division manner.

[0015] In the imaging device of the present technology described above, it is conceivable that focus control is performed using the more reliable result of the phase difference detection processing based on the second readout or the phase difference detection processing based on the fourth readout. Focus control is performed by selectively using either the defocus amount calculated as a result of phase difference detection by light-shielded pixels or the defocus amount calculated as a result of phase difference detection by photodiode divided pixels.

[0016] The imaging method of the present technology is an imaging method for an imaging device having an imaging element with photodiode split pixels, in which a first readout is performed in which the added value of a first pixel and a second pixel that constitute the photodiode split pixel is read out as a pixel value that constitutes an image, and a second readout is performed in which the value of the first pixel and the value of the second pixel used for phase difference detection are obtained from photodiode split pixels that are not the subject of readout in the first readout, and the first readout is performed after the second readout in one vertical period. This realizes an imaging device in which the first readout and the second readout are performed in a time-division manner, with the second readout being performed first. [Brief description of the drawings]

[0017] [Figure 1] 1 is a perspective view of an imaging device according to an embodiment of the present technology; [Diagram 2] FIG. 2 is a rear view of the imaging device according to the embodiment. [Diagram 3] 1 is a block diagram of a configuration of an imaging apparatus according to an embodiment. [Figure 4] 2 is an explanatory diagram of the internal configuration of a lens barrel of the imaging device according to the embodiment. FIG. [Diagram 5] 1 is an explanatory diagram of an imaging element having PD divided pixels according to an embodiment. [Figure 6] 1 is an explanatory diagram of a configuration example of PD divided pixels arranged in an on-axis region. FIG. [Figure 7] FIG. 11 is an explanatory diagram of a read operation in a comparative example. [Figure 8]FIG. 4 is an explanatory diagram of a read operation in a bright environment according to the first embodiment. [Figure 9] FIG. 2 is a schematic explanatory diagram of a readout state according to an embodiment. [Figure 10] FIG. 4 is an explanatory diagram of a readout operation in a dark environment according to the first embodiment. [Figure 11] 4 is a flowchart of control of readout and AF operations in the first embodiment. [Figure 12] FIG. 2 is a block diagram of a configuration related to calculation of an exposure amount according to an embodiment. [Figure 13] FIG. 2 is an explanatory diagram of FD addition and SF addition according to an embodiment. [Figure 14] 1 is an explanatory diagram of an image sensor having light-shielded pixels and PD divided pixels according to an embodiment. [Figure 15] 11 is an explanatory diagram of a configuration example of light-shielded pixels arranged in an on-axis region; FIG. [Figure 16] FIG. 11 is an explanatory diagram of a read operation in a bright environment according to the second embodiment. [Figure 17] FIG. 11 is an explanatory diagram of a readout operation in a dark environment according to the second embodiment. [Figure 18] 13 is a flowchart of control of readout and AF operations in the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] Hereinafter, embodiments will be described in the following order with reference to the accompanying drawings. <1. Configuration of imaging device> 2. Read Operation in Comparative Example 3. Read Operation of the First Embodiment <4. Read Operation of the Second Embodiment> <5. Summary and variations>

[0019] <1. Configuration of imaging device> The external appearance of an imaging device 1 according to this embodiment is shown in FIG. 1 and FIG. In the following examples, the subject side is described as the front and the photographer side is described as the rear, but these directions are for convenience of explanation, and the implementation of the present technology is not limited to these directions.

[0020] 1 and 2, the imaging device 1 includes a camera housing 2 in which required parts are arranged inside and outside, and a lens barrel 3 that is detachable from the camera housing 2 and attached to a front surface 2a. Note that the lens barrel 3 being detachable as a so-called interchangeable lens is just one example, and the lens barrel 3 may be a lens barrel that cannot be removed from the camera housing 2.

[0021] A rear monitor 4 is disposed on the rear portion 2b of the camera housing 2. On the rear monitor 4, a live view image, a reproduced image of a recorded image, and the like are displayed. The rear monitor 4 is, for example, a display device such as a liquid crystal display (LCD) or an organic EL (Electro-Luminescence) display. The rear monitor 4 is rotatable relative to the camera housing 2. For example, the rear monitor 4 is rotatable around an upper end portion of the rear monitor 4 as a rotation axis such that a lower end portion of the rear monitor 4 moves rearward. The right end or the left end of the rear monitor 4 may be the rotation axis. Furthermore, the rear monitor 4 may be rotatable about a plurality of axes.

[0022] An EVF (Electric Viewfinder) 5 is disposed on the top surface 2c of the camera housing 2. The EVF 5 includes an EVF monitor 5a and a frame-shaped surrounding portion 5b that protrudes rearward so as to surround the upper portion and the left and right sides of the EVF monitor 5a. The EVF monitor 5a is formed using an LCD, an organic EL display, etc. Note that an optical view finder (OVF) may be provided instead of the EVF monitor 5a.

[0023] The rear surface 2b and the top surface 2c are provided with various operators 6. The operators 6 include, for example, a playback menu start button, a decision button, a cross key, a cancel button, a zoom key, a slide key, and a shutter button 6S (release button).

[0024] The various types of operators 6 include various types of buttons, dials, composite operators that can be pressed and rotated, etc. The various types of operators 6 enable, for example, menu operation, playback operation, mode selection / switching operation, focus operation, zoom operation, and parameter selection / setting of shutter speed, F-number, etc.

[0025] The internal configuration of such an imaging device 1 is shown in Fig. 3. Also, Fig. 4 shows an example of the layout of part of the configuration in Fig. 3.

[0026] Inside and outside the camera housing 2 of the imaging device 1, an imaging element 7, a camera signal processing unit 8, a recording unit 9, a display unit 10, an output unit 11, an operation unit 12, a power supply unit 13, a camera control unit 14, a memory unit 15, etc. are provided. The lens barrel 3 comprises an optical system 16, a driver section 17, a barrel control section 18, an operation section 19, a memory section 20, and the like.

[0027] The optical system 16 is composed of various lenses such as an entrance end lens, a zoom lens, a focus lens, and a collecting lens, an aperture mechanism that controls exposure by adjusting the opening amount of the lens or iris (aperture) so that sensing is performed in a state where the signal charge is not saturated and is within the dynamic range, and a shutter unit such as a focal plane shutter. It should be noted that a portion of each component of the optical system 16 may be provided in the camera housing 2.

[0028] The imaging element 7 is, for example, a CCD (Charge Coupled Device) type or a CMOS (Complementary Metal-Oxide Semiconductor) type, and performs exposure control for light from a subject incident via an optical system 16. The imaging element 7 is configured to include a processing unit that performs, for example, CDS (Correlated Double Sampling) processing, AGC (Automatic Gain Control) processing, and A / D (Analog / Digital) conversion processing for electrical signals photoelectrically converted by the pixels. Therefore, the imaging element 7 outputs a captured image signal as digital data to the camera signal processing unit 8 and the camera control unit 14.

[0029] The sensor surface of the imaging element 7 is configured to have a sensing element in which a plurality of pixels are arranged two-dimensionally. 5, the imaging element 7 is configured with PD split pixels 21 arranged in a matrix in the row and column directions. Each PD split pixel 21 is configured with two split pixels.

[0030] The configuration of the PD divided pixel 21 is shown typically in FIG. The PD split pixel 21 includes two split pixels, namely, a left PD 40L which is the split pixel on the left side and a right PD 40R which is the split pixel on the right side, a pixel boundary metal 41 arranged in front of them, an inner lens 32, a color filter 33, and an on-chip microlens 34. The color filter 33 is one of a color filter 33R having a spectral sensitivity of red (R), a color filter 33G having a spectral sensitivity of green (G), and a color filter 33B having a spectral sensitivity of blue (B). Note that there is also a configuration example in which the inner lens 32, etc. is not provided.

[0031] As shown in the figure, the left PD 40L receives light that has passed through a right pupil region EPR of the exit pupil EP. The right PD 40R receives light that has passed through a left pupil region EPL. This achieves a pupil division function.

[0032] Such PD divided pixels 21 are arranged as R pixels, G pixels, and B pixels depending on the difference in the color filters 33, as shown in FIG. For example, in the case of a G pixel as one PD divided pixel 21, a signal obtained as an added value of the left PD 40L and the right PD 40R becomes a signal of one G pixel. Furthermore, phase difference detection can be performed using the values ​​of the left PD 40L and the right PD 40R.

[0033] Returning to Fig. 3, the camera signal processing unit 8 is composed of, for example, a microprocessor specialized for digital signal processing, such as a DSP (Digital Signal Processor), a microcomputer, and the like.

[0034] The camera signal processing unit 8 performs various types of signal processing on the digital signal (captured image signal) sent from the imaging element 7. Specifically, correction between the R, G, and B color channels, white balance correction, aberration correction, shading correction, and other processes are performed. In addition, the camera signal processing unit 8 performs various processes such as YC generation processing to generate (separate) a luminance (Y) signal and a color (C) signal from the R, G, and B image data, processing to adjust the luminance and color, knee correction, and gamma correction. Furthermore, the camera signal processing unit 8 performs conversion to a final output format by performing resolution conversion processing and codec processing for encoding for recording or communication. The image data converted to the final output format is stored in the memory unit 15. The image data is output to the display unit 10, whereby an image is displayed on the rear monitor 4 or the EVF monitor 5a. Furthermore, the image data is output from an external output terminal, whereby the image is displayed on a device such as a monitor provided outside the imaging device 1.

[0035] The camera signal processing unit 8 includes a phase difference detection unit 8a. The phase difference detection unit 8a performs phase difference detection from output signals of the left PD 40L and the right PD 40R of the PD division pixel 21. The phase difference detection unit 8a then calculates a defocus amount based on the detected phase difference information. The calculated defocus amount may be used for an autofocus (AF) function by being used to drive a focus lens provided in the optical system 16 via the lens barrel control unit 18. The defocus amount may also be used to present information regarding the focus state of a subject to a user.

[0036] The recording unit 9 is made up of, for example, a non-volatile memory, and stores image files (content files) such as still image data and video data, attribute information of the image files, thumbnail images, and the like. Image files are stored in formats such as Joint Photographic Experts Group (JPEG), Tagged Image File Format (TIFF), and Graphics Interchange Format (GIF). There are various possible actual forms for the recording unit 9. For example, the recording unit 9 may be configured as a flash memory built into the imaging device 1, or may be configured as a memory card (e.g., a portable flash memory) that can be attached to and detached from the imaging device 1 and an access unit that accesses the memory card for storage and reading. Also, the recording unit 9 may be realized as a hard disk drive (HDD) built into the imaging device 1.

[0037] The display unit 10 executes a process for providing various displays to the photographer. The display unit 10 is, for example, the rear monitor 4 or the EVF monitor 5a. The display unit 10 executes a process for displaying image data that has been converted to an appropriate resolution and input from the camera signal processor 8. This causes a so-called through image, which is an image captured during release standby, to be displayed. Furthermore, the display unit 10 realizes the display of various operation menus, icons, messages, etc. as a GUI (Graphical User Interface) on the screen based on instructions from the camera control unit 14. The display unit 10 is also capable of displaying a reproduced image of image data read out from a recording medium in the recording unit 9.

[0038] In this example, both the EVF monitor 5a and the rear monitor 4 are provided, but the implementation of the present technology is not limited to this configuration, and only one of the EVF monitor 5a and the rear monitor 4 may be provided, or either one or both of the EVF monitor 5a and the rear monitor 4 may be configured to be detachable.

[0039] The output unit 11 performs data communication and network communication with external devices in a wired or wireless manner, for example, transmitting captured image data (still image files and video files) to an external display device, recording device, playback device, etc. The output unit 11 may also function as a network communication unit. For example, the output unit 11 may communicate with various networks such as the Internet, a home network, and a LAN (Local Area Network) and transmit and receive various data between the output unit 11 and a server or terminal on the network.

[0040] The operation unit 12 provided on the camera housing 2 includes not only the various operators 6 described above, but also a rear monitor 4 that employs a touch panel system, and outputs operation information to the camera control unit 14 in response to various operations such as tapping and swiping by the photographer. The operation unit 12 may function as a receiving unit for an external operation device such as a remote controller separate from the imaging device 1.

[0041] The power supply unit 13 generates a power supply voltage (Vcc) required for each unit from, for example, an internally charged battery, and supplies it as an operating voltage. When the lens barrel 3 is attached to the imaging device 1, the power supply voltage Vcc from the power supply unit 13 is also supplied to the circuits within the lens barrel 3. The power supply unit 13 may be provided with a circuit for charging the battery and a circuit for generating a power supply voltage Vcc, using a DC voltage converted and input by an AC adapter connected to a commercial AC power supply as a power source.

[0042] The camera control unit 14 is configured by a microcomputer (arithmetic processing device) equipped with a CPU (Central Processing Unit), and performs overall control of the imaging device 1. For example, it controls the shutter speed in response to the operation of the photographer, issues instructions regarding various signal processes in the camera signal processing unit 8, performs imaging operations and recording operations in response to the user's operations, and performs playback operations of recorded image files. The camera control unit 14 switches between various shooting modes, etc. Examples of the various shooting modes include a still image shooting mode, a moving image shooting mode, and a continuous shooting mode in which still images are captured continuously.

[0043] The camera control unit 14 includes a user interface control unit (UI control unit) 14a for enabling the user to operate these functions. The UI control unit 14a performs a process for detecting operations on the operators 6 provided in the imaging device 1, a display process for the rear monitor 4, an operation detection process, and the like.

[0044] In addition, the camera control unit 14 issues instructions to a lens barrel control unit 18 in order to control various lenses included in the optical system 16 . For example, it performs processing to specify an aperture value to ensure the amount of light required for AF control, and issues instructions to operate the aperture mechanism according to the aperture value.

[0045] The camera control unit 14 is capable of acquiring information on various lenses included in the optical system 16 via the lens barrel control unit 18. The lens information includes, for example, the model number of the lens, the position and F-number of the zoom lens, or the exit pupil position. The camera control unit 14 is also capable of acquiring the aperture value of the aperture mechanism included in the optical system 16.

[0046] The memory unit 15 stores information and the like used in the processing executed by the camera control unit 14. The illustrated memory unit 15 collectively represents, for example, a Read Only Memory (ROM), a Random Access Memory (RAM), a flash memory, and the like. The memory unit 15 may be a memory area built into a microcomputer chip serving as the camera control unit 14, or may be configured by a separate memory chip.

[0047] The ROM, flash memory, etc. of the memory unit 15 store programs and the like used by the camera control unit 14. The ROM, flash memory, etc. store an OS (Operating System) for the CPU to control each unit, content files such as image files, and application programs and firmware for various operations. The camera control unit 14 executes the program to control the entire imaging device 1 and lens barrel 3.

[0048] The RAM of the memory unit 15 is used as a working area for the camera control unit 14 by temporarily storing data, programs, and the like used when the CPU of the camera control unit 14 executes various data processes.

[0049] The lens barrel control unit 18 of the lens barrel 3 is formed by, for example, a microcomputer, and outputs control signals to a driver unit 17 to actually drive various lenses of the optical system 16 based on instructions from the camera control unit 14. In addition, information communication between the camera control unit 14 and the lens barrel control unit 18 may be possible only when the lens barrel 3 is attached to the camera housing 2, or it may be possible via wireless communication when the lens barrel 3 is not attached to the camera housing 2.

[0050] The lens barrel control unit 18 transmits information on the exit pupil position and the pupil distance of the exit pupil to the camera control unit 14 based on the types and drive positions of the various lenses included in the optical system 16. Specifically, the lens barrel control unit 18 obtains information on the pupil distance from information stored in a ROM serving as the memory unit 20, and transmits the information to the camera control unit 14.

[0051] The driver unit 17 includes, for example, a motor driver for a zoom lens drive motor, a motor driver for a focus lens drive motor, an aperture mechanism driver for a motor that drives an aperture mechanism, and the like. Each driver supplies a drive current to the corresponding drive motor in response to an instruction from the lens barrel control unit 18 .

[0052] Operation unit 19 of lens barrel 3 indicates an operator provided on the side of lens barrel 3. Operation information by operation unit 19 is supplied to barrel control unit 18, and notified to camera control unit 14 via barrel control unit 18. In response to the operation of the operation unit 19, the barrel control unit 18 controls the operation of the optical system 16, and the camera control unit 14 performs various settings and controls the operations.

[0053] The operation unit 19 may function as a receiving unit for an external operation device such as a remote controller separate from the lens barrel 3 .

[0054] The memory unit 20 is configured with a ROM, a flash memory, etc., and stores programs, data, etc. used by the lens barrel control unit 18. The memory unit 20 stores an OS (Operating System) for the CPU to control each unit, application programs for various operations, firmware, etc. The information stored in the memory unit 20 also includes information such as the pupil distance of the exit pupil of the optical system 16 .

[0055] 2. Read Operation in Comparative Example Here, as a comparative example, a phenomenon will be described in which the response of the AF operation is delayed when pixel values ​​are read out from the image sensor 7 having the PD divided pixels 21. Note that the reading out is performed in a rolling shutter readout manner.

[0056] In the case of an image sensor 7 in which PD split pixels 21 each having a left PD 40L and a right PD 40R are arranged as shown in Fig. 5, each PD split pixel 21 can not only read out a normal pixel signal for image generation, but also obtain the left and right PD values ​​separately to obtain a phase difference signal. The defocus amount can be calculated from this phase difference signal, and AF (autofocus) control can be performed. For this reason, it is conceivable to perform two readouts for one horizontal line, such as performing (L+R) readout and L readout for one PD divided pixel 21. Note that "(L+R) readout" refers to readout of the added charges from the left PD 40L and right PD 40R. "L readout" refers to readout of charge from the left PD 40L, and "R readout" refers to readout of charge from the right PD 40R. Furthermore, the "L value" is the pixel signal value obtained by direct readout or calculation from the left PD 40L, and the "R value" is the pixel signal value obtained by direct readout or calculation from the right PD 40R. Furthermore, the (L+R) value is the added value of the pixel signal values ​​of the left PD 40L and right PD 40R.

[0057] The (L+R) value, that is, the sum of the charges read out from the left PD 40L and the right PD 40R, has the meaning of the pixel value of that PD divided pixel 21, and therefore becomes a signal used for image generation. Also, L readout is performed to obtain the signal value (L value) of the left PD 40L, and the signal value (R value) of the right PD 40R is obtained from the (L+R) value - L value. The phase difference of the pixel components divided into pupils can be obtained from this "L value" and "R value", and AF control can be performed based on this.

[0058] FIG. 7 illustrates a schematic diagram of such a read operation. 7, in the vertical period (V1, V2,...) defined by the vertical synchronization signal XVS, a straight line SL10 shows the readout timing of each horizontal line with the horizontal line number on the vertical axis. The first horizontal line number is "HL1" ​​and the last horizontal line number is "HLmax." The ● written on the straight line SL10 indicates the horizontal line that is also read out for phase difference detection.

[0059] For example, assuming that readout for phase difference detection is performed every 48 lines, the readout operation for each PD divided pixel 21 for each horizontal line is as follows. 1st horizontal line: (L+R) readout Second horizontal line: (L+R) readout 3rd horizontal line: (L+R) readout (The same procedure is repeated up to the 47th horizontal line (L+R).)

[0060] 48th horizontal line: L readout 48th horizontal line: (L+R) readout 49th horizontal line: (L+R) readout 50th horizontal line: (L+R) readout (The same procedure is repeated up to the 95th horizontal line (L+R))

[0061] 96th horizontal line: L readout 96th horizontal line: (L+R) readout 97th horizontal line: (L+R) readout 98th horizontal line: (L+R) readout

[0062] In this way, both L readout and (L+R) readout are performed on periodic horizontal lines (the 48th horizontal line and the 96th horizontal line in the above example). This is because if both L readout and (L+R) readout are performed on all horizontal lines, the readout time for one frame will become long.

[0063] However, performing readout for image generation and readout for phase difference detection in this way may cause a delay in the AF operation. After the L value and (L+R) value are read out, the R value is calculated to perform phase difference detection and defocus amount calculation, but this process is performed after the necessary L value and (L+R) value are obtained, and so occurs in the period shown as "phase difference detection" in Figure 7. In other words, this occurs after the readout of all horizontal lines for one frame (or some of them if thinning out is performed) has been completed. In other words, phase difference detection for AF operation can be performed after the period shown by the straight line SL10, which includes the readout for pixel generation, has passed. Based on the result of this phase difference detection, the camera control unit 14 performs lens drive control, and actually drives the focus lens to focus on the subject.

[0064] In such a case, as shown in the figure, the focusing state is not reflected in the next frame (at the timing of vertical period V2) but is delayed until the frame after that (at the timing of vertical period V3). This is because the start timing of phase difference detection is after one frame's worth of reading. Such a delay in AF operation is undesirable, as it can be easily felt by the user, particularly when capturing an image of a moving subject. Furthermore, in bright environments, the frame rate is increased by AE (Automatic Exposure) control, and when the 1V period becomes shorter, the delay in AF control described above tends to become more noticeable. Therefore, in this embodiment, a readout operation is performed so as not to cause such a delay in response of the AF operation.

[0065] 3. Read Operation of the First Embodiment The read operation according to the first embodiment is shown diagrammatically in FIG. 8, the horizontal direction is the time axis, and vertical periods V1, V2 defined by the vertical synchronization signal XVS are shown. For example, these vertical periods V1, V2, etc. are assumed to be when the user half-presses the shutter button 6S, and the camera control unit 14 causes the display unit 10 to display a live view image and performs AF control. One vertical period provides an image signal for one frame of the live view image.

[0066] In the figure, the vertical axis represents the horizontal line number ("HL1" ​​to "HLmax"), and solid lines RL1 and RL2 are shown. The solid lines RL1 and RL2 indicate the read timing for each horizontal line. As shown in the figure, in one vertical period, the readout (second readout) of the solid line RL2 is performed first, followed by the readout (first readout) of the solid line RL1. Although not shown in Figure 8, as shown in Figures 9A and 9B described below, the first readout and second readout target different horizontal lines, so exposure can be performed before each readout, and the timing and length of exposure for each horizontal line are not particularly specified, and can be set appropriately.

[0067] The first readout indicated by the solid line RL1 is a readout for generating a live view image, and (L+R) readout is performed from each PD divided pixel 21 as a pixel value. As an example, a description will be given of (L+R) readout being performed in a cycle of six horizontal lines in one vertical period.

[0068] 9A shows a schematic diagram of the first readout operation. Charges of L value+R value are read out from each horizontal line, and the target horizontal lines are four out of six lines. (L+R) readout is performed for each of the PD divided pixels 21 on each of the four horizontal lines. Furthermore, the (L+R) values ​​read out from the four PD divided pixels 21 in the vertical direction are added together (this type of addition is referred to as "vertical addition" to distinguish it from the addition of the L value and the R value) to generate one pixel signal.

[0069] In other words, the pixel value for generating the live view image is, for example, for every six horizontal lines, the vertical addition value of the PD divided pixels 21 of four of the six horizontal lines, and this is used as one pixel value for generating the live view image. Such reading is performed, for example, from the first horizontal line HL1 side toward the last horizontal line HLmax side in a cycle of six lines. For example, as a simple thinning readout, the (L+R) value can be read from one horizontal line out of every six horizontal lines to produce one pixel signal; however, vertical addition is advantageous in that it makes it easier to obtain a sufficient signal level even when the amount of received light is relatively small.

[0070] 9A is merely a schematic diagram, but vertical addition is performed on a plurality of pixels connected to the same vertical signal line VSL, which will be described later in FIG 13, and which are provided with color filters of the same color. This also applies to the vertical addition shown in FIG 9C. These also result in voltage addition on the vertical signal line VSL.

[0071] The second readout indicated by the solid line RL2 in FIG. 8 is a readout for phase difference detection, in which L readout and (L+R) readout are performed from the PD divided pixel 21 as pixel values. As shown diagrammatically in FIG. 9B, in a cycle of six horizontal lines in one vertical period, L readout and (L+R) readout are performed from the PD divided pixels 21 of at least one of the horizontal lines in which the first readout is not performed. That is, in the second readout, L readout and (L+R) readout are performed on the PD divided pixels 21 of one horizontal line out of every six horizontal lines. For example, after the L value is read from the left PD 40L, the image signal of the right PD 40R is read without resetting the charge. In other words, the L value and the R value are added in the FD (floating diffusion) section of the image sensor, and the (L+R) value is read out. Such reading is performed, for example, from the first horizontal line HL1 side toward the last horizontal line HLmax side in a cycle of six lines.

[0072] Such first and second readouts are performed in one vertical period, and as shown in FIG. 8, the second readout is performed first, followed by the first readout. When the second readout is completed, phase difference detection can be performed without waiting for the readout for image generation to be completed, and then the lens can be driven, so that the AF operation can be reflected at the timing of the next vertical period V2. This makes it possible to achieve good AF performance even for moving subjects, for example.

[0073] Incidentally, the readout operation as shown in Fig. 8 may be performed regardless of brightness, but it is also desirable to perform it when it is determined to be a bright environment. This is because, when the frame rate increases in a bright environment and the vertical period becomes shorter, there is a high possibility that the AF responsiveness will decrease, so by performing the second readout and the first readout as shown in Fig. 8, the AF operation responsiveness can be improved. On the other hand, when it is determined that the environment is dark, it is possible to perform the third readout indicated by the solid line RL3 as shown in FIG.

[0074] In this third readout, readout for image generation and readout for phase difference detection are performed on the PD divided pixels 21 on the same horizontal line. In other words, the readout period is not divided as in the first and second readouts described above.

[0075] 9C shows a schematic diagram of the third readout operation. As in the bright environment, thinning readout is performed at a period of six horizontal lines. To generate a live view image, a charge readout of L value + R value is performed, but as in Figure 9A, the horizontal lines targeted are four of the six lines, and (L+R) readout is performed for each of the PD split pixels 21 on each of the four horizontal lines, and the (L+R) values ​​read out from the four PD split pixels 21 in the vertical direction are vertically added together to obtain a single pixel value.

[0076] At the same time, as readout for phase difference detection, L readout is performed from these four PD split pixels 21. In this case, the L values ​​of the four PD split pixels 21 in the vertical direction are also vertically added together to obtain one L value. The R value is obtained by subtracting the vertically added L value from the vertically added (L+R) value. In this case, for example, as shown in FIG. 9B, it is possible to read out the L value and the (L+R) value from the PD split pixel 21 of one horizontal line without vertical addition, but it is possible to improve low-illumination performance by using the vertically added (L+R) value and the L value, respectively.

[0077] The third readout indicated by the solid line RL3 in FIG. 8, which is performed as shown in FIG. 9C, is as follows in the example of six horizontal line periods. 1st to 6th horizontal lines: L readout (4 / 6 line vertical sum readout) 1st to 6th horizontal lines: (L+R) readout (4 / 6 lines vertical sum readout) 7th to 12th horizontal lines: L readout (4 / 6 line vertical sum readout) 7th to 12th horizontal lines: (L+R) readout (4 / 6 lines vertical sum readout) (The same applies for every 6 horizontal lines.)

[0078] After such a third readout, phase difference detection is performed, and then lens drive can be performed. Phase difference detection starts after one frame's worth of pixel signals are read out, but in this case, the environment is dark. Since the frame rate is reduced in a dark environment and the vertical period is longer, the AF operation can be reflected at the timing of the next vertical period V2. This makes it possible to maintain good AF performance, for example. Furthermore, the number of pixels (number of horizontal lines) for which the (L+R) value readout and the L value readout are performed can be made smaller than when the second readout and the first readout are performed separately as in FIG. For example, in the example of Figure 9, the second readout and the first readout read out a total of five horizontal lines out of six, but the third readout reads out four horizontal lines out of six, resulting in a power saving effect.

[0079] 8 and 10 are executed under the control of the camera control unit 14. An example of the control process of the camera control unit 14 is shown in FIG.

[0080] 11, the camera control unit 14 monitors whether the shutter button 6S is half-pressed. If the shutter button 6S is not half-pressed, other processing (not shown) is performed. While the button is being pressed halfway, the process of FIG. 11 proceeds to step S102 and subsequent steps every vertical period.

[0081] When detecting that the shutter button 6S is half-pressed, the camera control unit 14 performs exposure calculation in step S102. This exposure calculation is performed to determine the shutter speed and the AGC gain for the read signal, that is, for AE control (automatic exposure control). FIG. 12 shows a processing block for calculating the amount of exposure as the exposure calculation.

[0082] A normal pixel output for generating an image signal and a phase difference pixel output as a phase difference signal for AF control are obtained from the image sensor 7. The normal pixel output here refers to the (L+R) value from the PD division pixel 21, and the phase difference pixel output refers to the L value and the R value.

[0083] The normal pixel output and the phase difference pixel output are each input to an output level detection circuit provided in the camera signal processing unit 8. The output level detection circuit calculates an output average value in an exposure calculation target area on the pixel based on the input normal pixel output and phase difference pixel output, and each output average value is output from the camera signal processing unit 8 and input to the camera control unit 14.

[0084] The camera control unit 14 performs exposure calculation according to the detection result output from the camera signal processing unit 8, and determines the shutter speed (or parameters capable of adjusting the exposure, such as the F-number and gain). The camera control unit 14 performs processing to set the determined shutter speed in the image sensor 7.

[0085] The exposure amount calculation performed by the camera control unit 14 may be performed based only on the normal pixel output, or may be performed based only on the phase difference pixel output. Also, it may be performed based on both the normal pixel output and the phase difference pixel output. In any case, the light and dark (illuminance) environment can be determined by the exposure amount calculation.

[0086] In step S103, the camera control unit 14 judges whether the environment is bright or dark. For example, it judges whether the EV (Exposure Value) obtained by the above exposure amount calculation is equal to or greater than a threshold thEV. For example, it judges whether the calculated value is equal to or greater than EV10, assuming that the threshold thEV=EV10.

[0087] If EV is equal to or greater than the threshold thEV, it is determined that the environment is bright and the process proceeds to step S111. In this case, the camera control unit 14 performs the readout control of Fig. 8. First, in step S111, the second readout is executed. In the imaging element 7, the readout shown in Fig. 9B is performed in a cycle of, for example, six horizontal lines. Also, in step S112, the camera control unit 14 causes the camera signal processing unit 8 to calculate the R value from the L value and (L+R) value read out every six horizontal lines, and starts the process of storing the L value and the R value.

[0088] The operations started by the control of steps S111 and S112 described above continue, and when the second readout is completed, the camera control unit 14 starts the first readout in step S113, that is, the readout operation shown in Fig. 9A. The (L+R) value obtained by vertically adding 4 / 6 lines for every six horizontal lines read out in this first readout is processed by the camera signal processing unit 8 as an image signal of a live view image.

[0089] The camera control unit 14 starts the first readout, and controls the execution of phase difference detection in step S130 without waiting for the end of the first readout. That is, after the L value and the R value are stored for each PD divided pixel 21, the camera signal processing unit 8 performs phase difference detection by the function of the phase difference detection unit 8a. Then, the defocus amount is transmitted to the camera control unit 14. In step S131, the camera control unit 14 instructs the lens barrel control unit 18 to drive the focus lens provided in the optical system 16 in accordance with the defocus amount. Then, in step S132, the lens drive is stopped. This causes the AF operation to be performed.

[0090] If EV is less than the threshold thEV in step S103, the camera control unit 14 determines that the environment is dark, and the process proceeds to step S121. In this case, the camera control unit 14 performs control to start readout as the third readout in Fig. 10. In the image sensor 7, the readout shown in Fig. 9C, that is, L readout (vertical addition readout of 4 / 6 lines) and (L+R) readout (vertical addition readout of 4 / 6 lines) for every 6 horizontal lines, is started. Also, in step S122, the camera control unit 14 causes the camera signal processing unit 8 to calculate the R value from the L value and (L+R) value read out every six horizontal lines, and starts the process of storing the L value and the R value.

[0091] The operations started by the control of steps S121 and S122 described above continue, and when the third readout is completed, the camera control unit 14 performs the processes in steps S130, S131, and S132 similar to those described above. This causes the AF operation to be performed based on the third readout.

[0092] Here, the difference between the addition as an (L+R) value and the vertical addition will be described with reference to Fig. 13. Fig. 13A and Fig. 13B show how signal charges are read out from the PD division pixel 21 to the vertical signal line VSL. The addition as an (L+R) value is the FD addition shown in FIG. 13A, and indicates that the charges of the left PD 40L and the right PD 40R constituting the PD split pixel 21 are added in the floating diffusion FD and read out.

[0093] On the other hand, the vertical addition performed on the (L+R) value or the L value is SF (source follower) addition shown in FIG. 13B, which indicates that the signals read out from the PD division pixels 21 are voltage-added on the vertical signal line VSL.

[0094] In the above explanation, an example was given in which the first, second, and third readouts are performed in a cycle of six horizontal lines, but six horizontal lines per cycle is just one example. It is also possible to change the number of horizontal lines per cycle depending on the brightness, for example. That is, when the number of horizontal lines in one period is N, the number N is variable. 9A and 9C, in the first and third readouts, vertical addition is performed on PD divided pixels 21 in the vertical direction on horizontal lines of less than (N-1) of N lines (four lines in the above example), but the number of lines to be vertically added may be variable according to the brightness. It is particularly preferable to reduce the number of pixels to be vertically added as the brightness increases and to increase the number of pixels to be vertically added as the brightness increases.

[0095] <4. Read Operation of the Second Embodiment> As the second embodiment, an example in which the imaging element 7 is of a hybrid type having the PD divided pixels 21 and the light-shielding pixels 23 will be described.

[0096] FIG. 14 shows an example of a pixel arrangement of the image sensor 7. A pixel row having light-shielded pixels 23 is formed as the first pixel row 22A. The first pixel rows 22A are arranged discretely in the vertical direction, and a plurality of second pixel rows 22B are arranged between the first pixel rows 22A. The first pixel rows 22A may be arranged regularly or irregularly, however, a regular arrangement can reduce design costs and manufacturing costs involved in manufacturing the image sensor 7.

[0097] Each PD split pixel 21 included in the second pixel row 22B is covered with a Bayer array color filter, and depending on the type of color filter, it has either red (R) spectral sensitivity, green (G) spectral sensitivity, or blue (B) spectral sensitivity.

[0098] The configuration of the light-shielded pixel 23 will be described with reference to the schematic diagram of FIG. The light-shielding pixel 23 includes a PD 30, a light-shielding section 31 arranged in front of the PD 30 (on the subject side), an inner lens 32 arranged in front of the light-shielding section 31, a color filter (cyan) 33 arranged in front of the inner lens 32, and an on-chip microlens 34 arranged in front of the color filter 33. It should be noted that the light-shielded pixel 23 does not necessarily need to be provided with the inner lens 32 or the color filter 33 .

[0099] The PD 30 is a light receiving element on which a portion of the light that has passed through the exit pupil EP is incident, but a light shielding portion 31 disposed in front of the PD 30 allows light to be received only in a portion of the light receiving region of the PD 30. That is, the light shielding portion 31 is formed so as to cover the left half area of ​​the PD 30. The light shielding portion 31 is formed with a right opening 35R.

[0100] The inner lens 32 and the on-chip microlens 34 are optical components provided for efficiently collecting, onto the PD 30, light that has passed through the exit pupil EP and is incident on one pixel. The color filter 33 is, for example, a filter having a spectral sensitivity of cyan (Cy).

[0101] 15, the PD 30 is configured to receive only light passing through a left region (left pupil region) which is the left half of the exit pupil EP. That is, light passing through a right region (right pupil region) which is the right half of the exit pupil EP is blocked by the light blocking portion 31 and does not reach the PD 30. This realizes a pupil division function.

[0102] 15, the light-shielded pixel 23 configured to receive light passing through the left pupil region receives light in a region biased to the right side on the light-receiving surface, and is therefore referred to as a light-shielded pixel 23R. That is, the light-shielded pixel 23R has a right opening 35R formed therein. 15, the light-shielded pixel 23L is configured to receive light passing through the right pupil region, and receives light in a region biased to the left side on the light-receiving surface. As shown in FIG. 14, the light-shielded portion 31 of the light-shielded pixel 23L has a left opening 35L.

[0103] As shown in FIG. 14, the distance between the light-shielded pixels 23R and 23L is, for example, the distance of two pixels, and they are arranged alternately. The signal output from the light-shielding pixel 23R and the signal output from the light-shielding pixel 23L are treated as a pair of phase difference signals by the camera signal processing unit 8 (or the camera control unit 14). That is, the phase difference detection unit 8a of the camera signal processing unit 8 can calculate the defocus amount using the phase difference between the signal output from the light-shielding pixel 23R and the signal output from the light-shielding pixel 23L. In addition, in order to enable the camera signal processing unit 8, etc. to identify a pair of phase difference signals, information regarding which light-shielding pixel 23R and which light-shielding pixel 23L are paired is stored in advance in a memory area accessible by the camera signal processing unit 8, etc.

[0104] FIG. 16 shows an example of a readout operation in a bright environment assuming such an image sensor 7, and FIG. 17 shows an example of a readout operation in a dark environment.

[0105] As shown in FIG. 16, in a bright environment, the fourth readout indicated by the solid line RL4 is performed first in the vertical period. This fourth readout is a readout for phase difference detection, and involves reading out (L+R) from the light-shielded pixels 23 as pixel values. Figure 9D As shown in FIG. 1, in one vertical period, a horizontal line having light-shielded pixels arranged according to a predetermined pupil distance is subjected to (L+R) readout of the horizontal line. The reason for performing (L+R) readout here is to enable significant signal readout from both light-shielded pixel 23R and light-shielded pixel 23L.

[0106] After the fourth readout is performed in this manner, the second readout is performed similarly as a readout for phase difference detection. Then, after the fourth readout and the second readout are performed, the first readout is started and phase difference detection is started, after which lens driving is performed. This improves the responsiveness of AF operation.

[0107] Even in a dark environment, as shown in Fig. 17, the fourth readout indicated by the solid line RL4 is performed first during the vertical period. Then, the third readout is performed. After the third readout, phase difference detection is started, and then lens driving is performed. Since the frame rate is reduced in a dark environment, this does not have much effect on AF responsiveness.

[0108] 16 and 17 are executed under the control of the camera control unit 14. An example of the control process of the camera control unit 14 is shown in FIG.

[0109] Steps S101, S102, and S103 in FIG. 18 are the same as those in FIG. 11, and therefore a duplicated description will be avoided. If the EV is equal to or greater than the threshold thEV and the environment is determined to be bright, the camera control unit 14 proceeds to step S110 and starts a fourth readout for the light-shielded pixels 23. The (L+R) value of the light-shielded pixels 23 that have been read out is stored in the camera signal processing unit 8.

[0110] Following the fourth readout, the camera control unit 14 starts the second readout, R value calculation, and storage processing of the L value and the R value in steps S111 and S112, similar to FIG. 11, and then starts the first readout in step S113. Then, the camera control unit 14 starts the first readout, and, without waiting for the first readout to finish, executes phase difference detection in step S130. In this case, both phase difference detection based on the signal of the PD divided pixel 21 and phase difference detection based on the signal of the light-shielded pixel 23 are performed.

[0111] If it is determined in step S103 that the EV is not equal to or greater than the threshold thEV and that the environment is dark, the camera control unit 14 proceeds to step S120 and starts a fourth readout of the light-shielded pixels 23. The (L+R) value of the light-shielded pixels 23 that has been read out is stored in the camera signal processing unit 8. Following the fourth readout, the camera control unit 14 starts the third readout, the R value calculation, and the storage process of the L value and the R value in steps S121 and S122, similarly to FIG. Then, the operations started by the control of steps S121 and S122 are continued, and when the third readout is completed, the camera control unit 14 proceeds to step S130 to execute phase difference detection. In this case, both phase difference detection based on the signal of the PD divided pixel 21 and phase difference detection based on the signal of the light-shielded pixel 23 are performed.

[0112] After proceeding to step S130 in a bright or dark environment as described above, the camera control unit 14 determines which of the phase difference detection results is more reliable based on the F-number in step S140. For example, it determines whether the F-number is equal to or greater than a threshold value thF. For example, the threshold value thF may be set to F11.

[0113] If the F-number is equal to or greater than the threshold value thF, the camera control unit 14 proceeds to step S141, calculates the defocus amount using the phase difference detection result by the PD divided pixels 21, and executes lens drive as AF control. Then, in step S132, the lens drive is stopped and the camera is brought into focus.

[0114] If the F-number is less than the threshold value thF, the camera control unit 14 proceeds to step S142, and calculates the defocus amount using the more reliable phase difference detection result between the phase difference detection result by the PD split pixel 21 and the phase difference detection result by the light-shielding pixel 23, and performs lens drive as AF control to achieve a focused state in step S132.

[0115] The light-shielded pixel 23 and the PD divided pixel 21 are different in the following respects. The light-shielded pixels 23 are pixels used only for phase difference detection. On the other hand, the PD divided pixels 21 can be used as normal pixels (pixels for image generation) while being used for phase difference detection. For this reason, the light-shielded pixels 23 are arranged discretely, and the number of pixels cannot be made too large. As a result, the PD divided pixel 21 is superior to the light-shielded pixel 23 in terms of low illumination performance and large F-number performance. On the other hand, the light-shielded pixel 23 is superior to the PD segment pixel 21 in that it has freedom in pupil correction design and has good off-axis performance. Considering this, it is considered that the advantages of the PD divided pixel 21 can be utilized in a low illuminance environment, and the advantages of the light-shielded pixel 23 can be utilized in a high illuminance environment.

[0116] Therefore, first, in step S140, the phase difference detection result of the PD divided pixel 21 which is advantageous in terms of large F-number performance is selected. In step S142, it is possible to determine the reliability based on the illumination environment. Specifically, one of the phase difference signals is selected depending on the amount of exposure. In a dark environment, the phase difference detection result of the PD divided pixel 21 is selected, and in a bright environment, the phase difference detection result of the light-shielded pixel 23 is selected. In the process of FIG. 18, the phase difference detection result based on the light-shielded pixels 23 and the phase difference detection result based on the PD divided pixels 21 are calculated in step S130, and then either one is selected for use in AF control. However, it is also possible to first select one and then perform phase difference detection for the selected one.

[0117] <5. Summary and variations> The above embodiment provides the following advantages. The imaging device 1 of the embodiment includes an imaging element 7 having a PD split pixel 21, and a camera control unit 14. The camera control unit 14 performs a first readout in which the sum (L+R) of the left PD 40L (first pixel) and the right PD 40R (second pixel) constituting the PD split pixel 21 is read out as a pixel value constituting an image, and a second readout in which an L value and an R value used for phase difference detection can be obtained from the PD split pixel 21 that is not the target for readout in the first readout. The camera control unit 14 performs the first readout after the second readout in one vertical period. By performing the second readout prior to the first readout within one vertical period, more time is provided for AF control based on phase difference detection, making it possible to control the focus state in the next frame, thereby improving the responsiveness of AF control. The readout that can obtain the L and R values ​​used in phase difference detection may be the readout of the (L+R) value and the L value, or the readout of the (L+R) value and the R value, or the readout of the L value and the R value.

[0118] In the first and second embodiments, an example has been described in which, when the camera control unit 14 determines that the environment is bright based on the light / dark determination, it performs the second readout and the first readout as readout for one vertical period. When the frame rate is high, the length of one vertical period becomes shorter, which can easily cause AF operation to not be able to keep up with the next frame. In such cases, it is particularly effective to perform pre-reading as the second readout and perform phase difference detection at an early stage to prevent a decrease in the responsiveness of AF operation. It should be noted that an example may be considered in which the second readout and the first readout are performed regardless of the light / dark environment, and the third readout is not performed.

[0119] In the first and second embodiments, when the camera control unit 14 determines that the environment is dark based on the light / dark determination, it reads out the (L+R) value, which is the sum of the left PD 40L and the right PD 40R, for the pixel to be generated as the readout for one vertical period, and also reads out the L value of the left PD 40L, thereby performing a third readout that can obtain the pixel values ​​that constitute the live view image and the L and R values ​​used for phase difference detection. When the frame rate is low, one vertical period is long, so the responsiveness of the AF operation is not a big problem even when the AF operation is performed as the third readout. In other words, the AF operation can often be performed in time for the next frame. Therefore, in a dark environment, the third readout is performed, which is simpler and consumes less power than performing the first and second readouts separately. In the third readout, for example, the (L+R) value may be read out and the R value of the right PD 40R may be read out.

[0120] In the first and second embodiments, an example has been given in which, in the third readout, the (L+R) value and the L value are read out as the vertical addition value of a plurality of PD divided pixels 21 in the vertical direction. In low-light environments, pixel values ​​are averaged and read out from multiple PD pixels in the vertical direction. This increases the signal level to determine the phase difference, preventing a deterioration in the S / N ratio and ensuring proper AF operation. In other words, it is possible to improve the low-light performance of AF operation using PD divided pixels. In addition, in the third readout, the total number of readout lines can be reduced, thereby reducing power consumption, by reading out the (L+R) value and the L value from the same PD divided pixel 21. This operation can also shorten the overall readout time, which may be advantageous for improving the frame rate.

[0121] In the first and second embodiments, in the first readout, the sum values ​​((L+R) values) of the PD divided pixels are vertically summed and read out for (N-1) or less horizontal lines in a period of N (e.g., 6) horizontal lines, and in the second readout, a readout (e.g., reading of the L value and (L+R) value) is performed that can obtain the L value and the R value from the PD divided pixels 21 of the horizontal lines that are not read out in the first readout. In the first readout, for example, 4 / 6 (L+R) values ​​are vertically added and read out in a period of 6 horizontal lines. This makes it possible to obtain pixel values ​​with appropriate signal levels for use in live view, etc. In this case, for example, the L value and (L+R) value are read out from one of the remaining two horizontal lines as the second readout. Since this readout is performed on the PD divided pixels 21 of the horizontal line that was not read out in the first readout, the exposure for the first readout and the second readout can be performed in common. In other words, there is no need to set separate exposure periods for the first readout and the second readout, and an efficient operation can be achieved as a readout operation in one vertical period. In the embodiment, the readout is performed periodically in units of 6 lines, but the periodic readout of N lines is an example position.

[0122] In the description of the first embodiment, it has been mentioned that the value of N lines as the period of the horizontal lines may be changed according to the brightness. This enables the generation of suitable images according to the brightness conditions and accurate phase difference detection.In addition, by changing the thinning rate according to the brightness, the number of pixels for the first and second readout can be reduced in bright conditions, and the readout time can be shortened.

[0123] In the first and second embodiments, the second readout involves reading out the added value of the left PD 40L and the right PD 40R, and reading out the value of the left PD 40L. For one horizontal line, for example, by reading out the left PD 40L and then reading out the sum of the left PD 40L and the right PD 40R, it is possible to obtain the values ​​of the left PD 40L and the right PD 40R for calculating the defocus amount. In the second readout, the added value of the left PD 40L and the right PD 40R may be read out, and the value of the right PD 40R may be read out. In the second readout, for example, the left PD 40L and the right PD 40R may be readout separately.

[0124] In the first and second embodiments, the added value obtained by the first readout is used to generate a live view image. If the first readout and the second readout are performed on different PD divided pixels 21, the pixel values ​​obtained by the first readout are not obtained as the pixel values ​​of all valid PD divided pixels 21. Therefore, the readout may be suitable for generating images with a relatively low resolution. It is particularly suitable for generating live view images. In addition, a case where the user views a live view image is assumed to be when the user half-presses the shutter button 6S and checks the subject while timing the release. In other words, this is an opportunity for an AF operation. In such a case, it is useful to perform the second readout prior to the first readout to improve the AF operation responsiveness.

[0125] In the second embodiment, the image sensor 7 has a light-shielding pixel 23 having a pupil division function by comprising a light-shielding portion that blocks one of a pair of light beams that pass through a pair of partial regions biased in opposite directions in a predetermined direction at the exit pupil, and a light-receiving element that receives the other light beam, and the camera control unit 14 controls so that a fourth readout is performed to read out the light-shielding pixel 23, and phase difference detection processing is performed using the value of the light-shielding pixel 23 obtained in the fourth readout. By using an imaging element 7 having a PD split pixel 21 and a light-shielding pixel 23, phase difference detection based on the PD split pixel 21 and phase difference detection based on the light-shielding pixel 23 can be performed, and can be selectively used depending on the situation, for example.

[0126] In the second embodiment, the fourth readout is performed before the second readout in one vertical period. The order of the second readout and the fourth readout is not necessarily limited, but by performing the fourth readout first, for example in situations where the reliability of the defocus amount based on light-shielded pixels is high, AF control can be performed based on the fourth readout, thereby improving responsiveness.

[0127] In the second embodiment, it has been described that focus control is performed using the more reliable result of the phase difference detection processing based on the second readout or the phase difference detection processing based on the fourth readout (see FIG. 18). This enables AF control that takes advantage of the respective advantages of PD split pixels and light-shielded pixels, improving AF reliability.

[0128] The program of the embodiment is a program that causes, for example, a CPU, a DSP, or a device including these to execute the processes shown in FIG. 11 and FIG. That is, the program of the embodiment is a program that causes a calculation processing device in an imaging device 1 that includes an image sensor 7 having PD divided pixels 21 to execute a process in which the first readout is performed after the second readout in one vertical period, with respect to a first readout in which the sum of the first and second pixels that constitute the PD divided pixels 21 is read out as a pixel value that constitutes an image, and a second readout in which the first and second pixel values ​​used for phase difference detection are obtained from PD divided pixels that are not the subject of readout in the first readout. The imaging device 1 described above can be realized by such a program.

[0129] A program for realizing such an imaging device 1 can be recorded in advance in a HDD serving as a recording medium built into a device such as the imaging device 1, or in a ROM or the like in a microcomputer having a CPU. Alternatively, the software may be temporarily or permanently stored (recorded) on a removable recording medium such as a flexible disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto Optical) disk, a DVD (Digital Versatile Disc), a Blu-ray Disc (registered trademark), a magnetic disk, a semiconductor memory, a memory card, etc. Such removable recording media may be provided as a so-called package software. Furthermore, such a program can be installed in a personal computer or the like from a removable recording medium, or can be downloaded from a download site via a network such as a LAN (Local Area Network) or the Internet.

[0130] Furthermore, such a program is suitable for widespread provision of the imaging device 1 of the embodiment. For example, by downloading the program to a mobile terminal device such as a smartphone or tablet equipped with a camera function, a mobile phone, a personal computer, a game device, a video device, a PDA (Personal Digital Assistant), or the like, these devices can function as the imaging device 1 of the present disclosure.

[0131] It should be noted that the effects described in this specification are merely examples and are not limiting, and other effects may also be obtained.

[0132] The present technology can also be configured as follows. (1) An imaging element having photodiode divided pixels; a control unit that performs a first readout in which a sum value of a first pixel and a second pixel that constitute a photodiode divided pixel is read out as a pixel value that constitutes an image, and a second readout in which a value of the first pixel and a value of the second pixel that are used for phase difference detection can be obtained from a photodiode divided pixel that is not a readout target in the first readout, and that causes the first readout to be performed after the second readout in one vertical period. Imaging device. (2) When the control unit determines that the environment is a bright environment based on the brightness determination, the control unit performs the second readout and the first readout as the readout for the one vertical period. The imaging device according to (1) above. (3) When the control unit determines that the environment is a dark environment by the light / dark determination, the control unit performs a third readout in which, for a pixel to be subjected to image generation, a sum value of the first pixel and the second pixel is read out and a value of one of the first pixel and the second pixel is read out as the readout for the one vertical period, thereby obtaining pixel values ​​constituting an image and values ​​of the first pixel and the second pixel used for phase difference detection. The imaging device according to (2) above. (4) In the third readout, the value of one of the first pixel and the second pixel and the sum are read out as a sum of values ​​of a plurality of photodiode divided pixels in the vertical direction. The imaging device according to (3) above. (5) In the first readout, the sum of the first pixel and the second pixel of the PD divided pixel is vertically summed and read out in (N-1) or less horizontal lines in a period of N horizontal lines, where N is a natural number of 3 or more; In the second readout, a readout is performed in which the value of the first pixel and the value of the second pixel can be obtained from the PD divided pixels of a horizontal line that is not readout in the first readout. An imaging device according to any one of (1) to (4) above. (6) The value of N for the period of the horizontal lines changes depending on the brightness. The imaging device according to (5) above. (7) In the second readout, a sum of the first pixel and the second pixel is read out, and a value of one of the first pixel and the second pixel is read out. An imaging device according to any one of (1) to (6) above. (8) The added value obtained by the first readout is used to generate a live view image. An imaging device according to any one of (1) to (7) above. (9) The imaging element includes: a light-shielding pixel having a pupil division function by including a light-shielding portion for shielding one of a pair of light beams that have passed through a pair of partial regions that are biased in opposite directions to each other in a predetermined direction in the exit pupil and a light-receiving element for receiving the other light beam, The control unit controls to perform a fourth readout for reading out the light-shielded pixels, A phase difference detection process is performed using the values ​​of the light-shielded pixels obtained in the fourth readout. An imaging device according to any one of (1) to (8) above. (10) As the readout in the one vertical period, the fourth readout is performed prior to the second readout. The imaging device according to (9) above. (11) A focus control is performed using the result of the phase difference detection process based on the second readout or the result of the phase difference detection process based on the fourth readout, whichever is more reliable. The imaging device according to (9) or (10) above. (12) As an imaging method of an imaging device including an imaging element having photodiode divided pixels, A first readout is performed in which the sum of a first pixel and a second pixel constituting a photodiode divided pixel is read out as a pixel value constituting an image, and a second readout is performed in which the value of the first pixel and the value of the second pixel used for phase difference detection are obtained from a photodiode divided pixel not targeted for reading in the first readout. In one vertical period, the first readout is performed after the second readout. Imaging method. [Explanation of symbols]

[0133] 1. Imaging device 2 Camera housing 3 Lens barrel 6S Shutter button 7. Image sensor 8 Camera signal processing section 8a Phase difference detector 14 Camera control section 21 PD split pixels 22A 1st pixel row 22B 2nd pixel row 23, 23R, 23L Light-shielding pixels 40L Left PD 40R Right PD

Claims

1. An imaging element having photodiode divided pixels; a control unit that performs a first readout in which a sum value of a first pixel and a second pixel that constitute a photodiode divided pixel is read out as a pixel value that constitutes an image, and a second readout in which a value of the first pixel and a value of the second pixel that are used for phase difference detection can be obtained from a photodiode divided pixel that is not a readout target in the first readout, and that causes the first readout to be performed after the second readout in one vertical period; When the control unit determines that the environment is a bright environment based on the brightness determination, the control unit performs the second readout and the first readout as the readout for the one vertical period. Imaging device.

2. When the control unit determines that the environment is a dark environment by the light / dark determination, the control unit performs a third readout in which, for a pixel to be subjected to image generation, a sum value of the first pixel and the second pixel is read out and a value of one of the first pixel and the second pixel is read out as the readout for the one vertical period, thereby obtaining pixel values ​​constituting an image and values ​​of the first pixel and the second pixel used for phase difference detection. The imaging device according to claim 1 .

3. In the third readout, the value of one of the first pixel and the second pixel and the sum are read out as a sum of values ​​of a plurality of photodiode divided pixels in the vertical direction. The imaging device according to claim 2 .

4. An imaging element having photodiode divided pixels; a control unit that performs a first readout in which a sum value of a first pixel and a second pixel that constitute a photodiode divided pixel is read out as a pixel value that constitutes an image, and a second readout in which a value of the first pixel and a value of the second pixel that are used for phase difference detection can be obtained from a photodiode divided pixel that is not a readout target in the first readout, and that causes the first readout to be performed after the second readout in one vertical period; In the first readout, the sum of the first pixel and the second pixel of the photodiode divided pixel is vertically summed and read out in (N-1) or less horizontal lines in a period of N horizontal lines, where N is a natural number of 3 or more; In the second readout, a readout is performed that can obtain the value of the first pixel and the value of the second pixel from the photodiode divided pixels of a horizontal line that is not readout in the first readout. Imaging device.

5. The value of N as the period of the horizontal lines is changed according to the brightness. The imaging device according to claim 4.

6. In the second readout, a sum of the first pixel and the second pixel is read out, and a value of one of the first pixel and the second pixel is read out. The imaging device according to claim 1 or 4.

7. The added value obtained by the first readout is used to generate a live view image. The imaging device according to claim 1 or 4.

8. An imaging element having photodiode divided pixels; a control unit that performs a first readout in which a sum value of a first pixel and a second pixel that constitute a photodiode divided pixel is read out as a pixel value that constitutes an image, and a second readout in which a value of the first pixel and a value of the second pixel that are used for phase difference detection can be obtained from a photodiode divided pixel that is not a readout target in the first readout, and that causes the first readout to be performed after the second readout in one vertical period; The imaging element includes: a light-shielding pixel having a pupil division function by including a light-shielding portion for shielding one of a pair of light beams that have passed through a pair of partial regions that are biased in opposite directions to each other in a predetermined direction in the exit pupil and a light-receiving element for receiving the other light beam, the control unit controls the readout of the one vertical period so that a fourth readout for reading out the light-shielded pixels is performed prior to the second readout; A phase difference detection process is performed using the values ​​of the light-shielded pixels obtained in the fourth readout. Imaging device.

9. An imaging element having photodiode divided pixels; a control unit that performs a first readout in which a sum value of a first pixel and a second pixel that constitute a photodiode divided pixel is read out as a pixel value that constitutes an image, and a second readout in which a value of the first pixel and a value of the second pixel that are used for phase difference detection can be obtained from a photodiode divided pixel that is not a readout target in the first readout, and that causes the first readout to be performed after the second readout in one vertical period; The imaging element includes: a light-shielding pixel having a pupil division function by including a light-shielding portion for shielding one of a pair of light beams that have passed through a pair of partial regions that are biased in opposite directions to each other in a predetermined direction in the exit pupil and a light-receiving element for receiving the other light beam, the control unit controls to perform a fourth readout for reading out the light-shielded pixels; a phase difference detection process is performed using the values ​​of the light-shielded pixels obtained in the fourth readout; A focus control is performed using the result of the phase difference detection process based on the second readout or the result of the phase difference detection process based on the fourth readout, whichever has higher reliability. Imaging device.

10. As an imaging method of an imaging device including an imaging element having photodiode divided pixels, A first readout is a first readout that reads out the sum of a first pixel and a second pixel that constitute a photodiode divided pixel as a pixel value that constitutes an image, and a second readout is a second readout that can obtain the value of the first pixel and the value of the second pixel used for phase difference detection from a photodiode divided pixel that is not the target of reading in the first readout. When a bright environment is determined by a light / dark determination, the first readout is performed after the second readout in one vertical period. Imaging method.

Images