Imaging apparatus and focus detection method

Abstract

To perform accurate focus detection in using an imaging optical system whose optical axis is not at the center of an image pick-up device.SOLUTION: An imaging apparatus 100 is removably attached with an imaging optical system. The imaging apparatus has: an image pick-up device 211 that has pixels each including photoelectric conversion parts 402, 403 divided to receive a flux of light that passes through divided two pupil areas, of an exit pupil of the imaging optical system; and focus detection means 218 that performs focus detection by using signals from the photoelectric conversion parts. The photoelectric conversion parts include a first photoelectric conversion part that receives a flux of light from two pupil areas divided in a first direction, and a second photoelectric conversion part that receives a flux of light from two pupil areas divided in a second direction different from the first direction. The focus detection means acquires optical axis information related to the position of an optical axis of the imaging optical system different from the center of the image pick-up device, and selects, from the first and second photoelectric conversion parts, a photoelectric conversion part used for focus detection on the basis of the optical axis information.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a focus detection technique in an image pickup apparatus. [Background technology]

[0002] Autofocus (AF) of the phase difference detection method, which obtains a focused state by moving a focus lens based on the focus detection result (defocus amount) of the imaging optical system, includes image-surface phase difference AF, which uses an image sensor for generating an image as a focus detection sensor. In image-surface phase difference AF, the defocus amount is calculated using the relative image shift amount of a pair of optical images formed by light beams passing through different regions of the exit pupil of the imaging optical system and a correction value and conversion coefficient according to the imaging optical system. The correction value and conversion coefficient are calculated using information indicating the angle of incidence of light from the imaging optical system to the image sensor and the state of vignetting of light by the imaging optical system, as disclosed in Patent Document 1, for example.

[0003] Some imaging optical systems have two optical systems arranged in parallel on the left and right to obtain two captured images having a parallax between them that can be viewed stereoscopically. Furthermore, some imaging devices, as disclosed in Patent Document 2, capture two optical images formed by two optical systems arranged in parallel with one imaging element. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2008-268403 A [Patent Document 2] JP 2011-205558 A Summary of the Invention [Problem to be solved by the invention]

[0005] When two optical images formed by two optical systems arranged in parallel are captured by one image sensor, the optical axes of the two optical systems are positioned offset from the center of the image sensor. In this case, a method similar to that of Patent Document 1, which is based on a general image capturing optical system in which the optical axis of one optical system is positioned at the center of the image sensor, cannot calculate an appropriate correction value or conversion coefficient. As a result, a highly accurate defocus amount cannot be obtained, and good AF cannot be performed.

[0006] The present invention provides an imaging device that is capable of obtaining highly accurate focus detection results when using an imaging optical system whose optical axis is not at the center of an image sensor, such as when using two optical systems arranged in parallel. [Means for solving the problem]

[0007] An imaging device according to one aspect of the present invention is capable of attaching and detaching an imaging optical system. The imaging device includes an imaging element having pixels including photoelectric conversion units divided to receive light beams passing through two divided pupil regions of an exit pupil of the imaging optical system, and a focus detection unit that performs focus detection using a signal from the photoelectric conversion units. The photoelectric conversion units include a first photoelectric conversion unit that receives light beams from two pupil regions divided in a first direction, and a second photoelectric conversion unit that receives light beams from two pupil regions divided in a second direction different from the first direction. The focus detection unit is characterized in that it obtains optical axis information related to the position of the optical axis of the imaging optical system different from the center of the imaging element, and selects a photoelectric conversion unit to be used for focus detection from the first and second photoelectric conversion units based on the optical axis information.

[0008] A focus detection method according to another aspect of the present invention includes the steps of: acquiring optical axis information relating to a position of an optical axis of an imaging optical system different from a center of an image sensor in the imaging device; and selecting a photoelectric conversion unit to be used for focus detection from the first and second photoelectric conversion units based on the optical axis information. Note that a program for causing a computer of the imaging device to execute processing according to the focus detection method also constitutes another aspect of the present invention. Effect of the Invention

[0009] According to the present invention, highly accurate focus detection results can be obtained when using an imaging optical system whose optical axis is not at the center of an image sensor, such as when using two optical systems arranged in parallel. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram showing the appearance of a camera according to a first embodiment. [Diagram 2] FIG. 2 is a diagram showing the internal configuration of the camera according to the first embodiment. [Diagram 3] FIG. 2 is a diagram showing the configuration of a lens unit in the first embodiment. [Figure 4] FIG. 2 is a diagram showing a pixel array of an image sensor in the first embodiment. [Diagram 5] 4 is a diagram showing the pupil distance between a lens and an image sensor in the first embodiment. [Figure 6] 4A to 4C are diagrams showing the lens aperture and the intensities of two optical images in the first embodiment. [Figure 7] 4 is a flowchart showing an AF process in the first embodiment. [Figure 8] 4A to 4C are diagrams showing examples of two optical images formed on an image sensor in the first embodiment. [Figure 9] 5 is a graph showing the relationship between the image shift amount and the defocus amount in the first embodiment. [Figure 10] 5 is a flowchart showing focus detection processing in the first embodiment. [Figure 11] 4 is a diagram showing the relationship between an image sensor and optical axes of two optical systems in the first embodiment. [Figure 12] 5A and 5B are diagrams showing examples of signal intensities and signal correction values ​​of focus detection pixels when two optical systems are used. [Figure 13] FIG. 11 is a diagram for explaining a second embodiment using a shift optical system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, embodiments of the present invention will be described with reference to the drawings. EXAMPLES

[0012] [Camera appearance configuration] 1(a) and (b) show the appearance of a digital camera 100 (hereinafter, simply referred to as a camera) as an imaging device of Example 1. Fig. 1(a) shows the camera 100 as seen obliquely from the front, and Fig. 1(b) shows the camera 100 as seen obliquely from behind.

[0013] The camera 100 has a shutter button 101, a power switch 102, a mode switch 103, a main electronic dial 104, a sub electronic dial 105, and a movie button 106 as operation units on its top surface, and further has a viewfinder display unit 107. The shutter button 101 is operated by a user to prepare for imaging and to instruct imaging. The power switch 102 is operated by a user to switch the power of the camera 100 between on and off. The mode switch 103 is operated by a user to switch the operation mode of the camera 100. The main electronic dial 104 is rotated by a user to change the setting values ​​such as the shutter speed and the aperture. The sub electronic dial 105 is rotated by a user to move the selection frame (cursor), advance images, etc. The movie button 106 is operated by a user to instruct starting and stopping movie imaging (recording). The viewfinder display unit 107 displays various setting values ​​such as the shutter speed and the aperture.

[0014] The camera 100 also has a display unit 108, a touch panel 109 as an operation unit, direction keys 110, a SET button 111, an AE lock button 112, a magnifying button 113, a playback button 114, a menu button 115, and a touch bar 119 on the back side. The camera 100 also has an eyepiece unit 116 and an eyepiece detection unit 118 on the back side. The display unit 108 displays images and various information. The touch panel 109 detects a touch operation by the user on the display surface (touch operation surface) of the display unit 108. The direction keys 110 are composed of operable keys (four-way keys) arranged on the top, bottom, left, and right sides, and can give instructions according to the position of the operated key. The SET button 111 is mainly operated by the user when deciding on a selected item. The AE lock button 112 is operated by the user when fixing the exposure state in an image capture standby state.

[0015] The enlargement button 113 is operated by the user in order to switch the enlargement mode of the live view (LV) display on and off in the imaging mode. When the enlargement mode is on, the displayed LV image is enlarged or reduced by operating the main electronic dial 104. The enlargement button 113 is also operated by the user in order to enlarge the playback image or increase the magnification ratio in the playback mode.

[0016] The playback button 114 is operated by the user to switch between the image capture mode and the playback mode. By operating the playback button 114 in the image capture mode, the mode is switched to the playback mode, and the latest image among the images recorded in the recording medium 228 described later can be displayed on the display unit 108. The menu button 115 is operated by the user when displaying a menu screen on which various settings can be made on the display unit 108. The user can intuitively make various settings using the menu screen displayed on the display unit 108, the direction key 110, and the SET button 111.

[0017] Eyepiece unit 116 is a portion where the user places his / her eye against eyepiece finder (peek-in finder) 117. The user can view an image displayed on an internal EVF (Electronic View Finder) 217 ​​(described below) through eyepiece unit 116. Eyepiece detection unit 118 is a sensor that detects whether the user places his / her eye against eyepiece unit 116.

[0018] The touch bar 119 is a line-shaped touch operation unit (line touch sensor) capable of receiving touch operations. The touch bar 119 is disposed at a position where the user can perform touch operations with the thumb of the right hand while holding the grip unit 120 of the camera 100 with the right hand. This allows the user to perform touch operations with the thumb of the right hand while holding the camera so that the user can operate the shutter button 101 with the index finger of the right hand while placing the eyepiece unit 116 close to the eyepiece unit 116 and looking into the eyepiece finder 117. The touch bar 119 can receive tap operations (operations of touching and releasing without moving within a predetermined period of time), left and right slide operations (operations of moving while touching), and the like. The touch bar 119 is an operation unit different from the touch panel 109, and does not have a display function. In this embodiment, the touch bar 119 functions as a multi-function bar.

[0019] The camera 100 also has the above-mentioned grip section 120, thumb rest section 121, terminal cover 122, lid 123, and communication terminal 124. The grip section 120 is a holding section formed in a shape that is easy to hold with the right hand when the user holds the camera 100. The shutter button 101 and the main electronic dial 104 are arranged at positions that can be operated with the index finger of the right hand when the camera 100 is held by gripping the grip section 120 with the little finger, ring finger, and middle finger of the right hand. In the same state, the sub electronic dial 105 and the touch bar 119 are arranged at positions that can be operated with the thumb of the right hand. The thumb rest section 121 is arranged at a position where it is easy to place the thumb of the right hand that holds the grip section 120 when none of the operation sections provided on the back of the camera 100 is operated. The thumb rest section 121 is made of a rubber member or the like for increasing the holding force (grip feeling).

[0020] Terminal cover 122 protects connectors such as a connection cable that connects camera 100 to an external device. Lid 123 protects recording medium 228 and the slot by closing the slot for storing recording medium 228, which will be described later. Communication terminal 124 is a terminal for communicating with lens unit 200, which will be described later and is detachable from camera 100.

[0021] [Internal structure of the camera] Figure 2 shows the internal configuration of the camera 100. In Figure 2, the same components as those shown in Figures 1(a) and (b) are denoted by the same reference numerals as in Figures 1(a) and (b). A lens unit 200 serving as an interchangeable lens is removably attached to the camera 100.

[0022] Lens unit 200 is a type of interchangeable lens that can be attached to and detached from camera 100. Lens unit 200 has an imaging optical system including two optical systems described below, but Fig. 2 shows the configuration of only one of the optical systems. Lens unit 200 has aperture 201, lens 202, aperture drive circuit 203, AF (autofocus) drive circuit 204, lens control unit 205, and communication terminal 206. Lens 202, which is one of the optical systems, is composed of a plurality of lens groups (not shown).

[0023] The aperture 201 is configured to be able to change the aperture diameter in order to adjust the amount of light. The aperture drive circuit 203 drives the aperture 201. The AF drive circuit 204 adjusts the focus by moving a focus lens included in the lens 202. The lens control unit 205 is capable of communicating with the camera 100 via a communication terminal 206 of the lens unit 200 and a communication terminal 124 of the camera 100, and controls the aperture drive circuit 203 and the AF drive circuit 204 based on instructions from a camera control unit 218 described later.

[0024] The camera 100 has a shutter 210, an imaging section (imaging element) 211, an A / D converter 212, a memory control section 213, an image processing section 214, a memory 215, a D / A converter 216, an EVF 217, a display section 108, and a camera control section 218.

[0025] The shutter 210 is a focal plane shutter that opens and closes based on instructions from a camera control unit 218 to control the exposure time of an image sensor 211. The image sensor 211 is composed of a photoelectric conversion element such as a CCD sensor or a CMOS sensor that converts an optical image into an electrical signal. The image sensor 211 also serves as a focus detection sensor for performing image plane phase difference AF.

[0026] The A / D converter 212 converts the imaging signal output from the imaging element 211 as an analog signal into imaging data as a digital signal. The image processing unit 214 performs image processing such as pixel interpolation, resizing, and color conversion on the imaging data from the A / D converter 212 or the memory control unit 213 to generate image data. The image processing unit 214 also performs predetermined arithmetic processing using the image data. The camera control unit 218 performs automatic exposure (AE) processing and AF processing based on the arithmetic results obtained by the image processing unit 214. Furthermore, the image processing unit 214 performs AWB (auto white balance) processing on the image data.

[0027] The imaging data from the A / D converter 212 is written to the memory 215 via the image processing unit 214 and the memory control unit 213, or written to the memory 215 via the memory control unit 213 without via the image processing unit 214. The memory 215 stores imaging data and image data to be displayed on the display unit 108 and the EVF 217. The memory 215 stores a predetermined number of still images and a predetermined length of moving image and sound. The memory 215 also serves as a memory for displaying images (video memory).

[0028] The D / A converter 216 converts the image data for display stored in the memory 215 into an analog signal and supplies it to the display unit 108 and the EVF 217. The display unit 108 and the EVF 217 perform display according to the analog signal from the D / A converter 216. This performs LV display, which is a display of a live view (LV) image. The display unit 108 and the EVF 217 are configured with a display device such as an LCD or an organic EL. A live view display is performed.

[0029] The camera control unit 218 is a computer configured with one or more processors and circuits. The camera control unit 218 controls the camera 100 and the lens unit 200. The camera control unit 218 executes each process shown in the flowchart described later according to a program recorded in the non-volatile memory 220. The camera control unit 218 also controls the memory 215, the D / A converter 216, the display unit 108, and the EVF 217 to perform image display control.

[0030] The camera 100 also has a system memory 219, a non-volatile memory 220, a system timer 221, a communication unit 222, an attitude detection unit 223, and an eyepiece detection unit 118. The system memory 219 is, for example, a RAM, and stores constants and variables for the operation of the camera control unit 218. Furthermore, programs and the like read from the non-volatile memory 220 are deployed in the system memory 219. The non-volatile memory 220 is an electrically erasable and recordable memory, and for example, an EEPROM is used. The non-volatile memory 220 stores constants and programs and the like for the operation of the camera control unit 218.

[0031] The system timer 221 measures the time used for various controls and the time of a built-in clock. The communication unit 222 transmits and receives image data, audio data, and the like to and from an external device connected wirelessly or via a wired cable. The communication unit 222 can also connect to a wireless LAN (Local Area Network) or the Internet. The communication unit 222 can also communicate with external devices via Bluetooth (registered trademark) or Bluetooth Low Energy.

[0032] The attitude detection unit 223 detects the attitude of the camera 100 with respect to the direction of gravity. The camera control unit 218 determines whether the attitude of the camera 100 when imaging for generating image data is performed is horizontal (normal) or vertical, based on the attitude detected by the attitude detection unit 223. The camera control unit 218 can also add orientation information according to the attitude detected by the attitude detection unit 223 to the image data, or rotate and record the image data according to the detected attitude. The attitude detection unit 223 is configured using an acceleration sensor, a gyro sensor, and the like. It is also possible to detect the movement (pan, tilt, etc.) of the camera 100 through the attitude detection unit 223.

[0033] The eyepiece detection unit 118 detects the approach of an object (the user's face) to the eyepiece unit 116 of the eyepiece finder 117 incorporating the EVF 217. The eyepiece detection unit 118 is configured using, for example, an infrared proximity sensor. In this case, infrared rays emitted from a light-projecting unit of the infrared proximity sensor are reflected by a nearby object and received by a light-receiving unit of the infrared proximity sensor. The distance from the eyepiece unit 116 to the object can be determined based on the amount of infrared rays received. The camera control unit 218 switches the display unit 108 and the EVF 217 between display and non-display depending on the state detected by the eyepiece detection unit 118.

[0034] The camera 100 also has an outside-finder display unit 107, an outside-finder display drive circuit 224, a power control unit 225, a power supply unit 226, a recording medium I / F 227, and an operation unit 229. The outside-finder display unit 107 is driven by the outside-finder display drive circuit 224, and displays various settings such as shutter speed and aperture. The power control unit 225 is composed of a battery detection circuit, a DC-DC converter, a switch circuit that switches between blocks to be energized, and the like, and detects whether a battery is installed, the type of battery, and the remaining battery level. The power control unit 225 also controls the DC-DC converter based on the detection results and instructions from the camera control unit 218, and supplies the required voltage to each unit including the recording medium 228 for the required period.

[0035] The power supply unit 226 is composed of primary batteries such as alkaline batteries and lithium batteries, secondary batteries such as NiCd batteries, NiMH batteries, and Li batteries, an AC adapter, etc. The recording medium I / F 227 is an interface with a recording medium 228 such as a semiconductor memory or a magnetic disk. The recording medium 228 records image data and audio data generated by capturing images. The recording medium 228 may be detachable from the camera 100, or may be built into the camera 100.

[0036] The operation unit 229 is an input unit that accepts operations by a user (user operations) and inputs instructions according to the user operations to the camera control unit 218. The operation unit 229 includes the above-mentioned shutter button 101, power switch 102, mode changeover switch 103, touch panel 109, and other operation members 230. The other operation members 230 include the main electronic dial 104, the sub electronic dial 105, the video button 106, the direction key 110, the SET button 111, the AE lock button 112, the enlargement button 113, the playback button 114, the menu button 115, and the touch bar 119.

[0037] The shutter button 101 has a first shutter switch 231 and a second shutter switch 232. The first shutter switch 231 is turned on by the user pressing the shutter button 101 halfway, and generates a first shutter switch signal SW1. When the first shutter switch signal SW1 is input, the camera control unit 218 starts image capture preparation processing such as AF processing, AE processing, and AWB processing. The second shutter switch 232 is turned on by pressing the shutter button 101 all the way, and generates a second shutter switch signal SW2. When the camera control unit 218 receives the second shutter switch signal SW2, it starts a series of image capture processing from reading a signal from the image sensor 211 to generating image data and writing it to the recording medium 228.

[0038] The mode changeover switch 103 is operated by a user to switch the operation mode of the camera control unit 218 among a still image capturing mode, a video capturing mode, and a playback mode. The touch panel 109 is a touch sensor that detects a touch operation by the user on the display surface of the display unit 108. As the touch panel 109, various types of touch sensors such as a resistive film type, a capacitive type, a surface acoustic wave type, an infrared type, an electromagnetic induction type, an image recognition type, and an optical sensor type can be used.

[0039] <Lens unit configuration> Fig. 3 shows a detailed configuration of the imaging optical system of lens unit 200 as viewed from above. In Fig. 3, lens unit 200 is attached to camera 100, and the components of camera 100 shown in the figure are the same as those denoted by the same reference numerals in Fig. 2. Lens unit 200 also has lens control unit 205 shown in Fig. 2.

[0040] The lens unit 200 is attached to the camera 100 by mechanically coupling a lens mount section 304 provided on the lens unit 200 to a camera mount section 250 provided on the camera 100. When the lens unit 200 is attached to the camera 100, the communication terminal 206 of the lens unit 200 is electrically connected to the communication terminal 124 of the camera 100, enabling communication between the lens control section 205 and the camera control section 218.

[0041] The imaging optical system is a two-eye type (stereo system) imaging optical system having a left-eye optical system 301L and a right-eye optical system 301R arranged in parallel on the left and right (top and bottom in the figure). Each optical system is a VR180 type fisheye lens with a wide viewing angle of approximately 180 degrees. However, a lens with a viewing angle of about 160 degrees narrower than 180 may also be used.

[0042] The left-eye optical system 301L and the right-eye optical system 301R are each composed of a plurality of lenses, two reflecting mirrors (prisms), and a diaphragm (not shown in FIG. 3) shown in FIG. 2. The object-side lenses of the left-eye optical system 301L and the right-eye optical system 301R are arranged so that their optical axes are spaced apart by a predetermined distance (baseline length), and the image-side lenses are arranged so that their optical axes are spaced apart by a distance shorter than the baseline length. In each optical system, two prisms are arranged between the object-side lens and the image-side lens, and light from the object-side lens is bent by reflection at the two prisms and guided to the image-side lens. The left-eye optical system 301L and the right-eye optical system 301R form an optical image for the left eye and an optical image for the right eye having parallax with each other in a left region 800L and a right region 800R on one imaging element (imaging surface 800) provided in the camera 100, respectively, as shown in FIG. 8. That is, in an imaging optical system consisting of only one optical system, its optical axis is located at the center of the imaging element, whereas in this embodiment, the positions L_C and R_C of the optical axes of the image side lenses of the left eye optical system 301L and right eye optical system 301R are shifted to the left and right, respectively, from the center of the imaging element (the intersection of the x-axis and y-axis).

[0043] The lens unit 200 is capable of focus adjustment by the AF process of the camera control unit 218 described above, and is also capable of focus adjustment by a user's focus operation, and is provided with a focus ring (not shown) for performing the focus operation. Two focus rings may be provided for each of the left eye optical system 301L and the right eye optical system 301R, or only one focus ring may be provided for performing the focus adjustment of the left eye optical system 301L and the right eye optical system 301R as a whole.

[0044] [Image sensor configuration] Fig. 4 shows a pixel array of the image sensor 211 in this embodiment. Fig. 4 shows a pixel array of a two-dimensional CMOS sensor serving as the image sensor 211 in a range of 4 columns x 4 rows of imaging pixels.

[0045] One pixel group is composed of 2 columns x 2 rows of imaging pixels, and each imaging pixel is provided with a color filter in a Bayer array. In one pixel group, an imaging pixel (red pixel) 400R having red spectral sensitivity is located at the upper left, and imaging pixels (green pixels) 400G1 and 400G2 having green spectral sensitivity are located at the upper right and lower left. Also, an imaging pixel (blue pixel) 400B having blue spectral sensitivity is located at the lower right. Furthermore, in order to perform imaging surface phase difference AF, each imaging pixel (focus detection pixel) has one microlens 401 and multiple (two arranged in 2 columns x 1 row or 1 column x 2 rows) photodiodes (first and second photoelectric conversion units) 402 and 403 as photoelectric conversion units. By arranging many pixel groups shown in FIG. 4 on the imaging surface, the imaging element can acquire imaging signals and focus detection signals from the entire imaging surface.

[0046] In each imaging pixel (focus detection pixel), an incident light beam is split into two by a microlens 401, and each light beam is received by a photodiode 402, 403. The signal read out from the photodiode 402 is an A signal, and the signal read out from the photodiode 403 is a B signal. The A and B signals read out from the photodiodes 402, 403 of a plurality of focus detection pixels (green, red, and blue pixels) in a focus detection area selected on the imaging surface are respectively combined to obtain an A image signal and a B image signal as a pair of focus detection signals. Also, an A+B signal obtained by adding the A and B signals read out from the photodiodes 402, 403 of each imaging pixel is combined and combined across all imaging pixels to generate an imaging signal.

[0047] In the red pixel 400R, the green pixel 400G1, and the blue pixel 400B, the photodiodes 402 and 403 are divided in the left-right direction (horizontal direction), and it is possible to detect the phase difference (image shift amount) in the horizontal direction. A focus detection pixel capable of detecting the image shift amount in the horizontal direction (first direction) in this way is called a side-eye pixel as a first pixel, and the photodiodes 402 and 403 provided in the side-eye pixel correspond to a first photoelectric conversion unit. On the other hand, in the green pixel 400G2, the photodiodes 402 and 403 are divided in the up-down direction (vertical direction), and it is possible to detect the image shift amount in the vertical direction. A focus detection pixel capable of detecting the image shift amount in the vertical direction (second direction) in this way is called a vertical-eye pixel as a second pixel. The photodiodes 402 and 403 provided in the vertical-eye pixel correspond to a second photoelectric conversion unit. By having a side-eye pixel and a vertical-eye pixel in this way, it is possible to realize high-precision focus detection regardless of the edge of the subject.

[0048] Although the A and B signals may be read out from the photodiodes 402 and 403, respectively, the A and B signals may be obtained by reading out the A+B signal and the A signal and taking the difference between them. The arrangement of the side-eye pixels and vertical-eye pixels may be other than the example shown in Fig. 4, and only a portion of all imaging pixels may be side-eye pixels and vertical-eye pixels as focus detection pixels.

[0049] In this embodiment, the case where the side-eye pixels and the vertical-eye pixels are received separately will be described, but one focus detection pixel may be provided with a total of four photodiodes, each divided into two in the horizontal and vertical directions. In other words, the side-eye pixels and the vertical-eye pixels may be the same focus detection pixel.

[0050] [Relationship between image shift and defocus amount] Next, the relationship between the image shift amount p calculated from a pair of focus detection signals (image A signal and image B signal) and the defocus amount d will be described with reference to Fig. 9. Here, a typical single-lens type imaging optical system having only one optical system and an optical axis located at the center of the image sensor will be described. 1300 denotes the imaging surface of the image sensor.

[0051] A microlens provided in the focus detection pixel divides the exit pupil of the imaging optical system into a first pupil region 1303 and a second pupil region 1304. The direction of pupil division in this case corresponds to the division direction of the photodiodes 402, 403 of the focus detection pixel shown in Fig. 4. For example, if the division direction of the photodiodes 402, 403 in a horizontal pixel is the vertical direction in Fig. 9, the division direction of the photodiodes 402, 403 in a vertical pixel is the direction perpendicular to the paper surface of Fig. 9.

[0052] The defocus amount d has a magnitude |d| which is the distance from the imaging position of an optical image (object image) formed by the imaging optical system to the imaging plane 1300, and a front-focus state in which the imaging position of the object image is on the object side of the imaging plane 1300 is represented by a negative sign (d<0). A back-focus state in which the imaging position is on the opposite side of the subject side of the imaging plane 1300 is represented by a positive sign (d>0). In a focused state in which the imaging position of the object image is on the imaging plane 1300, d=0. FIG. 9 shows a case in which the imaging optical system is in a front-focus state (d<0) with respect to the object plane 1302, and a case in which the imaging optical system is in a focused state with respect to the object plane 1301. The front-focus state (d<0) and the back-focus state (d>0) are collectively referred to as a defocus state (|d|>0).

[0053] In the front focus state (d<0), the light beams from the object 1302 that pass through the first pupil region 1303 and the second pupil region 1302 are once focused and then spread to widths Γ1 and Γ2 with the center positions G1 and G2 of the light beams as centers, respectively, to form a blurred object image on the imaging surface 1300. The blurred object images are received by the photodiodes 402 and 403 of each focus detection pixel, and a pair of focus detection signals having an image shift amount p as a phase difference are generated. The blur widths Γ1 and Γ2 increase approximately in proportion to an increase in the magnitude |d| of the defocus amount d. Similarly, the magnitude |p| of the image shift amount p (=the difference G1-G2 between the center positions of the light beams) also increases approximately in proportion to an increase in the magnitude |d| of the defocus amount d. In the back focus state (d>0), the image shift direction between the pair of focus detection signals is opposite to that in the front focus state, but the same is true.

[0054] A conversion coefficient K is used to convert the calculated image shift amount p into a defocus amount d. The conversion coefficient K depends on the incident angle of the light beam from the imaging optical system to the imaging element, the F-number of the imaging optical system, and the position of the optical axis, and is acquired (calculated) based on lens optical information including these pieces of information, which will be described later.

[0055] [Focus detection processing] 10 is a flowchart showing the focus detection process of the AF process in this embodiment. The camera control unit 218 as a computer executes this process according to a program. The camera control unit 218 corresponds to the focus detection unit.

[0056] In step S1401, the camera control unit 218 as a focus adjustment device reads out signals A and B from two photodiodes of each of multiple focus detection pixels in a focus detection area selected by the user or automatically selected by the camera control unit 218 using a predetermined algorithm. The signals A and B read out from these multiple focus detection pixels are then respectively combined to generate image signals A and B, which are paired focus detection signals.

[0057] Next, in step S1402, the camera control unit 218 performs pixel addition processing on each of the A image signal and the B image signal. Specifically, addition processing is performed in the column direction to reduce the amount of signal data, and further addition processing of signals from red, green, and blue pixels is performed to generate a Y signal. When the number of pixels added in this pixel addition processing is two, the pixel pitch is doubled and the Nyquist frequency is half that of when no addition is performed, and when the number of pixels added is three, the pixel pitch is tripled and the Nyquist frequency is one-third that of when no addition is performed.

[0058] Next, in step S1403, the camera control unit 218 performs shading correction processing (optical correction processing) on ​​the A image signal and the B image signal that have been subjected to the pixel addition processing in order to match their intensities. The shading correction value at this time is a value that depends on the imaging optical system, specifically, a value that depends on the incident angle of the light beam from the imaging optical system to the imaging element (is this OK?), the F-number of the imaging optical system, and the position of the optical axis. The camera control unit 218 calculates the shading correction value based on lens optical information received from the lens control unit 205, which will be described later. At this time, the camera control unit 218 also calculates a conversion coefficient based on the lens optical information (added).

[0059] Next, in step S1404, the camera control unit 218 performs bandpass filter processing on the A and B image signals to pass a specific frequency band. This processing is for improving the correlation (degree of signal agreement) between the A and B image signals to improve focus detection accuracy. Bandpass filters include differential filters (1, 4, 4, 4, 0, -4, -4, -4, -1) that cut DC components and extract edges, and additive filters (1, 2, 1) that suppress high-frequency noise components.

[0060] Next, in step S1405, the camera control unit 218 performs shift processing to relatively shift the A and B image signals after the band-pass filter processing in the pupil division direction, and calculates a correlation amount that indicates the degree of match between the A and B image signals.

[0061] Let A(k) be the kth A image signal after band-pass filter processing, B(k) be the B image signal, W be the range of number k corresponding to the focus detection area, s be the shift amount due to shift processing, and Γ be the shift range of shift amount s, then the correlation amount COR can be calculated by the following formula (1).

[0062]

number

[0063] The camera control unit 218 shifts the kth A image signal A(k) and the ksth B image signal B(ks) by shifting the signal with the shift amount s, and generates a shifted subtraction signal by matching them together. The camera control unit 218 then calculates the absolute value of the generated shifted subtraction signal, and calculates the sum of the number k within the range W corresponding to the focus detection area to calculate the correlation amount COR(s). At this time, the correlation amount calculated for each row may be added across multiple rows for each shift amount.

[0064] Next, in step S1406, the camera control unit 218 calculates a real-valued shift amount at which the correlation amount becomes a minimum value from the calculated correlation amount by sub-pixel calculation, and sets the shift amount as the image shift amount p. Then, the image shift amount p is multiplied by a conversion coefficient K to calculate a defocus amount (Def)d.

[0065] When calculating the defocus amount in this way, a signal correction value (shading correction value) and a conversion coefficient are used as focus detection adjustment values ​​corresponding to the imaging optical system. These focus detection adjustment values ​​depend on the incident angle and vignetting state of the light beam incident on the imaging element. In a twin-lens type imaging optical system as in this embodiment, two optical axes are positioned to the left and right of the center of the imaging element, so an accurate defocus amount cannot be obtained even if the same signal correction amount and conversion coefficient as those used in a single-lens type imaging optical system in which the optical axis is positioned at the center of the imaging element are used. In other words, for a twin-lens type imaging optical system, it is necessary to use an appropriate focus detection adjustment value corresponding to the optical axis that is shifted from the center of the imaging element.

[0066] The signal correction value in the focus detection adjustment value is not limited to the shading correction value, but may be any signal correction value for improving the focus detection accuracy.

[0067] [Signal correction value] 11 shows two optical systems (301L, 301R) having optical axes at positions shifted to the left and right from the center of the image sensor 211. The diameter of an image circle centered on each optical axis is approximately 1 / 2 the long side of the image sensor.

[0068] 12(a) shows an example of the change in light amount within the image sensor for focus detection signal 1 as the A image signal and focus detection signal 2 as the B image signal. Since the optical axes of the two optical systems are positioned to the left and right of the center of the image sensor, the change in light amount is discontinuous at the center of the image sensor. This is different from the case of using a single-lens type image sensor optical system, which shows a continuous change in light amount within the image sensor, and is characteristic of the case of using a twin-lens type image sensor optical system.

[0069] FIG. 12(b) shows an example of a signal correction value for adjusting the intensity of the focus detection signal 1 and the focus detection signal 2. In order to deal with discontinuous changes in the amount of light in the image sensor, the signal correction value also changes discontinuously at the boundary between the two image circles. The signal correction value is stored in a memory (such as the non-volatile memory 220) in the camera 100, with a signal correction value for a single-lens type imaging optical system and a signal correction value for a twin-lens type imaging optical system being separately stored. The camera control unit 218 switches the signal correction value to be used depending on whether the imaging optical system of the lens unit attached to the camera 100 is of the single-lens type or twin-lens type. The lens unit may notify the camera control unit 218 of whether the imaging optical system is of the single-lens type or twin-lens type by communication, or the camera control unit 218 may determine from individual information (lens ID) of the lens unit.

[0070] The signal correction value is calculated using lens optical information indicating the incidence angle and vignetting state of the light beam incident from the imaging optical system to the imaging element. In this case, in order to calculate the signal correction value for a two-lens type imaging optical system in which the center of the imaging element and the position of the optical axis are different, the positions of the two optical axes on the imaging element are important. In this embodiment, lens optical axis information (Lx, Ly) regarding the position of the optical axis of the optical system on the coordinate system of the imaging element is acquired, and the lens optical information expressed with the position of the optical axis of each optical system as the origin is converted to coordinates on the imaging element. The lens optical axis information regarding the position of the optical axis may be information directly indicating the position of the optical axis (Lx, Ly) or may be information that can be converted to the position of the optical axis. For example, suppose there is lens optical information that the angle of incidence at position r, 10 mm away from the optical axis, is 5 degrees, as shown in Table 1. In this case, suppose the coordinates of the optical axis of the twin-lens type optical system on the imaging element are Lx=5mm, Lx=0mm. In this case, it can be seen from the lens optical information that the angle of incidence at positions 10 mm away from the optical axis, such as x=15mm, y=0mm and x=-5mm, y=0mm, is 5 degrees.

[0071] In this way, the camera control unit 218 can convert the lens optical information expressed with the optical axis as the origin into coordinates on the image sensor by acquiring the lens optical axis information (Lx, Ly). Then, the camera control unit 218 calculates a signal correction value using the lens optical information converted into the coordinates on the image sensor.

[0072] [Table 1]

[0073] Conversion factor 5(a) and (b) show the lens aperture (hereinafter referred to as LPO aperture) at pupil distance LPO of the optical system (lens) and the lens aperture (hereinafter referred to as SPO aperture) at pupil distance SPO of the image sensor. An image sensor having the pixel array shown in FIG. 4 is arranged at x=0, z=0 in the figure. The lens pupil distance LPO is a distance determined by the exit pupil distance, which is the distance on the optical axis from the image plane of the sensor to the exit pupil at the position of the image of the aperture. The sensor pupil distance SPO is a distance determined by the size of the image plane and the shrink ratio of the microlens (401 in FIG. 4) arranged on the image plane. The shrink ratio is the amount by which the microlens is decentered toward the optical axis side from the center of the image sensor at the peripheral image height when the size of the microlens is made smaller than the size of the sensor. It is desirable for the LPO and SPO to match each other, but in an actual optical system, the LPO and SPO are generally designed to match within a range in which focus detection accuracy is maintained, and they often do not match completely. 5(a) and (b) show the case where the LPO is short relative to the SPO.

[0074] The LPO aperture DL is proportional to the F-number of the optical system. The SPO aperture DL' differs from DL by the ratio of the LPO to the SPO. DL′=DL×SPO / LPO This shows how the SPO aperture is formed when focus detection is performed at the central image height (x=0) as shown in FIG.

[0075] On the other hand, FIG. 5(b) shows the case where x=10 mm, i.e., focus detection is performed at the peripheral image height. Due to the change in image height from the central image height, the SPO aperture diameter DL' is no longer symmetrical with respect to the z-axis in the x direction. In general, when the image height is high on the x-axis, the aperture diameter DL' is no longer symmetrical with respect to the x-axis, and when the image height is high on the y-axis perpendicular to the x-axis and z-axis, the SPO aperture diameter DL' is no longer symmetrical with respect to the y-axis. When the image height changes in this way, the way the SPO aperture is formed changes. In addition, the shape of the SPO aperture changes significantly depending on the arrangement of multiple lenses in the optical system, especially at the maximum F-number, and it is no longer circular.

[0076] The upper diagrams of Fig. 6(a) to (c) show the incident angle characteristics PI_N (N = A to D) of the sensor superimposed on the SPO aperture (xy plane). In Fig. 6(a) to (c), the lower diagrams show cross sections along the pupil division direction of the upper diagrams. The incident angle characteristics PI_N show the incident angle characteristics of the two photodiodes 402 and 403 divided as shown in Fig. 4, and the whiter the coordinates, the higher the light intensity, and the darker the coordinates, the weaker the light intensity.

[0077] Figures 6(a) and (b) show the incident angle characteristics PI_A and PI_B of the side-eye pixels. The SPO aperture diameter DL' shown in the upper diagram of Figure 6(a) corresponds to DL' at the central image height shown in Figure 5(a). Generally, at the central image height, the shape of the SPO aperture determined by the diaphragm shape is circular, and the optical images PI_SA and PI_SB are linearly symmetrical with respect to the x-axis. PI_A and PI_B within the SPO aperture diameter DL' indicate the intensity of the optical image actually received by the sensor, and the optical images PI_SA and PI_SB in the pupil division direction (x direction) when the intensity is taken as the vertical axis are shown in the lower diagram of Figure 6(a). By calculating these with the center of gravity of the optical image PI_SA as G_SA and the center of gravity of the optical image PI_SB as G_SB, the center of gravity difference G_SB-G_SA can be calculated. In general, the conversion coefficient K is expressed as follows: K = SPO / (G_SA-G_SB) It is calculated as:

[0078] 10, the defocus amount d in the light direction is calculated by the image shift amount p×K, and a larger conversion coefficient K corresponds to a smaller center of gravity difference G_SB-G_SA. In general, the larger the center of gravity difference (absolute value of the correlation amount), the higher the detection accuracy at bit accuracy during defocusing, making it easier to improve focus detection accuracy.

[0079] The SPO aperture diameter DL' shown in the upper diagram of Fig. 6(b) corresponds to DL' at the high image height shown in Fig. 5(b). Since the shape of the SPO aperture, which is not symmetric in the x direction, which is the pupil division direction, limits the amount of light in that direction, it is difficult to create an intensity difference between the optical images PI_SA2 and PI_SB2 in the pupil division direction, as shown in the lower diagram of Fig. 6(b), and the center difference G_SA2-G_SB2 between them also becomes narrow. In such a case, the conversion coefficient K becomes large and the focus detection accuracy decreases.

[0080] The upper diagram in Fig. 6(c) shows the incident angle characteristics PI_C and PI_D of the vertical eye pixel superimposed on the SPO aperture (xy plane). The amount of light is limited in the xy plane by the shape of the SPO aperture, but the image shift amount detected by the vertical eye pixel is the image shift amount in the y direction. PI_C and PI_D within the SPO aperture diameter DL' indicate the intensity of the optical image actually received by the sensor, and the lower diagram in Fig. 6(c) shows the optical images PI_SC and PI_SD in the pupil division direction (y direction) when the intensity is taken as the vertical axis. The difference G_SC-G_SD between the center of gravity G_SC of the optical image PI_SC and the center of gravity G_SD of the optical image PI_SD is wider than the center of gravity difference G_SA2-G_SB2 at the horizontal eye pixel shown in Fig. 6(b), making it easier to maintain focus detection accuracy.

[0081] Thus, the focus detection accuracy varies depending on the lens aperture shape (shape of the SPO aperture) and the incident angle characteristics of the sensor. In general, when focus detection is performed at a high image height in the x direction, vertical eye pixels tend to be advantageous in terms of focus detection accuracy. Conversely, when focus detection is performed at a high image height in the y direction, the lens aperture shape is rotated 90 degrees from the shape shown in FIG. 6(b), so horizontal eye pixels tend to be advantageous in terms of focus detection accuracy. Lens optical information including the lens aperture shape, F value, and exit pupil distance (LPO) is transmitted from the lens unit 200 to the camera 100. The camera control unit 218 calculates the conversion coefficient K based on the lens optical information.

[0082] [AF processing] The flowchart in FIG. 7 shows the AF process (focus detection method) that the camera control unit 218 executes according to a program in this embodiment.

[0083] Upon receiving the first shutter switch signal SW1, the camera control unit 218 starts the process from step S100.

[0084] In step S101, the camera control unit 218 acquires lens optical axis information from the lens unit 200. Here, the acquired lens optical axis information indicates the positions R_C, L_C of the optical axes of the two optical systems (301L, 301R) on the imaging element (imaging surface), respectively. When two optical images 801L, 801R are formed side by side in the x direction on the imaging surface 800 by the two optical systems as shown in FIG. 8, the positions (x, y) of the optical axes of the two optical systems are expressed as follows: L_C=(-9,0) R_C=(+9,0) The camera control unit 218 acquires the lens optical axis information through communication with the lens control unit 205. Alternatively, the lens optical axis information may be stored in advance in a memory in the camera 100 in association with the lens ID, and the lens optical axis information may be acquired by reading out the lens optical axis information corresponding to the lens ID received from the lens control unit 205 when the lens unit 200 is attached to the camera 100.

[0085] Next, in step S102, the camera control unit 218 acquires camera optical axis information related to the position of the optical axis (center of the image sensor) in the camera 100. The center position of the image sensor may deviate from the designed center position due to manufacturing errors in the camera 100. In addition, when the camera 100 has a configuration in which the image sensor is shifted from the neutral position in a direction perpendicular to the optical axis to correct image blur caused by camera shake such as hand shake, the center position of the image sensor deviates from the center position at the neutral position according to the shift amount from the neutral position of the image sensor. The camera optical axis information may be information that directly indicates the center position of the image sensor by coordinates, or may be information that can be converted into the center position of the imaging surface. In addition, when an image blur correction configuration is used during video capture, it is desirable to acquire the camera optical axis information for each captured frame.

[0086] Next, in step S103, the camera control unit 218 compares the positions of the optical axes of the two optical systems indicated by the lens optical axis information acquired in step S101 with the position of the optical axis of the camera 100 acquired in step S102, and determines whether or not there is a difference (optical axis misalignment) between them. Here, since the optical axes of the two optical systems are misaligned to the left and right with respect to the center of the image sensor, the camera control unit 218 determines that there is optical axis misalignment and proceeds to step S104. If a single-lens type lens unit is attached to the camera 100 and the optical axis of the camera 100 is located at the design center, the process proceeds to step S106.

[0087] In step S104, the camera control unit 218 determines whether the direction of the optical axis shift is the y direction. For example, when the positions of the optical axes of the two optical systems are L_C=(-9,0) and R_C=(+9,0) as shown in Fig. 8 and the optical axis of the camera 100 is located at the center as designed, there is no optical axis shift in the y direction, so the camera control unit 218 proceeds to step S105. If the direction of the optical axis shift is the y direction, the camera control unit 218 proceeds to step S106.

[0088] In step S105, the camera control unit 218 determines whether the direction of the optical axis shift is the x direction. As shown in Fig. 8, the positions of the optical axes of the two optical systems are L_C = (-9, 0) and R_C = (+9, 0), and there is an optical axis shift in the x direction, so the camera control unit 218 proceeds to step S107. If the direction of the optical axis shift is not the x direction, the camera control unit 218 proceeds to step S106.

[0089] In step S106, the camera control unit 218 judges whether the image shift amount obtained from the side-eye pixels is reliable (hereinafter, simply referred to as the side-eye pixels being reliable). As described above, the side-eye pixels are advantageous in focus detection accuracy at high image heights in the y direction. However, when the contrast of the subject is low in the y direction, a pair of focus detection signals with a high degree of agreement cannot be obtained from the side-eye pixels, and as a result, a highly reliable image shift amount cannot be obtained. In addition, the reliability can be predicted from the lens optical information described above. For example, when the lens aperture shape is smaller in the x direction with respect to the side-eye pixels as shown in FIG. 6(b), the reliability is likely to decrease. In addition, when the F value is large, the conversion coefficient becomes large and the reliability is likely to decrease. Furthermore, when the difference between the SPO and the LPO is large, the reliability is also likely to decrease. If the side-eye pixels are reliable (the reliability is higher than a predetermined value), the camera control unit 218 proceeds to step S108, and if not, proceeds to step S107.

[0090] On the other hand, in step S107, the camera control unit 218 judges whether the image shift amount obtained from the vertical eye pixels is reliable (hereinafter, simply referred to as the vertical eye pixels being reliable). As described above, the vertical eye pixels are advantageous for focus detection accuracy at high image heights in the x direction. However, when the contrast of the subject is low in the x direction, a pair of focus detection signals with a high degree of agreement cannot be obtained from the vertical eye pixels, and as a result, a reliable image shift amount cannot be obtained. Also, as described in step S106, it is possible to predict reliability from lens optical information. If the vertical eye pixels are reliable (the reliability is higher than a predetermined value), the camera control unit 218 proceeds to step S109, and if not, proceeds to step S108.

[0091] In step S109, the camera control unit 218 sets (selects) the vertical eye pixel as the focus detection pixel to be used for focus detection. That is, it sets the second photoelectric conversion unit as the photoelectric conversion unit to be used for focus detection. Then, the process proceeds to step S110.

[0092] On the other hand, in step S108, the camera control unit 218 sets the side-eye pixel as the focus detection pixel to be used for focus detection. That is, it sets the first photoelectric conversion unit as the photoelectric conversion unit to be used for focus detection. Then, the process proceeds to step S110.

[0093] If it is determined in step S106 that the side-eye pixels are unreliable and furthermore in step S107 that the vertical-eye pixels are unreliable, then in step S108 the side-eye pixels are set to be used for focus detection. This is because, as shown in FIG. 4, in one pixel group, the vertical-eye pixel is the green pixel 400G2, whereas the side-eye pixels are the red, green and blue pixels 400R, 400G1 and 400B. Considering the signal-to-noise ratio (SN) of the signal from the image sensor, even if it is determined that the reliability of either the side-eye pixels or the vertical-eye pixels is low, it is highly likely that better focus detection accuracy can be obtained by using the signals from the larger number of side-eye pixels. Therefore, even when proceeding from step S107 to step S108, the side-eye pixels are set as focus detection pixels to be used for focus detection.

[0094] However, if it is determined in step S107 that the vertical eye pixels are not reliable, it may be possible to choose not to perform focus detection.

[0095] In step S110, the camera control unit 218 performs focus detection using signals from the focus detection pixels selected in step S108 or step S109. Specifically, the camera control unit 218 performs the focus detection process described in the flowchart of FIG. 10, and transmits a focus drive command including the obtained defocus amount to the lens control unit 205. Upon receiving the focus drive command, the lens control unit 205 performs focus drive of the left eye optical system 301L and the right eye optical system 301R according to the defocus amount. This allows a focused state for the subject to be obtained. Then, in step S1110, this process ends.

[0096] As described above, in this embodiment, when a twin-lens unit is used, appropriate signal correction values ​​and conversion coefficients are obtained according to the direction of deviation of the optical axes of the two optical systems from the center of the image sensor, making it possible to obtain highly accurate focus detection results and AF results for the twin-lens unit.

[0097] In a twin-lens type lens unit, the position of the optical axis of each of the two optical systems corresponding to Fig. 6(a) is (x,y) = (±9,0). In each of the two optical systems, at the image height of y = 0, the change in the x direction at the side-eye pixel causes the lens aperture shape and the centroid difference to change as shown in Fig. 6(b) . Therefore, when focus detection is performed at an image height of y = 0, there is little change in the centroid difference at the vertical-eye pixel due to the change in image height, and a stable signal can be obtained, so there is a high possibility that the reliability of the vertical-eye pixel will be high. EXAMPLES

[0098] In the first embodiment, a case where a twin-lens type imaging optical system is used is described. However, even if the optical axis of at least a part of the imaging optical system is decentered with respect to the center of the image sensor, such as a tilt / shift (TS) lens, high-precision focus detection can be performed as in the first embodiment.

[0099] The function of the TS lens will be described with reference to Figures 13(a) and (b). Figure 13(a) shows the relationship between the subject OBJ, the optical axis OA of the imaging optical system, and the imaging element 211 when the TS lens is not shifted.

[0100] 13(a), the optical axis OA of the imaging optical system coincides with the center of the imaging element 211. In this state, if the optical axis OA is tilted with respect to a rectangular building serving as the subject OBJ, a trapezoidal subject image is formed as shown in the optical image IM.

[0101] On the other hand, Fig. 13(b) shows the relationship between the subject OBJ, the optical axis OA of the imaging optical system, and the image sensor 211 when the TS lens is shifted. In the state of Fig. 13(b), the optical axis OA of the imaging optical system is decentered from the center of the image sensor 211. In this state, when the optical axis OA is made perpendicular to a rectangular building serving as the subject OBJ, a rectangular subject image is formed as shown in the optical image IM. In other words, the perspective of the subject image is adjusted and distortion is corrected.

[0102] 7, the camera control unit 218 acquires the amount of eccentricity (tilt amount or shift amount) x of the optical axis of the TS lens as lens optical axis information. Then, in steps S103 to S109, the camera control unit 218 sets either a side-eye pixel or a vertical-eye pixel as a focus detection pixel depending on the direction of optical axis deviation from the center of the image sensor indicated by the lens optical axis information.

[0103] This makes it possible to obtain appropriate signal correction values ​​and conversion coefficients according to the direction of deviation of the optical axis of the imaging optical system from the center of the image sensor when using a TS lens as a lens unit, thereby obtaining highly accurate focus detection results and AF results for the TS lens.

[0104] The above embodiment includes the following configurations.

[0105] (Configuration 1) An imaging device in which an imaging optical system can be attached and detached, an image sensor having pixels including photoelectric conversion units divided so as to receive light beams passing through two divided pupil regions of an exit pupil of the image pickup optical system; a focus detection unit that performs focus detection using a signal from the photoelectric conversion unit; the photoelectric conversion unit includes a first photoelectric conversion unit that receives light beams from the two pupil regions divided in a first direction, and a second photoelectric conversion unit that receives light beams from the two pupil regions divided in a second direction different from the first direction, The focus detection means acquires optical axis information regarding the position of the optical axis of the imaging optical system which is different from the center of the image sensor, and selects one of the first and second photoelectric conversion units to be used for the focus detection based on the optical axis information. (Configuration 2) The imaging device according to configuration 1, wherein the imaging element has, as the pixels, a first pixel including the first photoelectric conversion unit and a second pixel including the second photoelectric conversion unit. (Configuration 3) The imaging device described in configuration 1 or 2, characterized in that the focus detection means selects one of the first and second photoelectric conversion units to be used for focus detection based on the direction of the position of the optical axis of the imaging optical system relative to the center of the image sensor. (Configuration 4) The imaging device described in any one of configurations 1 to 3, characterized in that the focus detection means selects one of the first and second photoelectric conversion units to be used for the focus detection based on the optical axis information and information regarding the position of the center of the image sensor. (Configuration 5) The imaging device described in any one of configurations 1 to 4, characterized in that the focus detection means determines the reliability of the focus detection using each of the first and second photoelectric conversion units, and selects one of the first and second photoelectric conversion units to be used for the focus detection based on the optical axis information and the reliability. (Configuration 6) The imaging apparatus according to configuration 5, wherein the focus detection means acquires optical information of the imaging optical system and uses the optical information to determine the reliability. (Configuration 7) The imaging optical system has two optical systems arranged in parallel, each of the two optical systems has an optical axis at a position different from the center of the image sensor; 7. The imaging device according to any one of configurations 1 to 6, wherein the optical axis information is information regarding the positions of the optical axes of the two optical systems. (Configuration 8) the imaging optical system has a configuration in which an optical axis of at least a part of the imaging optical system moves to a different position with respect to a center of the imaging element, 7. The imaging device according to claim 1, wherein the optical axis information is information regarding the position of at least a portion of the optical axis. (Configuration 9) The imaging device according to configuration 4, further comprising a configuration for driving the imaging element in a direction perpendicular to the optical axis. (Other Examples) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0106] The embodiments described above are merely representative examples, and various modifications and alterations are possible for each embodiment when implementing the present invention. [Explanation of symbols]

[0107] 100 Cameras 211 Imaging unit (imaging element) 218 Camera control unit 200 Lens unit 301L Left eye optical system 301R Right eye optical system

Claims

1. An imaging device in which an imaging optical system can be attached and detached, an image sensor having pixels including photoelectric conversion units divided so as to receive light beams passing through two divided pupil regions of an exit pupil of the image pickup optical system; a focus detection unit that performs focus detection using a signal from the photoelectric conversion unit; the photoelectric conversion unit includes a first photoelectric conversion unit that receives light beams from the two pupil regions divided in a first direction, and a second photoelectric conversion unit that receives light beams from the two pupil regions divided in a second direction different from the first direction, an imaging device characterized in that the focus detection means acquires optical axis information regarding the position of the optical axis of the imaging optical system which is different from the center of the image sensor, and selects one of the first and second photoelectric conversion units to be used for the focus detection based on the optical axis information.

2. 2 . The imaging device according to claim 1 , wherein the image sensor includes, as the pixels, a first pixel including the first photoelectric conversion portion and a second pixel including the second photoelectric conversion portion.

3. 2. The imaging device according to claim 1, wherein the focus detection means selects one of the first and second photoelectric conversion units to be used for the focus detection based on a direction of a position of the optical axis of the imaging optical system relative to the center of the imaging element.

4. 2. The imaging device according to claim 1, wherein the focus detection means selects one of the first and second photoelectric conversion units to be used for the focus detection based on the optical axis information and information regarding the position of the center of the imaging element.

5. 2. The imaging device according to claim 1, wherein the focus detection means determines reliability of the focus detection using each of the first and second photoelectric conversion units, and selects one of the first and second photoelectric conversion units to be used for the focus detection based on the optical axis information and the reliability.

6. 6. The image pickup apparatus according to claim 5, wherein the focus detection means acquires optical information of the image pickup optical system, and uses the optical information to determine the reliability.

7. The imaging optical system includes two optical systems arranged in parallel, each of the two optical systems has an optical axis at a position different from the center of the image sensor; 2. The imaging apparatus according to claim 1, wherein the optical axis information is information regarding the positions of the optical axes of the two optical systems.

8. the imaging optical system has a configuration in which an optical axis of at least a part of the imaging optical system moves to a different position with respect to a center of the imaging element, 2. The imaging apparatus according to claim 1, wherein the optical axis information is information regarding the position of at least a portion of the optical axis.

9. The imaging apparatus according to claim 4 , wherein the imaging element is driven in a direction perpendicular to the optical axis.

10. An imaging device having an imaging element and a detachable imaging optical system, A focus detection method for an image pickup apparatus, wherein the image pickup element has pixels including photoelectric conversion units divided to receive light beams passing through two divided pupil regions of an exit pupil of the image pickup optical system, the photoelectric conversion units including a first photoelectric conversion unit that receives light beams from the two pupil regions divided in a first direction, and a second photoelectric conversion unit that receives light beams from the two pupil regions divided in a second direction different from the first direction, acquiring optical axis information regarding a position of an optical axis of the imaging optical system different from a center of the imaging element; and selecting one of the first and second photoelectric conversion units to be used for focus detection based on the optical axis information.

11. A program for causing a computer of the imaging device to execute a process according to the focus detection method according to claim 10.

Images