Imaging apparatus, method for controlling imaging apparatus, and program

Abstract

To provide an imaging apparatus that can achieve high distance measurement accuracy in controlling two optical systems arranged in parallel with each other.SOLUTION: An imaging apparatus (100) has: an image pick-up device (211) that has a plurality of pixels; acquisition means (218a) that acquires the defocus amount of an image forming optical system on the basis of a pair of signals from the plurality of pixels; and control means (218b) that controls the image forming optical system on the basis of the defocus amount. The image forming optical system has a first optical system (301R) and a second optical system (301L) arranged in parallel with the first optical system. The acquisition means acquires a first defocus amount in a first distance measurement area at a first image height with reference to a first optical axis of the first optical system, and a second defocus amount in a second distance measurement area at a second image height with reference to a second optical axis of the second optical system. The control means controls the image forming optical system on the basis of the average defocus amount of the first defocus amount and the second defocus amount.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present invention relates to an imaging device, a control method for an imaging device, and a program. [Background technology]

[0002] Conventionally, imaging devices capable of capturing stereoscopic images, such as head-mounted displays (HMDs), have been known. Patent Document 1 discloses a method for calculating an adjustment value and a method for displaying a focus evaluation value of a subject in phase-difference AF in a stereoscopic imaging device. Patent Document 2 discloses a method for performing phase-difference AF independently in each of two optical systems, left and right. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-189536 [Patent Document 2] Japanese Patent Application Publication No. 2023-83876 Summary of the Invention [Problem to be solved by the invention]

[0004] The distance measurement results calculated by phase-difference AF in each of the two optical systems arranged in parallel with each other will vary depending on the signal S / N ratio. In particular, as the distance measurement area where the distance measurement results are calculated becomes smaller, the overall signal S / N ratio decreases, increasing the distance measurement variation.

[0005] When a lens unit with two optical systems is attached to an imaging device, the area displayed by one optical system is half that of when a lens unit with only one optical system is attached, so it is desirable to halve the size of the ranging area. However, as mentioned above, reducing the ranging area increases the variation in ranging. For this reason, it is difficult to achieve high ranging accuracy when controlling two optical systems with the methods disclosed in Patent Documents 1 and 2.

[0006] SUMMARY OF THE INVENTION It is therefore an object of the present invention to provide an imaging device that can achieve high distance measurement accuracy when controlling two optical systems arranged in parallel with each other. [Means for solving the problem]

[0007] An imaging apparatus according to one aspect of the present invention includes an imaging element having a plurality of pixels that receive light beams that pass through a plurality of mutually different pupil partial regions in an imaging optical system; an acquisition unit that acquires a defocus amount of the imaging optical system based on a pair of signals from the plurality of pixels; and a control unit that controls the imaging optical system based on the defocus amount, wherein the imaging optical system includes a first optical system and a second optical system arranged in parallel with the first optical system, and the acquisition unit acquires a first defocus amount in a first ranging region at a first image height based on a first optical axis of the first optical system and a second defocus amount in a second ranging region at a second image height based on a second optical axis of the second optical system, and the control unit controls the imaging optical system based on an average defocus amount of the first defocus amount and the second defocus amount.

[0008] Other objects and features of the present invention will be described in the following embodiments. [Effects of the Invention]

[0009] According to the present invention, it is possible to provide an imaging device that can achieve high distance measurement accuracy when controlling two optical systems that are arranged in parallel with each other. [Brief explanation of the drawings]

[0010] [Figure 1] 1 is a diagram showing an example of the external configuration of an imaging apparatus according to an embodiment of the present invention. [Figure 2] FIG. 1 is a diagram illustrating an example of the internal configuration of an imaging system according to an embodiment of the present invention. [Figure 3] FIG. 1 is a schematic diagram showing an example of the configuration of an imaging system according to an embodiment of the present invention. [Figure 4] FIG. 2 is a schematic diagram showing an example of a pixel array in the present embodiment. [Figure 5] FIG. 2 is a diagram showing an example of a display image in the present embodiment. [Figure 6] 5 is a flowchart illustrating an example of an autofocus operation in the present embodiment. [Figure 7] 5 is a flowchart illustrating an example of an autofocus operation in the present embodiment. [Figure 8] 10A and 10B are diagrams illustrating the relationship between the defocus amount and the image shift amount in this embodiment. [Figure 9] 5A to 5C are diagrams illustrating an example of an autofocus operation in the present embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0012] <Camera body external configuration> First, the external configuration of a camera body (digital camera, imaging device) 100 in this embodiment will be described with reference to Figures 1(a) and 1(b). Figures 1(a) and 1(b) are diagrams showing an example of the external configuration of the camera body 100. Figure 1(a) is a perspective view of the camera body 100 seen from the front, and Figure 1(b) is a perspective view of the camera body 100 seen from the back.

[0013] The camera body 100 has, on its top surface, a shutter button 101, a power switch 102, a mode switch 103, a main electronic dial 104, a sub electronic dial 105, a video button 106, and an extra-viewfinder display 107. The shutter button 101 is an operation unit used to prepare for shooting or to give instructions to shoot. The power switch 102 is an operation unit used to turn the power of the camera body 100 on and off. The mode switch 103 is an operation unit used to switch between various modes. The main electronic dial 104 is a rotary operation unit used to change settings such as shutter speed and aperture. The sub electronic dial 105 is a rotary operation unit used to move the selection frame (cursor), advance images, etc. The video button 106 is an operation unit used to start and stop video shooting (recording). The extra-viewfinder display 107 displays various settings such as shutter speed and aperture.

[0014] The camera body 100 has, on its rear surface, a display unit 108, a touch panel 109, directional keys 110, a SET button 111, an AE lock button 112, a magnification button 113, a playback button 114, a menu button 115, an eyepiece unit 116, an eyepiece detection unit 118, and a touch bar 119. The display unit 108 displays images and various information. The touch panel 109 is an operation unit that detects touch operations on the display surface (touch operation surface) of the display unit 108. The directional keys 110 are an operation unit consisting of keys (four-way keys) that can be pressed up, down, left, and right. Operations can be performed according to the position of the directional keys 110 that are pressed. The SET button 111 is an operation unit that is mainly pressed to confirm a selection item. The AE lock button 112 is an operation unit that is pressed to fix the exposure state in a shooting standby state.

[0015] The enlargement button 113 is an operation unit for switching the enlargement mode on and off in the live view display (LV display) in the shooting mode. When the enlargement mode is on, the live view image (LV image) is enlarged or reduced by operating the main electronic dial 104. The enlargement button 113 is also used to enlarge the playback image or increase the magnification in the playback mode. The playback button 114 is an operation unit for switching between the shooting mode and the playback mode. In the shooting mode, pressing the playback button 114 switches to the playback mode, and the latest image of the images recorded on the recording medium 228 (described later) can be displayed on the display unit 108.

[0016] The menu button 115 is an operation unit that is pressed when a menu screen that allows various settings to be displayed on the display unit 108. The user can intuitively make various settings using the menu screen displayed on the display unit 108, the direction keys 110, and the SET button 111. The eyepiece unit 116 is a part for placing an eye on an eyepiece finder (peek-in type finder) 117. The user can view an image displayed on an internal EVF (Electronic View Finder) 217 ​​(described later) through the eyepiece unit 116. The eyepiece detection unit 118 is a sensor that detects whether the user has placed an eye on the eyepiece unit 116.

[0017] The touch bar 119 is a line-shaped touch operation unit (line touch sensor) capable of receiving touch operations. The touch bar 119 is positioned so that it can be touched by the thumb of the right hand when the grip unit 120 is held in the right hand (held with the little finger, ring finger, and middle finger of the right hand) so that the shutter button 101 can be pressed with the index finger of the right hand. That is, the touch bar 119 can be operated while the eyepiece unit 116 is placed near the viewfinder 117 and the camera is in a position ready to press the shutter button 101 at any time (shooting posture). The touch bar 119 can receive tap operations (operations in which the user touches the touch bar and then releases the touch bar without moving it within a predetermined period of time), slide operations to the left or right (operations in which the user touches the touch bar and then moves the touched position while keeping the touch), and the like. The touch bar 119 is an operation unit different from the touch panel 109 and does not have a display function. The touch bar 119 in this embodiment is a multi-function bar and functions as, for example, an M-Fn bar.

[0018] The camera body 100 also has a grip section 120, a thumb rest section 121, a terminal cover 122, a lid 123, a communication terminal 124, etc. The grip section 120 is a holding section formed in a shape that is easy to hold in the user's right hand when holding the camera body 100. When the user holds the camera body 100 by gripping the grip section 120 with the little finger, ring finger, and middle finger of the right hand, the shutter button 101 and main electronic dial 104 are located in positions that can be operated with the index finger of the right hand. Similarly, the sub electronic dial 105 and touch bar 119 are located in positions that can be operated with the thumb of the right hand.

[0019] Thumb rest 121 is a grip section provided on the rear side of camera body 100 in a position (thumb standby position) where the thumb of the right hand gripping grip 120 can be easily placed when none of the operation sections are being operated. Thumb rest 121 is made of a rubber member or the like to enhance holding power (grip feeling). Terminal cover 122 protects connectors such as connection cables that connect camera body 100 to external devices.

[0020] The lid 123 protects the recording medium 228 and the slot by closing the slot for storing the recording medium 228, which will be described later. The communication terminal 124 is a terminal for communicating with the detachable lens unit 200, which will be described later, of the camera body 100.

[0021] <Internal configuration of the imaging system> Next, the internal configuration of the imaging system 10 according to this embodiment will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of the internal configuration of the imaging system 10. In Fig. 2, the same components as those in Fig. 1 are given the same reference numerals, and their description will be omitted. The imaging system 10 has a camera body 100 and a lens unit (lens device) 200 that is detachable from the camera body 100. However, this embodiment is not limited to this, and can also be applied to an imaging device in which the camera body and lens unit are integrally configured.

[0022] First, the lens unit 200 will be described. The lens unit 200 is a type of interchangeable lens that can be attached to and detached from the camera body 100. The lens unit 200 is a single lens and is an example of a normal lens. The lens unit 200 has an aperture (aperture diaphragm) 201, a lens 202, an aperture drive circuit 203, an AF (autofocus) drive circuit 204, a lens system control circuit 205, a communication terminal 206, and the like.

[0023] The aperture 201 is configured to have an adjustable aperture diameter. The lens 202 is composed of multiple lenses. The aperture drive circuit 203 adjusts the amount of light by controlling the aperture diameter of the aperture 201. The AF drive circuit 204 drives the lens 202 to adjust the focus. The lens system control circuit 205 controls the aperture drive circuit 203, the AF drive circuit 204, etc. based on instructions from a system control unit 218 (described later). The lens system control circuit 205 controls the aperture 201 via the aperture drive circuit 203, and adjusts the focus by displacing the position of the lens 202 via the AF drive circuit 204. The lens system control circuit 205 can communicate with the camera body 100. Specifically, communication is performed via a communication terminal 206 of the lens unit 200 and a communication terminal 124 of the camera body 100. The communication terminal 206 is a terminal through which the lens unit 200 communicates with the camera body 100.

[0024] Next, we will explain the camera body 100. The camera body 100 has a shutter 210, an imaging unit 211, an A / D converter 212, a memory control unit 213, an image processing unit 214, a memory 215, a D / A converter 216, an EVF 217, a display unit 108, and a system control unit 218.

[0025] The shutter 210 is a focal plane shutter that can freely control the exposure time of the imaging unit 211 based on instructions from the system control unit 218. The imaging unit 211 has an imaging element (image sensor) that converts an optical image into an electrical signal. The imaging element is a photoelectric conversion element such as a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal-Oxide-Semiconductor) sensor. The imaging unit 211 may have an imaging surface phase difference sensor that outputs defocus amount information to the system control unit 218.

[0026] The A / D converter 212 converts the analog signal output from the imaging unit 211 into a digital signal. The image processing unit 214 performs predetermined processing (pixel interpolation, resizing such as reduction, color conversion, etc.) on the data from the A / D converter 212 or the data from the memory control unit 213. The image processing unit 214 also performs predetermined arithmetic processing using the captured image data, and the system control unit 218 performs exposure control and distance measurement control based on the obtained arithmetic results. This processing results in TTL (through-the-lens) AF processing, AE (auto exposure) processing, EF (flash pre-flash) processing, etc. The image processing unit 214 also performs predetermined arithmetic processing using the captured image data, and performs TTL AWB (auto white balance) processing based on the obtained arithmetic results.

[0027] The image data from the A / D converter 212 is written to the memory 215 via the image processing unit 214 and the memory control unit 213. Alternatively, the image data from the A / D converter 212 is written to the memory 215 via the memory control unit 213 without going through the image processing unit 214. The memory 215 stores image data obtained by the imaging unit 211 and converted into digital data by the A / D converter 212, as well as image data to be displayed on the display unit 108 and the EVF 217. The memory 215 has a storage capacity sufficient to store a predetermined number of still images and a predetermined period of moving images and audio. The memory 215 also serves as a memory for displaying images (video memory).

[0028] The D / A converter 216 converts image display data stored in the memory 215 into an analog signal and supplies it to the display unit 108 or the EVF 217. Therefore, the display image data written to the memory 215 is displayed on the display unit 108 or the EVF 217 via the D / A converter 216. The display unit 108 or the EVF 217 performs display in accordance with the analog signal from the D / A converter 216. The display unit 108 or the EVF 217 is, for example, an LCD or organic EL display. A digital signal that has been A / D converted by the A / D converter 212 and stored in the memory 215 is converted into an analog signal by the D / A converter 216 and then sequentially transferred to and displayed on the display unit 108 or the EVF 217, thereby performing live view display.

[0029] The system control unit 218 is a control unit made up of at least one processor and / or at least one circuit. That is, the system control unit 218 may be a processor, a circuit, or a combination of a processor and a circuit. The system control unit 218 controls the entire camera body 100. The system control unit 218 executes programs recorded in the nonvolatile memory 220 to realize each process in the flowcharts described below. The system control unit 218 also performs display control by controlling the memory 215, D / A converter 216, display unit 108, EVF 217, etc.

[0030] In this embodiment, the system control unit 218 has an acquisition unit 218a and a control unit 218b. The acquisition unit 218a acquires the defocus amount of the imaging optical system (imaging optical system) of the lens unit 300 based on a pair of signals (focus detection signals) from multiple pixels (focus detection pixels) of the image sensor of the imaging unit 211. The control unit 218b controls the imaging optical system based on the defocus amount acquired by the acquisition unit 218a.

[0031] As described below, the acquisition unit 218a acquires a first defocus amount in a first ranging region at a first image height based on the first optical axis of the first optical system and a second defocus amount in a second ranging region at a second image height based on the second optical axis of the second optical system. The control unit 218b controls the imaging optical system (first optical system and second optical system) based on an average defocus amount of the first defocus amount and the second defocus amount. Here, the first defocus amount is the defocus amount in the first ranging region at the first image height based on the first optical axis of the first optical system, and the second defocus amount is the defocus amount in the second ranging region at the second image height based on the second optical axis of the second optical system. The control unit 218b can control the imaging optical system so that the first optical system and the second optical system perform autofocus operations while maintaining the same focus positions.

[0032] The camera body 100 also has a system memory 219 , a non-volatile memory 220 , a system timer 221 , a communication unit 222 , an attitude detection unit 223 , and an eye proximity detection unit 118 .

[0033] The system memory 219 includes, for example, a RAM (Random Access Memory). Constants and variables for the operation of the system control unit 218, and programs read from the nonvolatile memory 220 are loaded in the system memory 219. The nonvolatile memory 220 is an electrically erasable and recordable memory, and an EEPROM is used for example. The nonvolatile memory 220 stores constants and programs for the operation of the system control unit 218. The programs here refer to programs for executing the flowcharts described below. The system timer 221 is a timing unit that measures the time used for various controls and the time of a built-in clock.

[0034] The communication unit 222 transmits and receives video signals and audio signals to and from external devices connected wirelessly or via a wired cable. The communication unit 222 can also connect to a wireless LAN (Local Area Network) or the Internet. The communication unit 222 can also communicate with external devices via Bluetooth (registered trademark) or Bluetooth Low Energy. The communication unit 222 can transmit images (including live images) captured by the imaging unit 211 and images recorded on a recording medium 228, and can receive image data and various other information from external devices.

[0035] The orientation detection unit 223 detects the orientation of the camera body 100 with respect to the direction of gravity. Based on the orientation detected by the orientation detection unit 223, it is possible to determine whether an image captured by the imaging unit 211 was captured with the camera body 100 held horizontally or vertically. The system control unit 218 can add orientation information corresponding to the orientation detected by the orientation detection unit 223 to the image file of the image captured by the imaging unit 211, or rotate and record the image. The orientation detection unit 223 can use, for example, an acceleration sensor or a gyro sensor. The orientation detection unit 223 can also be used to detect movement of the camera body 100 (panning, tilting, lifting, whether or not it is stationary, etc.).

[0036] The eyepiece detection unit 118 can detect the approach of an object to the eyepiece unit 116 of the eyepiece finder 117 that incorporates the EVF 217. The eyepiece detection unit 118 can use, for example, an infrared proximity sensor. When an object approaches, infrared light is emitted from a light-emitting unit of the eyepiece detection unit 118, reflected by the object, and received by a light-receiving unit of the infrared proximity sensor. The distance from the eyepiece unit 116 to the object can be determined based on the amount of received infrared light. In this way, the eyepiece detection unit 118 performs eyepiece detection, which detects the proximity of an object to the eyepiece unit 116.

[0037] The eye-contact detection unit 118 is an eye-contact detection sensor that detects the approach (eye-contact) and separation (eye-separation) of an eye (object) to the eyepiece unit 116 of the eyepiece finder 117. When an object is detected approaching the eyepiece unit 116 within a predetermined distance from a non-eye-contact state (non-approaching state), the eye-contact detection unit 118 detects that the eye has been placed in contact. On the other hand, when the object that was detected as approaching moves away from the eye-contact state (approaching state) by a predetermined distance or more, the eye-contact detection unit 118 detects that the eye has been separated. The threshold for detecting eye-contact and the threshold for detecting separation may be different, for example, by providing hysteresis. Furthermore, after detecting eye-contact, the eye-contact state is maintained until separation is detected. After detecting separation, the eye-contact state is maintained until the eye is placed in contact.

[0038] The system control unit 218 switches between display (display state) and non-display (non-display state) of the display unit 108 and the EVF 217 depending on the state detected by the eye proximity detection unit 118. Specifically, when at least in a shooting standby state and the display destination switching setting is automatic switching, the display destination is set to the display unit 108 and the display is turned on while the EVF 217 is not in view while the eye is not in view. Furthermore, when the eye is in view while the eye is in view, the display destination is set to the EVF 217 and the display is turned on while the display unit 108 is in view. Note that the eye proximity detection unit 118 is not limited to an infrared proximity sensor, and any other sensor may be used as long as it can detect a state that can be considered as eye proximity.

[0039] The camera body 100 also has an outside-finder display unit 107, an outside-finder display drive circuit 224, a power supply control unit 225, a power supply unit 226, a recording medium I / F 227, an operation unit 229, and the like.

[0040] The viewfinder display unit 107 displays various settings of the camera body 100, such as shutter speed and aperture, via an viewfinder display drive circuit 224. The power supply control unit 225 is composed of a battery detection circuit, a DC-DC converter, a switch circuit for switching between powered blocks, and other components, and detects whether a battery is installed, the battery type, and the remaining battery charge. The power supply control unit 225 also controls the DC-DC converter based on the detection results and instructions from the system control unit 218 to supply the required voltage for the required period to various components, including the recording medium 228. The power supply unit 226 may be a primary battery such as an alkaline battery or a lithium battery, a secondary battery such as a NiCd battery, a NiMH battery, or a Li battery, or an AC adapter. The recording medium I / F 227 is an interface with a recording medium 228, such as a memory card or a hard disk. The recording medium 228 is a memory card or the like for recording captured images, and may be composed of a semiconductor memory, a magnetic disk, or the like. The recording medium 228 may be removable or built-in.

[0041] The operation unit 229 is an input unit that accepts operations from the user (user operations) and is used to input various instructions to the system control unit 218. The operation unit 229 includes the shutter button 101, the power switch 102, the mode selector switch 103, the touch panel 109, and other operation units 230. The other operation units 230 include the main electronic dial 104, the sub electronic dial 105, the video button 106, the direction keys 110, the SET button 111, the AE lock button 112, the enlargement button 113, the playback button 114, the menu button 115, the touch bar 119, and the like.

[0042] The shutter button 101 has a first shutter switch 231 and a second shutter switch 232. The first shutter switch 231 is turned on when the shutter button 101 is pressed halfway (a shooting preparation instruction) during operation, and generates a first shutter switch signal SW1. The system control unit 218 starts shooting preparation processing such as AF processing, AE processing, AWB processing, and EF processing in response to the first shutter switch signal SW1. The second shutter switch 232 is turned on when the shutter button 101 is pressed fully (a shooting instruction) and generates a second shutter switch signal SW2. The system control unit 218 starts a series of shooting processing in response to the second shutter switch signal SW2, from reading out a signal from the imaging unit 211 to generating an image file including the captured image and writing it to the recording medium 228.

[0043] The mode selector switch 103 switches the operation mode of the system control unit 218 to one of still image capture mode, video capture mode, playback mode, etc. Modes included in the still image capture mode include auto capture mode, auto scene determination mode, manual mode, aperture priority mode (Av mode), shutter speed priority mode (Tv mode), and program AE mode (P mode). There are also various scene modes and custom modes that provide capture settings for different capture scenes. The user can directly switch to one of the above-mentioned capture modes using the mode selector switch 103. Alternatively, the user can first switch to a list screen of capture modes using the mode selector switch 103, and then selectively switch to one of the displayed modes using the operation unit 229. Similarly, the video capture mode may also include multiple modes.

[0044] The touch panel 109 is a touch sensor that detects various touch operations on the display surface of the display unit 108 (the operation surface of the touch panel 109). The touch panel 109 and the display unit 108 can be configured as an integrated unit. For example, the touch panel 109 is attached to the upper layer of the display surface of the display unit 108 so that its light transmittance does not interfere with the display of the display unit 108. By associating input coordinates on the touch panel 109 with display coordinates on the display surface of the display unit 108, a GUI (graphical user interface) can be configured that makes it appear as if the user is directly operating the screen displayed on the display unit 108. The touch panel 109 can be any of a variety of types, such as a resistive film type, a capacitive type, a surface acoustic wave type, an infrared type, an electromagnetic induction type, an image recognition type, or an optical sensor type. Depending on the type, there are types that detect a touch by contact with the touch panel 109, and types that detect a touch by the approach of a finger or a pen to the touch panel 109, but either type may be used.

[0045] The system control unit 218 can detect the following operations or states on the touch panel 109. It is compatible with commonly known functions and responds as a camera system according to the state of touch panel operation.

[0046] <Lens unit configuration> Next, an imaging system 30 according to this embodiment will be described with reference to Fig. 3. Fig. 3 is a schematic diagram showing an example of the configuration of imaging system 30 according to this embodiment. Fig. 3 shows a state in which lens unit 300 is attached to camera body 100. Note that in camera body 100 shown in Fig. 3, the same components as those described with reference to Fig. 2 are designated by the same reference numerals, and description thereof will be omitted.

[0047] The imaging system 30 includes a camera body 100 and a lens unit (lens apparatus) 300 that can be attached to and detached from the camera body 100. The lens unit 300 is a type of interchangeable lens that can be attached to and detached from the camera body 100. The lens unit 300 is a twin lens that can capture images with parallax between left and right images. The lens unit 300 has two optical systems, each with a wide viewing angle of approximately 180 degrees, and can capture images of the range of the forward hemisphere. Specifically, the two optical systems of the lens unit 300 can each capture an object within a field of view (angle of view) of 180 degrees in the left-right direction (horizontal angle, azimuth angle, yaw angle) and 180 degrees in the up-down direction (vertical angle, elevation angle, pitch angle).

[0048] The lens unit 300 includes a right-eye optical system 301R having a plurality of lenses and a reflecting mirror, a left-eye optical system 301L having a plurality of lenses and a reflecting mirror, and a lens system control circuit 303. The right-eye optical system 301R corresponds to an example of a first optical system, and the left-eye optical system 301L corresponds to an example of a second optical system arranged in parallel with the first optical system. The right-eye optical system 301R and the left-eye optical system 301L constitute an imaging optical system (image capturing optical system) of the lens unit 300. The right-eye optical system 301R has a first optical axis OA1, and the left-eye optical system 301L has a second optical axis OA2. Note that in the description of this embodiment, the left-eye optical system 301L may be referred to as the first optical system and the right-eye optical system 301R as the second optical system. In the right-eye optical system 301R and the left-eye optical system 301L, the lenses 302R and 302L located on the subject side face in the same direction, and the first optical axis OA1 and the second optical axis OA2 are approximately parallel to each other. In this embodiment, the imaging optical system of the lens unit 300 is not limited to a configuration including only two optical systems, but may include three or more optical systems.

[0049] The lens unit 300 of this embodiment is a VR180 lens for capturing images for so-called VR180, a VR image format that enables binocular stereoscopic viewing. The VR180 lens has fisheye lenses that enable the right-eye optical system 301R and the left-eye optical system 301L to capture a range of approximately 180 degrees. Note that the VR180 lens may be a lens that can capture a wide viewing angle range of approximately 160 degrees, which is narrower than the 180-degree range, as long as the right-eye optical system 301R and the left-eye optical system 301L can acquire images that can be displayed as binocular VR in VR180. The VR180 lens can form a right image (first image) formed via the right-eye optical system 301R and a left image (second image) formed via the left-eye optical system 301L, which has parallax from the right image, on one or two image sensors of the attached camera body.

[0050] The lens unit 300 is equipped with a focus ring for focus adjustment. Although not shown, the lens unit 300 is equipped with two focus rings: one for adjusting the focus of the right image formed via the right-eye optical system 301R, and the other for adjusting the focus of the left image formed via the left-eye optical system 301L. Alternatively, the lens unit 300 is equipped with two focus rings: one for adjusting the focus of the right and left images simultaneously, and the other for adjusting the focus of either the right or left image.

[0051] Furthermore, the lens unit 300 is attached to the camera body 100 via the lens mount section 304 and the camera mount section 305 of the camera body 100. By attaching the lens unit 300 to the camera body 100, the system control section 218 and the lens system control circuit 303 are electrically connected via the communication terminal 124 of the camera body 100 and the communication terminal 306 of the lens unit 300.

[0052] In this embodiment, a right image formed via the right-eye optical system 301R and a left image formed via the left-eye optical system 301L, which has parallax from the right image, are formed side by side on the imaging unit 211 of the camera body 100. That is, two optical images formed by the right-eye optical system 301R and the left-eye optical system 301L are formed on a single imaging element. The imaging unit 211 converts the formed subject image (optical signal) into an analog electrical signal. In this way, by using the lens unit 300, two images with parallax can be simultaneously acquired (as a set) from two locations (optical systems), the right-eye optical system 301R and the left-eye optical system 301L. Furthermore, by dividing the acquired images into an image for the left eye and an image for the right eye and displaying them in VR, the user can view a stereoscopic VR image with a range of approximately 180 degrees, known as VR180.

[0053] In this embodiment, the lens system control circuit 303 controls the driving of the right eye optical system 301R and the left eye optical system 301L based on commands from the system control unit 218 of the camera body 100.

[0054] <Image sensor configuration> Next, the pixel arrangement of the image sensor in the image capturing unit 211 in this embodiment will be described with reference to Fig. 4. Fig. 4 is a schematic diagram showing an example of the pixel arrangement of the image sensor. Fig. 4 shows the pixel arrangement of a two-dimensional CMOS sensor used as the image sensor in a range of 4 columns x 4 rows of imaging pixels (a range of 8 columns x 4 rows as the array of focus detection pixels).

[0055] In this embodiment, the pixel group 400 is composed of pixels arranged in 2 columns and 2 rows, and is covered with color filters in a Bayer array. In the pixel group 400, a pixel 400R having a spectral sensitivity of R (red) is arranged in the upper left position, pixels 400G having a spectral sensitivity of G (green) are arranged in the upper right and lower left positions, and a pixel 400B having a spectral sensitivity of B (blue) is arranged in the lower right position. Furthermore, the image sensor in the image capturing unit 211 of this embodiment performs focus detection using an image capture plane phase difference method, so each pixel has multiple photodiodes (photoelectric conversion units) for one microlens 401. In this embodiment, each pixel is composed of two photodiodes 402 and 403 arranged in a 2 column x 1 row.

[0056] As shown in FIG. 4, the imaging element in the imaging unit 211 has a large number of pixel groups 400 arranged on the imaging surface, each of which is made up of 2 columns x 2 rows of pixels (4 columns x 2 rows of photodiodes), thereby enabling acquisition of imaging signals and focusing signals.

[0057] In each pixel having such a configuration, a light beam is separated by a microlens 401 and an image is formed on a first photodiode 402 and a second photodiode 403. A signal (A+B signal) obtained by adding together signals from the two photodiodes 402 and 403 is used as an imaging signal, and a pair of focus detection signals (A and B image signals) read out from each of the photodiodes 402 and 403 is used as a focusing signal. Note that the imaging signal and the focusing signal may be read out separately, but in consideration of the processing load, the following may also be used. That is, the imaging signal (A+B signal) and a focusing signal (e.g., signal A) from one of the photodiodes 402 and 403 may be read out, and the other focusing signal (e.g., signal B) may be obtained by taking the difference.

[0058] In this embodiment, each pixel has two photodiodes 402, 403 for one microlens 401, but the number of photodiodes is not limited to two and may be more. Also, a plurality of pixels may be provided with different positions of the opening of the light receiving portion relative to the microlens 401. In other words, any configuration may be used as long as it results in the acquisition of two signals for phase difference detection, such as image A and image B signals, which enable phase difference detection.

[0059] 4 shows a configuration in which all pixels have multiple photodiodes, but this embodiment is not limited to this. For example, a configuration in which focus detection pixels as shown in FIG. 4 are provided discretely within a normal pixel region (image capturing pixel region) that constitutes the image sensor in the image capturing unit 211 may also be used.

[0060] <Relationship between defocus amount and image shift amount> Next, the relationship between the defocus amount calculated from a pair of signals (image A signal and image B signal) acquired by the image sensor of this embodiment and the image shift amount will be described with reference to Fig. 8. Fig. 8 is a diagram showing the relationship between the defocus amount and the image shift amount between a pair of focus detection signals (image A signal and image B signal). In this embodiment, a general optical system in which the imaging center and the optical axis center coincide will be described.

[0061] The image sensor (not shown) of this embodiment is disposed on an image pickup surface 800 , and the exit pupil of the imaging optical system is divided into two regions: a first pupil partial region 803 and a second pupil partial region 804 .

[0062] The defocus amount d is defined as the distance from the subject's imaging position to the imaging plane, with magnitude |d|, and a front-focus state where the subject's imaging position is closer to the subject than the imaging plane is given a negative sign (d<0), and a back-focus state where the subject's imaging position is on the opposite side of the imaging plane is given a positive sign (d>0). The in-focus state where the subject's imaging position is on the imaging plane (focus position) is d=0. In Figure 8, subject 801 is in-focus, and subject 802 is in front-focus. The front-focus state (d<0) and back-focus state (d>0) are combined to form the defocus state (|d|>0).

[0063] In a front-focus state (d<0), a light beam from the subject 802 that passes through the first pupil partial region 803 (second pupil partial region 804) is first focused and then spreads to a width Γ1 (Γ2) around the center of gravity G1 (G2) of the light beam, forming a blurred image on the imaging surface 800. The blurred image is received by the first photodiode 402 (second photodiode 403) that constitutes each pixel arranged on the image sensor, and a pair of focus detection signals (A and B image signals) is generated. Therefore, the pair of focus detection signals (A and B image signals) is recorded as a subject image of the subject 1302 blurred to a width Γ1 (Γ2) at the center of gravity G1 (G2) on the imaging surface 1300. The blur width Γ1 (Γ2) of the subject image increases roughly proportionally as the magnitude of the defocus amount d, |d|, increases.

[0064] Similarly, the magnitude |p| of the image shift amount p (= the difference G1-G2 in the center of gravity positions of the light beams) of the subject image between the first focus detection signal and the second focus detection signal also increases roughly proportionally as the magnitude |d| of the defocus amount d increases. The same is true in the back-focus state (d>0), although the direction of image shift of the subject image between a pair of focus detection signals is opposite to that in the front-focus state.

[0065] Therefore, by determining in advance the conversion coefficient K for converting the image shift amount p into the defocus amount d, it is possible to calculate the defocus amount d from the image shift amount p of the subject image between a pair of focus detection signals.

[0066] <Display image> Next, referring to FIG. 5, an example of an image (display image) displayed on the display unit 108 of the present embodiment will be described. FIG. 5 is a diagram showing an example of a display image in the present embodiment.

[0067] In the present embodiment, it is assumed that the lens unit has an optical system having two left and right optical axes so that stereoscopic vision is possible. For this reason, two images (display images), namely, a left image (500L) and a right image (500R) of the display image 502, are displayed on the display unit 108. The display image 502 includes a distance measurement area 501R displayed based on the right image and a distance measurement area 501L corresponding to the distance measurement area 501R in the left image. Note that the distance measurement frame indicates the outer shape of the distance measurement area, and the center coordinates of the distance measurement frame are used as the image height.

[0068] The distance measurement areas 501R and 501L are arranged at positions where the relative coordinates from each optical axis position substantially coincide with each other. This is because when the subject is far compared to the focal length, the same subject is located at the position where the relative coordinates from each optical axis position coincide. In the present embodiment, in order to perform an autofocus operation (AF operation) based on the distance measurement area 501R, the distance measurement area 501L is not displayed, but the distance measurement area 501L may be displayed.

[0069] <AF operation> Next, referring to FIG. 6, an example of the AF operation in the present embodiment will be described. FIG. 6 is a flowchart showing an example of the AF operation in the present embodiment. Each step in FIG. 6 is mainly executed by the system control unit 2,18 or the lens system control circuit 205.

[0070] When AF starts, first, in step S601, the system control unit 218 sets a ranging area (first ranging area) 501R. Then, in step S602, the system control unit 218 acquires a ranging result (first defocus amount) 601R. However, this embodiment is not limited to this, and the ranging area 501L may be set in step S601 and the ranging result 601L may be acquired in step S602. In this case, the ranging result 601L corresponds to the first defocus amount, and the ranging result 601R acquired in step S604, which will be described later, corresponds to the second defocus amount.

[0071] Next, in step S603, the system control unit 218 determines whether the distance measurement result 601R acquired in step S602 is within the range of the def threshold, i.e., whether the absolute value of the first defocus amount is within the range of the first threshold. This determines whether the distance measurement area 501R is blurred or near-in-focus. If the distance measurement result 601R (absolute value of the first defocus amount) is outside the range of the def threshold, the system control unit 218 determines that the distance measurement area 501R is blurred. If the distance measurement result 601R is outside the range of the def threshold, i.e., blurred, there is a possibility that the distance measurement area (first distance measurement area) 501R and the distance measurement area (second distance measurement area) 501L are different images (images that do not include the same subject area) due to parallax. Therefore, the process proceeds to step S608, where the system control unit 218 drives the lens (drives the imaging optical system) according to the distance measurement result 601R. On the other hand, if the distance measurement result 601R is within the range of the def threshold, the system control unit 218 determines that the distance measurement area 501R is in a near-focus state, and proceeds to step S604.

[0072] In step S604, the system control unit 218 acquires a distance measurement result (second defocus amount) 601L in the distance measurement area 501L. Subsequently, in step S605, the system control unit 218 determines whether the absolute value of the difference between the distance measurement result (first distance measurement result) 601R and the distance measurement result (second distance measurement result) 601L is within a threshold value (second threshold value). If there is an obstruction in front of either the right-eye optical system 301R or the left-eye optical system 301L, a large difference will occur between the two distance measurement results 601R, 601L. In this case, it is not preferable to average the distance measurement results 601R, 601L (to acquire an average defocus amount).

[0073] If the absolute value of the difference between the distance measurement result 601R and the distance measurement result 601L is within the threshold range, the process proceeds to step S606. In step S606, the system control unit 218 averages the two distance measurement results 601R and 601L and performs focusing control by driving the lens using the averaged distance measurement result (average defocus amount). When two distance measurement results with the same degree of distance measurement variation σ are averaged, the variation becomes σ / √2, improving focusing accuracy. On the other hand, if the absolute value of the difference between the distance measurement result 601R and the distance measurement result 601L is outside the threshold range, the process proceeds to step S607. In this case, the system control unit 218 determines that there is an obstruction in front of the left eye optical system 301L, for example, and in step S607, performs focusing control by driving the lens using only the distance measurement result (first defocus amount) 601R. However, the focusing accuracy in step S607 is inferior to that in step S606 because the lens is driven using only the distance measurement result 601R.

[0074] In FIG. 6, the distance measurement result 601L may be acquired as the first defocus amount in step S602, and the distance measurement result 601R may be acquired as the second defocus amount in step S604.

[0075] In this embodiment, the control unit 218b determines whether to perform averaging processing based on the absolute value of the difference between the distance measurement result 601R and the distance measurement result 601L. That is, the control unit 218b determines whether to control the imaging optical system using the first defocus amount or the average defocus amount. However, this embodiment is not limited to this, and a step of confirming that both the distance measurement result 601R and the distance measurement result 601L are in a near-focus state may be provided. In this case, it is confirmed that at least one of the absolute values ​​of the two distance measurement results 601R, 601L is within a predetermined defocus amount (first threshold).

[0076] According to this embodiment, the two distance measurement results are averaged, thereby improving the focusing accuracy. Therefore, in this embodiment, when the averaging process is performed, the distance measurement area may be smaller than when the averaging process is not performed. That is, in this embodiment, the first distance measurement area when the imaging optical system is controlled using the average defocus amount is preferably smaller than the first distance measurement area when the imaging optical system is controlled using the first defocus amount.

[0077] <Close-up response> When the subject is very close (when the focus lens that makes up the imaging optical system is at a very close position), the left and right parallax shift becomes large, so if the ranging areas are placed at positions where the relative coordinates from each optical axis position are approximately the same, the images corresponding to the left and right ranging areas will be different. For this reason, when the subject is very close, it is preferable to detect the same subject in the left image and the right image, and determine ranging areas 501L and 501R of the left image (500L) and right image (500R) so that they include the same subject.

[0078] Fig. 7 is a flowchart including close-up correspondence as a modified example of this embodiment. Fig. 7 differs from Fig. 6 in that it includes steps S701 and S702. Note that steps S601 to S608 in Fig. 7 are common to Fig. 6, and therefore their description will be omitted.

[0079] If it is confirmed in step S603 that the distance measurement result 601R is in a near-focus state, the process proceeds to step S701. In step S701, the system control unit 218 determines whether the focus lens (lens 202) is in the closest position. If the focus lens is in the closest position, the process proceeds to step S702. In step S702, the system control unit 218 uses pattern matching or the like to place the distance measurement area 501L of the left image (500L) at a position where the same image as the right image (500R) can be acquired (determines the distance measurement area 501L). Then, the system control unit 218 acquires the distance measurement result 601L for the left image (500L).

[0080] Thus, in this embodiment, when the focus position of the imaging optical system is close, the control means 218b obtains the first image height and the second image height by performing pattern matching between the first image in the first ranging area and the second image in the second ranging area.

[0081] <Response when subject is detected> As mentioned above, the ranging area should basically be located at a position where the relative coordinates from each optical axis position are approximately the same. However, in recent years, advances in machine learning have improved the accuracy of subject detection, so if the same subject is detected by the left and right optical systems, the subject detection position may be used as the ranging area.

[0082] Referring to FIG. 9, an example will be described in which the same subject (e.g., a bird) is detected by the left and right optical systems, and the subject detection position is used as the ranging area. FIG. 9 is a diagram showing an example of AF operation. In FIG. 9, 901L indicates the optical axis center of the left eye optical system 301L, and 901R indicates the optical axis center of the right eye optical system 301R. 902L indicates the subject detection position (subject area) in the left image, and 902R indicates the subject detection position (subject area) in the right image. The ranging areas are located at the subject detection positions 902L and 902R. In this manner, in this embodiment, the relative position of the first ranging area with respect to the first optical axis OA1 and the relative position of the second ranging area with respect to the second optical axis OA2 are the same subject area detected via the first optical system and the second optical system, respectively.

[0083] Note that the above-described flowcharts are applicable except for the method of arranging the distance measurement areas. Furthermore, if subject detection is performed using only one image, the distance measurement result for the one image where subject detection was performed may be used to perform AF operation.

[0084] In this embodiment, a bird is shown as the subject, but the subject is not limited to this and can be widely applied to subjects that can be generally detected. In this embodiment, a representative form has been described, but other forms may also be used.

[0085] (Other embodiments) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0086] According to each embodiment, it is possible to provide an image capture device, an image capture device control method, and a program that can achieve high distance measurement accuracy when controlling two optical systems arranged in parallel with each other. Furthermore, according to each embodiment, it is possible to achieve high distance measurement accuracy even when the distance measurement area is small.

[0087] The disclosure of each embodiment includes the following configurations and methods. (Configuration 1) an image sensor having a plurality of pixels that receive light beams passing through a plurality of pupil partial regions different from one another in the imaging optical system; an acquisition unit that acquires a defocus amount of the imaging optical system based on a pair of signals from the plurality of pixels; a control unit that controls the imaging optical system based on the defocus amount, the imaging optical system includes a first optical system and a second optical system arranged in parallel with the first optical system, the acquisition unit acquires a first defocus amount in a first ranging area at a first image height based on a first optical axis of the first optical system and a second defocus amount in a second ranging area at a second image height based on a second optical axis of the second optical system, The imaging apparatus is characterized in that the control means controls the imaging optical system based on an average defocus amount of the first defocus amount and the second defocus amount. (Configuration 2) 2. The imaging device according to configuration 1, wherein the relative position of the first ranging area with respect to the first optical axis and the relative position of the second ranging area with respect to the second optical axis are equal to each other. (Configuration 3) The imaging device described in configuration 1, characterized in that the relative position of the first ranging area with respect to the first optical axis and the relative position of the second ranging area with respect to the second optical axis are the same subject area detected via the first optical system and the second optical system, respectively. (Configuration 4) The imaging device according to any one of configurations 1 to 3, wherein the control unit determines whether to control the imaging optical system using the first defocus amount or to control the imaging optical system using the average defocus amount. (Configuration 5) The control means When the absolute value of the first defocus amount is greater than a first threshold, the imaging optical system is controlled using the first defocus amount; 5. The imaging apparatus according to configuration 4, wherein when the absolute value of the first defocus amount is smaller than the first threshold, the imaging optical system is controlled using the average defocus amount. (Configuration 6) The control means When an absolute value of a difference between the first defocus amount and the second defocus amount is greater than a second threshold, the imaging optical system is controlled using the first defocus amount; The imaging device according to configuration 4 or 5, characterized in that when the absolute value of the difference between the first defocus amount and the second defocus amount is smaller than the second threshold, the imaging optical system is controlled using the average defocus amount. (Configuration 7) The imaging device described in any one of configurations 4 to 6, wherein the first ranging area when the imaging optical system is controlled using the average defocus amount is smaller than the first ranging area when the imaging optical system is controlled using the first defocus amount. (Configuration 8) The imaging device described in any one of configurations 1 to 7, wherein the control unit acquires the first image height and the second image height by performing pattern matching between a first image in the first ranging area and a second image in the second ranging area when the focus position of the imaging optical system is at a close range. (Configuration 9) The imaging device described in any one of configurations 1 to 8, wherein the control means controls the imaging optical system so that the first optical system and the second optical system perform autofocus operations while maintaining the same focus position. (Method 1) a light receiving step of receiving light beams passing through a plurality of pupil partial regions different from each other in the imaging optical system using a plurality of pixels of the image sensor; an acquisition step of acquiring a defocus amount of the imaging optical system based on a pair of signals from the plurality of pixels; a control step of controlling the imaging optical system based on the defocus amount, the imaging optical system includes a first optical system and a second optical system arranged in parallel with the first optical system, In the obtaining step, a first defocus amount in a first ranging area at a first image height based on a first optical axis of the first optical system and a second defocus amount in a second ranging area at a second image height based on a second optical axis of the second optical system are obtained, A control method for an imaging apparatus, wherein the control step controls the imaging optical system based on an average defocus amount of the first defocus amount and the second defocus amount. (Configuration 10) A program that causes a computer to execute the method for controlling an imaging device according to Method 1.

[0088] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. [Explanation of symbols]

[0089] 100 Camera body (imaging device) 211 Imaging unit (imaging element) 218a Acquisition method 218b Control means 301R Right eye optical system (1st optical system) 301L Left eye optical system (second optical system)

Claims

1. an image sensor having a plurality of pixels that receive light beams passing through a plurality of pupil partial regions different from one another in the imaging optical system; an acquisition unit that acquires a defocus amount of the imaging optical system based on a pair of signals from the plurality of pixels; a control unit that controls the imaging optical system based on the defocus amount, the imaging optical system includes a first optical system and a second optical system arranged in parallel with the first optical system, the acquisition unit acquires a first defocus amount in a first ranging area at a first image height based on a first optical axis of the first optical system and a second defocus amount in a second ranging area at a second image height based on a second optical axis of the second optical system, The imaging apparatus is characterized in that the control means controls the imaging optical system based on an average defocus amount of the first defocus amount and the second defocus amount.

2. 2. The imaging device according to claim 1, wherein the relative position of the first distance measurement area with respect to the first optical axis and the relative position of the second distance measurement area with respect to the second optical axis are equal to each other.

3. 2. The imaging device according to claim 1, wherein the relative position of the first ranging area with respect to the first optical axis and the relative position of the second ranging area with respect to the second optical axis are the same subject area detected via the first optical system and the second optical system, respectively.

4. 4. The imaging device according to claim 1, wherein the control unit determines whether to control the imaging optical system using the first defocus amount or the average defocus amount.

5. The control means When the absolute value of the first defocus amount is greater than a first threshold, the imaging optical system is controlled using the first defocus amount; 5. The imaging apparatus according to claim 4, wherein when the absolute value of the first defocus amount is smaller than the first threshold value, the imaging optical system is controlled using the average defocus amount.

6. The control means When an absolute value of a difference between the first defocus amount and the second defocus amount is greater than a second threshold value, the imaging optical system is controlled using the first defocus amount; 5. The imaging apparatus according to claim 4, wherein when the absolute value of the difference between the first defocus amount and the second defocus amount is smaller than the second threshold value, the imaging optical system is controlled using the average defocus amount.

7. 5. The imaging device according to claim 4, wherein the first ranging area when the imaging optical system is controlled using the average defocus amount is smaller than the first ranging area when the imaging optical system is controlled using the first defocus amount.

8. The imaging device according to any one of claims 1 to 3, characterized in that when the focus position of the imaging optical system is at a close range, the control unit obtains the first image height and the second image height by performing pattern matching between a first image in the first ranging area and a second image in the second ranging area.

9. 4. The imaging device according to claim 1, wherein the control unit controls the imaging optical system so that the first optical system and the second optical system perform autofocus operations while maintaining the same focus position.

10. a light receiving step of receiving light beams passing through a plurality of pupil partial regions different from each other in the imaging optical system using a plurality of pixels of the image sensor; an acquisition step of acquiring a defocus amount of the imaging optical system based on a pair of signals from the plurality of pixels; a control step of controlling the imaging optical system based on the defocus amount, the imaging optical system includes a first optical system and a second optical system arranged in parallel with the first optical system, In the obtaining step, a first defocus amount in a first ranging area at a first image height based on a first optical axis of the first optical system and a second defocus amount in a second ranging area at a second image height based on a second optical axis of the second optical system are obtained, a control step of controlling the imaging optical system based on an average defocus amount of the first defocus amount and the second defocus amount, in the control step;

11. A program causing a computer to execute the method for controlling an imaging device according to claim 10.

Images