Imaging apparatus, control method for imaging apparatus, and program

Abstract

To provide an imaging apparatus that can properly perform in-focus determination using a reference value according to an imaging condition for each of a plurality of optical systems constituting a compound-eye optical system.SOLUTION: An imaging apparatus includes: imaging means (202) that captures a subject image formed by each of a first optical system (110) and a second optical system (120) arranged in parallel with the first optical system, constituting a compound-eye optical system; focus detection means (206) that detects a defocus amount of each of the first optical system and the second optical system regarding the subject image; and evaluation means (208) that evaluates an imaging state regarding the subject image. The evaluation means sets a first reference value that is used to determine an in-focus state of the first optical system and a second reference value that is used to determine an in-focus state of the second optical system according to the imaging state.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The present invention relates to an imaging device, a control method for an imaging device, and a program. [Background technology]

[0002] Patent Document 1 discloses an imaging device that includes an imaging optical system for a left-eye image and an imaging optical system for a right-eye image, and that is capable of performing focus adjustment by independently controlling each imaging optical system. Patent Document 2 discloses an imaging device that includes a first imaging device and a second imaging device, and detects the focus position of a subject based on contrast information output from the first imaging device. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2013-130674 A [Patent Document 2] JP 2014-134697 A Summary of the Invention [Problem to be solved by the invention]

[0004] In the imaging devices disclosed in Patent Documents 1 and 2, various factors tend to cause differences in imaging conditions for each optical system constituting the compound eye optical system. Therefore, even if each optical system detects focus on the same subject, there is a possibility that an appropriate focus determination result will not be obtained.

[0005] Therefore, an object of the present invention is to provide an imaging device that is capable of appropriately performing focus determination by using a reference value according to the imaging conditions for each of the multiple optical systems that make up the compound eye optical system. [Means for solving the problem]

[0006] An imaging device according to one aspect of the present invention has an imaging means for forming a subject image formed by a first optical system constituting a compound eye optical system and a second optical system arranged in parallel to the first optical system, a focus detection means for detecting the amount of defocus of each of the first optical system and the second optical system with respect to the subject image, and an evaluation means for evaluating an imaging state with respect to the subject image, wherein the evaluation means respectively sets a first reference value used for determining a focus state of the first optical system and a second reference value used for determining the focus state of the second optical system according to the imaging state.

[0007] Other objects and features of the present invention will be described in the following embodiments. Effect of the Invention

[0008] According to the present invention, it is possible to provide an imaging device capable of appropriately performing focus determination by using a reference value according to the imaging conditions for each of the multiple optical systems that make up the compound eye optical system. [Brief description of the drawings]

[0009] [Figure 1] FIG. 2 is a configuration diagram of an imaging system in each embodiment. [Diagram 2] 5A and 5B are explanatory diagrams of an image circle formed on an imaging surface of an imaging unit in each embodiment. [Diagram 3] 3 is a pixel array diagram of an imaging unit in each embodiment. FIG. [Figure 4] 3 is a diagram showing a pixel structure of an imaging unit in each embodiment. FIG. [Diagram 5] 5 is an explanatory diagram of a phase difference between an A image signal and a B image signal in a focused state in each embodiment. FIG. [Figure 6] 5 is an explanatory diagram of the phase difference between the A and B image signals in a non-focused state in each embodiment. FIG. [Figure 7] 5 is a flowchart showing an operation for determining a focus determination criterion in the first embodiment. [Figure 8] 10 is a flowchart showing an operation for determining a focus determination criterion in the second embodiment. [Figure 9(A)] 13 is a flowchart showing an operation of determining a focus determination criterion in the third embodiment. [Figure 9(B)] 13 is a flowchart showing an operation of determining a focus determination criterion in the third embodiment. [Figure 9(C)] 13 is a flowchart showing an operation of determining a focus determination criterion in the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

[0011] As described below, the imaging device (imaging system) of each embodiment has a compound eye optical system including a first optical system and a second optical system arranged in parallel with the first optical system, and has an imaging means for forming an object image formed by each of the first optical system and the second optical system. In a conventional compound eye optical system, differences in imaging conditions of each optical system are likely to occur due to various factors, and even if each optical system tries to detect the focus of the same object, it is possible that the same focus detection result cannot be obtained in all optical systems.

[0012] For example, as the object of interest moves away from the intermediate position of each optical system, the object distance of each optical system is likely to differ, and parallax is likely to occur. Also, the closer the object distance, the more likely parallax is to occur. If there is parallax, the appearance of the object that is focus detected will differ, and as a result, differences will occur in the focus detection results of each optical system. If the focus position is controlled based on the focus detection results, differences will easily occur in the degree of focus of each optical system.

[0013] Also, for example, when shooting with a fisheye lens with a short focal length, the outdoor sky or the indoor ceiling is likely to be mixed into the imaging area, and high-brightness areas such as the sun and its reflected light, or indoor lighting are likely to occur in the imaging area. In particular, when shooting with a compound eye optical system for virtual reality (VR) shooting, reflected light is likely to occur only in some optical systems, and high-brightness areas are likely to occur in the imaging area. As a result, differences occur in the focus detection results of each optical system, and when the focus position is controlled based on the focus detection results, differences in the focus degree of each optical system are likely to occur.

[0014] In addition, for example, when there is another object in front of the main subject that acts as an obstruction, parallax occurs in the images obtained by each optical system, so the degree of obstruction by the other object differs for each optical system, resulting in differences in the focus detection results of each optical system. In this case, if the focus position is controlled based on these focus detection results, while some optical systems can focus, other optical systems may be out of focus due to the obstruction, focus on the other object, or result in false focus due to perspective competition between the main subject and the other object.

[0015] In addition, the influence of the other object acting as an obstruction may include the obstruction of the main subject by the other object itself, and the appearance of the shadow of the other object may differ in each optical system, and low brightness areas due to the shadow may occur in the imaging area only in some optical systems. As a result, differences occur in the focus detection accuracy of each optical system, and when the focus position is controlled based on the focus detection result, differences in the focus degree of each optical system are likely to occur.

[0016] As described above, in a compound eye optical system, some optical systems may perform erroneous focus detection. However, when focus detection is performed in each optical system of the compound eye optical system and focus determination is performed based on the same determination criteria for all optical systems, if a focus notification such as a focus sound or a focus display on the display unit is performed, the subject can only be focused when shooting under favorable conditions where the above-mentioned problems do not occur. In this case, when the above-mentioned problems occur in the subject, some optical systems may not reach the focus determination, making it impossible to move on to shooting operations, or there may be a possibility that the focus control may continue to be waited for completion because the focus notification is not given.

[0017] The main object of each embodiment is to solve such problems. Each embodiment will be described in detail below.

[0018] (First embodiment) <Image capture system configuration> First, an imaging system (digital camera system) 10 according to a first embodiment of the present invention will be described with reference to Fig. 1. Fig. 1 is a configuration diagram of the imaging system 10.

[0019] The imaging system 10 includes a lens device 100 having a twin-lens optical system as an example of a compound-eye optical system in which a plurality of different optical systems (imaging optical systems) are provided in one lens barrel, and a camera body (imaging device) 200. The lens device 100 is detachably attached to the camera body 200 via a mount unit 101, which is a mechanical lens mounting mechanism. The mount unit 101 is provided with an electrical contact unit 102. The electrical contact unit 102 has terminals for communication bus lines such as a communication clock line, a data communication line, and a data receiving line. These enable communication between the lens device 100 and the camera body 200. Note that this embodiment can also be applied to an imaging device in which the camera body and the lens device are integrally configured.

[0020] The lens device 100 has a left optical system (first optical system) 110 and a right optical system (second optical system) 120 arranged in parallel with the left optical system 110. The left optical system 110 and the right optical system 120 form a compound eye optical system. The left optical system 110 and the right optical system 120 each include a plurality of lenses including a focus lens, and a prism or a reflector that bends incident light. The left optical system 110 and the right optical system 120 each have a left aperture (first aperture diaphragm) 111 and a right aperture (second aperture diaphragm) 121 that limit the incident light. The left optical system 110 and the right optical system 120 are symmetrically arranged with members having the same optical properties so that their optical axes are parallel to each other at a predetermined base length. The left optical system 110 and the right optical system 120 are full-circle fisheye lenses, and each has a hemispherical wide viewing angle of approximately 180 degrees on the front surface.

[0021] The lens device 100 has a left lens drive unit 112 that includes a drive system (not shown) consisting of an ultrasonic motor, a stepping motor, a DC motor, or the like for controlling the focus of the left optical system 110 and controls the drive system. Similarly, the lens device 100 has a right lens drive unit 122 that controls the focus of the right optical system 120. The focus of the left optical system 110 is adjusted by the left lens drive unit 112 and its drive system. Similarly, the focus of the right optical system 120 is adjusted by the right lens drive unit 122 and its drive system.

[0022] The lens device 100 has a left aperture drive unit 113 that controls the aperture of the left aperture 111. Similarly, the lens device 100 has a right aperture drive unit 123 that controls the aperture of the right aperture 121. In this embodiment, the left optical system 110 and the right optical system 120 each function as a plurality of different imaging optical systems.

[0023] The lens device 100 has a drive characteristic information recording unit 104. The drive characteristic information recording unit 104 records various optical design values ​​such as the focus lenses of the left optical system 110 and the right optical system 120, the left aperture 111 and the right aperture 121, and the like, and has information on the subject distance that roughly corresponds to the focal length and the focal position. The drive characteristic information recording unit 104 also records focus sensitivity information based on optical characteristics, which is made up of coefficients or a mathematical expression including coefficients, and indicates the correspondence between the image plane movement amount determined by the defocus amount and the actual drive amount of the focus lens, defined for each focal position.

[0024] The left lens driving unit 112 , the left aperture driving unit 113 , the right lens driving unit 122 , the right aperture driving unit 123 , and the driving characteristic information recording unit 104 are connected to a lens controller 103 including a CPU that controls the overall operation of the lens apparatus 100 .

[0025] The lens device 100 has a left lens position detection unit 114 that acquires a phase waveform of the drive system included in the left lens drive unit 112 from the left lens drive unit 112 via the lens controller 103 and detects position information of the left optical system 110. The lens device 100 also has a right lens position detection unit 124 that acquires a phase waveform of the drive system included in the right lens drive unit 122 from the right lens drive unit 122 via the lens controller 103 and detects position information of the right optical system 120.

[0026] The camera body 200 has a camera controller 201 including a CPU that controls the overall operation of the camera body 200. The camera body 200 communicates with the lens apparatus 100 via the electrical contact unit 102, transmits control requests regarding the focus of the left optical system 110 and the right optical system 120, and receives the control results. The camera body 200 also transmits control requests regarding the openings of the left diaphragm 111 and the right diaphragm 121, and receives the control results.

[0027] The incident light is guided to an imaging unit 202 including an image sensor, CDS, ADC, and a processor for performing development calculations (not shown) in the camera body 200 from an optical path via the left optical system 110 and the left aperture 111, and an optical path via the right optical system 120 and the right aperture 121. The imaging unit (imaging means) 202 obtains compound eye imaging data by photoelectrically converting the incident subject image and performing development calculations. That is, the imaging unit 202 forms subject images formed by the left optical system 110 and the right optical system 120, respectively, and obtains imaging data corresponding to the subject image formed by the left optical system 110 and imaging data corresponding to the subject image formed by the right optical system 120.

[0028] Here, the image circle on the imaging surface of the imaging unit 202 will be described with reference to FIG. 2. FIG. 2 is an explanatory diagram of image circles IC_L and IC_R of effective angles of view formed by the left optical system 110 and the right optical system 120, respectively, when the imaging unit 202 side is viewed from the subject side on the imaging surface of the imaging unit 202. In this embodiment, each optical system is a full-circle fisheye lens having a hemispherical wide viewing angle of approximately 180 degrees, and a circular image of each optical system is formed on the imaging surface. As shown in FIG. 2, the image circles IC_L and IC_R formed by the left optical system 110 and the right optical system 120 on the imaging surface of the imaging unit 202 are arranged side by side so as not to overlap each other. On the imaging surface of the imaging unit 202, optical images having parallax are formed side by side by the compound eye optical system. With such a configuration, each optical image of the compound eye optical system can be acquired simultaneously.

[0029] In FIG. 1, the camera body 200 has an operation unit 203 for inputting various operations to the camera body 200, and a display unit 204 for displaying an image of a subject captured by the imaging unit 202 and various operation statuses. The operation unit 203 is composed of a two-step type switch and a plurality of buttons (not shown). The first step (SW1) of the two-step type switch is a switch for starting a shooting preparation operation such as photometry using an imaging signal, focus detection, and focus control. The second step switch (SW2) is a switch for starting an imaging operation such as charge accumulation and charge readout in the imaging unit 202 to obtain a still image. When SW2 is pressed down in one go, a shooting preparation operation such as photometry and focus detection performed by SW1 is performed, and then a still image is obtained. In addition, by continuing to press SW2, the shooting preparation operation such as photometry and focus detection performed by SW1 and the acquisition of a still image by SW2 are repeated at a predetermined cycle, thereby realizing continuous shooting of still images.

[0030] The display unit 204 is composed of a display member such as a liquid crystal panel or an organic EL panel, and a microcomputer and VRAM (not shown) for displaying digital images. The operation unit 203 also includes a plurality of buttons (not shown) for receiving various operational inputs for the camera body 200, such as an operation input for turning the power on and off, for different purposes.

[0031] In this embodiment, when the power supply of the camera body 200 is turned on, the imaging unit 202 starts periodic imaging, for example, at 60 fps, and the captured subject image is developed and calculated and displayed on the display unit 204. In this embodiment, the display unit 204 has a configuration in which two optical images as shown in FIG. 2 are displayed on one display unit 204. However, this embodiment is not limited to this, and a configuration in which a substantially corresponding partial area is cut out from each subject image of the compound eye optical system and displayed on an external head mounted display having different display units for the left eye and the right eye is also possible. When an operation input is made by operating SW2 of the operation unit 203, the imaging unit 202 converts the captured still image into a predetermined known data format, and records it in the recording unit 205.

[0032] The imaging unit 202 has an imaging element (not shown) having a plurality of pixels each having a photoelectric conversion unit, and a CDS (not shown) that subtracts noise charges from charges accumulated in the imaging element. The imaging unit 202 also has an ADC (A / D conversion unit) (not shown) that converts an analog electrical signal (image signal) obtained by sequentially transferring charges output from the CDS for each pixel into digital image data. The imaging element may be, for example, a CCD (Charge Coupled Device) or CMOS (Complementary Metal-Oxide-Semiconductor) image sensor. The CDS and ADC (not shown) are not limited to being independent of the imaging element as described above, and may be incorporated in the imaging element. At least some of the pixels of the imaging element have a configuration capable of outputting a pair of pupil-split image signals as digital image data for focus detection. The pair of image data is transmitted to a focus detection unit (focus detection means) 206 connected to the camera controller 201.

[0033] The focus detection unit 206 performs a correlation calculation on a pair of image data for focus detection obtained by the imaging unit 202, and calculates a correlation amount for each phase shift amount. The focus detection unit 206 calculates a defocus amount of the subject image by a focus detection calculation using a known phase difference detection method, based on the phase shift amount that maximizes the correlation amount and the optical characteristics of the lens device 100. In this manner, the focus detection unit 206 detects the defocus amount of each optical system related to the subject image formed by each of the left optical system 110 and the right optical system 120. The pair of image data for focus detection obtained by the imaging unit 202 will be described in detail later.

[0034] The focus control unit 207 controls the focal positions of the compound eye optical systems (left optical system 110 and right optical system 120) so that subject images formed by each of the compound eye optical systems are focused on the imaging unit 202. Specifically, the focus control unit 207 acquires the current focal positions of the focus lenses (not shown) included in each optical system from the left lens position detection unit 114 and the right lens position detection unit 124. The focus control unit 207 also acquires, from the drive characteristic information recording unit 104, focus sensitivity information for calculating the displacement amount of the focus lens corresponding to the defocus amount according to the focal position.

[0035] The focus control unit 207 also acquires the defocus amount of each optical system from the focus detection unit 206, and calculates the target drive amount of the focus lens of each optical system using the focus sensitivity information. The focus control unit 207 also issues a focus control command to the left lens drive unit 112 and the right lens drive unit 122 to drive the focus lens by the calculated target drive amount of each optical system, thereby moving the focus position of each optical system to the in-focus position.

[0036] The imaging evaluation unit (evaluation means) 208 evaluates the imaging situation (imaging state) of each subject image of the compound eye optical system. Specifically, the imaging evaluation unit 208 has a focus evaluation mode that controls whether to perform focus determination for all of the subject images of the compound eye optical system or for only some of them. The imaging evaluation unit 208 selects and sets the focus evaluation mode according to an operation input from the operation unit 203. The focus evaluation mode includes an "all optical system focus evaluation mode" in which focus determination is performed for the entire optical system, and a "partial optical system focus evaluation mode" in which focus determination is substantially performed for only some of the compound eye optical system. In the initial setting, the all optical system focus evaluation mode is selected in the imaging evaluation unit 208.

[0037] The imaging evaluation unit 208 also accepts evaluation settings for selecting the various evaluation viewpoints described above according to operation input from the operation unit 203 by the user at any time while the camera body 200 is powered on. The imaging evaluation unit 208 also determines focus determination reference values ​​for each optical system according to the selection status of the focus determination mode described above. The focus determination thresholds as focus determination references (reference values) determined by the imaging evaluation unit 208 include a first focus determination threshold (first reference value) that is basically selected, and a second focus determination threshold (second reference value) that substantially invalidates focus determination.

[0038] The first focus determination threshold is a threshold corresponding to the permissible circle of confusion diameter based on the size of an image sensor (not shown) of the imaging unit 202 and the aperture value of the left aperture 111 and the right aperture 121, and is defined by the following formula (1).

[0039] T1 = k1 × F × δ (1) however, T1: First focus determination threshold k1: any coefficient (e.g., value 0.5) F: Aperture value of each optical system δ: Permissible circle of confusion diameter The second focus determination threshold is expressed by, for example, a value ∞ (infinity) or a maximum value that can be set. Note that in this embodiment, the focus determination threshold is not limited to the above-mentioned two types, and the number of types may be further changed. Each focus determination threshold is a threshold expressed by an absolute value of a defocus amount. The focus determination threshold is compared with the defocus amount detected by the focus detection unit 206, and whether or not the focus is achieved is determined based on whether or not it is included in a range width indicated by the positive and negative values ​​of each focus determination threshold with either the in-focus position or the focus control position as the center of the range.

[0040] The focus determination unit (focus determination means) 209 determines whether the focal position of each optical system is focused on the subject based on the defocus amount of each optical system detected by the focus detection unit 206 and the focus determination threshold of each compound eye optical system determined by the imaging evaluation unit 208. That is, the focus determination unit 209 determines the focus state of each of the left optical system 110 and the right optical system 120. As described later, when the defocus amount of the left optical system 110 is within a first range set using a first reference value (for example, a first focus determination threshold), the focus determination unit 209 determines that the left optical system 110 is in a focused state. Also, when the defocus amount of the right optical system 120 is within a second range set using a second reference value (for example, a second focus determination threshold), the focus determination unit 209 determines that the right optical system 120 is in a focused state.

[0041] When the focus determination unit 209 determines that each optical system is in focus on the subject, the notification unit (notification means) 210 notifies the user that focus has been achieved by a predetermined sound or display. The display at the time of focus determination is a rectangular frame indicating the focus detection area superimposed in a predetermined color on the imaging data displayed on the display unit 204. The notification unit 210 also has a built-in speaker (not shown), and when the focus determination unit 209 determines that each optical system is in focus on the subject, the notification unit 210 emits a predetermined short tone.

[0042] <Configuration of imaging unit 202> Next, the configuration of the imaging unit 202 will be described with reference to Fig. 3 and Figs. 4(a) and (b). Fig. 3 is a pixel array diagram of the imaging unit 202, and shows a schematic diagram of the light receiving surface of the imaging unit 202 as viewed from the incident light side. Fig. 4 is a pixel structure diagram of the imaging unit 202.

[0043] Each of the pixels provided in the imaging unit 202 has a first focus detection pixel A and a second focus detection pixel B onto which a pair of light rays split by the exit pupil plane are respectively incident. A light beam that passes through a first pupil region of the exit pupil is incident on the first focus detection pixel A, and a light beam that passes through a second pupil region of the exit pupil is incident on the second focus detection pixel B. Microlenses ML for collecting light are arranged in front of the first focus detection pixel A and the second focus detection pixel B. Pixels each consisting of a set of the first focus detection pixel A, the second focus detection pixel B, and a microlens ML are arranged in a number of h pixels horizontally and v pixels vertically on the light receiving surface of the image sensor.

[0044] The charges from the first focus detection pixel A and the second focus detection pixel B can be used for the image signal for focus detection. Adding the charges from the first focus detection pixel A and the second focus detection pixel B results in a charge as imaging pixel C as shown in FIG. 4(b), which can be used for the signal of the captured image. A light beam that has passed through the same microlens ML is incident on the first focus detection pixel A and the second focus detection pixel B. Each pixel has a red, green, or blue color filter (not shown), and the color filters are arranged in a known Bayer array.

[0045] Microlenses ML are formed on the light incident sides of the first focus detection pixel A and the second focus detection pixel B. The smooth layer 301 is a flat surface for forming the microlenses ML. Light blocking layers 302a and 302b are arranged to prevent light rays from being incident at unnecessary oblique angles on the first focus detection pixel A and the second focus detection pixel B.

[0046] 4(a) and (b) is equivalent to the pupil of the imaging optical system being divided symmetrically when viewed from the first focus detection pixel A and when viewed from the second focus detection pixel B. Because the position of the pupil of the imaging optical system is different when viewed from the first focus detection pixel A and when viewed from the second focus detection pixel B, there is parallax between the light beam received by the first focus detection pixel A and the light beam received by the second focus detection pixel B.

[0047] <Principle of focus detection using image plane phase difference method> In a pair of a row of a plurality of first focus detection pixels A (first focus detection pixel row) and a row of a plurality of second focus detection pixels B (second focus detection pixel row), two images that are closer to each other are formed as the number of pixels in the image sensor increases. When the imaging optical system is in focus on the subject, a row of image signals (A image signal) formed by the first focus detection pixel row and a row of image signals (B image signal) formed by the second focus detection pixel row match each other. In contrast, if the imaging optical system is out of focus, a phase difference occurs between the A image signal and the B image signal. The direction of the phase difference is reversed between a front-focus state in which the imaging position is closer to the intended focal plane and a back-focus state in which the imaging position is farther from the intended focal plane.

[0048] Fig. 5 is an explanatory diagram of the phase difference between the A and B image signals in a focused state at a certain pixel. Fig. 6 is an explanatory diagram of the phase difference between the A and B image signals in a defocused state before focusing at a certain pixel. In Figs. 5 and 6, the first focus detection pixel A is conceptually represented as A and the second focus detection pixel B as B. A light beam from a specific point on the subject is split into a light beam ΦLa that passes through a split pupil corresponding to the first focus detection pixel A and is incident on the first focus detection pixel A, and a light beam ΦLb that passes through a split pupil corresponding to the second focus detection pixel B and is incident on the second focus detection pixel B.

[0049] These two light beams are incident on the same point on the subject, and therefore when the imaging optical system is in focus, they pass through the same microlens and reach a single point on the image sensor consisting of a pair of first focus detection pixel A and second focus detection pixel B, as shown in Fig. 5. Therefore, the pair of image signals A and B match each other.

[0050] On the other hand, as shown in Fig. 6, when the focus is shifted by Δx in the optical axis direction, the reach of both light beams ΦLa and ΦLb is shifted from each other by the change in the angle of incidence of the light beams ΦLa and ΦLb on the microlens. Fig. 6 shows a state in which the reach of both light beams ΦLa and ΦLb is shifted by Δy in comparison with a state in which the focus is shifted by Δx in the optical axis direction. Therefore, a phase difference occurs between the pair of image signals A and B. By calculating the correlation amount for each image shift amount for the pair of image signals A and B, and calculating the phase difference based on the correlation amount, focus detection using the phase difference method can be performed.

[0051] <Operation of imaging system> Next, with reference to Fig. 7, the operation of determining the focus determination criterion (reference value) for each subject image of the compound eye optical system (left optical system 110 and right optical system 120) in the imaging system 10 of this embodiment will be described. Fig. 7 is a flowchart showing the operation of determining the focus determination criterion. This operation is realized by the camera controller 201 controlling each unit, and unless the source of the operation is specifically specified, various operations are performed by the camera controller 201.

[0052] In the imaging system 10 of this embodiment, after transitioning to a state in which various operational inputs of the operation unit 203 can be received, the power is turned on and each subject image of the compound eye optical system captured by the imaging unit 202 is displayed on the display unit 204. Then, SW1 is pressed to request a shooting preparation operation such as a focus detection operation, and this operation starts and the process proceeds to step S701. In addition, by turning on the power, the camera controller 201 confirms that the lens device 100 is attached to the camera body 200 from the energized state of the electrical contact unit 102 provided in the mount unit 101, and recognizes a state in which focus control and aperture control are possible.

[0053] First, in step S701, the focus detection unit 206 detects the defocus amount for a predetermined focus detection area of ​​each object image of the compound eye optical system. The focus detection area is a predetermined area near the center of each object image, or an imaging area specified by a user's operation input via the operation unit 203. In addition, the focus detection in this step is performed for each object image of the compound eye optical system, but is not limited to this. In this step, a simplified form in which focus detection is performed only for object images of some optical systems may be used. The focus detection unit 206 outputs the detected defocus amount of each optical system to the focus control unit 207 and the imaging evaluation unit 208. After outputting the defocus amount, the process proceeds to step S702.

[0054] In step S702, the focus control unit 207 commands each of the compound eye optical systems to perform focus control based on the defocus amount of each optical system detected by the focus detection unit 206. Specifically, the focus control unit 207 calculates a target driving amount of each focus lens of the left optical system 110 and the right optical system 120, and commands the left lens driving unit 112 and the right lens driving unit 122 to perform focus control. Then, the focus control unit 207 receives callback communication of completion of focus control from the left lens driving unit 112 and the right lens driving unit 122 through the left lens driving unit 112 and the right lens driving unit 122. After the focus control unit 207 receives the completion of focus control, the process proceeds to step S703.

[0055] In step S703, the focus detection unit 206 performs the same defocus amount detection operation as in step S701 to check the degree of focus after the focus control in the previous step. Note that the focus detection in this step is performed for each subject image of each optical system. After outputting the detected defocus amount of each optical system to the focus control unit 207 and the image capture evaluation unit 208, the process proceeds to step S704.

[0056] In step S704, the camera controller 201 determines whether the focus evaluation mode set by the imaging evaluation unit 208 is the all-optical system focus evaluation mode, which performs focus determination based on focus detection results of all optical systems in the compound eye optical system, among the two modes. If it is determined that the focus evaluation mode is the all-optical system focus evaluation mode, the process proceeds to step S705. On the other hand, if it is determined that the focus evaluation mode is the partial optical system focus evaluation mode, which performs focus determination based substantially only on focus detection results of some optical systems in the compound eye optical system, the process proceeds to step S706.

[0057] In step S705, the imaging evaluation unit 208 sets a focus determination threshold as a focus determination criterion according to the focus evaluation mode. This step is performed when the all-optical system focus evaluation mode is set, and the imaging evaluation unit 208 sets all focus determination thresholds of the compound eye optical system, that is, the focus determination thresholds of each subject image of the left optical system 110 and the right optical system 120, to the first focus determination threshold. After setting the focus determination threshold, the process proceeds to step S709.

[0058] In step S706, the imaging evaluation unit 208 compares the defocus amount of each subject image in all optical systems of the compound eye optical system detected by the focus detection unit 206, and determines the optical system with the smallest defocus amount. That is, in this step, the imaging evaluation unit 208 compares the defocus amount of each subject image of the left optical system 110 and the right optical system 120 to determine which is smaller. If the defocus amount of the subject image of the left optical system 110 is smaller, the process proceeds to step S707. On the other hand, if the defocus amount of the subject image of the right optical system 120 is smaller, the process proceeds to step S708.

[0059] Steps S707 and S708 are processing steps in which, when the partial optical system focus evaluation mode is set, the imaging evaluation unit 208 sets a focus determination threshold as a focus determination criterion according to the focus evaluation mode.

[0060] In step S707, the image capturing evaluation unit 208 sets the focus determination threshold so that focus determination is performed only for the subject image of the left optical system 110 in which the defocus amount is detected to be smaller. Specifically, the image capturing evaluation unit 208 sets the focus determination threshold of the subject image of the left optical system 110 in which the defocus amount is detected to be smaller and the focal position is closer to the focus to a first focus determination threshold (first reference value) that is basically selected. In addition, the image capturing evaluation unit 208 sets the focus determination threshold of the subject image of the right optical system 120, which is not closer to the focus than the left optical system 110, to a second focus determination threshold (second reference value) that substantially invalidates the focus determination. The second focus determination threshold is, for example, a value of ∞ or a maximum value that can be set. By setting the focus determination threshold in this way, even if a difference occurs in the focus degree in each optical system due to various parallaxes, the above-mentioned difference in the focus degree can be allowed by manual setting by the user. After setting the focus determination threshold, the process proceeds to step S709.

[0061] In step S708, the image capturing evaluation unit 208 sets the focus determination threshold so that focus determination is performed only for the subject image of the right optical system 120 in which the defocus amount is detected to be smaller. Specifically, the image capturing evaluation unit 208 sets the focus determination threshold of the subject image of the right optical system 120 in which the defocus amount is detected to be smaller and the focal position is closer to in-focus to the first focus determination threshold. In addition, the image capturing evaluation unit 208 sets the focus determination threshold of the subject image of the left optical system 110, which is not closer to in-focus than the right optical system 120, to the second focus determination threshold that substantially invalidates the focus determination. By setting the focus determination threshold in this manner, even if a difference in the focus degree occurs in each optical system due to various parallaxes, the above-mentioned difference in the focus degree can be allowed by the user's manual setting, as in step S707. After setting the focus determination threshold, the process proceeds to step S709.

[0062] In step S709, the focus determination unit 209 determines a focus determination range (first range or second range) based on each focus determination threshold (first reference value or second reference value) as a focus determination criterion for each optical system of the compound eye optical system. Specifically, the focus determination unit 209 determines the width of the range indicated by the positive and negative values ​​of the focus determination threshold as the focus determination range, with the focus position based on the focus detection result of the focus detection unit 206 as the center of the range. After the focus determination range is determined, the process proceeds to step S710.

[0063] In step S710, the focus determination unit 209 determines whether or not the focus position of each optical system after focus control detected in step S703 is within the focus determination range determined in step S709. If the defocus amounts of all optical systems subject to focus detection, that is, the left optical system 110 and the right optical system 120, are within the focus determination ranges, the process proceeds to step S711. On the other hand, if the defocus amount of at least one of the optical systems is not within the focus determination range, the process ends. In step S711, the notification unit 210 notifies the user that focus has been achieved by a predetermined sound or display, and ends the process.

[0064] By performing the operation of determining the focus determination standard of this embodiment, even if the focus degree of each subject image of the compound eye optical system differs due to the influence of parallax, the user can change the setting of the focus evaluation mode to set the focus notification according to the user's intention. In addition, since the focus notification is performed according to the user's intention, it becomes easy for the user to perform the shooting operation by operating SW2.

[0065] <Modification> In this embodiment, when the partial optical system focus evaluation mode is set in steps S707 and S708, a second focus determination threshold that substantially invalidates the focus determination is set as the focus determination threshold for the optical system with the larger detected defocus amount. This setting operation can be performed so that the optical system with the larger detected defocus amount does not affect the focus notification. However, this embodiment is not limited to this, and a different operation form may be adopted as long as it is a processing operation that does not affect the focus notification by the optical system with the larger detected defocus amount.

[0066] Second embodiment Next, a second embodiment of the present invention will be described. In the first embodiment, the focus determination range is set based on each focus determination threshold of the compound eye optical system with the focus position based on the focus detection result of the focus detection unit 206 as the center of the range, and the focus determination is performed. On the other hand, in this embodiment, the focus determination range of the other optical systems is set with the focus position after focus control in the optical system with the best focus degree after focus control as the center of the range, with the range width based on each focus determination threshold of the compound eye optical system. By setting the focus determination range in this way, even if the focus position is slightly out of the focus determination range when the focus position based on the focus detection result is set as the center of the range, it is possible to determine that the focus is achieved, even if the focus positions after focus control of each optical system are close to each other. By such focus determination, it is possible to easily obtain a unified focus determination when the focus positions after focus control of each optical system are close to each other.

[0067] The basic configuration of the imaging system in this embodiment is the same as that of the imaging system 10 in the first embodiment described with reference to Fig. 1. However, the operation of the imaging evaluation unit (evaluation means) 208 is such that a function to be described later is added to the first embodiment.

[0068] In the imaging evaluation unit 208 in this embodiment, in addition to the first and second focus determination thresholds described in the imaging evaluation unit 208 in the first embodiment, a third focus determination threshold is provided as a focus determination threshold that is a looser standard than the first focus determination threshold. The third focus determination threshold, like the first focus determination threshold, is a threshold according to the permissible circle of confusion diameter based on the size of an image sensor (not shown) of the imaging unit 202 and the aperture value of the left aperture 111 or the right aperture 121, and is defined by the following formula (2).

[0069] T3 = k3 × F × δ (2) however, T3: Third focus threshold k3: any coefficient greater than k1 (e.g., value 1.0) F: Aperture value of each optical system δ: Permissible circle of confusion diameter <Operation of imaging system> With reference to Fig. 8, the operation of determining the focus determination standard for each subject image of the compound eye optical system in the imaging system 10 of this embodiment will be described. Fig. 8 is a flowchart showing the operation of determining the focus determination standard in this embodiment. This operation is realized by the camera controller 201 controlling each unit, and various operations are performed by the camera controller 201 unless the source of the operation is specifically specified.

[0070] After the imaging system 10 of this embodiment transitions to a state in which various operational inputs from the operation unit 203 can be received, the power is turned on and the subject image captured by the imaging unit 202 is displayed on the display unit 204. Then, when SW1 is pressed to request a shooting preparation operation such as a focus detection operation, this operation starts and the process proceeds to step S801. Furthermore, when the power is turned on, the camera controller 201 confirms that the lens device 100 is attached to the camera body 200, and recognizes a state in which focus control and aperture control are possible.

[0071] First, in step S801, the focus detection unit 206 detects the defocus amount for a predetermined focus detection area of ​​each subject image of the compound eye optical system. The focus detection area is a predetermined area near the center of each subject image or an imaging area specified by a user's operation input via the operation unit 203. The focus detection unit 206 outputs the detected defocus amount of each optical system to the focus control unit 207 and the imaging evaluation unit 208. After outputting the defocus amount, the process proceeds to step S802.

[0072] In step S802, the focus control unit 207 issues a focus control command to each of the compound eye optical systems based on the detected defocus amount of each optical system. Specifically, the focus control unit 207 calculates a target drive amount of each focus lens of the left optical system 110 and the right optical system 120, and issues a focus control command to the left lens drive unit 112 and the right lens drive unit 122. Then, the focus control unit 207 receives a callback communication of the completion of the focus control from the left lens drive unit 112 and the right lens drive unit 122 through the left lens drive unit 112 and the right lens drive unit 122. After the focus control unit 207 receives the completion of the focus control, the process proceeds to step S803.

[0073] In step S803, the focus detection unit 206 performs the same defocus amount detection operation as in step S801 to check the degree of focus after the focus control in the previous step. Also, the focus detection in this step is performed for each subject image of each optical system. After outputting the detected defocus amount of each optical system to the focus control unit 207 and the image capture evaluation unit 208, the process proceeds to step S804.

[0074] In step S804, the camera controller 201 determines whether the focus evaluation mode set by the imaging evaluation unit 208 is the all-optical system focus evaluation mode, which performs focus determination based on focus detection results of all optical systems in the compound eye optical system, among the two modes. If it is determined that the focus evaluation mode is the all-optical system focus evaluation mode, the process proceeds to step S805. On the other hand, if it is determined that the focus evaluation mode is the partial optical system focus evaluation mode, which performs focus determination based substantially only on focus detection results of some optical systems in the compound eye optical system, the process proceeds to step S808.

[0075] In step S805, the imaging evaluation unit 208 sets the optical system with the smallest detected defocus amount as the reference optical system, and determines the in-focus position in the reference optical system as the reference focal position. After determining the reference focal position, the process proceeds to step S806.

[0076] In step S806, the imaging evaluation unit 208 sets a focus determination threshold as a focus determination criterion according to the focus evaluation mode. This step is performed when the all-optical system focus evaluation mode is set, and the imaging evaluation unit 208 sets different focus determination thresholds for the reference optical system determined in step S805 and the other optical systems. Specifically, the imaging evaluation unit 208 sets the focus determination threshold of the reference optical system to a first focus determination threshold. In addition, the imaging evaluation unit 208 sets the focus determination thresholds of the other optical systems, excluding the reference optical system, among the compound eye optical systems to a third focus determination threshold that is larger than the first focus determination threshold and has a looser criterion. After setting the focus determination threshold, the process proceeds to step S807.

[0077] In step S807, the focus determination unit 209 determines a focus determination range based on the focus determination threshold of the subject image of each optical system set by the imaging evaluation unit 208. Specifically, the focus determination unit 209 determines the width of a range indicated by the positive and negative values ​​of the focus determination threshold of the reference optical system, with the focus position based on the focus detection result of the focus detection unit 206 as the focus determination range of the reference optical system with the smallest detected defocus amount. In addition, the focus determination unit 209 determines the width of a range indicated by the positive and negative values ​​of the focus determination threshold of the other optical systems, with the focus position in the reference optical system, which is the control result of the focus control unit 207, as the focus determination range of the other optical systems except the reference optical system, as the center of the range. After determining the width of the range, the process proceeds to step S810.

[0078] In step S808, the imaging evaluation unit 208 sets a focus judgment threshold as a focus judgment criterion according to the focus evaluation mode. This step is performed when the partial optical system focus evaluation mode is set, and the imaging evaluation unit 208 sets the focus judgment threshold of a part of the compound eye optical system, that is, the focus judgment threshold of each subject image of either the left optical system 110 or the right optical system 120, to the first focus judgment threshold. The imaging evaluation unit 208 also sets the focus judgment threshold of the other optical system to the second focus judgment threshold that substantially invalidates the focus judgment. After setting the focus judgment threshold, the process proceeds to step S809.

[0079] In step S809, the focus determination unit 209 determines a focus determination range based on the focus determination threshold of the subject image of each optical system set by the imaging evaluation unit 208. Specifically, the focus determination unit 209 determines, as the focus determination range, the width of a range indicated by positive and negative values ​​of the focus determination threshold, with the focus position based on the focus detection result of the focus detection unit 206 as the center of the range. After the focus determination range is determined, the process proceeds to step S810.

[0080] In step S810, the focus determination unit 209 determines whether or not the defocus amounts of the subject images detected in the predetermined focus detection areas of the compound eye optical system are within the focus determination ranges of the respective optical systems based on the focus determination ranges determined by the focus determination unit 209. If all of the defocus amounts are within the focus determination ranges, the process proceeds to step S811. On the other hand, if at least one of the defocus amounts is not within the focus determination range, the process ends. In step S811, the notification unit 210 notifies the user that focus has been achieved by a predetermined sound or display, and ends the process.

[0081] By the operation of determining the in-focus determination standard of this embodiment, when the focus positions after focus control of each optical system are close to each other, even if the focus position after focus control is slightly out of the focus determination range when the focus position is set to the center of the range, it can be determined that the focus is achieved. Then, by determining that the focus is achieved, it becomes easier to transition to a focus notification or a subsequent shooting operation.

[0082] <Modification> In this embodiment, in step S805, the imaging evaluation unit 208 determines the optical system with the smallest detected defocus amount as the reference optical system and determines the focus position in the reference optical system as the reference focal position, but this is not limited to this. If SW2 is kept pressed and the defocus amount detection operation for checking the focus degree after focus control is performed multiple times within a predetermined time or a predetermined number of times or more, the imaging evaluation unit 208 may make a different determination.

[0083] Specifically, the imaging evaluation unit 208 calculates a predicted focus position by known approximation calculation such as the least squares method based on not only the most recent defocus amount detection result but also the continuous history of multiple defocus amount detection results obtained individually by each optical system. The imaging evaluation unit 208 may then determine the focus position of the reference optical system as the reference focal position by using the optical system that has obtained the detected defocus amount with the smallest difference from the calculated predicted focus position as the reference optical system. By performing such an operation, it is possible to more accurately determine the focus when focus detection and imaging are continuously repeated for a subject moving in the focus shift direction.

[0084] Third embodiment Next, a third embodiment of the present invention will be described. In each of the above-mentioned embodiments, focus determination is performed based on the focus detection result (imaging state) of each subject image, regardless of the imaging state of each subject image of the compound eye optical system. On the other hand, in this embodiment, the focus determination criterion may be adaptively determined according to the imaging state (imaging state) such as the brightness of each subject image of the compound eye optical system, the subject type, the subject distance, or the presence or absence of occlusion by another subject, taking into account the influence of parallax of the compound eye optical system. By performing such an operation, even if focus detection is hindered in some optical systems of the compound eye optical system due to the influence of parallax, it is possible to appropriately proceed with the focus determination.

[0085] The basic configuration of the imaging system in this embodiment is the same as that of the imaging system 10 in the first embodiment described with reference to Fig. 1. However, the operation of the imaging evaluation unit (evaluation means) 208 is such that an operation function described later is added to the first embodiment.

[0086] Specifically, the imaging evaluation unit 208 evaluates whether an area of ​​each subject image of the compound eye optical system that is equal to or larger than a predetermined area or equal to or larger than a predetermined area ratio is occluded by another subject in order to evaluate the imaging state of each subject image of the compound eye optical system. Alternatively, the imaging evaluation unit 208 evaluates whether each subject image of the compound eye optical system includes a high brightness area equal to or larger than a predetermined level or a low brightness area equal to or smaller than a predetermined level that is equal to or larger than a predetermined area. Alternatively, the imaging evaluation unit 208 evaluates whether a subject in a focus detection area that is a focus detection target is a person's face or pupil, or a vehicle such as a train, a four-wheeled vehicle, or a two-wheeled vehicle, from the feature amount of the subject image using a known subject recognition method.

[0087] Alternatively, the imaging evaluation unit 208 acquires information on each object distance that roughly corresponds based on the focal position of each object image of the compound eye optical system from the driving characteristic information recording unit 104. Then, the imaging evaluation unit 208 evaluates whether the object distance is closer than a predetermined object distance threshold and the parallax of each optical image by the compound eye optical system is greater than a predetermined value. The parallax of each optical image by the compound eye optical system is affected by the base length of each optical system of the compound eye optical system and the object distance. In addition, the imaging evaluation unit 208 determines the focus determination reference value of each optical system according to at least one evaluation result of the above-mentioned evaluation viewpoints. In addition, the imaging evaluation unit 208 of this embodiment has the third focus determination threshold described using formula (2) in the imaging evaluation unit 208 of the second embodiment.

[0088] <Operation of imaging system> With reference to Fig. 9(A) to Fig. 9(C), the operation of determining the focus determination standard for each subject image of the compound eye optical system in the imaging system 10 of this embodiment will be described. Fig. 9(A) to Fig. 9(C) are flowcharts showing the operation of determining the focus determination standard. This operation is realized by the camera controller 201 controlling each component, and each operation is performed by the camera controller 201 unless the source of the operation is specifically specified.

[0089] After the imaging system 10 of this embodiment transitions to a state in which various operational inputs of the operation unit 203 can be received, the power is turned on, the imaging unit 202 captures an image, and the subject image is displayed on the display unit 204. Then, this operation starts when SW1 is pressed to request a shooting preparation operation such as a focus detection operation, and the process proceeds to step S901. Furthermore, when the power is turned on, the camera controller 201 confirms that the lens device 100 is attached to the camera body 200 based on the energized state of the electrical contact unit 102 provided in the mount unit 101, and recognizes a state in which focus control and aperture control are possible.

[0090] First, in step S901, the focus detection unit 206 detects the defocus amount for a predetermined focus detection area of ​​each subject image of the compound eye optical system. The focus detection area is a predetermined area near the center of each subject image, or an imaging area specified by a user's operation input via the operation unit 203. In addition, the focus detection in this step is performed for each subject image of the compound eye optical system, but is not limited to this, and this step may be a simplified form in which focus detection is performed only for subject images of some optical systems. The focus detection unit 206 outputs the detected defocus amount of each optical system to the focus control unit 207 and the imaging evaluation unit 208. After outputting the defocus amount, the process proceeds to step S902.

[0091] In step S902, the focus control unit 207 issues a focus control command to each of the compound eye optical systems based on the detected defocus amount of each optical system. Specifically, the focus control unit 207 calculates a target drive amount of each focus lens of the left optical system 110 and the right optical system 120, and issues a focus control command to the left lens drive unit 112 and the right lens drive unit 122. Then, the focus control unit 207 receives a callback communication of the completion of the focus control from the left lens drive unit 112 and the right lens drive unit 122 through the left lens drive unit 112 and the right lens drive unit 122. After the focus control unit 207 receives the completion of the focus control, the process proceeds to step S903.

[0092] In step S903, the focus detection unit 206 performs the same defocus amount detection operation as in step S901 again to check the degree of focus after the focus control in the previous step. Note that the focus detection in this step is performed for each subject of each optical system. After outputting the detected defocus amount of each optical system to the focus control unit 207 and the image capture evaluation unit 208, the process proceeds to step S904.

[0093] In step S904, the image evaluation unit 208 determines the image area of ​​a representative main subject in each focus detection area of ​​each optical system of the compound eye optical system by a known object recognition method based on feature points of color and brightness. The image evaluation unit 208 also evaluates whether the main subject of each optical system of the compound eye optical system is occluded by another object in an image area having a predetermined area or a predetermined area ratio to the focus detection area. An example of the predetermined area ratio is a value of 30% or 40% of the area of ​​the focus detection area expressed in pixel units. After the evaluation, the process proceeds to step S905.

[0094] In step S905, the camera controller 201 determines whether the focus evaluation mode set by the imaging evaluation unit 208 is the all-optical system focus evaluation mode, which performs focus determination based on focus detection results of all optical systems in the compound eye optical system, among the two modes. If it is determined that the focus evaluation mode is the all-optical system focus evaluation mode, the process proceeds to step S906. On the other hand, if it is determined that the focus evaluation mode is the partial optical system focus evaluation mode, which performs focus determination based substantially only on focus detection results of some optical systems in the compound eye optical system, the process proceeds to step S918.

[0095] In step S906, if the image capturing evaluation unit 208 determines that the condition for blocking by another object in step S905 is satisfied for all optical systems targeted for focus detection, the process proceeds to step S917. If the image capturing evaluation unit 208 determines that the condition for blocking by another object is satisfied for only some of the optical systems targeted for focus detection, the process proceeds to step S916. If there is no other object blocking the main object, the process proceeds to step S907.

[0096] In step S907, the image capturing evaluation unit 208 generates a pixel-by-pixel luminance histogram for each focus detection area of ​​the compound eye optical system for each optical system. In the luminance histogram, the count is divided at least at a boundary between a predetermined high luminance and a predetermined low luminance for determining that the luminance is abnormal. The count may be divided by finer luminance gradations. An example of the predetermined high luminance is a value such as 230 or 240 when the luminance of the digital imaging data is expressed with an 8-bit depth of values ​​0 to 255. Also, an example of the predetermined low luminance is a value such as 20 or 10, while the minimum luminance value based on the dark current of the imaging element is a value of 0 when the luminance of the digital imaging data is expressed with an 8-bit depth of values ​​0 to 255. After the luminance histogram is generated for each optical system, the process proceeds to step S908.

[0097] In step S908, in each focus detection area of ​​the compound eye optical system, it is evaluated whether a high luminance area having a predetermined high luminance level or more or a low luminance area having a predetermined low luminance level or less is included in a predetermined area or a predetermined area ratio or more in pixel units in each optical system. This evaluation is performed by the image capture evaluation unit 208. The predetermined area ratio or the predetermined area has two types of thresholds, a first area ratio threshold or first area threshold and a second area ratio threshold or second area threshold, which are described later, and are provided by the image capture evaluation unit 208. An example of the predetermined area ratio or the predetermined area is a first area ratio threshold having an intermediate value such as 30% or 40% with respect to the area of ​​the focus detection area expressed in pixel units, or a first area threshold having an area equal to this threshold. Also, a second area ratio threshold having an almost extreme value larger than the first area ratio threshold such as almost 100%, or a second area threshold having an area equal to this threshold. If the imaging evaluation unit 208 determines that all optical systems subject to focus detection in the compound eye optical system satisfy the second area threshold or more or the second area ratio or more, the process proceeds to step S917. If the imaging evaluation unit 208 determines that some focus detection regions of the optical systems subject to focus detection satisfy the first area threshold or more and less than the second area threshold or the first area ratio or more and less than the second area ratio, the process proceeds to step S916. In other cases, the process proceeds to step S909.

[0098] In step S909, the imaging evaluation unit 208 acquires information on each roughly corresponding subject distance based on the focal position of each subject image of the compound eye optical system from the driving characteristics information recording unit 104. After acquiring the information, the process proceeds to step S910.

[0099] In step S910, the image capturing evaluation unit 208 evaluates whether the acquired subject distance is closer than a predetermined subject distance threshold and whether the distance condition is such that the parallax of each optical image by the compound eye optical system is greater than a predetermined value. The parallax of each optical image by the compound eye optical system is affected by the base length of each optical system of the compound eye optical system and the subject distance. If the shortest distance among the acquired subject distances of each optical system is closer than the predetermined subject distance threshold, the process proceeds to step S915. On the other hand, if not, the process proceeds to step S911.

[0100] In step S911, the image capturing evaluation unit 208 determines a predetermined object type from the color and brightness features of each object image of the compound eye optical system by a known object recognition method. Specifically, the image capturing evaluation unit 208 determines whether the object to be the focus detection target is a person's face or eye, or a vehicle such as a train, a four-wheeled vehicle, or a two-wheeled vehicle. After determining the object type of each optical system, the process proceeds to step S912.

[0101] In step S912, the imaging evaluation unit 208 determines whether the subject types of the subject images of the determined optical systems are the same. If the subject types are not the same, different subject images are detected for each optical system due to parallax or the like, and unified focus control for the same subject is not possible as a compound eye optical system. If the subject types of the optical systems determined by the imaging evaluation unit 208 are not the same, proceed to step S917. On the other hand, if the subject types of the optical systems determined by the imaging evaluation unit 208 are the same, proceed to step S913.

[0102] In step S913, it is determined whether the subject type of the subject image of each optical system determined by the imaging evaluation unit 208 is a vehicle such as a train, a four-wheeled vehicle, or a two-wheeled vehicle. If the subject type is a vehicle, the process proceeds to step S915. If the subject type is determined to be another subject type, the process proceeds to step S914.

[0103] In step S914, the image capturing evaluation unit 208 sets the first focus determination threshold value, which is basically selected, as the focus determination threshold value serving as the focus determination criterion for each optical system of the compound eye optical system. After setting the focus determination threshold value, the process proceeds to step S928.

[0104] In step S915, the image capturing evaluation unit 208 sets a third focus determination threshold value, which is a looser standard than the first focus determination threshold value, as the focus determination criterion for each optical system of the compound eye optical system. This is a condition under which the main subject in the focus detection area of ​​each optical system is determined to be a vehicle, and generally, a vehicle is more likely to be captured as the same subject in a long distance in the depth direction than other subjects. Therefore, even if there is a difference between the focus position after focus control and the focus position, there is a higher possibility that the main subject, the vehicle, is in focus somewhere than other subjects. Therefore, in this embodiment, when the main subject is a vehicle, the parallax of the compound eye optical system is easily noticeable by capturing it long in the depth direction, and the degree of focus in each optical system is likely to vary, so that the third focus determination threshold value is set to make it easier to determine focus. After setting, proceed to step S928.

[0105] In step S916, the image capturing evaluation unit 208 sets a second focus determination threshold that substantially invalidates the focus determination as a focus determination criterion for some optical systems among the optical systems of the compound eye optical system in which occlusion by another object or brightness abnormality has been detected. The second focus determination threshold is specifically expressed as a value of ∞ or a maximum value that can be set. In addition, the image capturing evaluation unit 208 sets a third focus determination threshold that is larger than the first focus determination threshold and has a looser criterion as a focus determination criterion for the remaining optical systems among the compound eye optical system. After setting the focus determination threshold, the process proceeds to step S928.

[0106] Step S917 is a processing step that is performed when all optical systems targeted for focus detection in the compound eye optical system are occluded by another object, or have high-luminance or low-luminance areas of a predetermined area or area ratio, or when the types of objects in each optical system are not the same. In this step, the image capture evaluation unit 208 sets the value 0 to the focus determination threshold value as the focus determination criterion for all optical systems targeted for focus detection in the compound eye optical system. By setting the value 0, when determining the focus determination range in the subsequent processing step, there is no focus determination range, which leads to a determination that the image is not in focus. After setting the focus determination threshold value, the process proceeds to step S928.

[0107] Steps S918 to S927 are operations performed when the focus evaluation mode recognized by the imaging evaluation unit 208 is a partial optical system focus evaluation mode that performs focus determination substantially based only on focus detection results of a portion of the optical system of the compound eye optical system. In these processing steps, the imaging evaluation unit 208 performs various imaging evaluations according to the selected focus evaluation mode, which will be described later, and determines one representative optical system that is most suitable for the focus evaluation by the imaging evaluation unit 208. Then, the imaging evaluation unit 208 makes the focus determination threshold for the optical system that is most suitable for focus evaluation different from the focus determination threshold for the remaining optical systems. Then, the focus determination unit 209 performs focus determination based on the optical system that is substantially most suitable for focus evaluation. Hereinafter, each process of steps S918 to S927 that performs the above-mentioned operations will be described.

[0108] In step S918, the image capturing evaluation unit 208 determines whether the occlusion condition by another object in step S904 is satisfied in all optical systems targeted for focus detection. If the image capturing evaluation unit 208 determines that the occlusion condition is satisfied in all optical systems targeted for focus detection, the process proceeds to step S917 to essentially change the in-focus determination to out-of-focus. On the other hand, if the image capturing evaluation unit 208 determines that the occlusion condition is not satisfied in at least one of the optical systems targeted for focus detection, the process proceeds to step S919.

[0109] Step S919 is a processing step that is performed when at least one optical system determined to be non-applicable in step S918 exists among the compound eye optical systems. In step S919, the imaging evaluation unit 208 generates a pixel-by-pixel luminance histogram for each focus detection area in the optical system that is not applicable in step S918 among the optical systems that are focus detection targets of the compound eye optical system. In the luminance histogram, the count is divided at least into a predetermined high luminance and a predetermined low luminance for determining that the luminance is abnormal. The count may be divided into finer luminance gradations. An example of the predetermined high luminance is a value such as 230 or 240 when the luminance of the digital imaging data is expressed with an 8-bit depth of values ​​0 to 255. Also, an example of the predetermined low luminance is a value such as 20 or 10, while the minimum luminance value based on the dark current of the imaging element is a value of 0 when the luminance of the digital imaging data is expressed with an 8-bit depth of values ​​0 to 255. After the luminance histogram is generated, the process proceeds to step S920.

[0110] In step S920, in each focus detection area of ​​the optical system, it is evaluated whether a high luminance area having a predetermined high luminance level or more or a low luminance area having a predetermined low luminance level or less is included in a predetermined area or a predetermined area ratio or more in pixel units in each focus detection area of ​​the optical system. The evaluation is performed by the image capture evaluation unit 208. The predetermined area ratio and the predetermined area each have two types of thresholds, a first area ratio threshold or a first area threshold described later, and a second area ratio threshold or a second area threshold described later, which are set by the image capture evaluation unit 208. An example of the predetermined area ratio and the predetermined area is a first area ratio threshold having an intermediate value such as 30% or 40% with respect to the area of ​​the focus detection area expressed in pixel units, or a first area threshold having an area equal to this threshold. Also, a second area ratio threshold having an almost extreme value larger than the first area ratio threshold such as almost 100%, or a second area threshold having an area equal to this threshold.

[0111] If the imaging evaluation unit 208 determines that the second area threshold or the second area ratio is satisfied in all focus detection areas of the optical system, the process proceeds to step S917 in order to substantially change the focus determination to out of focus. If the imaging evaluation unit 208 determines that the first area threshold or the first area ratio threshold is satisfied in all focus detection areas of the optical system, except for the condition for proceeding to step S917 described above, the process proceeds to step S927. In other cases, the process proceeds to step S921. Note that there is a similar processing step, step S908, but the determination condition for whether a high brightness area or a low brightness area is included is different from that of step S908. This is because the operations of steps S918 to S926, including this step, are different from the operations of steps S906 to S916, including step S908, in that the imaging evaluation unit 208 determines one representative optical system that is most suitable for focus evaluation.

[0112] Step S921 is a processing step that is performed when at least one of the compound eye optical systems is determined not to meet the luminance condition described in step S920. In step S921, for the optical systems not meeting the luminance condition in step S920 among the optical systems targeted for focus detection in the compound eye optical system, the image capturing evaluation unit 208 acquires information on the approximate corresponding subject distance based on the focal position of the main subject of each optical system from the drive characteristics information recording unit 104. After acquiring the information, the process proceeds to step S922.

[0113] In step S922, the image capturing evaluation unit 208 evaluates whether the acquired object distances are closer than a predetermined object distance threshold value for the main object in each focus detection area of ​​the optical system, and whether the distance condition is such that the parallax of each optical image by the compound eye optical system is greater than a predetermined value. The parallax of each optical image by the compound eye optical system is affected by the base length of each optical system of the compound eye optical system and the object distance. If the farthest distance among the acquired object distances in the optical system is closer than the predetermined object distance threshold value, the process proceeds to step S927. On the other hand, if not, the process proceeds to step S923.

[0114] Step S923 is a processing step that is performed when there is at least one optical system that does not meet the distance condition described in step S922. In step S923, the image capture evaluation unit 208 determines a predetermined object type from the color and brightness features of each object image of each optical system in the optical systems of the compound eye optical system that do not meet the distance condition of step S922, using a known object recognition method. Specifically, the image capture evaluation unit 208 determines whether the object to be detected for focus detection is a person's face or eye, or a vehicle such as a train, a four-wheeled vehicle, or a two-wheeled vehicle. After determining the object type, the process proceeds to step S924.

[0115] In step S924, it is determined whether the object types determined in the optical systems not corresponding to the distance condition in step S922 are the same. If there are multiple non-corresponding optical systems in step S922 and the object types are not the same, different object images are detected for each optical system due to parallax, etc., and unified focus control for the same object is not performed as a compound eye optical system. If the object types in the non-corresponding optical systems are not the same, the process proceeds to step S917 to substantially bring the focus determination to non-focus. If there is only one non-corresponding optical system in step S922, or if there are multiple non-corresponding optical systems and the object types are determined to be the same in step S922, the process proceeds to step S925.

[0116] In step S925, for the optical systems not corresponding in step S922, it is determined whether the object type of the object image of each optical system determined by the imaging evaluation unit 208 is a vehicle such as a train, a four-wheeled vehicle, or a two-wheeled vehicle. If the object type is corresponding, the process proceeds to step S927. On the other hand, if it is determined to be another object type, the process proceeds to step S926.

[0117] Step S926 is a processing step that is performed when at least one optical system determined to be non-applicable in the subject type condition in step S925 exists among the compound eye optical systems. In step S926, the optical system with the smallest detected defocus amount among the optical systems determined to be non-applicable in the subject type condition in step S925 is set as a representative optical system, and the imaging evaluation unit 208 sets different focus determination thresholds for the representative optical system and the other optical systems. Specifically, the imaging evaluation unit 208 sets the focus determination threshold of the representative optical system to a first focus determination threshold that is basically selected. In addition, the imaging evaluation unit 208 sets the focus determination thresholds of the other optical systems other than the representative optical system among the compound eye optical systems to a second focus determination threshold that substantially invalidates the focus determination. By setting in this way, the imaging evaluation unit 208 can make the focus determination threshold for the optical system most suitable for focus evaluation different from the focus determination threshold for the remaining other optical systems. After setting the focus determination threshold, the process proceeds to step S928.

[0118] Step S927 is a processing step that is performed when at least one of the compound eye optical systems is determined to be the optical system in step S922, or when the optical system is determined to be the optical system in step S925. In step S927, the optical system with the smallest detected defocus amount among the optical systems that were the subject of condition determination in the previous step is set as a representative optical system, and the imaging evaluation unit 208 sets different focus determination thresholds for the representative optical system and the other optical systems. Specifically, the imaging evaluation unit 208 sets the focus determination threshold of the representative optical system to a third focus determination threshold that is a larger value that is a looser standard than the first focus determination threshold that is basically selected. In addition, the imaging evaluation unit 208 sets the focus determination thresholds of the other optical systems other than the representative optical system among the compound eye optical systems to a second focus determination threshold that substantially invalidates the focus determination. By setting in this way, the imaging evaluation unit 208 can make the focus determination threshold for the optical system that is most suitable for focus evaluation different from the focus determination thresholds for the remaining other optical systems. After setting the focus determination threshold, the process proceeds to step S928.

[0119] In step S928, the focus determination unit 209 determines a focus determination range based on the focus determination threshold as a focus determination criterion for each optical system of the compound eye optical system. Specifically, the focus determination unit 209 determines, as the focus determination range, the width of a range indicated by positive and negative values ​​relative to the focus determination threshold expressed in absolute values, with the focus position based on the focus detection result of the focus detection unit 206 as the center of the range. After determination, the process proceeds to step S929.

[0120] In step S929, the in-focus determination unit 209 compares and determines whether the defocus amounts detected in the predetermined focus detection areas of each optical system are within the in-focus determination range determined in step S928. If both defocus amounts are within the in-focus determination range, the process proceeds to step S930. On the other hand, if not, the operation ends. In step S930, the notification unit 210 notifies the user that in-focus has been achieved by a predetermined sound or display, and the operation ends.

[0121] By performing the above-described focus determination standard determination operation, even if differences occur in the degree of focus of each subject image of the compound eye optical system due to the influence of parallax, the imaging situation is automatically evaluated, and it is possible to appropriately perform focus notification while taking into account the influence of parallax of the compound eye optical system. In addition, by being able to appropriately perform focus notification, for example, in repeated shooting operations when SW2 operation is continued, it is possible to automatically and appropriately transition to shooting operations after focus control.

[0122] <Modification> In this embodiment, in step S913, the image capture evaluation unit 208 changes the setting of the focus determination threshold depending on whether the type of the main subject in the focus detection area of ​​each optical system is a vehicle or not. However, this embodiment is not limited to this, and the image capture evaluation unit 208 may change the setting of the focus determination threshold depending on whether the type of the main subject in the focus detection area of ​​each optical system is a face or an eye. Specifically, when the main subject is a face or an eye, a slight focus deviation tends to be more easily recognized than other subjects. Therefore, when the main subject is a face or an eye, a fourth focus determination threshold that is stricter than the first focus determination threshold that is basically selected may be newly created and set.

[0123] By setting it in this manner, it is possible to set a focus determination threshold that takes into account the effects of parallax in the compound eye optical system for general subjects, while setting a strict focus determination threshold for subjects where focus errors such as the face or eyes tend to be easily noticeable, regardless of the effects of parallax.

[0124] (Other variations) In each of the above-mentioned embodiments, the digital camera is described as an example in which each subject image of the compound eye optical system is incident on one imaging unit 202. That is, the imaging unit 202 has one imaging element that forms the subject images formed by the left optical system 110 and the right optical system 120 in different regions. However, each embodiment is not limited to this, and each subject image of the compound eye optical system may be incident on a plurality of independent imaging units. That is, the imaging unit 202 may have a first imaging element that forms the subject image formed by the left optical system 110 and a second imaging element that forms the subject image formed by the right optical system 120.

[0125] In each embodiment, a digital camera having a twin-lens optical system has been described as an example of a compound eye optical system in which a plurality of different optical systems are provided in one lens barrel, but each embodiment is not limited to this and can be similarly applied to a digital camera having three or more optical systems.

[0126] (Other embodiments) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0127] According to each embodiment, it is possible to provide an imaging device, a control method for an imaging device, and a program that are capable of appropriately determining focus by using a reference value according to the imaging conditions for each of the multiple optical systems that make up the compound eye optical system.

[0128] The disclosure of each embodiment includes the following configurations and methods. (Configuration 1) an imaging means for forming a subject image formed by a first optical system constituting a compound eye optical system and a second optical system arranged in parallel with the first optical system; a focus detection unit that detects a defocus amount of each of the first optical system and the second optical system with respect to the object image; evaluation means for evaluating an imaging state relating to the subject image; an evaluation means for evaluating a focus state of the first optical system and a second reference value for evaluating a focus state of the second optical system in accordance with the imaging state; (Configuration 2) a focus determination unit that determines a focus state of each of the first optical system and the second optical system, The focus determination means When the defocus amount of the first optical system is within a first range set using the first reference value, it is determined that the first optical system is in the in-focus state; The imaging device described in configuration 1, characterized in that when the defocus amount of the second optical system is within a second range set using the second reference value, the second optical system is determined to be in the in-focus state. (Configuration 3) The imaging device described in configuration 1 or 2, further comprising a notification means for notifying a user that the first optical system and the second optical system are in the in-focus state when the first optical system and the second optical system are determined to be in the in-focus state. (Configuration 4) 4. The imaging apparatus according to any one of configurations 1 to 3, wherein the evaluation means evaluates the imaging state based on the defocus amounts of the first optical system and the second optical system. (Configuration 5) 4. The imaging device according to any one of configurations 1 to 3, wherein the evaluation means evaluates the imaging state based on the aperture value of each of the first optical system and the second optical system. (Configuration 6) The evaluation means includes: Identifying an imaging area of ​​the subject, and determining whether or not the imaging area is obscured by another subject entering the imaging area in front of the subject; The imaging device described in any one of configurations 1 to 3, characterized in that when occlusion occurs in the imaging region at a rate greater than or equal to a predetermined area ratio, at least one of the first reference value or the second reference value is changed so that the in-focus state is more likely to be determined. (Configuration 7) 4. The imaging apparatus according to any one of configurations 1 to 3, wherein the evaluation means evaluates the imaging state based on whether or not the subject image includes a high-luminance area or a low-luminance area having a predetermined area or more. (Configuration 8) The imaging device described in configuration 7, characterized in that the evaluation means changes at least one of the first reference value or the second reference value so that the subject image is more likely to be determined to be in focus when the subject image includes a high-brightness area or a low-brightness area having an area equal to or larger than the specified area. (Configuration 9) The evaluation means includes: extracting feature points of the subject image and determining the type of the subject; The imaging apparatus according to any one of configurations 1 to 3, wherein the imaging state is evaluated based on the object type. (Configuration 10) The imaging device described in configuration 9, characterized in that when the evaluation means determines that the subject type is a human face, it changes at least one of the first reference value or the second reference value so that the in-focus state is less likely to be determined. (Configuration 11) The imaging device described in configuration 9, characterized in that when the evaluation means determines that the subject type is a vehicle, it changes at least one of the first reference value or the second reference value so that the in-focus state is more likely to be determined. (Configuration 12) The evaluation means includes: determining a distance between the imaging device and the subject using a focal position of at least one of the first optical system and the second optical system and an output signal of the focus detection means; The imaging device described in any one of configurations 1 to 3, characterized in that when the distance is closer than a predetermined distance threshold, at least one of the first reference value or the second reference value is changed so that the in-focus state is more likely to be determined. (Configuration 13) 13. The imaging device according to any one of configurations 1 to 12, wherein the first optical system for which the first reference value is set is more in-focus than the second optical system for which the second reference value is set. (Configuration 14) 14. The imaging device according to any one of configurations 1 to 13, wherein the second reference value is set so as to make the in-focus state more likely to be determined than the first reference value. (Configuration 15) The imaging device described in any one of configurations 1 to 14, characterized in that the imaging means has a first imaging element that forms the subject image formed by the first optical system, and a second imaging element that forms the subject image formed by the second optical system. (Configuration 16) 15. The imaging device according to any one of configurations 1 to 14, wherein the imaging means has a single imaging element that focuses the subject images formed by the first optical system and the second optical system in different areas. (Method 1) an imaging step of forming a subject image formed by a first optical system constituting a compound eye optical system and a second optical system arranged in parallel with the first optical system; a focus detection step of detecting a defocus amount of each of the first optical system and the second optical system with respect to the object image; an evaluation step of evaluating an imaging condition related to the subject image, A control method for an imaging device, characterized in that in the evaluation step, a first reference value used for determining the focus state of the first optical system and a second reference value used for determining the focus state of the second optical system are set depending on the imaging state. (Configuration 17) A program for causing a computer to execute the imaging device control method according to Method 1.

[0129] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. [Explanation of symbols]

[0130] 200 Camera unit (imaging device) 110 Left optical system (1st optical system) 120 Right optical system (second optical system) 202 Imaging unit (imaging means) 206 Focus detection unit (focus detection means) 208 Imaging evaluation unit (evaluation means)

Claims

1. an imaging means for forming a subject image formed by at least a first optical system constituting a compound eye optical system and a second optical system arranged in parallel with the first optical system; a focus detection unit that detects a defocus amount of each of the first optical system and the second optical system with respect to the object image; evaluation means for evaluating an imaging state relating to the subject image; an evaluation means for evaluating a focus state of the first optical system and a second reference value for evaluating a focus state of the second optical system, each of which is set according to the imaging state;

2. a focusing determination unit that determines a focusing state of each of the first optical system and the second optical system, The focus determination means When the defocus amount of the first optical system is within a first range set using the first reference value, it is determined that the first optical system is in the in-focus state; 2. The imaging device according to claim 1, wherein the second optical system is determined to be in the in-focus state when the defocus amount of the second optical system is within a second range set using the second reference value.

3. 2. The imaging apparatus according to claim 1, further comprising a notification unit that, when it is determined that the first optical system and the second optical system are each in the in-focus state, notifies a user that the first optical system and the second optical system are in the in-focus state.

4. 4. The imaging apparatus according to claim 1, wherein the evaluation unit evaluates the imaging state based on the defocus amount of each of the first optical system and the second optical system.

5. 4. The imaging apparatus according to claim 1, wherein the evaluation unit evaluates the imaging state based on the aperture values ​​of the first optical system and the second optical system.

6. The evaluation means includes: Identifying an imaging area of ​​the subject, and determining whether or not the imaging area is obscured by another subject entering the imaging area in front of the subject; The imaging device according to any one of claims 1 to 3, characterized in that, when occlusion occurs in the imaging region at a rate equal to or greater than a predetermined area ratio, at least one of the first reference value or the second reference value is changed so that the in-focus state is more likely to be determined.

7. 4. The imaging apparatus according to claim 1, wherein the evaluation means evaluates the imaging state based on whether or not the subject image includes a high-luminance area or a low-luminance area having a predetermined area or more.

8. The imaging device according to claim 7, characterized in that the evaluation means changes at least one of the first reference value or the second reference value so that the subject image is more likely to be determined to be in focus when the subject image includes a high-brightness area or a low-brightness area having an area equal to or larger than the specified area.

9. The evaluation means includes: extracting feature points of the subject image and determining the type of the subject; 4. The imaging apparatus according to claim 1, wherein the imaging state is evaluated based on the object type.

10. 10. The imaging device according to claim 9, wherein the evaluation means, when determining that the subject type is a human face, changes at least one of the first reference value or the second reference value so that the in-focus state is less likely to be determined.

11. The imaging device according to claim 9, characterized in that, when the evaluation means determines that the subject type is a vehicle, it changes at least one of the first reference value or the second reference value so that the in-focus state is more likely to be determined.

12. The evaluation means includes: determining a distance between the imaging device and the subject using a focal position of at least one of the first optical system and the second optical system and an output signal of the focus detection means; 4. The imaging device according to claim 1, wherein at least one of the first reference value and the second reference value is changed so that the in-focus state is more likely to be determined when the distance is closer than a predetermined distance threshold.

13. 4. The imaging device according to claim 1, wherein the first optical system for which the first reference value is set is more in-focus than the second optical system for which the second reference value is set.

14. 4. The imaging apparatus according to claim 1, wherein the second reference value is set so that the in-focus state is more likely to be determined than the first reference value.

15. 4. The imaging device according to claim 1, wherein the imaging means has a first imaging element that forms the subject image formed by the first optical system, and a second imaging element that forms the subject image formed by the second optical system.

16. 4. The imaging device according to claim 1, wherein the imaging means includes a single imaging element that forms the subject images formed by the first optical system and the second optical system in different areas.

17. an imaging step of forming a subject image formed by a first optical system constituting a compound eye optical system and a second optical system arranged in parallel with the first optical system; a focus detection step of detecting defocus amounts of the first optical system and the second optical system with respect to the object image; an evaluation step of evaluating an imaging condition related to the subject image, A control method for an imaging device, characterized in that in the evaluation step, a first reference value used for determining the focus state of the first optical system and a second reference value used for determining the focus state of the second optical system are set depending on the imaging state.

18. A program for causing a computer to execute the method for controlling an imaging apparatus according to claim 17.

Images