Camera device, control method and storage medium

Abstract

The present invention relates to an imaging device, a control method and a storage medium. The imaging device includes: a tilt drive unit configured to change the tilt of at least one of an image sensor and an imaging optical system to change the angle between the imaging surface of the image sensor and the main surface of the imaging optical system; a focus drive unit configured to drive a focus lens of the imaging optical system; an area setting unit configured to set a plurality of areas in a captured image; a defocus information detection unit configured to detect defocus information for each of the plurality of areas; and a control unit configured to select a control mode from a plurality of control modes based on the defocus information about each of the plurality of areas, the plurality of control modes being used to perform focus control using at least one of the tilt drive unit and the focus drive unit.

Description

Technical Field

[0001] The present disclosure relates to an imaging apparatus capable of tilt control. Background Art

[0002] In recent years, imaging equipment has been installed for safety purposes or in places such as factory lanes. Such imaging equipment is required to obtain images with a deep depth of field. A technique called the Scheimpflug principle is known, which adjusts the focal plane by tilting the imaging plane relative to a plane orthogonal to the imaging optical axis used to capture an image of a subject (i.e., by performing tilt control) so as to deepen the depth of field of the imaging scene.

[0003] Japanese Patent Laid-Open (“JP”) 2003-75716 discloses an imaging device that detects each focus position of a plurality of subjects based on a contrast evaluation value, and determines a focus position and a tilt control amount so that the plurality of subjects become focused. JP 2020-154283 discloses an imaging device that includes a plurality of control modes and focuses on a plurality of subjects by selecting one of the plurality of control modes based on the number of subjects, wherein each control mode controls at least one of a focus position and a tilt angle. Each of these techniques can detect the defocus amount immediately by applying an imaging plane phase difference method, and faster focusing is expected.

[0004] However, when using the imaging apparatus disclosed in JP 2003-75716 or JP 2020-154283, control based on contrast evaluation values ​​may temporarily cause the subject to be out of focus during the process of detecting a plurality of focus positions, or it may take some time to acquire the focus state. In JP 2003-75716 and JP 2020-154283, it is not considered that a high-precision defocus amount is not always detected when the imaging plane phase difference method is used, and a control method when the imaging plane phase difference method is used is not disclosed. Summary of the invention

[0005] The present invention provides an image pickup apparatus that can focus on a plurality of subjects at high speed and with high precision by performing optimal focus control based on defocus information about the plurality of subjects.

[0006] According to one aspect of an embodiment of the present disclosure, a camera device includes: a tilt drive unit, which is configured to change the tilt of at least one of an image sensor and a camera optical system to change the angle between the camera surface of the image sensor and the main surface of the camera optical system; a focus drive unit, which is configured to drive the focus lens of the camera optical system; an area setting unit, which is configured to set a plurality of areas in a captured image; a defocus information detection unit, which is configured to detect defocus information for each of the plurality of areas; and a control unit, which is configured to select a control mode from a plurality of control modes based on the defocus information about each of the plurality of areas, and the plurality of control modes are used to perform focus control using at least one of the tilt drive unit and the focus drive unit.

[0007] A control method of an image pickup apparatus corresponding to the above-described image pickup apparatus and a storage medium storing a computer program that enables a computer to execute the control method also constitute other aspects of the embodiment.

[0008] Further features of the present invention will become apparent from the following description of exemplary embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 is a block diagram showing the configuration of an imaging apparatus.

[0010] Figure 2 is a diagram showing a pixel arrangement of an image sensor.

[0011] Figure 3 (A) to (D) are diagrams showing a method of calculating a bit shift amount.

[0012] Figure 4 It is a diagram showing a method of calculating the conversion coefficient K.

[0013] FIG. 5A to FIG. 5C is a diagram showing tilt control.

[0014] Figure 6 is a list of controls based on defocus information and its reliability according to the first embodiment.

[0015] Figure 7 is a diagram showing a focus-tilt calculation mode.

[0016] Figure 8 is a flowchart of the process according to the first embodiment.

[0017] 9A to 9C 2 is a diagram showing a focus maintaining tilt mode.

[0018] Fig.10 2 is a diagram showing a focus maintaining tilt mode.

[0019] Fig.11 is a list of control based on defocus information and its reliability according to the second embodiment.

[0020] Fig.12 is a flowchart of the process according to the second embodiment.

[0021] Fig.13 is a flowchart of the process according to the third embodiment.

[0022] Fig.14 is a flowchart of the process according to the third embodiment. DETAILED DESCRIPTION

[0023] Referring now to the drawings, a detailed description of embodiments of the present disclosure will be given.

[0024] Overall configuration

[0025] Figure 1 The lens unit as the imaging optical system includes a zoom lens 101 that moves along the optical axis direction to change the focal length, a focus lens 102 that moves along the optical axis direction to control the focus, and an aperture unit 103 that adjusts the light amount.

[0026] The light passing through the imaging optical system forms a subject image as an optical image on the image sensor 106 via a band pass filter (BPF) 104 and a color filter 105. The BPF 104 can advance in and retract from the optical path of the imaging optical system. The image sensor 106 photoelectrically converts the subject image into an imaging signal. The analog electric signal (imaging signal) output from the image sensor 106 is subjected to gain adjustment by an automatic gain control (AGC) 107, converted into a digital imaging signal by an A / D converter 108, and input to a camera signal processing unit 109.

[0027] The camera signal processing unit 109 performs various image processing (e.g., gamma processing and white balance adjustment) on the digital camera signal to generate an image signal. By displaying the image signal as an image on a display unit (not shown), the captured image can be displayed. The image signal is output to a monitoring device 111 connected to the camera apparatus through wired or wireless communication via the communication unit 110. The communication unit 110 receives a command from an external control device such as an external PC, and sends a control signal such as a command to a tilt / focus control unit (control unit) 115 in the camera apparatus.

[0028] In this embodiment, a system including the monitoring device 111 is referred to as an imaging device. Figure 1The units other than the monitoring device 111 are accommodated in one housing and included in the monitoring camera.

[0029] The focus detection unit (defocus information detection unit) 112 performs focus detection by a phase difference detection method by using the focus detection data acquired for each target subject area from the camera signal processing unit 109. More specifically, the camera signal processing unit 109 generates a pair of image data formed by light beams passing through a pair of pupil areas of the imaging optical system as focus detection data. The focus detection unit 112 detects the defocus amount, the defocus direction, and the reliability of the defocus amount and the defocus direction, etc. based on the offset between the pair of image data. That is, the focus detection unit 112 performs focus detection by an imaging plane phase difference method based on the output from the image sensor 106 without using a dedicated AF sensor. A detailed description of the focus detection operation by the imaging plane phase difference method performed by the focus detection unit 112 is given below.

[0030] The subject determination unit (area setting unit) 113 detects a specific subject in the captured image by using image recognition of the image signal output from the camera signal processing unit 109. The specific subject is a subject arbitrarily specified by the user or set in advance as a default, and is, for example, a person, a face, a car, etc.

[0031] The defocus information output from the focus detection unit 112 and the subject information output from the subject judgment unit 113 are input to the tilt / focus drive amount calculation unit 114. The tilt / focus drive amount calculation unit 114 calculates the optimal tilt angle and focus position for the scene by using the defocus information, the subject information, and the tilt angle and focus position output from the tilt / focus control unit 115. The calculated tilt angle and focus position are sent to the tilt / focus control unit 115, and the tilt / focus control unit 115 controls the tilt angle and focus position. The tilt / focus control unit 115 includes a built-in CPU as a computer and as a control unit, and performs various operations on the entire device according to a computer program stored in a memory (not shown). The tilt / focus control unit 115 performs focus control using AF or manual focus (MF) based on an instruction acquired from an external control device such as an external PC via the communication unit 110. The tilt / focus control unit 115 receives the current position information output from the image sensor driving unit 116 and the focus driving unit 117, and transmits the current position information to the tilt / focus driving amount calculation unit 114. The tilt / focus control unit 115 transmits the tilt angle and the focal position calculated by the tilt / focus driving amount calculation unit 114 to the image sensor driving unit 116 and the focus driving unit 117 so as to indicate the driving position.

[0032] The image sensor driving unit 116 functions as a tilt driving unit, and tilts the image sensor 106 based on the tilt angle indicated by the tilt / focus control unit 115. That is, the image sensor driving unit 116 tilts the image sensor 106 to change the tilt angle, which is the angle between the imaging surface of the image sensor 106 and the principal surface of the imaging optical system. In this embodiment, the rotation axis of the tilt of the image sensor 106 is a horizontal axis passing through the center of the captured image, that is, an axis along the longitudinal direction of the image sensor 106. The image sensor 106 is tilted relative to the imaging optical system around the rotation axis. In the tilt control, instead of the image sensor 106, some lenses of the imaging optical system may be tilted relative to the image sensor 106.

[0033] The focus driving unit 117 controls the position of the focus lens 102 based on the set focal position instructed by the tilt / focus control unit 115 .

[0034] Description of focus detection operation by imaging plane phase difference method

[0035] Will refer to Figure 2 A description is given of the pixel configuration of the image sensor 106 . Figure 2 is a diagram showing the pixel arrangement of the image sensor 106 .

[0036] In the image sensor 106, each pixel 201 is divided into two photoelectric conversion units 201a and 201b in the X direction. Individual photoelectric conversion signals output from the photoelectric conversion units 201a and 201b and the sum of the photoelectric conversion signals output from the two photoelectric conversion units 201a and 201b can be read from the image sensor 106 individually.

[0037] By subtracting the photoelectric conversion signal output from one photoelectric conversion unit (for example, the photoelectric conversion unit 201a) from the sum of the photoelectric conversion signals output from the two photoelectric conversion units 201a and 201b, a signal corresponding to the photoelectric conversion signal output from the other photoelectric conversion unit (the photoelectric conversion unit 201b) can be acquired. Each of the photoelectric conversion signals output from the photoelectric conversion units 201a and 201b is used as focus detection data in phase difference AF. The sum of the photoelectric conversion signals output from the two photoelectric conversion units 201a and 201b is used as normal shooting image data.

[0038] The focus detection unit 112 performs correlation calculation to obtain the relative image shift amount between the A image and the B image generated as described above. As a result, the focus detection unit 112 can detect the bit shift amount [bit] as the correlation of a pair of image signals. By multiplying the bit shift amount by the conversion coefficient, the bit shift amount is converted into the defocus amount [mm] of the predetermined area.

[0039] This embodiment reads the output signal from one of the photoelectric conversion units 201a and 201b and the sum of the output signals from the two photoelectric conversion units 201a and 201b from each pixel of the image sensor 106. For example, in the case of reading the output signal from the photoelectric conversion unit 201a and the sum of the output signals from the photoelectric conversion units 201a and 201b, the output signal from the photoelectric conversion unit 201b can be obtained by subtracting the output signal from the photoelectric conversion unit 201a from the sum of the output signals from the photoelectric conversion units 201a and 201b. As a result, both the A image and the B image can be obtained, and focus detection using the imaging surface phase difference method can be realized. Generally, the sum of the output signals from the photoelectric conversion units 201a and 201b forms one pixel of the output image, that is, one output pixel. Such an image sensor is well known as disclosed in JP2004-134867, and therefore a detailed description thereof is omitted.

[0040] Next, a description will be given of a focus detection operation using the imaging plane phase difference method. The focus detection operation described below is mainly performed by the focus detection unit 112. The focus detection unit 112 sets a focus detection position. Thereafter, the focus detection unit 112 reads focus detection data from the set focus detection position. By using the signals read from the pixels in the focus detection position set by the focus detection unit 112, the focus detection unit 112 generates signals of the A image and the B image, respectively. Next, the focus detection unit 112 detects the bit shift amount P [bit] by performing correlation calculation to obtain the relative image shift amount between the A image and the B image.

[0041] Reference Figure 3 (A) to (D) describe examples of correlation calculation methods. Figure 3 (A) to (D) are explanatory diagrams showing a correlation calculation method, which is a method for calculating a bit offset. Figure 3 In (A) to (D), the vertical axis indicates the signal value and the horizontal axis indicates the bit (position).

[0042] Assume that signals of the A image and the B image are read from pixels (focus detection pixels) of the image sensor 106. The camera signal processing unit 109 digitally filters each of the A image and the B image in order to reduce noise. Figure 3 (A) shows the state after filtering. Figure 3As shown in (A) to (D) of , the focus detection unit 112 calculates the correlation amount COR while performing a bit shift on the A image signal or the B image signal or both the A image signal and the B image signal. The correlation amount COR is an area where the A image and the B image overlap, a value obtained by subtracting the area of ​​the B image from the area of ​​the A image, or a calculated value indicating the degree of correlation, etc., but is not limited to these. Hereinafter, a description will be given of a case where the correlation amount COR is an area where the A image and the B image overlap.

[0043] If the A image and the B image coincide, the area where the A image and the B image overlap is large, so the correlation becomes highest and the correlation amount COR is large. Here, the offset [bit] when the correlation amount COR takes the maximum value is the bit offset P [bit].

[0044] Next, the focus detection unit 112 acquires the conversion coefficient K. By multiplying the conversion coefficient K by the bit shift amount P, the defocus amount DEF is acquired.

[0045] Will refer to Figure 4 Describe the calculation of the conversion factor K. Figure 4 : is an explanatory diagram showing the calculation of the conversion coefficient K. Figure 4 , the Z axis indicates the optical axis direction of the imaging optical system, and Z=0 indicates the plane of the image sensor 106, that is, the imaging plane.

[0046] Pupil intensity distribution PI_A and pupil intensity distribution PI_B, which are light amount distributions of focus detection light beams of the A image and the B image at Z=Zep, are projected images of focus detection signals output from the photoelectric conversion units 201a and 201b, respectively, on the exit pupil plane.

[0047] Figure 4 PI_A and PI_B in PI_A indicate one-dimensional pupil intensity distributions. The distance between the centers of gravity of the pupil intensity distributions PI_A and PI_B is defined as the base length BL. The ratio of the exit pupil distance Zep to the base length BL indicates the amount of change [mm] in the bit shift amount [bit] between the A image and the B image in the optical axis direction.

[0048] Therefore, the conversion coefficient K can be expressed by the following equation (1).

[0049] K=Zep / BL (1)

[0050] Next, the focus detection unit 112 calculates the defocus amount DEF [mm] using the following equation (2).

[0051] DEF=P×K (2)

[0052] The focus detection unit 112 determines the reliability of the defocus amount. For example, the focus detection unit 112 makes a determination about the reliability based on the degree of consistency between the A image and the B image (hereinafter referred to as the two-image consistency). If the A image signal and the B image signal have a high contrast and the A image signal and the B image signal are similar in shape (that is, if the two-image consistency is high) or if the main subject is in focus, the focus detection unit 112 determines that the state is a state of high reliability (that is, the defocus amount is reliable).

[0053] If the two-image consistency is lower than a predetermined value, but the correlation obtained by relatively shifting the A image signal and the B image signal has a certain trend, the focus detection unit 112 determines that the state is a state in which the defocus amount is unreliable but the defocus direction is reliable. This determination is usually made, for example, in a state in which the main subject is slightly out of focus.

[0054] If none of the above applies to the state, the focus detection unit 112 determines that the state is a state where both the defocus amount and the defocus direction are unreliable. This determination is usually made, for example, in a state where the contrast between the A image signal and the B image signal is low, the consistency between the two images is low, and the subject is seriously out of focus.

[0055] Description of Tilt Control

[0056] Reference FIG. 5A to FIG. 5C Describe the tilt control. FIG. 5A to FIG. 5C It is an explanatory diagram showing tilt control. Figure 5A The state in which the optical system (imaging optical system) and the image sensor 106 are parallel to each other is shown. The focusing distance L is focused, and the focal plane is parallel to each of the optical system and the image sensor 106. Figure 5B The image sensor 106 is rotated about the image sensor rotation axis. Figure 5A The state shown is a state in which tilt control is performed. When tilt control is performed, according to the Scheimpflug principle, the focal plane also rotates around the focal plane rotation axis corresponding to the image sensor rotation axis. Therefore, all objects from short distance to long distance can be focused on a specific plane. According to the Scheimpflug principle, when the main surface of the optical system and the imaging surface of the image sensor 106 intersect on a straight line, the focal plane also intersects them on the same straight line.

[0057] The tilt angle b is calculated by the following equation (3) according to the Scheimpflug principle, where f represents the focal length, L represents the focusing distance, and α represents the depression angle.

[0058] b=tan -1 (f / (Ltanα)) (3)

[0059] Figure 5CFIG. 4 shows a scene in which subject X and subject Y exist. Figure 5C As shown, in this scene, the focal plane can be controlled so that the faces of the subject X and the subject Y are in focus. In the following, the "subject" includes a part of the subject, such as the face of the subject. Therefore, in addition to the tilt control, focus control in which the focus lens 102 is driven can also be performed. The optimal focal plane (that is, the optimal tilt angle and the optimal focus position) differs depending on the subject, so it is difficult for the user to manually adjust the focus. Therefore, the tilt / focus control unit 115 automatically performs focus control to optimize the tilt angle and the focus position so that the focal plane becomes a focal plane that brings both subjects X and Y into focus.

[0060] First embodiment

[0061] Hereinafter, the processing performed by the imaging apparatus according to the first embodiment will be described. The imaging apparatus according to the first embodiment has three focus control modes (control modes), including a focus mode (first mode), a tilt mode (second mode), and a focus-tilt calculation mode (third mode).

[0062] In the focus mode, control is performed so as to focus on the subject by driving the focus lens 102. In the tilt mode, control is performed so as to focus on the subject by performing tilt driving of the image sensor 106. In the focus-tilt calculation mode, control is performed so as to focus on the subject based on target values ​​calculated for the focus position and the tilt angle, respectively. Based on the defocus information detected by the focus detection unit 112 for each subject and its reliability, the tilt / focus control unit 115 performs focus control on two subjects while selecting the optimal focus control mode.

[0063] A specific description will be given of the control method according to this embodiment. Figure 6 is a list of control based on defocus information about each of the two subjects X and Y and its reliability.

[0064] If the defocus amounts of both objects X and Y are reliable, the tilt / focus control unit 115 performs focus control in the focus-tilt calculation mode. Figure 7 An example of calculating the tilt angle and the focal position in the focus-tilt calculation mode is described. Figure 7 is an explanatory diagram showing the focus-tilt calculation mode. Figure 5C There are subject X and subject Y as in the figure. The relationship between the current tilt angle and the position of the focus lens 102 is Figure 7. x indicates the defocus amount for focusing on the focal plane of the subject X, and y indicates the defocus amount for focusing on the focal plane of the subject Y. k1 [μm] indicates the distance from the tilt axis to the subject X on the image sensor 106, and k2 [μm] indicates the distance from the tilt axis to the subject Y on the image sensor 106.

[0065] The following equations (4) and (5) are established, where α [°] represents a tilt angle for focusing on the subject X and the subject Y, and β represents a focus correction amount of the focus lens 102 on the focal plane.

[0066] x-β=k1×tanα+β (4)

[0067] y=k2×tanα-β (5)

[0068] When the simultaneous equations (4) and (5) are solved, the tilt angle α and the focus correction amount β are expressed by the following equations (6) and (7), respectively.

[0069] α=tan -1 ((x+y) / (k1+k2)) (6)

[0070] β=(k2×x-k1×y) / (k1+k2) (7)

[0071] The driving amount of the focus lens 102 can be simply calculated by dividing the focus correction amount β by the sensitivity FS of the focus lens 102 .

[0072] On the other hand, the driving amount of the focus lens 102 can be accurately calculated by solving a high-order equation or a polynomial based on the sensitivity FS. However, the present embodiment is not limited thereto, and the driving amount may be calculated by other methods.

[0073] Therefore, if the defocus amounts (x and y) of both subjects X and Y are reliable, the tilt angle and focus position can be calculated, and the subjects X and Y can be quickly focused. If the defocus amounts of both subjects X and Y are less than a predetermined value, it can be determined that the two subjects are already focused, and the processing ends.

[0074] Next, a case where the defocus amounts of both subjects X and Y are unreliable will be described. If both defocus directions of the two subjects are reliable, the tilt / focus control unit 115 selects the focus mode or the tilt mode according to the defocus direction. If the defocus directions are consistent (i.e., the defocus directions are the same direction), the tilt / focus control unit 115 controls in the focus mode because the focus is controlled in a direction such that both subject X and subject Y become focused by driving the focus lens 102. On the other hand, if the defocus directions are inconsistent (i.e., the defocus directions are opposite directions), the tilt / focus control unit 115 controls in the tilt mode because the focus can be controlled in a direction such that both subject X and subject Y become focused by performing a tilt drive of the image sensor 106.

[0075] If the defocus amount and the defocus direction of either the subject X or the subject Y are unreliable, the tilt / focus control unit 115 controls in the focus mode or the tilt mode. If the defocus amount or the defocus direction of one of the subject X and the subject Y is reliable, the tilt / focus control unit 115 controls in the focus mode or the tilt mode in the direction in which the one subject becomes focused. If the defocus amount or the defocus direction of both the subject X and the subject Y is unreliable, the tilt / focus control unit 115 controls in the focus mode or the tilt mode in a predetermined direction. In these cases, the focus detection unit 112 can determine the defocus direction by acquiring the defocus change and the contrast change of the subject whose defocus direction is unreliable during the control. In the present embodiment, if the defocus direction of one subject is reliable and the one subject is focused, the tilt / focus control unit 115 selects the tilt mode, otherwise, the tilt / focus control unit 115 selects the focus mode. By performing control in a different manner as described above, the present embodiment increases focus variation of another object so that defocus variation and contrast variation are easily acquired.

[0076] Figure 8 is a flowchart showing the above-mentioned processing according to the present embodiment.

[0077] In step S801, the subject determination unit 113 sets two subjects (areas). The two subjects may be automatically set by face detection, automatic detection, etc., or may be specified by the user.

[0078] In step S802 , the focus detection unit 112 detects defocus information (defocus amount and defocus direction) for each subject set in step S801 , and determines the reliability thereof, that is, whether the defocus amount and defocus direction are reliable.

[0079] In step S803, the focus detection unit 112 determines whether both defocus amounts detected for the two subjects are reliable. If both are reliable, the process proceeds to step S804. In step S804, the focus detection unit 112 determines whether both defocus amounts are less than a predetermined value. If both are less than the predetermined value, it can be determined that the two subjects are in focus, and the process ends. If either or both of the defocus amounts are equal to or greater than the predetermined value, the process proceeds to step S805. In step S805, the tilt / focus control unit 115 performs control in a focus-tilt calculation mode that calculates respective target values ​​of the focal position and the tilt angle, and performs focus control on the two subjects based on the target values. Thereafter, the process returns to step S802 again.

[0080] If the result of the determination in step S803 is "No", the process proceeds to step S806. In step S806, the focus detection unit 112 determines whether both of the defocus directions detected for the two subjects are reliable. If both defocus directions are reliable, the process proceeds to step S807. In step S807, the focus detection unit 112 determines whether the detected defocus directions are consistent. If the detected defocus directions are consistent, the process proceeds to step S808, and the tilt / focus control unit 115 controls in the focus mode. If the detected defocus directions are inconsistent, the process proceeds to step S809, and the tilt / focus control unit 115 controls in the tilt mode. Thereafter, the process returns to step S802 again. As the tilt / focus control unit 115 continues to control in the focus direction in the focus control mode of step S808 or step S809, it gradually becomes possible to obtain a reliable defocus amount for each subject, and it is more likely that a determination result of "Yes" is obtained in step S803.

[0081] If the result of the determination in step S806 is "No", the processing proceeds to step S810, and the focus detection unit 112 determines whether the defocus amount of either of the two subjects is less than a predetermined value. That is, the focus detection unit 112 determines whether there is a focused subject. If there is a focused subject, the processing proceeds to step S811, and the tilt / focus control unit 115 controls in the tilt mode. If there is no focused subject, the processing proceeds to step S812, and the tilt / focus control unit 115 controls in the focus mode. Thereafter, the processing returns to step S802 again. As the tilt / focus control unit 115 continues to control in the focus direction in the focus control mode of step S811 or step S812, it gradually becomes possible to obtain reliable defocus amounts for each subject. In addition, the defocus direction can be obtained by acquiring defocus changes and contrast changes. As a result, it is more likely that a determination result of "Yes" is obtained in step S803 or step S806.

[0082] In the above-mentioned first embodiment, defocus information about two subjects is detected, and focus control is performed while switching the mode among three focus control modes of focus mode, tilt mode and focus-tilt calculation mode based on the reliability of the defocus information. If the defocus amount of the two subjects is reliable, focus control is performed at high speed in the focus-tilt calculation mode, and if either or both of the defocus amounts of the two subjects are unreliable, the best mode is selected from the focus mode and the tilt mode based on the defocus directions of the two subjects, which makes it possible to perform focus control on the two subjects at a higher speed and with higher accuracy. In the first embodiment, in order to focus on the two subjects with certainty, the focus control is ended when both defocus amounts are less than a predetermined value. Alternatively, the focus control may be ended, for example, when a predetermined time has passed since the start of the focus control. Alternatively, the focus control may be ended according to the number of reversals of the focus drive or the tilt drive. In addition to this, various modifications are available as a process for ending the focus control.

[0083] In the present embodiment, a case where the number of the plurality of subjects is two is described, but the number of subjects may be three or more.

[0084] For example, in a case where there are three or more subjects, if the defocus amounts of two or more subjects are reliable, the tilt / focus control unit 115 performs focus control in the focus-tilt calculation mode.

[0085] In the case where there are three or more subjects, if the defocus directions of two or more subjects are reliable, the tilt / focus control unit 115 selects the focus mode or the tilt mode according to the defocus directions. If all the defocus directions are consistent, the tilt / focus control unit 115 controls in the focus mode. On the other hand, if the defocus directions are inconsistent, the tilt / focus control unit 115 controls in the tilt mode.

[0086] If the defocus direction of less than two subjects among three or more subjects is reliable, the tilt / focus control unit 115 controls in the focus mode or the tilt mode. In a state where the defocus direction of one subject is reliable and the one subject is focused, the tilt / focus control unit 115 selects the tilt mode, and in other states, the tilt / focus control unit 115 selects the focus mode.

[0087] In the case where there are three or more subjects, it is not always possible to perform focus control so that all subjects are in focus. All subjects become focused only when all subjects are on the same focal plane. However, generally, subjects at a long distance have a deep depth of field for tilt control compared to subjects at a short distance, so if focus control is given priority to subjects at a short distance, focus control that brings all subjects into focus can be performed. Therefore, in the case where the number of subjects is three or more, the tilt / focus control unit 115 can cause the subject judgment unit 113 to set two subjects while giving priority to two subjects at a short distance. Thereafter, the tilt / focus control unit 115 can cause the focus detection unit 112 to detect defocus information about each of the two set subjects, and can perform the above-mentioned processing for the case where there are two subjects X and Y on the two set subjects. Alternatively, two subjects of interest to the user can be selected. As described above, when there are three or more subjects, if two subjects are selected, it is not necessary to detect defocus information of all the subjects, so that the detection load can be reduced.

[0088] Second embodiment

[0089] Hereinafter, a description will be given of the processing performed by the imaging apparatus according to the second embodiment. The imaging apparatus according to this embodiment has a total of four focus control modes, including a focus maintenance tilt mode (fourth mode) in addition to the focus mode (first mode), the tilt mode (second mode), and the focus-tilt calculation mode (third mode) of the imaging apparatus of the first embodiment. Based on the defocus information detected by the focus detection unit 112 for each subject and its reliability, the tilt / focus control unit 115 performs focus control on two subjects while selecting the optimal focus control mode.

[0090] The focus maintenance tilt mode will be described. The focus maintenance tilt mode is a mode in which the tilt / focus control unit 115 performs focus control by using the tilt drive unit so as to prevent the focus change of a specific subject out of two subjects. In other words, the focus maintenance tilt mode is a mode in which the tilt / focus control unit 115 performs focus control by driving the focus lens 102 while performing tilt control so that the focal plane is rotated while maintaining the specific subject in a focused state.

[0091] FIG. 9A to FIG. 10 It is an explanatory diagram showing the focus maintaining tilt mode. 9A to 9C An example of performing tilt control from a state where the subject at point A is in focus is shown. Fig. 9A A state in which the optical system and the image sensor 106 are parallel to each other is shown. Fig. 9B Shows that Fig. 9A As described above, when tilt control is performed, the focal plane rotates around the position corresponding to the tilt rotation axis. Therefore, the subject at point A which is not on the rotation axis deviates relative to the focal plane and becomes out of focus. If control is performed in a state where the subject which is not on the tilt rotation axis is focused, focus control can be performed so that the subject does not become out of focus. Fig. 9C Shows that Fig. 9A The state in which the tilt control and the focus control are performed is a state in which the focus plane can be rotated while maintaining the in-focus state of the subject at point A. By the focus control, the focal plane can be rotated.

[0092] Will refer to Fig.10 A description is given of an example of calculating a focus correction amount in focus control performed according to tilt control. For a subject area located below the position of the rotation axis in a captured image, a focus correction amount for maintaining a focused state even when tilt control is performed is calculated. The positional relationships between the focus lens 102 and the image sensor 106 before and after the tilt control are as follows, respectively. Fig.10 As shown in the upper and lower parts of . In the case where k [μm] represents the distance from the tilt axis to the subject on the image sensor 106, and the image sensor 106 is driven by tilt control so that the tilt angle changes from α1 [°] to α2 [°] (here, because the image sensor 106 is parallel to the lens, α1 [°] is 0 [°]), the change amount β on the focal plane represents the focus correction amount. The focus correction amount β is calculated by the following equation (8).

[0093] β=(tanα2-tanα1)×k (8)

[0094] A specific description will be given of the control method of this embodiment. Fig.11 is a list of controls based on defocus information about each of the two subjects X and Y and their reliability. In the following, only the points different from the first embodiment will be described. In the present embodiment, in a case where the defocus amounts of both subjects X and Y are unreliable and the focus-tilt calculation mode is not available, if the defocus amount of one subject is less than a predetermined value (i.e., the one subject is in focus), the focus maintenance tilt mode is selected. In the focus maintenance tilt mode, focus control can be performed on another subject while preventing the focused subject from being out of focus as described above. Therefore, in the second embodiment, the focusing quality is higher than that in the first embodiment.

[0095] Fig.12 1 is a flowchart showing the processing in the present embodiment described above. Portions similar to those described in the first embodiment are denoted by the same reference numerals, and description thereof will be omitted.

[0096] If the judgment result in step S803 is "No", the processing proceeds to step S1201. In step S1201, the focus detection unit 112 determines whether the defocus amount of any one of the two subjects X and Y is less than a predetermined value. That is, the focus detection unit 112 determines whether there is a focused subject. If there is a focused subject, the processing proceeds to step S1202, and the tilt / focus control unit 115 controls in the focus maintenance tilt mode. After that, the processing returns to step S802 again. As the tilt / focus control unit 115 continues to control in the focus direction of the other subject in the focus maintenance tilt mode in step S1202, it gradually becomes possible to obtain a reliable defocus amount of the other subject, and it is more likely to obtain a judgment result of "Yes" in step S803.

[0097] If the result of the determination in step S1201 is "No", the process proceeds to step S1203. In step S1203, the focus detection unit 112 determines whether both of the defocus directions detected for the two subjects X and Y are reliable. If both defocus directions are reliable, the process proceeds to step S1204. In step S1204, the focus detection unit 112 determines whether the detected defocus directions are consistent. If the defocus directions are consistent, the process proceeds to step S1205, and the tilt / focus control unit 115 performs control in the focus mode. If the defocus directions are inconsistent, the process proceeds to step S1206, and the tilt / focus control unit 115 performs control in the tilt mode. Thereafter, the process returns to step S802 again. As the tilt / focus control unit 115 continues to control in the focus direction in the focus control mode of step S1205 or step S1206, it gradually becomes possible to obtain reliable defocus amounts for each subject, and a judgment result of "yes" is more likely to be obtained in step S803 or step S1201.

[0098] If the judgment result in step S1203 is "No", the processing proceeds to step S1207, and the tilt / focus control unit 115 controls in the focus mode. After that, the processing returns to step S802 again. As the tilt / focus control unit 115 continues to control in the focus direction in the focus mode in step S1207, it gradually becomes possible to obtain a reliable defocus amount for each subject. In addition, the defocus direction can be obtained by obtaining the defocus change and the contrast change. Therefore, it is more likely to obtain a judgment result of "Yes" in step S803, step S1201, or step 1203.

[0099] In the above-mentioned second embodiment, defocus information about two subjects is detected, and focus control is performed while switching the mode among four focus control modes of focus mode, tilt mode, focus-tilt calculation mode, and focus maintenance tilt mode based on the reliability of the defocus information. In the second embodiment, in the focus maintenance tilt mode, focus control can be performed on an unfocused subject while preventing the focused subject from being out of focus, so the focusing quality is higher than that in the first embodiment.

[0100] In the second embodiment, a case where the number of subjects is two is described, but the number of subjects may be three or more. In a case where the defocus amount of one subject among three or more subjects is reliable and the one subject is focused, the tilt / focus control unit 115 controls in the focus maintenance tilt mode. In other cases, similar processing to that in the first embodiment is applied.

[0101] Third embodiment

[0102] Hereinafter, a process performed by an image pickup apparatus according to a third embodiment will be described. In an image pickup apparatus according to this embodiment, a focus control stop process for a case where the defocus amount cannot be sufficiently reduced or the defocus amount cannot be detected is added to the image pickup apparatus described in the first embodiment or the second embodiment.

[0103] Fig.13 and Fig.14 1 is a flowchart showing the processing in this embodiment. The same reference numerals are given to similar processing to that in the first embodiment, and description thereof will be omitted.

[0104] First, refer to Fig.13 After the control is performed in various focus control modes in step S805, step S808, step S809, step S811, and step S812, the process shifts to the stop determination of the focus process in step S1301. Fig.14 The specific processing of the stop determination in step S1301 is shown.

[0105] First, in step S1401, it is determined whether the focus control mode is the focus-tilt calculation mode. If the focus control mode is the focus-tilt calculation mode, it is counted as the number of executions of the focus-tilt calculation mode in step S1402. In the focus-tilt calculation mode, control is performed so that the calculated defocus amount becomes zero, but depending on, for example, the lack of accuracy of the drive system or the threshold setting, even when the control is repeated multiple times, the defocus amount may not become less than a predetermined value and the focus control may not end. For this reason, the number of executions of the focus-tilt calculation mode is counted so that the counted number can be used for a later stop judgment (step S1409).

[0106] If it is determined in step S1401 that the focus control mode is not the focus-tilt calculation mode (i.e., if the focus control mode is the focus mode or the tilt mode), it is determined in step S1403 whether the focus control mode is different from the previous focus control mode. If the focus control mode is different from the previous focus control mode, it is counted as the number of switching between focus control modes in step S1404. If the focus control mode is frequently switched, it is considered that oscillation occurs near the focus area. Therefore, the number of switching between focus control modes is counted so that the counted number can be used for the stop judgment later (step S1409).

[0107] If it is determined in step S1403 that the focus control mode is the same as the last focus control mode (i.e., if the focus control mode is maintained in the focus mode or the tilt mode), it is determined in step S1405 whether the driving direction is different from the last driving direction. If the driving direction is different from the last driving direction, it is counted as the number of reversals of the focus drive or the tilt drive in step S1406. If the focus drive or the tilt drive is frequently reversed, it is considered that oscillation occurs near the focus area. Therefore, the number of reversals is counted so that the counted number can be used for the stop judgment later (step S1409).

[0108] If it is determined in step S1405 that the driving direction is the same as the last driving direction, it is further determined in step S1407 whether this state continues for a predetermined number of times. If this state is maintained, the driving in the same direction can be continued in the focus mode or the tilt mode, and this state can be in the process of focusing (not near the focus area). Therefore, in step S1408, the number of switching times and the number of inversions between the focus control modes are reset to zero.

[0109] In step S1409, it is determined whether the focus processing is to be stopped (stop determination). Specifically, the execution times of the focus-tilt calculation mode, the switching times between the focus control modes, and the reversal times of the focus drive or the tilt drive counted in steps S1402, S1404, and S1406 are compared with respective thresholds (TH1, TH2, and TH3). If any of the counted times exceeds the threshold, it is determined that the focus processing is to be stopped, and the focus processing is stopped in step S1410.

[0110] In the third embodiment described above, a focus control stop process is added, which is performed when the defocus amount cannot be sufficiently reduced or cannot be detected. As a result, the focus control is not continued unnecessarily, which provides an effect of preventing oscillation of focus change and improving the durability of the drive unit.

[0111] Besides this, as the process of ending the focus control, various modifications are available.

[0112] Other embodiments

[0113] The embodiments of the present invention may also be implemented by providing software (program) for performing the functions of the above-described embodiments to a system or device via a network or various storage media, and a computer or a central processing unit (CPU) or a microprocessing unit (MPU) of the system or device reads and executes the program.

[0114] According to the various embodiments, it is possible to provide an image pickup apparatus capable of focusing on a plurality of subjects at high speed and with high accuracy by performing optimal focus control based on defocus information on the plurality of subjects.

[0115] While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.

Claims

1. A camera device, comprising: a tilt driving unit configured to change the tilt of at least one of the image sensor and the imaging optical system to change the angle between the imaging surface of the image sensor and the main surface of the imaging optical system; a focus drive unit configured to drive a focus lens of the imaging optical system; an area setting unit configured to set a plurality of areas in the captured image; a defocus information detection unit configured to detect defocus information for each of the plurality of regions; as well as a control unit configured to select one control mode from a plurality of control modes for performing focus control using at least one of the tilt drive unit and the focus drive unit based on defocus information about each of the plurality of areas, the plurality of control modes being used to perform focus control using at least one of the tilt drive unit and the focus drive unit, The multiple control modes include at least three of the following modes: a first mode, in which the focus control is performed by controlling the focus drive unit; a second mode, in which the focus control is performed by controlling the tilt drive unit; a third mode, in which the focus control is performed by controlling the tilt drive unit and the focus drive unit based on respective target values ​​of a focal position and a tilt angle, the target value being calculated based on the defocus information; and and a fourth mode in which the focus control is performed while maintaining the in-focus state of a specific area among the plurality of areas by simultaneously controlling the tilt drive unit and the focus drive unit.

2. The imaging device according to claim 1, wherein: The control unit performs the focus control in the third mode when the defocus amounts in two or more than two of the plurality of areas are reliable.

3. The imaging device according to claim 1, wherein: In the case where the defocus directions in two or more than two of the plurality of regions are reliable, If the defocus directions in the two or more areas are consistent, the control unit performs the focus control in the first mode; as well as If the defocus directions in the two or more areas are inconsistent, the control unit performs the focus control in the second mode.

4. The imaging device according to claim 1, wherein: In a case where the defocus directions in less than two of the plurality of areas are reliable, the control unit performs the focus control in the first mode or the second mode.

5. The imaging device according to claim 1, wherein: In a case where the defocus direction in only one of the plurality of areas is reliable, the control unit performs the focus control in the second mode if the defocus amount in the one area is reliable and is smaller than a predetermined value.

6. The imaging device according to claim 1, wherein: In a case where the defocus amount in only one area among the plurality of areas is reliable, the control unit performs the focus control in the fourth mode if the defocus amount in the one area is smaller than a predetermined value.

7. The imaging device according to claim 1, wherein: If all defocus amounts in the plurality of areas are smaller than a predetermined value, the control unit ends the focus control.

8. The imaging device according to claim 1, wherein: The control unit ends the focus control if a predetermined time has elapsed from the start of the focus control.

9. The imaging device according to claim 1, wherein: The defocus information includes a defocus amount and a defocus direction.

10. The imaging device according to claim 1, wherein The number of the plurality of regions is 2.

11. The imaging device according to any one of claims 1 to 10, wherein: The control unit: causing the area setting unit to set two areas among the plurality of areas; causing the defocus information detection unit to detect defocus information for each of the two areas; as well as One control mode is selected from the plurality of control modes based on defocus information about each of the two areas.

12. The imaging device according to any one of claims 1 to 6, wherein: If the focus control has been performed a predetermined number of times or more in the third mode, the control unit ends the focus control.

13. The imaging device according to claim 12, wherein: The control unit ends the focus control if the plurality of control modes have been switched a predetermined number of times or more.

14. The imaging device according to claim 13, wherein: The control unit ends the focus control if the defocus direction in all of the plurality of areas has been reversed a predetermined number of times or more.

15. The imaging device according to claim 14, wherein: The control unit ends the focus control if at least one of the focus driving unit and the tilt driving unit has been reversed a predetermined number of times or more.

16. The imaging device according to claim 15, wherein: When driving is performed continuously for a predetermined number of times or more than a predetermined number of times in the same direction in the first mode or the second mode, the control unit resets at least one of the following numbers: the number of switching between the multiple control modes, the number of reversals of the defocusing direction, and at least one of the number of reversals of the focus drive unit and the tilt drive unit.

17. A control method for an imaging device, the imaging device comprising a tilt drive unit and a focus drive unit, the tilt drive unit being configured to change the tilt of at least one of an image sensor and an imaging optical system to change the angle between an imaging plane of the image sensor and a principal plane of the imaging optical system, the focus drive unit being configured to drive a focus lens of the imaging optical system, the control method comprising: Set multiple areas in the captured image; detecting defocus information for each of the plurality of regions; as well as selecting one control mode from a plurality of control modes for performing focus control using at least one of the tilt drive unit and the focus drive unit based on defocus information about each of the plurality of areas, The multiple control modes include at least three of the following modes: a first mode, in which the focus control is performed by controlling the focus drive unit; a second mode, in which the focus control is performed by controlling the tilt drive unit; a third mode, in which the focus control is performed by controlling the tilt drive unit and the focus drive unit based on respective target values ​​of a focal position and a tilt angle, the target value being calculated based on the defocus information; and and a fourth mode in which the focus control is performed while maintaining the in-focus state of a specific area among the plurality of areas by simultaneously controlling the tilt drive unit and the focus drive unit. 18 . A non-transitory computer-readable storage medium storing a computer program, the computer program causing a computer to execute the control method according to claim 17 .

Images