Image pickup apparatus, control method for image pickup apparatus, and program

Abstract

To provide an image pickup apparatus capable of quickly adjusting focus after completing switching of the image capturing orientation, thereby, enabling a smooth transition to image capturing.SOLUTION: An image pickup apparatus (20) includes: imaging means (201) configured to photoelectrically convert an optical image formed by an optical system including a focus lens (104) and output image data; and control means (204) configured to control the focus lens. The control means acquires information on a start of change in an imaging direction from pivoting drive means (30) configured to perform pivoting drive in at least one of a pan direction or a tilt direction, and drives the focus lens to a predetermined position using the start of the change in imaging direction as a trigger.SELECTED DRAWING: Figure 10(a)

Description

[Technical field]

[0001] The present invention relates to an imaging device, a control method for an imaging device, and a program. [Background technology]

[0002] Conventionally, imaging devices are known in which an imaging unit is mounted on a gimbal that can rotate in the pan / tilt / roll directions. With such imaging devices, it is possible to capture images while suppressing the user's hand shake. In addition, for example, by rotating the pan axis to change the orientation of the imaging unit, the user can switch the capture orientation between a selfie state and a state other than a selfie state while holding the gimbal. When the capture orientation is switched, the state of the subject changes suddenly, so in order to smoothly transition to capture after the capture orientation is switched, it is preferable to quickly adjust the focus in response to the change in the subject.

[0003] Patent Document 1 discloses a method of acquiring the angular velocity of at least one of the three axial directions of a gimbal, and performing autofocus (AF) when the angular velocity remains below a predetermined value for a predetermined period of time. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2020-003730 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, the method disclosed in Patent Document 1 can improve stability during normal shooting, but when the shooting orientation is changed by rotating the axis of the gimbal, it is not possible to speed up the focus adjustment after the shooting orientation is changed. Also, when changing the shooting orientation, the state of the subject to be shot next cannot be accurately grasped until the shooting orientation change is completed. For this reason, even if AF is performed during the shooting orientation change, the expected focus adjustment state is not necessarily achieved after the shooting orientation change, and shooting cannot be smoothly transitioned to after the shooting orientation change.

[0006] Therefore, an object of the present invention is to provide an imaging device that, when changing the shooting orientation by rotating the gimbal axis, can quickly adjust the focus after the shooting orientation change is completed, and can smoothly transition to shooting. [Means for solving the problem]

[0007] An imaging device according to one aspect of the present invention has an imaging means for photoelectrically converting an optical image formed by an optical system including a focus lens to output image data, and a control means for controlling the focus lens, wherein the control means obtains information regarding the start of a change in the shooting direction from a rotation drive means that performs rotation drive in at least either a pan direction or a tilt direction, and drives the focus lens to a predetermined position using the start of the change in the shooting direction as a trigger.

[0008] Other objects and features of the present invention will be described in the following embodiments. Effect of the Invention

[0009] According to the present invention, when the shooting orientation is changed by rotating the gimbal axis, an imaging device can be provided that can quickly adjust the focus after the shooting orientation change is completed, allowing a smooth transition to shooting. [Brief description of the drawings]

[0010] [Figure 1]FIG. 1 is a block diagram of an imaging device according to first, third, fourth and sixth embodiments. [Diagram 2] FIG. 2 is a plan view of a light receiving pixel in the first and third to sixth embodiments, as viewed from the lens unit side. [Diagram 3] 11A to 11C are diagrams illustrating a shooting state when the shooting direction is changed in each embodiment. [Figure 4] 5 is a flowchart showing a moving image shooting process in the first to sixth embodiments. [Diagram 5] 5 is a flowchart showing a process for setting a shooting orientation state in the first, second, and fourth to sixth embodiments. [Figure 6] 13 is a flowchart showing a process for setting a destination to which a focus lens is driven to a predetermined position in the first to third and fifth embodiments. [Figure 7] 13 is a flowchart showing a focus lens drive control process in the first, fifth and sixth embodiments. [Figure 8] 5 is a flowchart showing successive AF processing in the first and third to sixth embodiments. [Figure 9(a)] 13A to 13C are diagrams showing states in a time series when the shooting direction is changed as a comparative example. [Figure 9(b)] 13A to 13C are diagrams showing states in a time series when the shooting direction is changed as a comparative example. [Figure 10(a)] 5A to 5C are diagrams illustrating states in a chronological order when a shooting direction is changed in the first embodiment. [Figure 10(b)] 5A to 5C are diagrams illustrating states in a chronological order when a shooting direction is changed in the first embodiment. [Figure 11(a)] 10A to 10C are diagrams illustrating states in a chronological order when the shooting direction is changed in the comparative example and the first embodiment. [Figure 11(b)] 10A to 10C are diagrams illustrating states in a chronological order when the shooting direction is changed in the comparative example and the first embodiment. [Figure 12] FIG. 11 is a block diagram of an imaging device according to a second embodiment. [Figure 13]FIG. 11 is a plan view of a light receiving pixel according to the second embodiment, as viewed from the lens unit side. [Figure 14] 13 is an explanatory diagram of an AF area used in focus detection processing based on a phase difference in the second embodiment. FIG. [Figure 15] 15 is a diagram showing a pair of image signals obtained from the AF area shown in FIG. 14. [Figure 16] 16 is a diagram showing the relationship between the shift amount and the correlation amount of the pair of image signals shown in FIG. 15. [Figure 17] 16 is a diagram showing the relationship between the shift amount and the correlation change amount of the pair of image signals shown in FIG. 15. [Figure 18] 10 is a flowchart showing a focus lens drive control process in the second embodiment. [Figure 19] 10 is a flowchart showing successive AF processing in the second embodiment. [Figure 20] 13 is a flowchart showing a process for setting a shooting orientation state in the third embodiment. [Figure 21] 13 is a flowchart showing a focus lens drive control process in the third embodiment. [Figure 22(a)] 13A to 13C are diagrams illustrating states in a chronological order when a shooting direction is changed in the third embodiment. [Figure 22(b)] 13A to 13C are diagrams illustrating states in a chronological order when a shooting direction is changed in the third embodiment. [Figure 23] 13 is a flowchart showing a process for setting a shooting direction change speed in the third embodiment. [Figure 24(a)] 13A to 13C are diagrams illustrating states in a chronological order when a shooting direction is changed in the third embodiment. [Figure 24(b)] 13A to 13C are diagrams illustrating states in a chronological order when a shooting direction is changed in the third embodiment. [Figure 25(a)] 13A to 13C are diagrams illustrating states in a chronological order when the shooting direction is changed in the third embodiment. [Figure 25(b)] 13A to 13C are diagrams illustrating states in a chronological order when the shooting direction is changed in the third embodiment. [Figure 26] 13 is a flowchart showing a process for setting a destination to which a focus lens is driven to a predetermined position in the fourth embodiment. [Figure 27] 13 is a flowchart showing a focus lens drive control process in the fourth embodiment. [Figure 28] FIG. 13 is a block diagram of an imaging device according to a fifth embodiment. [Figure 29] 20 is a flowchart showing a focus lens drive control process in the sixth embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

[0012] (First embodiment) First, a first embodiment of the present invention will be described.

[0013] [Configuration of imaging device] 1 is a block diagram showing an example of the functional configuration of a digital camera as an example of an imaging device according to this embodiment. The digital camera according to this embodiment is a lens-integrated camera, and has a lens unit (lens device) 10 equipped with an optical system (imaging optical system), and a camera unit (imaging device body) 20. The digital camera according to this embodiment is also a gimbal-integrated camera, and the lens unit 10 and camera unit 20 are mounted on a gimbal unit 30. Note that in this embodiment, the lens unit 10, camera unit 20, and gimbal unit 30 are integrated, but at least one of them may be configured to be detachable.

[0014] Lens unit 10 has an optical system (first lens group 101, aperture 102, second lens group 103, focus lens group (focus lens) 104, and a drive / control system. Thus, lens unit 10 includes focus lens 104 and is a photographing lens that forms an optical image of a subject.

[0015] The first lens group 101 is disposed at the tip of the lens unit 10 and is held so as to be movable in the optical axis direction. The diaphragm (aperture diaphragm) 102 has the function of adjusting the amount of light during shooting. The diaphragm 102 and the second lens group 103 are movable together in the optical axis direction, and move in conjunction with the first lens group 101 to achieve a zoom function. The focus lens 104 is also movable in the optical axis direction, and the subject distance (focusing distance) at which the lens unit 10 focuses changes depending on the position. Focus adjustment is performed to adjust the focusing distance of the lens unit 10 by controlling the position of the focus lens 104 in the optical axis direction.

[0016] The drive / control system includes a zoom actuator 105, an aperture actuator 106, and a focus actuator 107. The drive / control system also includes a zoom drive circuit 108, an aperture drive circuit 109, a focus drive circuit 110, and a lens control unit 111. The zoom drive circuit 108 drives the first lens group 101 and the third lens group 103 in the optical axis direction using the zoom actuator 105, and controls the angle of view of the optical system of the lens unit 10. The aperture drive circuit 109 drives the aperture 102 using the aperture actuator 106, and controls the aperture diameter and opening / closing operation of the aperture 102. The focus drive circuit 110 drives the focus lens 104 in the optical axis direction using the focus actuator 107, and changes the focal distance of the optical system of the lens unit 10. The focus drive circuit 110 also detects the current position of the focus lens 104 using the focus actuator 107.

[0017] The lens control unit 111 controls the zoom driving circuit 108, the aperture driving circuit 109, and the focus driving circuit 110. The lens control unit 111 also communicates with a camera control unit (control means) 204. For example, the lens control unit 111 detects the position of the focus lens 104, and notifies the camera control unit 204 of focus lens position information. The lens control unit 111 also controls the zoom driving circuit 108, the aperture driving circuit 109, and the focus driving circuit 110 in response to a processing command from the camera control unit 204.

[0018] The camera unit 20 has an image sensor 201, and controls the entire camera including the lens unit 10 and the gimbal unit 30. The image sensor 201 is composed of a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor and peripheral circuits, and is arranged with m pixels in the horizontal direction and n pixels in the vertical direction (m and n are integers of 2 or more). The image sensor 201 is an imaging means that photoelectrically converts an optical image formed by the optical system of the lens unit 10 and outputs image data (imaging signal).

[0019] FIG. 2(a) is a diagram showing an example of an imaging pixel array of the imaging element 201 in this embodiment, and shows a state in which a range of 6 vertical rows (Y direction) and 8 horizontal columns (X direction) of a two-dimensional C-MOS area sensor is observed from the lens unit 10 side. The imaging element 201 is provided with color filters in a Bayer array, and red (R) and green (G) color filters are alternately arranged from the left in the pixels of odd rows, and green (G) and blue (B) color filters are alternately arranged from the left in the pixels of even rows. The pixel 211R will be described with reference to FIG. 2(b). The circle 211i represents an on-chip microlens, and the rectangle 211X arranged inside the on-chip microlens is a photoelectric conversion unit. The pixels 211Gr, 211Gb, and 211B have the same configuration.

[0020] The image sensor drive circuit 202 controls the operation of the image sensor 201, and also A / D converts the acquired image signal and transmits it to a camera control unit 204. The image processing circuit 203 generates contrast AF data and display / recording image data from the image data output by the image sensor 201. In addition, the image processing circuit 203 performs general image processing performed in digital cameras, such as gamma conversion, white balance adjustment processing, color interpolation processing, and compression encoding processing, on the image data acquired by the image sensor 201.

[0021] The camera control unit 204 performs all calculations and control related to the camera unit 20, and controls the image sensor drive circuit 202 and the image processing circuit 203. The camera control unit 204 requests the lens control unit 111 to obtain a focus lens position, to drive the aperture, focus lens, and zoom at a predetermined drive amount, and to obtain optical information specific to the lens unit 10. The camera control unit 204 includes a read only memory (ROM) 204a and a random access memory (RAM) 204b. The camera control unit 204 also includes an electrically erasable programmable read-only memory (EEPROM) 204c. The ROM 204a stores a program that controls the camera operation. The RAM 204b stores variables. The EEPROM 204c stores various parameters and various setting information of the camera unit 20 set by the user.

[0022] The camera control unit 204 receives control commands for the lens control unit 111 and processing commands from the user at the operation unit 303 via the gimbal control unit 301 of the gimbal unit 30 in order to change the operations of the image sensor drive circuit 202 and the image processing circuit 203. The camera control unit 204 also displays an imaging signal on a display unit 302 or records an imaging signal in a memory 304 via the gimbal control unit 301.

[0023] The contrast focus detection unit 205 performs focus detection processing using signal data obtained by the image processing circuit 203. The contrast focus detection unit 205 generates a focus signal by extracting a predetermined frequency component from the luminance signal generated by the image processing circuit 203 by passing the luminance signal through a band pass filter (BPF). Furthermore, the contrast focus detection unit 205 detects a line peak value for each horizontal line within a predetermined imaging range for the focus signal. Furthermore, the contrast focus detection unit 205 integrates the line peak value for each horizontal line in the vertical direction within the predetermined imaging range to generate an integral evaluation value. The camera control unit 204 performs AF by adjusting the position of the focus lens 104 via the lens control unit 111 so as to search for a peak position where the integral evaluation value becomes large. The subject detection unit 206 performs subject detection based on image data obtained by the image processing circuit 203. In this embodiment, subject detection that estimates the position of the target subject in the image data is used to select the focus adjustment result of the contrast focus detection unit 205 so that the camera control unit 204 drives the focus lens 104 via the lens control unit 111. For the subject to be detected, for example, a person's face and the pupils contained therein, an animal's body and the face / pupils contained therein, or the entire vehicle and characteristic parts contained therein (such as the driver or cockpit of the vehicle) are detected. In addition, a subject present at a position specified by the user in the captured image screen is detected through a user's touch operation on the display 302, etc.

[0024] The gimbal unit 30 is a rotation drive means that rotates at least in either the pan direction or the tilt direction. The gimbal unit 30 has a gimbal control unit 301, a display unit 302, an operation unit 303, a memory 304, an inertial measurement unit 305, an angle detector 306, a yaw axis rotation mechanism 307, a pitch axis rotation mechanism 308, and a roll axis rotation mechanism 309.

[0025] The gimbal control unit 301 controls the operation of each unit of the gimbal unit 30. It also exchanges information with the camera control unit 204, and provides information to change the operation of the image sensor drive circuit 202 and the image processing circuit 203. It also controls the zoom drive circuit 108, the aperture drive circuit 109, and the focus drive circuit 110 to the lens control unit 111, and via the camera control unit 204. It receives an image pickup signal from the camera control unit 204, and displays it on the display 302 or records it in the memory 304.

[0026] The display 302 is composed of an LCD (liquid crystal display) and displays information about the camera's shooting mode, a preview image before shooting and a confirmation image after shooting, and a focus state display image when detecting focus. The display 302 has a touch operation function, and the camera can be operated by directly touching the display 302. The operation unit 303 is composed of a power switch, a focus adjustment start switch, a shooting trigger switch, a zoom operation switch, a gimbal operation switch, and the like. The memory 304 is a removable flash memory, and records the captured image. The inertial measurement unit 305 has an acceleration sensor and a gyro sensor, and detects the acceleration of the entire camera device including the lens unit 10, the camera unit 20, and the gimbal unit 30 in three axial directions, namely, forward / backward, left / right, and up / down. It also detects the angular velocity in three axial directions, namely, the yaw axis, the pitch axis, and the roll axis.

[0027] The yaw axis rotation mechanism 306 has a yaw axis motor that drives the lens unit 10 and the camera unit 20 in the yaw axis direction. The pitch axis rotation mechanism 307 has a pitch axis motor that drives the lens unit 10 and the camera unit 20 in the pitch axis direction. The roll axis rotation mechanism 308 has a roll axis motor that drives the lens unit 10 and the camera unit 20 in the roll axis direction.

[0028] The gimbal control unit 301 controls the yaw axis rotation 306, the pitch axis rotation mechanism 307, and the roll axis rotation mechanism 308 to suppress camera shake during shooting by the user. When the user issues a command to drive the yaw axis rotation 306, the pitch axis rotation mechanism 307, or the roll axis rotation mechanism 308 in response to the operation of the gimbal operation switch of the operation unit 303, the gimbal control unit 301 performs control according to the command. The gimbal control unit 301 also controls the yaw axis rotation 306, the pitch axis rotation mechanism 307, and the roll axis rotation mechanism 308 to frame the subject detected by the subject detection unit 206 within the imaging screen.

[0029] As shown in Figs. 3(a) and (b), the yaw axis rotation mechanism 306 or pitch axis rotation mechanism 307 is controlled by using the operation of the operation unit 303 or the display unit 302 as a trigger to change the shooting direction (shooting direction by the image sensor 201) while the user is holding the camera. For example, in Fig. 3(a), the user is shooting a distant subject A in the shooting direction A. In this state, the yaw axis rotation mechanism 306 is rotated by operating the operation unit 303 or the display unit 302, and in Fig. 3(b), the shooting direction can be switched to the shooting direction B of the user (subject B) while still holding the camera device. Examples of operations for changing the shooting direction include pressing a shooting direction switching button on the operation unit 303 or touching a shooting direction switching icon on the touch panel of the display unit 302, but other forms are also acceptable.

[0030] [Details of various processes performed by the camera unit 20] In the camera section 20, the camera control section 204 performs the following processing according to an imaging processing program, which is a computer program. FIG. 4 is a flowchart showing the procedure of the video shooting processing. S means step. The camera control section 204 executes various video shooting controls and AF control by repeatedly performing the video shooting processing. Note that, although only the video shooting processing will be described in this embodiment, this embodiment is also applicable to the shooting processing of still images.

[0031] In S401, the camera control unit 204 determines whether or not an instruction to start video shooting (video shooting instruction) has been input by touching an operation means such as the operation unit 303 or the display 302 via the gimbal control unit 301. A video shooting instruction is notified when a video shooting switch on the operation unit 303 is pressed or a video shooting icon on the display 302 is pressed when video shooting is not in progress. If a video shooting instruction has been notified, the camera control unit 204 advances the process to S402, and if a video shooting instruction has not been notified, the camera control unit 204 advances the process to S407.

[0032] In S402, the camera control unit 204 performs a video shooting process, records the video image in the memory 304 via the gimbal control unit 301, and proceeds to S403. In S403, the camera control unit 204 performs a shooting orientation state setting process, and proceeds to S404. The shooting orientation state setting process will be described in detail later.

[0033] In S404, the camera control unit 204 performs an AF area setting process, and proceeds to S405. In the AF area setting process, the position of the subject in the imaging screen to be AFed is set. If the subject detection unit 206 detects a subject, the AF area is set based on the position and size of the detected subject, and the AF area is updated when the position and size of the detected subject change. If a subject is not detected, the user sets a fixed AF area by touch operation on the operation unit 303 or display 302, or sets multiple AF areas within the screen.

[0034] In S405, the camera control unit 204 causes the contrast focus detection unit 205 to perform focus state detection processing. Processing is performed to acquire integral evaluation value information used for AF, and the process proceeds to S406. In S406, the camera control unit 204 executes focus lens drive control processing, and ends the video shooting process. The focus lens drive control processing will be described in detail later.

[0035] In S407, the camera control unit 204 determines whether or not a video is already being shot. If a video is being shot, the process proceeds to S408. If a video is not being shot, the process proceeds to S403. In S408, the camera control unit 204 determines whether or not an instruction to stop video shooting (video shooting stop instruction) has been input by touch operation on the operation unit 303 or the display unit 302 via the gimbal control unit 301. The video shooting stop instruction is notified when the video shooting switch of the operation unit 303 is pressed or the video shooting icon of the display unit 302 is pressed during video shooting. If the video shooting stop instruction has been notified, the camera control unit 204 proceeds to S409, and if the video shooting stop instruction has not been notified, the camera control unit 204 continues the video shooting process by proceeding to S402. In S409, the camera control unit 204 performs a process to stop video shooting via the gimbal control unit 301 to stop recording of moving images in the memory 304, and proceeds to S403.

[0036] Next, the shooting orientation setting process performed by the camera control unit 204 in S403 in Fig. 4 will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the shooting orientation setting process.

[0037] In S501, the camera control unit 204 determines whether the shooting orientation is being changed. The shooting orientation being changed is the content described with reference to Figs. 3(a) and 3(b), and indicates a state in which the shooting orientation is being changed by rotating the yaw axis rotation mechanism 306 or the pitch axis rotation mechanism 307 by the gimbal control unit 301. That is, it is a state in which the shooting orientation is being changed from the state of Fig. 3(a) to the state of Fig. 3(b), or from the state of Fig. 3(b) to the state of Fig. 3(a). If the shooting orientation is not being changed, the process proceeds to S502, and if the shooting orientation is being changed, the process proceeds to S507. In the following description of the embodiment, the yaw axis rotation mechanism 306 is basically rotated, but it may be replaced with a configuration in which the pitch axis rotation mechanism 307 is rotated.

[0038] In S502, the camera control unit 204 determines whether or not there is an instruction to change the shooting orientation. As described with reference to Figures 3(a) and (b), examples of an instruction to change the shooting orientation include pressing a shooting orientation switching button on the operation unit 303, touching a shooting orientation switching icon on the touch panel of the display unit 302, etc. If there is no instruction to change the shooting orientation, the process proceeds to S503, and if there is an instruction to change the shooting orientation, the process proceeds to S504.

[0039] In S503, the camera control unit 204 sets the shooting orientation to an unchanged state and ends the shooting orientation state setting process. In S504, the camera control unit 204 receives an instruction to change the shooting orientation (information (instruction) regarding the start of changing the shooting orientation), starts switching the shooting orientation (changing the shooting orientation), and proceeds to S505. In S505, the camera control unit 204 clears the predetermined position drive completion state and proceeds to S506. The predetermined position drive completion state is information used for determination in the focus lens drive control process described later with reference to FIG. 7, and details will be described later. In S506, the camera control unit 204 performs a focus lens predetermined position drive destination setting process and proceeds to S508. Details of the focus lens predetermined position drive destination setting process will be described later.

[0040] In S507, which is reached if the shooting orientation is in the changing state in S501, the camera control unit 204 determines whether the change of the shooting orientation is complete. If the change of the shooting orientation is not complete, the process proceeds to S508, and if the change of the shooting orientation is complete, the process proceeds to S509. In S508, the camera control unit 204 sets the shooting orientation to the changing state and ends the shooting orientation state setting process. In S509, the camera control unit 204 sets the shooting orientation to the change completed state and ends the shooting orientation state setting process. Each shooting orientation state set in FIG. 5 is information used for determination in the focus lens drive control process described later with reference to FIG. 7, and details will be described later.

[0041] Next, a process for setting a destination of the focus lens to a predetermined position performed by the camera control unit 204 in S506 in Fig. 5 will be described with reference to Fig. 6. Fig. 6 is a flowchart showing the process for setting a destination of the focus lens to a predetermined position.

[0042] In S601, the camera control unit 204 determines whether the shooting direction after switching is toward the user (toward the photographer). If the shooting direction is toward the user, the process proceeds to S602, and if the shooting direction is not toward the user, the process proceeds to S603. As an example of determining whether the shooting direction is toward the user, it can be determined based on whether the shooting direction is the same as the direction in which the display is located, as shown in FIG. 3(b).

[0043] In S602, the camera control unit 204 sets the focus drive destination of the predetermined position drive to A, and ends the process of setting the focus lens predetermined position drive destination. In S603, the camera control unit 204 sets the focus drive destination of the predetermined position drive to B, and ends the process of setting the focus lens predetermined position drive destination.

[0044] Driving to a specified position is one of the operations in the focus lens drive control process described later with reference to Figure 7, and is an operation of driving the focus lens 104 to a preset drive destination without using an integral evaluation value used in the AF described above.

[0045] The positional relationship between the focus drive destinations A and B of the predetermined position drive is set so that the focus lens position of B is more capable of focusing on a subject on the infinity side than that of A. Details of the operation will be described later, but when the shooting direction is toward the user, that is, when the focus drive destination is A, there is a high possibility that the user will be photographed, that is, there is a high possibility that there is a focus lens position that can focus on a subject distance of about the length of the person's arm. On the other hand, when the shooting direction is not toward the user, that is, when the focus drive destination is B, although any shooting subject is possible, there is a high possibility that people, objects, landscapes, etc. other than the user will be photographed. Therefore, there is a high possibility that there is a focus lens position that can focus on the infinity side more than the focus drive destination A.

[0046] In this way, as a method of setting the focus position drive destination in the focus lens specified position drive destination setting process, it is preferable to set a focus lens position that is likely to be close to the position where focusing can be achieved, assuming the state of the shooting orientation after switching. As an example of specific focus lens movement destinations A and B, A is set to a focus lens position corresponding to a subject distance of 1 meter, and B is set to a focus lens position corresponding to a subject distance of 3 meters.

[0047] Next, the focus lens drive control process performed by the camera control unit 204 in S406 in Fig. 4 will be described with reference to Fig. 7. Fig. 7 is a flowchart showing the focus lens drive control process.

[0048] In S701, the camera control unit 204 determines whether or not the shooting orientation is being changed. If the shooting orientation is being changed, the process proceeds to S702, and if the shooting orientation is not being changed, the process proceeds to S707. In S702, the camera control unit 204 determines whether or not the predetermined position drive has been completed. If the predetermined position drive has been completed, the process proceeds to S703, and if the predetermined position drive has not been completed, the process proceeds to S704. In S703, the camera control unit 204 stops the focus lens and ends the focus lens drive control process.

[0049] In S704, the camera control unit 204 determines whether the focus drive destination has been reached by the predetermined position drive. If the focus drive destination has been reached by the predetermined position drive, the process proceeds to S705, and if the focus drive destination has not yet been reached, the process proceeds to S706. In S705, the camera control unit 204 sets a predetermined position drive completion state and proceeds to S703, where the focus lens is stopped. In S706, the camera control unit 204 controls the focus lens to be driven to the focus movement destination of the predetermined position drive, and ends the focus lens drive control process.

[0050] In S707, which is performed when the shooting orientation is not being changed in S701, that is, when the shooting orientation is not changed or the shooting orientation change is completed, the camera control unit 204 determines whether the shooting orientation change is completed. If the shooting orientation change is completed, the process proceeds to S708, and if the shooting orientation change is not completed, that is, when the shooting orientation change is not completed, the process proceeds to S710. In S708, the camera control unit 204 changes the shooting orientation to the unchangeable state and proceeds to S709. In S709, the camera control unit 204 executes high-speed AF processing and ends the focus lens drive control processing. The high-speed AF processing performs focusing with an emphasis on quickly achieving focus.

[0051] In S710, which is reached if the shooting orientation has not been changed in S707, the camera control unit 204 determines whether or not high-speed AF processing is being performed in S709. If high-speed AF processing is being performed, the process proceeds to S709 to continue high-speed AF processing, and if high-speed AF processing is not being performed, the process proceeds to S711.

[0052] In S711, the camera control unit 204 executes continuous AF processing and ends the focus lens drive control processing. Although a detailed explanation of the continuous AF processing will be given later, unlike the concept of the high-speed AF processing in S709, it assumes that recording will be performed during video shooting, suppresses unnecessary focusing movement, or performs focusing that is not too high speed.

[0053] Next, the successive AF processing performed by the camera control unit 204 in S711 in Fig. 7 will be described with reference to Fig. 8. Fig. 8 is a flowchart showing the successive AF processing.

[0054] In S801, the camera control unit 204 determines whether or not the camera is in a focusing stopped state. If the camera is not in a focusing stopped state, the process proceeds to S802. If the camera is in a focusing stopped state, the process proceeds to S806. In S802, the camera control unit 204 determines whether or not the peak position of the integral evaluation value generated by the contrast focus detection unit 205 has been detected. If the peak position of the integral evaluation value has been detected, that is, if a focus lens position assumed to be the focusing position has been detected, the process proceeds to S803. If the peak position of the integral evaluation value has not been detected, the process proceeds to S805.

[0055] In S803, the camera control unit 204 transitions to a focus stop state and proceeds to S804. At this time, the integral evaluation value at the time of transition to the focus stop state is stored and used in the processing described later. In S804, the camera control unit 204 controls to stop the focus lens and ends the continuous AF processing.

[0056] In S805, the camera control unit 204 has not detected the peak position of the integral evaluation value, i.e., has not detected the focus lens position assumed to be the in-focus position, so drives the focus lens in the direction in which the integral evaluation value increases, continues to control the focus lens to search for the in-focus position, and ends the continuous AF process.

[0057] In S806, the camera control unit 204 refers to the integral evaluation value held when the camera transitioned to the focus stop state in S803, and determines whether or not a predetermined change or more has occurred compared to the latest integral evaluation value. If there is a predetermined change or more, there is a possibility that blurring has occurred or the subject has changed to a different subject due to a change in the distance between the user and the subject since the previous transition to the focus stop state. In this way, it is determined whether or not the integral evaluation value has changed by a predetermined amount since the transition to the focus stop state, and if there has been no change, the process proceeds to S807, and if there has been a change, the process proceeds to S809.

[0058] In S807, the camera control unit 204 determines that no blurring has occurred because there has been little change in the integral evaluation value since the transition to the focus stop state, maintains the focus stop state, and advances the process to S808. In S808, the camera control unit 204 controls the focus lens to stop in the same manner as in S804, and ends the continuous AF process.

[0059] In S809, the camera control unit 204 determines whether the integration evaluation value has changed by a predetermined amount or more since the transition to the focusing stopped state in S806, and then determines whether that state has continued for a predetermined time. If the change in the integration evaluation value is a predetermined amount or more and has continued for the predetermined time, the process proceeds to S810, and if not, the process proceeds to S807 to continue the focusing stopped state.

[0060] In S810, the camera control unit 204 cancels the focusing stop state and proceeds to S811. After determining that the integral evaluation value has changed by a predetermined amount or more since the transition to the focusing stop state, if that state continues for a predetermined period of time, it is determined that there is blurring or a change in the subject, and focusing by AF is performed. In S811, the camera control unit 204 drives the focus lens in the direction in which the integral evaluation value increases, as in S805. It then controls the focus lens to search for a focusing position, and ends the continuous AF process.

[0061] Next, the characteristic parts of this embodiment will be described in detail. First, the function of changing the shooting direction has been described in Figs. 3(a) and (b), but the problem of AF at this time will be described. When switching from the shooting direction A in Fig. 3(a) to the shooting direction B in Fig. 3(b), and when switching from the shooting direction B to the shooting direction A, a change in the subject to be photographed occurs in both cases. In particular, when a large change in the subject distance occurs due to the switching, the focus lens must also be moved according to the change in the subject distance, otherwise the focus will be blurred after the shooting direction is switched. However, in a mechanism that performs AF based on the imaging signal of the imaging element 201, the shooting subject and the subject distance after the shooting direction is switched cannot be detected unless the shooting direction is switched. In other words, in order to perform AF on the shooting subject after the shooting direction is switched, it is necessary to wait for the shooting direction to be switched. It is preferable to quickly bring the camera into focus after the shooting direction is switched, because it allows the user to immediately move to shooting.

[0062] Figures 9(a) and 9(b) are examples of time-series states when the shooting direction is changed as in Figures 3(a) and 3(b). The states are an example of the shooting direction, the state of the camera device, the gimbal control state, the focus lens drive control state, the focus lens position and the movable range of the focus lens in the in-focus state in each of the shooting directions X and Y, and the transition of the focus lens position. Diagrams in the same format as Figures 9(a) and 9(b) will be used hereafter.

[0063] 3(a) and (b), the shooting direction A is the direction toward the background, i.e., the direction toward which the display 303 is not present, and the shooting direction B is the direction toward the user, i.e., the direction toward which the display 303 is present. The shooting orientations X and Y in Fig. 9(a) and Fig. 9(b) may be replaced with either the shooting orientations A and B in Fig. 3(a) and (b). For example, the shooting orientations X and Y in Fig. 9 may be either the shooting orientations A and B in Fig. 3 or the shooting orientations B and A, respectively.

[0064] 9(a) and 9(b) show an example in which a problem occurs when this embodiment is not applied. In a state of shooting orientation X, the operation of starting switching of the shooting orientation is used as a trigger, and the gimbal unit control state changes over time from switching of the shooting orientation (rotating on the yaw axis) to completion of switching of the shooting orientation, and then changes to a state of shooting orientation Y. The transition of the drive control state of the focus lens and the focus lens position at that time is shown.

[0065] In the example of FIG. 9(a), the focus lens drive control state is a focus lens stop state from the start of switching the shooting orientation to the completion of switching, and is in a high-speed AF state after the switching of the shooting orientation is completed. When the drive of the focus lens is controlled as in this example, at the time of completion of switching from the shooting orientation X to the shooting orientation Y, the focus lens position is the in-focus focus lens position in the shooting orientation X before the switching of the shooting orientation. In other words, it is assumed that the focus lens position is located far from the in-focus focus lens position in the shooting orientation Y, and the subject captured in the shooting orientation Y is largely blurred. After this, the subject captured in the shooting orientation Y is AF-ed by high-speed AF, but since the movement stroke of the focus lens is long and it takes time to become in focus, there is a problem that the start of shooting in the shooting orientation Y is delayed.

[0066] FIG. 9(b) is different from FIG. 9(a) in that the focus lens drive control state is a continuous AF state from the start of the switching of the shooting orientation to the completion of the switching. When the drive of the focus lens is controlled as in this example, the focus lens can be driven by AF at the completion of the switching from the shooting orientation X to the shooting orientation Y. However, since the subject captured in the shooting orientation Y cannot be captured within the angle of view unless the switching of the shooting orientation is completed near the completion of the switching of the shooting orientation, the focus lens cannot necessarily be driven to a good position for the state of the shooting orientation Y. As shown in FIG. 9(b), at the completion of the switching of the shooting orientation, the focus lens may be positioned farther away from the subject captured in the shooting orientation Y in some cases. In such a case, as in the explanation of FIG. 9(a), the subject captured in the shooting orientation Y is AF-tracked by high-speed AF after this, but since the moving stroke of the focus lens is long and it takes time to achieve a focused state, there is a problem that the start of shooting in the shooting orientation Y is delayed. An example of an improvement by applying this embodiment to the problem explained with reference to FIG. 9(a) and FIG. 9(b) will be described later.

[0067] Fig. 10(a) and Fig. 10(b) are diagrams showing the state of each time series when the shooting direction is changed in this embodiment. In Fig. 10(a) and Fig. 10(b), unlike Fig. 9(a) and Fig. 9(b), the focus lens is driven to a predetermined position immediately after the shooting direction is changed as the focus lens drive control state. The difference between Fig. 10(a) and Fig. 10(b) is whether the focus lens position to which the predetermined position is driven does not pass the in-focus focus lens position in the shooting direction Y or whether it passes it.

[0068] In the focus lens predetermined position drive, a focus drive destination for the predetermined position drive is determined in S506, that is, in the process of setting the focus lens predetermined position drive destination in Fig. 6, upon receiving an instruction to change the shooting orientation in S502 in Fig. 5. Furthermore, in the focus lens drive control process in Fig. 7, as the focus lens drive when it is determined in S701 that the shooting orientation is being changed, the focus lens is controlled to be driven to the focus drive destination for the predetermined position drive in S706 after the determinations in S702 and S703.

[0069] 10(a) and 10(b), the focus lens is driven to a predetermined position before the switching of the shooting direction is completed, and the focus lens drive control state is set to a focus lens stop state until the switching of the shooting direction is completed. In FIG. 7, when the focus drive destination is reached by the predetermined position drive in S704, the predetermined position drive completion state is set in S705, and the focus lens is controlled to stop in S703. Furthermore, when the shooting direction is being changed in S701, the predetermined position drive is completed in S702, so the focus lens stop state is maintained in S703. After that, in FIG. 10(a) and FIG. 10(b), high-speed AF is performed when the switching of the shooting direction is completed. Since it is determined that the shooting direction is not being changed in S701 of FIG. 7, and the shooting direction change is completed in S707, high-speed AF processing is performed in S709 via S708.

[0070] As a concept of controlling in this way, as described with reference to FIG. 6, when the shooting direction is toward the user, it is highly likely that the user will be photographed, that is, there is a high possibility that there is a focus lens position that can focus on a subject distance of about the length of a person's arm. On the other hand, when the shooting direction is not toward the user, although any shooting subject is possible, there is a high possibility that a person, object, landscape, etc. other than the user will be photographed. A focus movement destination A or B for predetermined position drive according to the switching destination of each shooting direction is set, and control is performed so that the focus lens is moved to that focus lens position in advance during switching of the shooting direction. When switching of the shooting direction is completed, it is highly likely that the focus lens has been moved to a position close to the focus position of the subject captured in the shooting direction Y. By starting high-speed AF from that point, the movement stroke of the focus lens can be shortened compared to FIG. 9(a) and FIG. 9(b), and the focus position can be reached in a shorter time. This allows the user to switch to shooting in a shorter time when switching the shooting direction.

[0071] FIG. 10(a) shows a case where the focus lens position of the predetermined position drive destination does not pass the in-focus focus lens position in the shooting direction Y. However, the predetermined position drive only moves the focus lens to a position aimed at a subject that is expected to be photographed after the shooting direction switching is completed, and it is assumed that the focus lens may pass the in-focus focus lens position in the shooting direction Y as shown in FIG. 10(b). Even in such a case, the concept is the same as in FIG. 10(a), and it is possible to move the focus lens to a position close to the in-focus position of the subject captured in the shooting direction Y when the shooting direction switching is completed. In both FIG. 10(a) and FIG. 10(b), the movement stroke of the focus lens during AF after the shooting direction switching is completed can be shortened, and the in-focus position can be reached in a shorter time.

[0072] 11(a) and 11(b) show another example of the state of each time series when the shooting direction is changed. As in FIG. 10(a) and FIG. 10(b), the focus lens is driven to a predetermined position immediately after the shooting direction is changed. However, FIG. 11(a) and FIG. 11(b) show a case where the focus lens cannot reach the target focus lens position at the timing when the shooting direction is changed. Such a case occurs, for example, when the rotation speed of the yaw axis rotation mechanism 306 of the gimbal unit 30 is fast, when the drive speed at which the focus lens 104 can be driven is slow, or when the drive stroke of the focus lens that must be driven is long.

[0073] 11(a) and 11(b) show cases where the driving speed of the focus lens is slower than those shown in FIG. 9(a), FIG. 9(b), FIG. 10(a), and FIG. 10(b). FIG. 11(a) shows an example where the present embodiment is not partially applied, and shows a case where the focus lens continues to be driven to a predetermined position even when the timing of the switching of the shooting orientation is completed, and then the camera shifts to high-speed AF. As described above, when the switching of the shooting orientation is completed, the camera is capable of AF on the subject captured in the shooting orientation Y, so that AF is considered to be preferable. However, in FIG. 11(a), even after the switching of the shooting orientation is completed, the camera moves to the focus lens position of the predetermined position drive destination and then AF is performed, so that the focus lens is driven extra.

[0074] On the other hand, Fig. 11B shows an example in which this embodiment is applied, which improves the problem in Fig. 10A. In Fig. 11B, at the timing when the photographing orientation switching is completed, the focus lens position of the predetermined position drive destination is not reached by the predetermined position drive, but the focus lens position is shifted to high-speed AF.

[0075] As described above, the purpose of the predetermined position drive is to move the focus lens to a position as close as possible to the subject captured at the time of shooting orientation Y after the shooting orientation switching is completed. Even if the planned predetermined position drive destination is not reached, it can be said that the purpose is achieved to some extent. Also, as described above, when the shooting orientation switching is completed, AF is possible for the subject captured in the shooting orientation Y, so it is considered preferable to perform AF at that time, and the system transitions to high-speed AF when the shooting orientation switching is completed. By controlling in this way, the in-focus position can be reached in a short time by suppressing unnecessary focus lens movement, and the user can transition to shooting in a short time when switching the shooting orientation.

[0076] In addition, in the flow of performing the control of Fig. 11(b), first, it is determined that the shooting orientation is being changed in S701 of Fig. 7. After that, it is determined that the predetermined position drive has not been completed in S702, and that the focus drive destination has not been reached in the predetermined position drive in S704, and control is performed to drive the focus lens to the focus drive destination of the predetermined position drive in S706. After the predetermined position drive is performed in Fig. 11(b), the shooting orientation switching is completed in S509 of Fig. 5, without determining whether the focus drive destination has been reached in the predetermined position drive in S704, and it is determined that the shooting orientation is not being changed in S701 of Fig. 7.

[0077] Thereafter, in S707, it is determined that the shooting orientation change has been completed, and in S708, the shooting orientation is set to an unchanged state, and then high-speed AF processing is performed in S709. Thereafter, if the shooting orientation is not set to an in-change state again in S508 in Fig. 5, the shooting orientation is set to an unchanged state in S503, and after the determinations of S701 and S707 in Fig. 7, if high-speed AF processing is being performed in S710, high-speed AF processing is performed in S709. If high-speed AF processing is completed in S710, the process transitions to continuous AF processing in S711, and, as described above, it is assumed that recording will occur during video shooting, and unnecessary focusing movement is suppressed, or focusing is performed at a not too high speed.

[0078] As described above, in this embodiment, in the function of switching the shooting orientation by rotating the axis of the gimbal, the focus lens is controlled to be driven to a predetermined position when the switching of the shooting orientation starts. The focus lens drive destination when driving to the predetermined position is determined based on the subject distance that is likely to be photographed after the switching of the shooting orientation. By driving to the predetermined position, the focus lens is driven in advance to a focus lens position that is assumed to be likely to be photographed after the switching of the shooting orientation while the shooting orientation is being switched. After that, when the switching of the shooting orientation is completed and the subject after the switching of the shooting orientation is captured and AF is possible, the AF drive is controlled to be performed.

[0079] In this embodiment, immediately after the switching of the shooting orientation is completed, it is preferable to perform high-speed AF so as to quickly achieve a focused state. In addition, depending on the rotation speed of the gimbal axis, the driveable speed of the focus lens, and the length of the stroke to drive the focus lens in the predetermined position drive, the predetermined position drive may not be completed before the switching of the shooting orientation is completed. In that case, the predetermined position drive is stopped at the point where the switching of the shooting orientation is completed, and the control is performed so as to transition to AF. By controlling in this way, when the switching function of the shooting orientation is executed, the focus lens can reach the focus position in a shorter time after the switching of the shooting orientation, and the user can transition to shooting in a shorter time.

[0080] Second embodiment Next, a second embodiment of the present invention will be described, in which the description of the same configuration as the first embodiment will be omitted.

[0081] [Configuration of imaging device] Fig. 12 is a block diagram showing an example of the functional configuration of a digital camera as an example of an imaging device in this embodiment. The lens unit 10 and its internal configuration in Fig. 12 are similar to the lens unit 10 and its internal configuration in Fig. 1, so detailed description will be omitted. The gimbal unit 30 and its internal configuration in Fig. 12 are similar to the gimbal unit 30 and its internal configuration in Fig. 1, so detailed description will be omitted. 204, 204a, 204b, 204c, and 206 of the camera unit 40 in Fig. 12 are similar to 204, 204a, 204b, 204c, and 206 of the camera unit 20 in Fig. 1, respectively, so detailed description will be omitted.

[0082] The image sensor 201 of this embodiment in Fig. 12 has a pupil division function and is capable of phase-difference AF using image data. In addition to the operation of 202 in Fig. 1, the image sensor drive circuit 202 also performs processing such as A / D conversion on data for phase-difference AF. In addition to the operation of 203 in Fig. 1, the image processing circuit 203 also generates data for phase-difference AF. The camera unit 40 in Fig. 12 has an image plane phase-difference focus detection unit 405 instead of the contrast focus detection unit 205 in Fig. 1.

[0083] The image plane phase difference focus detection section 405 performs focus detection processing by a phase difference detection method using focus detection data obtained by the image processing circuit 203. More specifically, the image processing circuit 203 generates pairs of image data formed by light beams passing through two pairs of pupil regions as focus detection data. The image plane phase difference focus detection section 405 then detects the amount of focus deviation based on the amount of deviation of each pair of image data. In this way, the image plane phase difference focus detection section 405 of this embodiment performs phase difference AF (image plane phase difference AF) based on the output of the image sensor 201 without using a dedicated AF sensor.

[0084] [Details of the image plane phase difference focus detection unit 405] The operation of the imaging surface phase difference focus detection unit 405 will be described in detail. Fig. 13(a) is a diagram showing the imaging pixel array of the imaging element 201 in this embodiment, showing a range of 6 vertical rows (Y direction) and 8 horizontal columns (X direction) of a two-dimensional C-MOS area sensor observed from the lens unit 10 side. The imaging element 201 is provided with color filters in a Bayer array, with red (R) and green (G) color filters arranged alternately from left to right in the pixels of odd rows, and green (G) and blue (B) color filters arranged alternately from left to right in the pixels of even rows.

[0085] 13B, the pixel 1311R will be described. A circle 1311i represents an on-chip microlens, and a plurality of rectangles 1311A and 1311B arranged inside the on-chip microlens are photoelectric conversion units. The pixels 1311Gr, 1311Gb, and 1311B have the same configuration.

[0086] In the image sensor 201 of this embodiment, there are pixels (1311R, 1311Gr, 1311Gb, 1311B) in which the photoelectric conversion unit of the imaging pixel is divided into two in the X direction. The photoelectric conversion signals in the individual photoelectric conversion units can be used as data for phase difference AF, or can be used to generate parallax images that form a 3D (3-Dimensional) image. The sum of the photoelectric conversion signals can also be used as normal captured image data.

[0087] Here, pixel signals when performing phase difference AF will be described. In this embodiment, the microlens 1311i in FIG. 13(b) and the divided photoelectric conversion units 1311A and 1311B divide the emitted light beam of the photographing optical system into pupils. The photoelectric conversion units 1311A and 1311B in FIG. 13(b) are used as a pair. This makes it possible to perform focus detection based on the image shift amount (phase difference) in the X direction.

[0088] A phase difference AF by focus detection based on the image shift amount in the X direction will be described. In FIG. 13(b), the signal of the photoelectric conversion unit 1311A arranged in a plurality of pixels 1311R within a predetermined range arranged in the same pixel row is defined as an AF A image. And the signal of the photoelectric conversion unit 1311B is defined as an AF B image. The output of the photoelectric conversion unit 1311A and the photoelectric conversion unit 1311B uses a pseudo luminance (Y) signal calculated by adding the outputs of green, red, blue, and green included in the unit array of the color filter. However, the AF A image and the AF B image may be arranged for each color of red, blue, and green. By detecting the relative image shift amount of a pair of image signals, which are the AF A image and the AF B image generated in this way, by correlation calculation, it is possible to detect a prediction, which is the correlation degree of the pair of image signals. The camera control unit 204 can detect the defocus amount of a predetermined area by multiplying the prediction by a conversion coefficient. The sum of the outputs of photoelectric conversion unit 1311A and photoelectric conversion unit 1311B forms one pixel (output pixel) of the output image.

[0089] Next, the focus detection process will be described in detail. Fig. 14 shows an example of an AF area 1402 on a pixel array 1401 of the image sensor 201 in this embodiment during focus detection.

[0090] Shift areas 1403 on both sides of the AF area 1402 are areas necessary for the correlation calculation. Therefore, area 1404 which is the combination of the AF area 1402 and the shift area 1403 is the pixel area necessary for the correlation calculation. In the figure, p, q, s, and t each represent coordinates in the X direction, with p and q representing the X coordinates of the start point and end point of the pixel area 1404, and s and t representing the X coordinates of the start point and end point of the AF area 1402.

[0091] 15(a) to (c) show an example of a pair of image signals for AF acquired from multiple pixels included in the AF area 1402 shown in FIG. 14. A solid line 1501 is one of the AF A images, and a dashed line 1502 is the AF B image. FIG. 15(a) shows the AF A image and the AF B image before shifting, and FIG. 15(b) and (c) show the AF A image and the AF B image shifted in the plus direction and the minus direction, respectively, from the state of FIG. 15(a). When calculating the correlation amount of the pair of AF A images 1501 and B images 1502, both of the AF A image 1501 and the AF B image 1502 are shifted by one bit in the direction of the arrow.

[0092] 15B and 15C, the AF A image 1501 and the B image 1502 are each shifted by one bit to calculate the sum of the absolute values ​​of the differences between the AF A image 1501 and the B image 1502. When the shift amount is i, the maximum shift amount in the negative direction is ps, the maximum shift amount in the positive direction is qt, x is the start coordinate of the AF area 1402, and y is the end coordinate of the AF area 1402, the correlation amount COR can be calculated by the following formula (1).

[0093]

number

[0094] Fig. 16(a) shows an example of the relationship between the shift amount and the correlation amount COR. In Fig. 16(a), the horizontal axis shows the shift amount, and the vertical axis shows the correlation amount COR. Among extreme values ​​1602, 1603 in a correlation amount 1601 that changes with the shift amount, the degree of match between a pair of AF images A and B is highest at a shift amount corresponding to a smaller correlation amount.

[0095] Next, a method for calculating the correlation change amount will be described. The difference between the correlation amounts for every other shift in the waveform of the correlation amount 1601 shown in Fig. 16(a) is calculated as the correlation change amount. If the shift amount is i, the maximum shift amount in the negative direction is ps, and the maximum shift amount in the positive direction is qt, the correlation change amount ΔCOR can be calculated by the following formula (2).

[0096]

number

[0097] Fig. 17(a) shows an example of the relationship between the shift amount and the correlation change amount ΔCOR. In Fig. 17(a), the horizontal axis represents the shift amount, and the vertical axis represents the correlation change amount ΔCOR. The correlation change amount 1701, which changes with the shift amount, goes from positive to negative at portions 1702 and 1703. The state in which the correlation change amount becomes 0 is called a zero cross, and the degree of match between the pair of AF images A and B is highest. Therefore, the shift amount that gives the zero cross is the amount of image shift.

[0098] Fig. 17(b) shows an enlarged view of the portion indicated by 1702 in Fig. 17(a). Reference numeral 1704 denotes a portion of the correlation change amount 1701. A method for calculating the image shift amount will be described with reference to Fig. 17(b).

[0099] The shift amount (k-1+α) that gives the zero crossing is divided into an integer part β (=k-1) and a decimal part α. The decimal part α can be calculated by the following formula (3) based on the similarity between triangles ABC and ADE in the figure.

[0100]

number

[0101] The integer part β can be calculated from FIG. 17(b) using the following formula (4).

[0102]

number

[0103] Then, the amount of image shift, that is, a prediction which is the degree of correlation between a pair of image signals, can be detected from the sum of α and β.

[0104] As shown in Fig. 17(a), when there are multiple zero crossings of the correlation change amount ΔCOR, the one in the vicinity of which the change in the correlation change amount ΔCOR has a greater steepness is determined to be the first zero crossing. This steepness is an index showing the ease of AF, and the larger the value, the easier it is to perform accurate AF. The steepness maxder can be calculated using the following formula (5).

[0105]

number

[0106] As described above, in this embodiment, when there are multiple zero crossings in the correlation change amount, the first zero crossing is determined based on the steepness of the zero crossings, and the shift amount that gives the first zero crossing is set as a prediction.

[0107] Next, a method for calculating the reliability of the image shift amount will be described. The reliability of the image shift amount can be defined by the degree of coincidence (two-image coincidence) fnclvl between a pair of AF images A and B and the steepness of the correlation change amount described above. The two-image coincidence is an index that represents the accuracy of the image shift amount, and in the correlation calculation method of this embodiment, the smaller the value, the better the accuracy.

[0108] Fig. 16(b) is an enlarged view of the portion indicated by 1602 in Fig. 16(a), and 1604 is a portion of the correlation amount 1601. The two-image coincidence degree fnclvl can be calculated by the following formula (6).

[0109]

number

[0110] [Details of various processes performed by the camera unit 40] In this embodiment, the focus lens drive control process of S406 in Fig. 4 is performed according to the flowchart shown in Fig. 18. Fig. 18 is a flowchart showing the focus lens drive control process. The processes from S1801 to S1808 and from S1812 to S1814 in Fig. 18 are similar to the processes from S701 to S708 and from S709 to S711 in Fig. 7, and therefore detailed descriptions thereof will be omitted.

[0111] In S1809 of FIG. 18, the camera control unit 204 determines whether the reliability of the defocus amount is higher than a predetermined level. If the reliability of the defocus amount is higher than the predetermined level, the process proceeds to S1812, and high-speed AF processing is executed in the same manner as in the process of S709 of FIG. 7. On the other hand, if the reliability of the defocus amount is not higher than the predetermined level, the process proceeds to S1810. It is preferable that the reliability threshold of the defocus amount set in S1809 is set to the maximum value of a reliability range in which not only the calculated defocus amount but also the defocus direction is unreliable as the predetermined reliability. The reliability of the defocus amount may be calculated using both the two-image coincidence degree and the steepness of the image shift amount, or may be calculated using only one of them. In addition, other indices such as a signal level may be used.

[0112] In S1810, the camera control unit 204 determines whether or not the state in which the reliability of the defocus amount is lower than a predetermined value has continued for a predetermined time. If the state in which the reliability of the defocus amount is lower than a predetermined value has not continued for the predetermined time, the process proceeds to S1810, and if the predetermined time has elapsed, the process proceeds to S1812, where high-speed AF processing is executed. In S1811, the camera control unit 204 controls the focus lens to stop in the same manner as in S1803, and ends the focus lens drive control processing.

[0113] Unlike the configuration for contrast focus detection described in the first embodiment, this embodiment allows image plane phase difference detection. Therefore, when AF is performed, the focus position and direction can be determined without driving the focus lens to search for the increase / decrease direction of the integral evaluation value. On the other hand, if the reliability of the defocus amount is so low that the defocus direction is also unreliable, careless AF execution may cause unnecessary focus lens driving, and it may take time to focus. For example, there may be cases where the reliability of the defocus amount is insufficient, such as when the subject is too dark / too bright after switching the shooting direction due to a significant difference in brightness of the subject before and after switching the shooting direction, or when the subject moves and blurs significantly.

[0114] In such a case, in S1809, it is determined whether the reliability of the defocus amount is higher than a predetermined level, and instead of immediately executing high-speed AF processing in S1812, the determination is made in S1810, and control is performed so that the focus lens is temporarily stopped in S1811. Thereafter, when the reliability of the defocus amount is improved, for example, by adjusting the exposure for shooting or reducing the blurring of the subject, and it becomes possible to determine in S1809 that the reliability of the defocus amount is higher than a predetermined level, high-speed AF processing is performed in S1812. By controlling in this manner, unnecessary movement of the focus lens can be suppressed.

[0115] In addition, in S1810, it is determined whether or not the state in which the reliability of the defocus amount is lower than a predetermined value has continued for a predetermined time. If the state in which the reliability of the defocus amount is lower than a predetermined value continues, it is possible that, for example, the focus lens drive destination of the predetermined position drive is deviated from the in-focus focus lens position of the subject after the shooting direction is switched, causing out-of-focus. Therefore, if the state in which the reliability of the defocus amount is lower than a predetermined value continues for a predetermined time in S1810, control is performed in S1812 to search for a focus lens position where focus can be achieved. It is preferable that the threshold value of the predetermined time set in S1810 is set in consideration of a balance between the case in which the reliability of the defocus amount described above is increased before the high-speed AF process is performed and the case in which the in-focus position needs to be searched for by the high-speed AF process due to the occurrence of out-of-focus.

[0116] Next, the continuous AF process performed by the camera control unit 204 in S1814 of Fig. 8 will be described with reference to Fig. 19. Fig. 19 is a flowchart showing the continuous AF process. The processes of S1901, S1904, S1905, S1911, S1912, and S1914 in Fig. 19 are similar to the processes of S801, S803, S804, S807, S808, and S810 in Fig. 8, respectively, and therefore detailed description thereof will be omitted.

[0117] In S1902 of FIG. 19, the camera control unit 204 determines whether the reliability of the defocus amount is equal to or greater than a predetermined value. If it is equal to or greater than the predetermined value, the process proceeds to S1903, and if it is not equal to or greater than the predetermined value, the process proceeds to S1908. The reliability threshold of the defocus amount set in S1902 is preferably set to the maximum value of a reliability range in which not only the calculated defocus amount but also the defocus direction is unreliable as the predetermined reliability. Note that the reliability of the defocus amount may be calculated using both the degree of coincidence between the two images and the steepness of the image shift amount, or may be calculated using only one of them. Other indices such as a signal level may also be used.

[0118] In S1903, the camera control unit 204 determines whether the defocus amount is within the focal depth, and if it is within the focal depth, proceeds to S1904, and if it is not within the focal depth, proceeds to S1906. In S1906, the camera control unit 204 determines that a focused state has not yet been obtained, performs lens drive setting for driving the focus lens 104 based on the defocus amount, and proceeds to S1907. In S1907, the camera control unit 204 transmits a drive command for the focus lens 104 to the lens control unit 111 based on the defocus amount and the information of the lens drive setting set in S1906, drives the focus lens, and ends the continuous AF process.

[0119] In S1908, the camera control unit 204 cannot use the defocus amount to drive the focus lens 104 because the reliability of the defocus amount is low. Therefore, the camera control unit 204 performs search drive to calculate the defocus amount while moving the focus lens 104 toward its movable end in order to detect the position of the focus lens 104 at which a defocus amount with high reliability can be obtained. The camera control unit 204 first performs lens drive setting for search drive. The lens drive setting for search drive is a setting of the drive speed of the focus lens 104, the direction in which to start driving, etc. In S1909, the camera control unit 204 drives the focus lens by transmitting a control command for the focus lens 104 to the lens control unit 111 based on the lens drive setting for search drive set in S1908, and ends the continuous AF process.

[0120] In S1910, the camera control unit 204 determines whether the defocus amount is within the focal depth. If it is within the focal depth, the process proceeds to S1911 and the focus stop state is maintained, and if it is not within the focal depth, the process proceeds to S1913. In S1913, the camera control unit 204 determines whether the state in which the defocus amount is not within the focal depth has continued for a predetermined time. If it is, the process proceeds to S1914 and the focus stop state is released, and if it is not, the process proceeds to S1911 and the focus stop state is maintained.

[0121] (Third embodiment) Next, a third embodiment of the present invention will be described, in which the description of the same configuration as the first embodiment will be omitted.

[0122] [Details of various processes performed by the camera unit 20] In this embodiment, the shooting orientation state setting process of S403 in Fig. 4 follows the flowchart shown in Fig. 20. Fig. 20 is a flowchart showing the shooting orientation state setting process. The processes from S2001 to S2006 and from S2008 to S2010 in Fig. 20 are similar to the processes from S501 to S506 and from S507 to S509 in Fig. 5, respectively, and therefore detailed explanations will be omitted.

[0123] 20, the camera control unit 204 performs various processes when there is an instruction to change the shooting direction in S2002, and then performs a process of setting the shooting direction change speed, and proceeds to S2009. Details of the process of setting the shooting direction change speed will be described later.

[0124] Next, in this embodiment, the focus lens drive control process of S406 in Fig. 4 follows the flowchart shown in Fig. 21. Fig. 21 is a flowchart showing the focus lens drive control process. The processes from S2101 to S2105 and from S2109 to S2111 in Fig. 21 are similar to the processes from S701 to S705 and from S706 to S708 in Fig. 7, respectively, and therefore detailed descriptions are omitted. Also, the processes from S2113 to S2115 in Fig. 21 are similar to the processes from S709 to S711 in Fig. 7, respectively, and therefore detailed descriptions are omitted.

[0125] In S2106, the camera control unit 204 determines whether or not video shooting processing is in progress after determining in S2104 that focus drive by predetermined position drive is being performed. If video shooting processing is in progress, the process proceeds to S2108, and if video shooting processing is not in progress, the process proceeds to S2107. In S2107, the camera control unit 204 sets the focus lens drive speed during predetermined position drive to fsA, and proceeds to S2109. In S2108, the camera control unit 204 sets the focus lens drive speed during predetermined position drive to fsB, and proceeds to S2109.

[0126] Regarding the focus lens drive speed during drive to the predetermined position, fsA set in S2107 and fsB set in S2108 will be described in detail later, but in terms of relationship, fsA is set to be slower than fsB. That is, when it is determined in S2106 that video shooting processing is in progress, the focus lens drive speed during drive to the predetermined position is set to be slower. In S2112, the camera control unit 204 determines whether or not video shooting processing is in progress, similar to S2106. If video shooting processing is in progress, the process proceeds to S2113, and if video shooting processing is not in progress, the process proceeds to S2115.

[0127] Fig. 22(a) and Fig. 22(b) are diagrams showing the state of each time series when the shooting direction is changed in this embodiment. Fig. 22(a) and Fig. 22(b) show an example in which the driving method of the focus lens is changed during preparation for video shooting in Fig. 22(a) and during video shooting in Fig. 22(b), based on the example of Fig. 10(a).

[0128] In FIG. 22(a) during preparation for video shooting, after the determination in S2106, the focus lens drive speed during drive to the predetermined position is set to fsB in S2108. The focus lens drive speed fsB is faster than fsA, as described above. After determining that the change in shooting orientation is complete in S2110, after the determination in S2112, control is performed to execute high-speed AF processing in S2113. During preparation for video shooting, unlike during video shooting described later, it is assumed that the focus lens drive state will not be recorded as a video. Therefore, placing more importance on focusing as quickly as possible, the focus lens drive speed during drive to the predetermined position is set faster, and high-speed AF processing is also executed for the AF operation after the shooting orientation change is completed.

[0129] On the other hand, in FIG. 22(b) during video shooting, after the judgment in S2106, the focus lens drive speed during driving to the predetermined position is set to fsA in S2107. As described above, the focus lens drive speed fsA is preferably slower than fsB. After the shooting direction change is judged to be complete in S2110, the control is performed to execute continuous AF processing in S2115 after the judgment in S2112. During video shooting, it is assumed that the focus lens drive state is also recorded as a video. For example, if the focus lens is driven too fast, or the focus lens passes the focus lens position once and returns, etc., and such behavior is recorded, the video may not look good. In addition, if the focus lens is driven fast, the drive sound of the focus lens may become louder, and this may be recorded in the video. Therefore, a slower focus lens drive speed is set with emphasis on the quality of the drive of the focus lens rather than on quickly focusing. In addition, the AF operation after the shooting direction change is completed also executes continuous AF processing with emphasis on the quality of the drive of the focus lens.

[0130] Next, the process of setting the shooting direction change speed performed by the camera control unit 204 in S2007 in Fig. 20 will be described with reference to Fig. 23. Fig. 23 is a flowchart showing the process of setting the shooting direction change speed.

[0131] In S2301, the camera control unit 204 determines whether or not a moving image shooting process is in progress. If a moving image shooting process is not in progress, the process proceeds to S2301, and if a moving image shooting process is in progress, the process proceeds to S2303.

[0132] In S2302, the camera control unit 204 sets jsA as the speed at which the gimbal axis is rotated when changing the shooting orientation to the gimbal control unit 301. The gimbal control unit 301 sets the rotation speed of the yaw axis rotation mechanism 306 when switching the shooting orientation to jsA, and ends the shooting orientation change speed setting process.

[0133] In S2303, the camera control unit 204 sets jsB to the gimbal control unit 301 as the speed at which the gimbal axis is rotated when the shooting orientation is changed. The gimbal control unit 301 sets the rotation speed of the yaw axis rotation mechanism 306 when switching the shooting orientation to jsB, and proceeds to S2304. In S2304, the camera control unit 204 calculates the time ft for the focus lens to reach the destination of the predetermined position drive, and proceeds to S2305. In S2305, the camera control unit 204 calculates the time jt until switching of the shooting orientation is completed, and proceeds to S2306. At this time, jt is calculated based on the gimbal axis rotation speed jsB set in S2303.

[0134] In S2306, the camera control unit 204 determines whether jt is shorter than ft. If jt is shorter than ft, that is, it indicates that the switching of the shooting orientation is completed earlier than the completion of the focus lens drive by the predetermined position drive. If jt is shorter than ft, the process proceeds to S2307, and if not, the gimbal axis rotation speed jsB set in S2303 is confirmed and the setting process of the shooting orientation change speed is terminated.

[0135] In S2307, the camera control unit 204 sets jsC to the gimbal control unit 301 as the speed at which the gimbal axis is rotated when the shooting orientation is changed. The gimbal control unit 301 sets the rotation speed of the yaw axis rotation mechanism 306 when switching the shooting orientation to jsC, and ends the shooting orientation change speed setting process. As described above, in the description of this embodiment, the shooting orientation is basically changed by rotating the yaw axis rotation mechanism 306, but the shooting orientation may be changed by rotating the pitch axis rotation mechanism 307.

[0136] The gimbal axis rotation speed will be described in detail later, but in terms of the relationship between jsA and jsB, jsB is set to a speed slower than jsA. Also, jsC controls to determine the gimbal axis rotation speed based on ft calculated in S2304 so that the time until the shooting direction switching is completed is equivalent to ft. When jsC is set in S2307, the time jt until the shooting direction switching is completed calculated at the speed of jsB in S2305 is shorter than ft in S2306, so jsC will be a slower speed than jsB.

[0137] Fig. 24(a) and Fig. 24(b) are diagrams showing the state of each time series when the shooting direction is changed in this embodiment. As in Fig. 22(a) and Fig. 22(b), Fig. 24(a) shows an example of preparation for video shooting, and Fig. 24(b) shows an example of video shooting. However, unlike Fig. 22, a change has been made to the method of switching the shooting direction.

[0138] In FIG. 24(a) during preparation for video shooting, after the judgment in S2301, the gimbal axis rotation speed is set to jsA in S2302. The gimbal axis rotation speed jsA is faster than jsB as described above. During preparation for video shooting, unlike during video shooting described below, it is assumed that the state when switching the shooting direction by rotating the gimbal axis will not be recorded as a video. For this reason, emphasis is placed on switching the shooting direction as quickly as possible, and the gimbal axis rotation speed is set to be faster.

[0139] On the other hand, in FIG. 24(b) during video shooting, after the judgment in S2302, in S2303, the gimbal axis rotation speed when changing the shooting direction is set to jsB. As described above, the gimbal axis rotation speed jsB is preferably slower than jsA. Note that in the example of FIG. 24(b), the focus lens is driven to the specified position before the shooting direction is changed, that is, the case is assumed in which the case is not applicable in S2306 and the speed of jsB is set. During video shooting, as described above, it is assumed that the state of changing the shooting direction by rotating the gimbal axis is also recorded as a video. If the gimbal axis is rotated too quickly when the shooting direction is changed, the video may not look good. In addition, if the gimbal axis rotation speed is fast, the driving sound of the gimbal axis rotation may become louder, and this may be recorded in the video. Therefore, a slower gimbal axis rotation speed is set with an emphasis on the quality of the angle of view change when the shooting direction is changed and the suppression of the driving sound, rather than on changing the shooting direction quickly.

[0140] Figures 25(a) and 25(b) are figures showing the state of each time series when the shooting direction is changed, and Figure 25(a) shows the case where the processing of S2307 in Figure 23 is not applied during video shooting, while Figure 25(b) shows the case where it is applied.

[0141] FIG. 25(a) shows a case where the focus lens has not been driven to a predetermined position when the switching of the shooting orientation is completed under the conditions during video shooting. In such a case, as described above, the drive of the focus lens to the predetermined position is interrupted (stopped) and control is performed to execute continuous AF. However, FIG. 25(b) further adds the control of the flowchart of FIG. 23, and controls so that the time of switching the shooting orientation is synchronized with the completion time of the drive of the focus lens to the predetermined position during video shooting. In S2304, the arrival time ft of the focus lens to the destination of the drive to the predetermined position is calculated, and further in S2305, the time jt until the switching of the shooting orientation is completed is calculated.

[0142] In Fig. 25(b), it is determined in advance in S2306 that jt is shorter than ft, i.e., that the focus lens has not been driven to the specified position when the shooting orientation is switched, and the gimbal axis rotation speed is set to jsC in S2307. As described above, jsC is controlled to set the gimbal axis rotation speed adjusted to match the time when the focus lens has been driven to the specified position, so that it matches the time when the shooting orientation is switched. This makes it possible to set jsC that is slower than the gimbal axis rotation speed jsB set in S2303, thereby further improving the quality of the angle of view change when the shooting orientation is switched and suppressing drive noise.

[0143] (Fourth embodiment) Next, a fourth embodiment of the present invention will be described, in which the description of the same configuration as the first embodiment will be omitted.

[0144] [Details of various processes performed by the camera unit 20] In this embodiment, the process of setting the destination of driving the focus lens to a predetermined position in S506 in Fig. 5 follows the flowchart in Fig. 26. Fig. 26 is a flowchart showing the process of setting the destination of driving the focus lens to a predetermined position. The processes from S2605 to S2607 in Fig. 26 are similar to the processes from S601 to S603 in Fig. 6, respectively, and therefore detailed descriptions thereof will be omitted.

[0145] In S2601, the camera control unit 204 determines whether or not the size of the image sensor 201 is larger than a predetermined size (whether or not the image sensor size is larger than a predetermined size). If the size of the image sensor 201 is larger than a predetermined size, the process proceeds to S2604, and if the size of the image sensor 201 is not larger than a predetermined size, the process proceeds to S2602. In S2602, the camera control unit 204 determines whether or not the set aperture as the state of the aperture 102 is brighter than a predetermined value. If the set aperture (aperture value, F-number) is brighter than a predetermined value (predetermined aperture value, predetermined F-number), the process proceeds to S2604, and if the set aperture is not brighter than a predetermined value, the process proceeds to S2603.

[0146] In S2603, the camera control unit 204 determines whether the focal length (set focal length) as the state of the lens group is more telephoto than the predetermined position. If the focal length is more telephoto than the predetermined position, the process proceeds to S2604, and if the focal length is not more telephoto than the predetermined position, the process proceeds to S2608. In S2604, the camera control unit 204 sets a predetermined position drive use state, and in S2605 to S2607, sets a predetermined position drive destination for the focus lens predetermined position drive process. In S2608, the camera control unit 204 sets a predetermined position drive unused state, and ends this process without setting the focus lens predetermined position drive destination.

[0147] In this embodiment, the focus lens drive control process of S406 in Fig. 4 is based on the flowchart of Fig. 27. Fig. 27 is a flowchart showing the focus lens drive control process. The processes of S2701 and S2703 to S2712 in Fig. 27 are similar to the processes of S701 to S711 in Fig. 7, respectively, and therefore detailed descriptions thereof will be omitted.

[0148] In S2702, the camera control unit 204 determines whether or not the camera is in a predetermined position drive use state. If the predetermined position drive use state is in use, the process proceeds to S2703, and processing related to the predetermined position drive is performed. On the other hand, if the predetermined position drive is not used, the process proceeds to S2708, and even if it is determined in S2701 that the shooting orientation is changing, control is performed so that processing related to the predetermined position drive is not performed. At this time, if the shooting orientation is changing, the determinations of S2708 and S2711 are made, and continuous AF processing is performed in S2712.

[0149] With reference to Figs. 3(a) and (b), it has been described that when the shooting direction is switched from A to B or when the shooting direction is switched in the opposite direction, a change in the subject to be photographed occurs. In this case, if a large change in the subject distance occurs, the possibility of blurring of the focus after switching the shooting direction is also as described above, unless the focus lens is moved according to the change in the subject distance. On the other hand, if the optical system is not in a state where blurring of the focus is noticeable, the possibility of blurring of the focus occurring is small. That is, as described with reference to Fig. 26, it is assumed that the degree of blurring of the focus caused by the change in the subject distance is small when the conditions such as the image sensor size being large, the set aperture being brighter than the specified, and the set focal length being more telephoto than the specified are not met. In such a case, the process of S2608 in Fig. 26 and S2702 in Fig. 27 may be performed so that the focus lens is not driven to the specified position.

[0150] Fifth embodiment Next, a fifth embodiment of the present invention will be described. Note that a description of the same configuration as the first embodiment will be omitted.

[0151] [Configuration of imaging device] Fig. 28 is a block diagram showing an example of the functional configuration of a digital camera and a gimbal as an example of an imaging device in this embodiment. In this embodiment, the lens unit 10 and the camera unit 50 are separable (detachable) from the gimbal unit (rotation drive means) 60, and can communicate with each other. The lens unit 10 and its internal configuration in Fig. 28 are similar to the lens unit 10 and its internal configuration in Fig. 1, so detailed explanations are omitted. 201 to 203 and 204a, 204b, 204c, 205, and 206 of the camera unit 50 in Fig. 28 are similar to 201 to 203 and 204a, 204b, 204c, 205, and 206 of the camera unit 20 in Fig. 1, so detailed explanations are omitted.

[0152] As described above, the gimbal unit 60 in FIG. 28 can be separated from the lens unit 10 and the camera unit 50. 303 and 305 to 308 of the gimbal unit 60 are similar to 303 and 305 to 308 of the gimbal unit 30 in FIG. 1, and therefore detailed description thereof will be omitted. The gimbal unit 60 in FIG. 28 does not have the display 302 and memory 304 of the gimbal unit 30 in FIG. 1. On the other hand, the camera unit 50 in FIG. 28 has a display 507, an operation unit 508, and a memory 509. The display 507 and memory 509 of the camera unit 50 in FIG. 28 have the same functions as the display 302 and memory 304 of the gimbal unit 30 in FIG. 1, and detailed description thereof will be omitted. The operation unit 508 of the camera unit 50 basically operates the lens 10 and the entire camera unit 50. The operation unit 303 of the gimbal unit 60 basically operates the entire gimbal unit 60.

[0153] The camera control unit 204 of the camera unit 50 and the gimbal control unit 301 of the gimbal unit 60 in Fig. 28 have a function of communicating with each other, and can exchange information similar to that between the camera control unit 204 of the camera unit 20 and the gimbal unit 30 in Fig. 1. The communication method may be any method, such as wireless communication or wired communication.

[0154] [Details of various processes performed by the camera unit 50] In this embodiment, the various processes performed by camera unit 50 are similar to those described in detail as the various processes performed by camera unit 20 in the first embodiment. However, this is not limited to an imaging device in which the camera unit and the gimbal unit are integrally configured as in the first embodiment, and can also be applied to a configuration in which the camera unit and the gimbal unit are separable from each other as in this embodiment.

[0155] Sixth embodiment Next, a sixth embodiment of the present invention will be described. Note that a description of the same configuration as the first embodiment will be omitted.

[0156] [Details of various processes performed by the camera unit 20] In this embodiment, the process of setting the destination of the focus lens to be driven to a predetermined position in S506 in Fig. 5 follows the flowchart of Fig. 29. Fig. 29 is a flowchart showing the process of setting the destination of the focus lens to be driven to a predetermined position.

[0157] In S2901, the camera control unit 204 acquires user registration information stored in the EEPROM 204c. The user registration information stores the focus lens predetermined positions registered (set) by the user in the shooting directions A and B shown in Figs. 3(a) and (b). The user can register at least one focus lens predetermined position when shooting a video, etc. After acquiring the user registration information, the process proceeds to S2902.

[0158] In S2902, the camera control unit 204 acquires AF information, which is the result of automatic focus adjustment (AF) stored in the RAM 204b (the result of past AF). The information stored as the AF information includes at least two predetermined positions of the focus lens when AF is performed in each of the shooting directions A and B, together with the AF time (Taf) when AF is performed. When the AF information is used in the process of setting the drive destination of the focus lens to the predetermined position, if the time difference Δt (t-Taf) between the current time (t) and the most recent AF time (Taf1) stored is within a predetermined time, the most recent AF time (Taf1) is referenced. Note that the premise for referencing Taf1 is that the power switch of the operation unit 303 has not been operated during the time from Taf1 to t, or that the power off process such as the camera not being automatically turned off in the camera control unit 204 has not been performed.

[0159] Also, when it is determined that the time difference Δt is longer than a predetermined time and time has passed after AF, or when the above-mentioned power off process is performed, the camera control unit 204 uses at least two pieces of AF information stored in the RAM 204b. That is, in this case, the camera control unit 204 uses at least two pieces of AF information to calculate and acquire the focus lens predetermined positions for each of the shooting directions A and B. Here, the method of calculating the focus lens predetermined position uses the average value (average position), the most frequent value (most frequent position), or a weighted arithmetic average according to Taf of at least two pieces of AF information, but is not limited to these methods. After acquiring the AF information, the process proceeds to S2903.

[0160] In S2903, the camera control unit 204 acquires shooting record information. The shooting record information is information obtained from recorded information such as a shot video and at least two or more still images. It is calculated using a calculation method such as an average value (average position), a mode (most frequent position) of focus lens predetermined position information for each of shooting directions A and B in the video (while shooting is continuing), or a weighted arithmetic average for frames when a predetermined subject is detected.

[0161] When the length of the recorded video time (duration of shooting) is longer than a predetermined time, it is assumed that the user shot the video at a distance that the user can hold stably, or that the camera was fixed to a tripod or the like. In this case, the camera control unit 204 calculates and acquires an average value (average position) as the focus lens predetermined position information for each of the shooting directions A and B. On the other hand, when the length of the recorded video time (duration of shooting) is shorter than a predetermined time, it is assumed that the user shot the video spontaneously at a distance different from the usual shooting distance, and there are many frames in which the subject of interest is present. In this case, the camera control unit 204 calculates and acquires a mode (most frequent position) as the focus lens predetermined position information for each of the shooting directions A and B. Furthermore, the focus lens predetermined position information for each of the shooting directions A and B may be calculated using the above-mentioned calculation method regardless of the length of the recorded video time.

[0162] For still images, the distance is calculated using the above-mentioned calculation method or the like, using the focus lens predetermined position information for each of the shooting directions A and B in the multiple still images captured. In this embodiment, the above-mentioned calculation method is described, but the calculation method is not limited to these. Also, when a predetermined number or more of recorded videos or still images are collected, it may be determined that the shooting is performed with the composition or the like most frequently used by the user, and the most frequent distance may be used. After acquiring the shooting record information, the process proceeds to S2904.

[0163] In S2904, the camera control unit 204 acquires shooting environment information. The shooting environment information is information such as brightness at the time of shooting, and is acquired from recorded information such as recorded moving images and still images (image data output from the image sensor 201). It may also be acquired from measurement information of an unillustrated device that measures environmental brightness. From the acquired shooting environment information, which is brightness information at the time of shooting, a determination is made as follows.

[0164] First, when the brightness during shooting is a first brightness that is brighter than a predetermined value, it is assumed that the shooting is mainly outdoor infinity shooting such as landscapes. At this time, the camera control unit 204 acquires the first focus position of this embodiment as information on the predetermined position of the focus lens 104 (focus lens predetermined position information). Next, when the brightness during shooting is a second brightness that is darker than the first brightness, it is assumed that the shooting is indoors and closer to infinity such as a portrait. At this time, the camera control unit 204 acquires the second focus position closer to the first focus position as focus lens predetermined position information.

[0165] In the case where the brightness during shooting is a third brightness that is darker than the second brightness, and there is a signal with a luminance value higher than a predetermined value in the image signal of the recorded information, it is assumed that the entire angle of view is dark, but the luminance value is high due to building lights, etc., and that the shooting is a night scene shooting, etc. Then, the camera control unit 204 acquires a third focus position as the focus lens predetermined position information. In addition, in the case where the brightness during shooting is a third brightness that is darker than the second brightness, and there is no signal with a luminance value higher than a predetermined value in the image signal of the recorded information, it is assumed that the shooting is a landscape or a starry sky in a dark place, but also assumes a near subject. In order to avoid starting from a blurred focus lens position when assuming a near subject, a fourth focus position that is closer than the third focus position is acquired as the focus lens predetermined position information. After acquiring the shooting environment information, the process proceeds to S2905.

[0166] In S2905, the camera control unit 204 determines focus drive destinations A and B to be used by a predetermined position changing means in S2907 and S2908 described later from the information acquired from S2901 to S2904. When only one piece of information is acquired from S2901 to S2904, that information is determined as the focus drive destinations A and B. When at least two pieces of information are acquired from S2901 to S2904, the camera unit 50 automatically determines the destination.

[0167] The priority of selecting one piece of information from S2901 to S2904 is determined as follows. The first priority is the user registration information acquired in S2901 determined by the user's will. The second priority is the AF information acquired in S2902, which is the previous focus position information stored in the camera. The third priority is the shooting record information acquired in S2903, which is the focus position information obtained from the record information. Finally, the fourth priority is the shooting environment information acquired in S2904, which is information related to shooting. In addition, when selecting from a plurality of pieces of information from S2901 to S2904, the calculation is performed using a calculation method such as a weighted average value, an average value, or a mode value according to the above-mentioned priority, and the focus drive destinations A and B are determined. Note that the calculation method is not limited to the above-mentioned calculation method. In addition, the user may set the priority when at least two pieces of information from S2901 to S2904 are obtained. After the focus drive destinations A and B are calculated and determined, the process proceeds to S2906. The processes from S2906 to S2908 are similar to the processes from S601 to S603 in Fig. 6, respectively, and therefore detailed description thereof will be omitted. This ends the process of setting the destination of driving the focus lens to the predetermined position in Fig. 29.

[0168] In this embodiment, the setting of the focus lens predetermined position drive destination in the standby state before video recording or still image shooting has been described. However, this can also be implemented when the shooting direction A, B is changed during video recording, and during video recording, the focus position is set to be slightly offset from the focus position in the standby state so as not to become more blurred. When the focus position setting is on the far side, it is offset slightly to the near side, and when the focus position setting is on the near side, it is offset slightly to the far side.

[0169] In this embodiment, the predetermined position of the focus lens may be changed so that the amount of focus movement during video recording is shorter than when video recording is not being performed. Also, in this embodiment, the predetermined position of the focus lens may be changed so that the amount of focus movement becomes shorter as the aperture 102 is narrowed.

[0170] (Other embodiments) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0171] In the imaging device of each embodiment, the focus lens is driven to a predetermined position when the start of the change in the shooting direction is used as a trigger. Therefore, according to each embodiment, when the shooting direction is changed by rotating the axis of the gimbal, it is possible to provide an imaging device, a control method for the imaging device, and a program that can quickly adjust the focus after the shooting direction change is completed and smoothly transition to shooting.

[0172] The disclosure of each embodiment includes the following configurations and methods. (Configuration 1) an imaging unit that photoelectrically converts an optical image formed by an optical system including a focus lens and outputs image data; A control unit for controlling the focus lens, The control means acquiring information regarding the start of a change in the shooting direction from a rotation drive means that performs rotation drive in at least one of a pan direction and a tilt direction; the start of the change in the shooting direction is used as a trigger to drive the focus lens to a predetermined position. (Configuration 2) The control means acquiring information regarding the completion of the change of the photographing direction from the rotation driving means; The imaging device according to configuration 1, wherein the focus lens is controlled so as to perform automatic focus adjustment using the image data, with the completion of the change in the imaging direction as a trigger. (Configuration 3) The control means The imaging device described in configuration 2 is characterized in that when the information regarding the completion of the change in the shooting direction is obtained, if the driving of the focus lens to the specified position is not completed, the driving of the focus lens is interrupted and the focus lens is controlled to perform the automatic focus adjustment. (Configuration 4) 4. The imaging apparatus according to any one of configurations 1 to 3, wherein the control means drives the focus lens at a slower speed during moving image recording than when moving image recording is not being performed. (Configuration 5) The imaging device according to any one of configurations 1 to 4, wherein the rotation drive means, while driving the focus lens to the predetermined position, makes the rotation speed during video recording slower than when not recording a video. (Configuration 6) The turning drive means is acquiring time information for completing the driving of the focus lens to the predetermined position; The imaging device described in any one of configurations 1 to 5, characterized in that, during video recording, if it is determined using the time information that the time required for the rotation drive to be completed is shorter than the time required for the drive to be completed, the rotation speed is slowed down so that the time required for the rotation drive to be completed matches the time required for the drive to the specified position to be completed. (Configuration 7) The imaging device according to any one of configurations 2 to 6, characterized in that, when performing the automatic focus adjustment based on phase difference information in the image data, if reliability of the phase difference information acquired after the end of the change of the shooting direction is lower than a predetermined reliability, the control means does not perform the automatic focus adjustment until a predetermined time has elapsed or the reliability becomes higher than the predetermined reliability. (Configuration 8) The imaging device according to any one of configurations 1 to 7, wherein the control means drives the focus lens to the predetermined position when the image sensor size of the imaging means is larger than a predetermined size, the aperture value is brighter than a predetermined aperture value, or the focal length is longer than a predetermined focal length. (Configuration 9) 9. The imaging device according to any one of configurations 1 to 8, wherein the control means changes the predetermined position. (Configuration 10) 3. The imaging device according to claim 2, wherein the control means changes the predetermined position by using a result of past autofocus adjustment before driving the focus lens to the predetermined position. (Configuration 11) The control means If a predetermined time has elapsed since the automatic focus adjustment was performed or if the power supply of the imaging device has not been turned off, the result of the immediately preceding automatic focus adjustment is used as the result of the past automatic focus adjustment, The imaging device described in configuration 10, characterized in that if it has not been within the specified time since the automatic focus adjustment was performed and the imaging device is powered off, the results of at least two previous automatic focus adjustments are used as the result of the past automatic focus adjustment. (Configuration 12) 10. The imaging device according to configuration 9, wherein the control means changes the predetermined position by using past focus lens position information before driving the focus lens to the predetermined position. (Configuration 13) The control means If the duration of shooting is longer than a predetermined time, an average position of the focus lens during shooting is used as the past focus lens position information; 13. The imaging device according to claim 12, wherein when the duration of shooting is shorter than the predetermined time, the most frequent position of the focus lens during the shooting is used as the past focus lens position information. (Configuration 14) The imaging device according to configuration 12 or 13, characterized in that when the number of recorded moving images or still images is equal to or greater than a predetermined number, the control means uses the most frequent position related to the recorded moving images or still images as the past focus lens position information. (Configuration 15) The imaging device according to configuration 9, wherein the control means changes the predetermined position by using brightness information obtained from the past image data before driving the focus lens to the predetermined position. (Configuration 16) The control means If the brightness information is information indicating a first brightness, a first focus position is obtained as the predetermined position; The imaging device described in configuration 15, characterized in that when the brightness information indicates a second brightness that is darker than the first brightness, a second focus position that is closer than the first focus position is acquired as the specified position. (Configuration 17) When the brightness information is information indicating a third brightness that is darker than the second brightness, the control means When the image data includes a signal having a luminance value higher than a predetermined luminance value, a third focus position is obtained as the predetermined position; The imaging device described in configuration 16, characterized in that when the image data does not contain a signal with a luminance value higher than the predetermined luminance value, a fourth focus position closer than the third focus position is acquired as the predetermined position. (Configuration 18) 10. The imaging device according to claim 9, wherein the control means changes the predetermined position so that the amount of focus movement during moving image recording is shorter than when moving image recording is not being performed. (Configuration 19) 10. The imaging apparatus according to claim 9, wherein the control means changes the predetermined position so that the focus movement amount becomes shorter as the aperture is narrowed. (Configuration 20) 10. The imaging apparatus according to claim 9, wherein the control means changes the predetermined position to a focus position set by a user. (Configuration 21) 21. The imaging device according to any one of configurations 1 to 20, wherein the imaging device is integrally formed with the rotation drive means. (Configuration 22) The imaging device according to any one of configurations 1 to 20, wherein the imaging device is detachable from the rotation drive means. (Method 1) an imaging step of photoelectrically converting an optical image formed by an optical system including a focus lens and outputting image data; A control step of controlling the focus lens, In the control step, obtaining an instruction to start changing the shooting direction from a rotation driving means that performs rotation driving in at least one of a pan direction and a tilt direction; a control method for an imaging apparatus, the control method comprising the steps of: driving the focus lens to a predetermined position using the start of the change in the imaging direction as a trigger; (Configuration 23) A program for causing a computer to execute the control method according to method 1.

[0173] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention.

[0174] In the description of each embodiment, the imaging device configuration includes a gimbal held by the user as shown in Fig. 3. However, the imaging device may have a configuration that is not held by the user. For example, a network camera with a rotation mechanism or a broadcast camera may be used. [Explanation of symbols]

[0175] 20, 40, 50 Camera unit (imaging device) 30, 60 Gimbal section (rotation drive means) 104 Focus lens (optical system) 201 Imaging element (imaging means) 204 Camera control unit (control means)

Claims

1. an imaging unit that photoelectrically converts an optical image formed by an optical system including a focus lens and outputs image data; A control unit for controlling the focus lens, The control means acquiring information regarding the start of a change in the shooting direction from a rotation drive means that performs rotation drive in at least one of a pan direction and a tilt direction; the start of the change in the shooting direction is used as a trigger to drive the focus lens to a predetermined position.

2. The control means acquiring information regarding the completion of the change of the photographing direction from the rotation driving means; The imaging device according to claim 1 , wherein the focus lens is controlled so as to perform automatic focus adjustment using the image data, with the completion of the change in the imaging direction as a trigger.

3. The control means The imaging device described in claim 2, characterized in that when the information regarding the completion of the change in the shooting direction is obtained, if the driving of the focus lens to the specified position is not completed, the driving of the focus lens is interrupted and the focus lens is controlled to perform the automatic focus adjustment.

4. 2. The image pickup apparatus according to claim 1, wherein the control means drives the focus lens at a slower speed during moving image recording than when moving image recording is not being performed.

5. 2. The image pickup apparatus according to claim 1, wherein the rotation drive means makes the rotation speed of the focus lens during moving image recording slower than when the moving image is not being recorded, while the focus lens is being driven to the predetermined position.

6. The turning drive means is acquiring time information for completing the driving of the focus lens to the predetermined position; The imaging device according to claim 1, characterized in that, during video recording, when it is determined using the time information that the time required for the rotation drive to be completed is shorter than the time required for the drive to be completed, the rotation speed is slowed down so that the time required for the rotation drive to be completed matches the time required for the drive to the specified position to be completed.

7. 3. The imaging device according to claim 2, wherein, when performing the automatic focus adjustment based on phase difference information in the image data, if reliability of the phase difference information acquired after the end of the change in the shooting direction is lower than a predetermined reliability, the control means does not perform the automatic focus adjustment until a predetermined time has elapsed or the reliability becomes higher than the predetermined reliability.

8. 2. The imaging device according to claim 1, wherein the control means drives the focus lens to the predetermined position when the imaging element size of the imaging means is larger than a predetermined size, the aperture value is brighter than a predetermined aperture value, or the focal length is longer than a predetermined focal length.

9. 2. The imaging apparatus according to claim 1, wherein the control means changes the predetermined position.

10. 3. The image pickup apparatus according to claim 2, wherein the control means changes the predetermined position by using a result of a past automatic focus adjustment before driving the focus lens to the predetermined position.

11. The control means If a predetermined time has elapsed since the automatic focus adjustment was performed or if the power supply of the imaging device has not been turned off, the result of the immediately preceding automatic focus adjustment is used as the result of the past automatic focus adjustment, 11. The imaging device according to claim 10, wherein if the predetermined time has not elapsed since the automatic focus adjustment was performed and if the imaging device is powered off, the results of at least two previous automatic focus adjustments are used as the result of the previous automatic focus adjustment.

12. 10. The imaging apparatus according to claim 9, wherein the control means changes the predetermined position by using past focus lens position information before driving the focus lens to the predetermined position.

13. The control means If the duration of shooting is longer than a predetermined time, an average position of the focus lens during shooting is used as the past focus lens position information; 13. The imaging apparatus according to claim 12, wherein when the duration of shooting is shorter than the predetermined time, a most frequent position of the focus lens during the shooting is used as the past focus lens position information.

14. 13. The imaging device according to claim 12, wherein when the number of recorded moving images or still images is equal to or greater than a predetermined number, the control means uses a most frequent position related to the recorded moving images or still images as the past focus lens position information.

15. 10. The imaging apparatus according to claim 9, wherein the control means changes the predetermined position by using brightness information obtained from the past image data before driving the focus lens to the predetermined position.

16. The control means If the brightness information is information indicating a first brightness, a first focus position is obtained as the predetermined position; The imaging device according to claim 15, characterized in that when the brightness information indicates a second brightness that is darker than the first brightness, a second focus position that is closer than the first focus position is acquired as the specified position.

17. When the brightness information is information indicating a third brightness that is darker than the second brightness, the control means When the image data includes a signal having a luminance value higher than a predetermined luminance value, a third focus position is obtained as the predetermined position; The imaging device described in claim 16, characterized in that when the image data does not contain a signal with a luminance value higher than the specified luminance value, a fourth focus position closer than the third focus position is acquired as the specified position.

18. 10. The imaging apparatus according to claim 9, wherein the control means changes the predetermined position so that a focus movement amount during moving image recording is shorter than that when moving image recording is not being performed.

19. 10. The image pickup apparatus according to claim 9, wherein the control means changes the predetermined position so that the focus movement amount becomes shorter as the aperture becomes narrower.

20. 10. The imaging apparatus according to claim 9, wherein the control means changes the predetermined position to a focus position set by a user.

21. 21. The imaging device according to claim 1, wherein the imaging device is integral with the rotation drive means.

22. 21. The imaging device according to claim 1, wherein the imaging device is detachable from the rotation drive means.

23. an imaging step of photoelectrically converting an optical image formed by an optical system including a focus lens and outputting image data; A control step of controlling the focus lens, In the control step, obtaining an instruction to start changing the shooting direction from a rotation driving means that performs rotation driving in at least one of a pan direction and a tilt direction; a control method for an imaging apparatus, the control method comprising the steps of: driving the focus lens to a predetermined position using the start of the change in the imaging direction as a trigger;

24. A program causing a computer to execute the control method according to claim 23.

Images