Control device, optical device, and control method

Abstract

The invention discloses a control apparatus, an optical apparatus, and a control method. A control device includes a control unit configured to acquire an object position and an object speed from an image generated using an output of an image sensor configured to photoelectrically convert an object image formed by an optical system including an optical element; the controller performs a first control to move at least one of an optical element and an image sensor based on an object speed and a detection result of a motion of an optical device caused by panning the optical device, the optical device including at least one of an optical system and the image sensor, and performs a second control to move at least one of the optical element and the image sensor based on the object speed and the detection result of the motion of the optical device. The second control moves the at least one of the optical element and the image sensor based on the object position before imaging such that the object image moves to a predetermined position or direction on the image sensor.

Description

Technical Field

[0001] The present disclosure relates to panning-shot (or tracking shot) assist control for an optical device such as a digital camera. Background Art

[0002] In panning of a moving object using a camera, the shutter speed is generally reduced in order to dynamically express the moving object. However, the reduced shutter speed may cause image blur. Japanese Patent Publication No. 2007-139952 discloses an object tracking method that calculates the object speed from the difference between the object moving speed on an image sensor and the panning speed at which a user moves an image pickup device, and decenters the optical system during panning imaging to correct the error between the object speed and the panning speed. Thus, panning can be achieved while maintaining the position of the object within the imaging screen (captured image).

[0003] Image stabilization methods that optically reduce image blur are classified into a lens shift method (optical image stabilization: OIS) configured to move a correction lens (shift lens) relative to the optical axis, and an in-camera sensor shift method (in-body image stabilization: IBIS) configured to move an image sensor relative to the optical axis. Japanese Patent No. 6410431 discloses a camera system that improves image stabilization performance by utilizing OIS and IBIS at a ratio that effectively uses their movable ranges. Summary of the invention

[0004] A control device according to one aspect of the present disclosure includes: an acquisition unit configured to acquire an object position and an object speed from an image generated using an output of an image sensor configured to photoelectrically convert an object image formed by an optical system including an optical element; and a control unit configured to: perform a first control, the first control moving at least one of the optical element and the image sensor based on the object speed and a detection result of a movement of the optical device caused by panning the optical device, the optical device including at least one of the optical system and the image sensor, and perform a second control, the second control moving at least one of the optical element and the image sensor based on the object position before imaging, so that the object image moves to a predetermined position or direction on the image sensor. An optical system and a control method corresponding to the above control device also constitute another aspect of the present disclosure.

[0005] Further features of various embodiments of the present disclosure will become apparent from the following description of the embodiments with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Figure 1 is a block diagram illustrating a configuration of an imaging system according to one or more aspects of the present disclosure.

[0007] Figure 2 is a block diagram illustrating a configuration of an image stabilization system according to one or more aspects of the present disclosure.

[0008] Figure 3A , Figure 3B and Figure 3C Panning is illustrated.

[0009] Figure 4A , Figure 4B and Figure 4C A method for calculating an object vector according to one or more aspects of the present disclosure is illustrated.

[0010] Figure 5 An object angular velocity, a pan angular velocity, and a motion (amount of shift) in OIS during panning according to one or more aspects of the present disclosure are illustrated.

[0011] Fig. 6A and Figure 6B Successful and unsuccessful images obtained by panning according to one or more aspects of the present disclosure are illustrated.

[0012] Fig. 7A and Figure 7B Illustrated is a pre-pan composition adjustment control according to one or more aspects of the present disclosure.

[0013] Figure 8 is a flow chart illustrating a pan assist process according to one or more aspects of the present disclosure.

[0014] Fig. 9 The object angular velocity, pan angular velocity, and movement (shift amount) in OIS and IBIS during composition adjustment control and panning are illustrated.

[0015] Fig.10 The composition adjustment control in the second embodiment and the object angular velocity during panning, the panning angular velocity, and the movement (shift amount) in OIS and IBIS are illustrated. DETAILED DESCRIPTION

[0016] In the following, the term "unit" may refer to a software context, a hardware context, or a combination of software and hardware contexts. In the software context, the term "unit" refers to a function, application, software module, function, routine, instruction set, or program that can be executed by a programmable processor such as a microprocessor, a central processing unit (CPU), or a specially designed programmable device or controller. The memory contains instructions or programs that cause the CPU to perform operations corresponding to the unit or function when executed by the CPU. In the hardware context, the term "unit" refers to a hardware element, circuit, component, physical structure, system, module, or subsystem. According to a specific embodiment, the term "unit" may include mechanical, optical, or electrical components or any combination thereof. The term "unit" may include active (e.g., transistors) or passive (e.g., capacitors) components. The term "unit" may include a semiconductor device having a substrate and other material layers with different conductivity concentrations. It may include a CPU or a programmable processor that can execute a program stored in a memory to perform a specified function. The term "unit" may include a logic element (e.g., AND, OR) implemented by a transistor circuit or any other switching circuit. In the combination of software context and hardware context, the term "unit" or "circuit" refers to any combination of software context and hardware context as described above. In addition, the term "element", "component", "part" or "device" may also refer to a "circuit" when integrated with or without packaging materials.

[0017] Referring now to the drawings, a description will be given of embodiments according to the present disclosure. First embodiment

[0018] Figure 1 The configuration of the imaging system according to the first embodiment is illustrated.The camera system includes an interchangeable lens 101 as a lens device and an image pickup device (hereinafter referred to as a camera body) 100 as an optical device to which the interchangeable lens 101 is detachably and communicably connected.

[0019] The camera body 100 includes a camera MPU 102 as a computer, an operation unit 103 , an image sensor 104 , a camera-side contact terminal 105 , a camera-side gyro sensor 106 , an acceleration sensor 109 , and a rear display 116 .

[0020] The camera MPU 102 is a controller configured to control the entire camera system composed of the camera body 100 and the interchangeable lens 101, and controls various operations such as automatic exposure (AE), automatic focus (AF), and imaging according to input from the operation unit 103 to be described later. The camera MPU 102 transmits various commands and information with the lens MPU 110 via the camera side contact terminal 105 and the lens side contact terminal 112 provided on the interchangeable lens 101. The camera side contact terminal 105 and the lens side contact terminal 112 include power terminals for supplying power from the camera body 100 to the interchangeable lens 101.

[0021] The operation unit 103 includes a mode selection dial that can be operated by a user (photographer) to select various imaging modes, a release button that can be operated by the user to instruct an imaging preparation operation and an imaging operation, and the like. By operating the mode selection dial, an imaging mode such as a still image capturing mode, a moving image capturing mode, and a panning assist mode described later can be selected. In the case where the release button is pressed halfway, the first switch (Sw1) is turned on, and in the case where the release button is pressed completely, the second switch (Sw2) is turned on. When Sw1 is turned on, AE and AF are performed as an imaging preparation operation. When Sw2 is turned on, AE setting ends, AF stops, and an instruction to start imaging (exposure) is issued. When an exposure start instruction is given, the camera MPU 102 first turns on Sw2-1 as an exposure preparation signal, and then turns on Sw2-2 as an exposure instruction signal that instructs the start of actual exposure at a predetermined time after the instruction to start the imaging operation is issued. When the set exposure time has passed and imaging has ended, Sw2-1 and Sw2-2 are turned off. The camera MPU 102 notifies the lens MPU 110 of the on-state and off-state of Sw1 , Sw2 - 1 , and Sw2 - 2 through communication.

[0022] The image sensor 104 includes a photoelectric conversion element such as a CCD sensor or a CMOS sensor, and generates an imaging signal by photoelectrically converting (capturing) an object image formed by an imaging optical system (described later). The camera MPU 102 generates a captured image (image data) using the imaging signal from the image sensor 104.

[0023] The camera-side gyro sensor 106 is a shake sensor configured to detect angular shake (camera shake) applied to the camera body 100 due to hand-held shake or the like, and outputs a camera shake detection signal as an angular velocity signal. The camera MPU 102 controls the driving of the image sensor actuator (driving unit) 107 based on the camera shake detection signal or the like, and moves (shifts) the image sensor 104 in a direction orthogonal to the optical axis of the imaging optical system. At this time, the camera MPU 102 performs feedback control of the image sensor actuator 107 so that the position of the image sensor 104 detected by the image sensor position detector 108 (the amount of movement relative to the position on the optical axis as the center of the shift) approaches the target position. Thus, image stabilization (IBIS) using a sensor shift method by shifting the image sensor 104 is performed.

[0024] The camera MPU 102 also recognizes an object, separates the object from the background, performs calculations to acquire the moving direction and moving speed of the object, and the like based on the image data from the image sensor 104 and the lens information received from the lens MPU 110 .

[0025] The acceleration sensor 109 is used to detect the posture of the camera body 100 , and detect shift shake that is difficult to detect with the above-described camera-side gyro sensor 106 .

[0026] The rear display 116 serving as a display unit displays image data as an image or video obtained by the camera MPU 102 through the image sensor 104. In the following description, imaging (exposure) refers to imaging for recording for obtaining a still image or a moving image for recording. Before imaging (before exposure), the user can observe the image displayed on the rear display 116 as a viewfinder image (live view image). After imaging (after exposure), the image data can be displayed on the rear display 116 as a still image or a moving image for recording.

[0027] The interchangeable lens 101 includes an imaging optical system (not shown), a lens MPU 110 as a computer, a lens-side contact terminal 112, and a lens-side gyro sensor 111. The lens-side gyro sensor 111 is a shake sensor configured to detect an angular shake (lens shake) of the interchangeable lens 101 and output a lens shake detection signal as an angular velocity signal.

[0028] The lens MPU 110 controls the driving of the lens actuator (driving unit) 113 based on the lens shake detection signal and the OIS correction ratio (described later), thereby moving (shifting) the correction lens (shift lens) 114, which is an optical element that is part of the imaging optical system, in a direction orthogonal to the optical axis of the imaging optical system. At this time, the lens MPU 110 performs feedback control of the lens actuator 113 so that the position of the correction lens 114 detected by the lens position sensor 115 (the amount of movement relative to the position on the optical axis as the shift center) approaches the target position. Thus, image stabilization (OIS) is performed by shifting the correction lens 114.

[0029] Figure 2 The configuration of the image stabilization system in the camera system according to the present embodiment is illustrated. The image stabilization system includes a camera image stabilizer 201 provided on the camera body 100 side and a lens image stabilizer 209 provided on the interchangeable lens 101 side. The camera image stabilizer 201 is a part of the camera MPU 102, and the lens image stabilizer 209 is a part of the lens MPU 110.

[0030] The camera gyro offset remover 202 removes an offset component from a camera shake detection signal (angular velocity signal) output from the camera side gyro sensor 106 mounted on the camera body 100. The camera side angle converter 203 converts the angular velocity signal output from the camera gyro offset remover 202 into an angle signal. The camera information memory 204 stores camera information such as the drivable (or displaceable) amount of the IBIS and the size of the image sensor 104. The camera information is used for the drive control of the IBIS and is transmitted from the lens communication transmitter 205 to the lens MPU 110. The lens communication transmitter 205 also transmits to the lens MPU 110 object information (e.g., object position, object velocity, etc.) obtained from the object recognition processing unit 217 provided in the camera MPU 102 separately from the image stabilization system.

[0031] The lens communication receiver 206 receives OIS information on image stabilization (information such as OIS sensitivity and OIS correction ratio indicating the relationship between the shift amount of the correction lens 114 and the image stabilization amount) transmitted from the camera communication transmitter 213 of the interchangeable lens 101. The camera-side cooperative control unit 207 determines the image stabilization amount to be implemented by IBIS based on the camera information read out from the camera information memory 204 and the OIS information received through the lens communication receiver 206. The image sensor drive control unit 208 generates a drive control signal for shifting the image sensor 104 in IBIS based on the angle signal output from the camera-side angle converter 203 and the image stabilization amount determined by the camera-side cooperative control unit 207.

[0032] When the pan assist mode is set, the camera-side cooperative control unit 207 determines the adjustment shift amount of the image sensor 104 based on the object position indicated by the object information from the object recognition processing unit 217. The image sensor drive control unit 208 generates a drive control signal for shifting the image sensor 104 by the adjustment shift amount.

[0033] The lens gyro offset remover 210 removes an offset component from a lens shake detection signal (angular velocity signal) output from a lens side gyro sensor 111 mounted on the interchangeable lens 101. The lens side angle converter 211 converts the angular velocity signal output from the lens gyro offset remover 210 into an angle signal. The camera communication receiver 214 receives object information and information about the drive amount of the correction lens 114 transmitted from the lens communication transmitter 205 of the camera body 100. The lens information memory 212 stores information about the OIS drivable amount and the OIS sensitivity. The lens information memory 212 also stores IBIS sensitivity information indicating the relationship between the shift amount of the image sensor 104 and the image stabilization amount.

[0034] The lens side cooperative control unit 215 performs cooperative control of OIS and IBIS based on the information read from the lens information memory 212 and the information received through the camera communication receiver 214. At this time, the lens side cooperative control unit 215 calculates a correction ratio which is a ratio of the image stabilization amounts corrected by OIS and IBIS (ratio of control regarding OIS and IBIS). The correction lens drive control unit 216 generates a drive control signal for shifting the correction lens 114 in OIS based on the angle signal from the lens side angle converter 211. In the case where the pan assist mode is set in the camera body 100, the correction lens drive control unit 216 generates a drive control signal for shifting the correction lens 114 to perform object tracking control (first control) based on the object information received via the camera communication receiver 214.

[0035] A description will now be given of panning. Figure 3A , Figure 3B and Figure 3C The motion of the object and the camera (system) in a pan shot of an object (train) passing in front of the photographer is illustrated in time series. In a pan shot, the camera is moved (panned) to match the object movement speed even during the exposure period, and a captured image is obtained in which the object motion stops and the background flows. Figure 3A , Figure 3B and Figure 3CAs illustrated in , even if the photographer attempts to pan the camera to match the subject motion, a difference may actually occur between the speed at which the camera is panned (pan speed) and the speed at which the subject moves (subject speed). There is a correlation between fluctuations in the subject speed and fluctuations in the output of the shake sensor provided to the camera.

[0036] like Figure 3B As shown in the figure, the following equation holds:

[0037] D = βLπθ / 180 (1)

[0038] Where θ [degrees] is the angular displacement of the camera motion, L is the object distance, β is the imaging magnification, and D is the jitter displacement of the object image.

[0039] Therefore, the following equation holds:

[0040] Va = βLπωa / 180 (2)

[0041] Where Va is the object velocity, and ωa is the panning angular velocity of the camera detected by the shake sensor (the detection result of the motion).

[0042] Figure 4A and Figure 4B The diagram illustrates an object image formed on the image sensor 104. When the object recognition processing unit 217 processes image data obtained by photoelectrically converting the object image using the image sensor 104, the object and the background in the image data can be separated, and object information indicating the size, type (car, train, bird, person, etc.), position, and moving speed of the object can be obtained.

[0043] Figure 4A and Figure 4B The subject images formed on the image sensor 104 at a first timing in panning and a second timing later than the first timing by a predetermined sampling rate are respectively illustrated. Figure 4C The diagram shows Figure 4A and Figure 4B The motion vector data is obtained by performing a comparison process on each area of ​​the object image divided into a grid shape as shown in FIG. Figure 4C In the captured image, the position of the object (train) does not change much, and a small value is output as the motion vector data. On the other hand, the background (buildings, etc.) moves at a speed comparable to the object speed, so a large value of motion vector data is obtained. The object speed Va is calculated based on the shift amount between corresponding pixels at the specified sampling rate.

[0044] By subtracting the angular velocity ωa from the output ω of the shake sensor during exposure (imaging) for panning, the angular velocity ωo for good panning—that is, for accurate tracking of a moving object—can be calculated as expressed in the following equation (3):

[0045] ωo = ω - ωa = ω-180Va / (βLπ) (3)

[0046] Figure 5 The top figure in the figure shows the Figure 3A , Figure 3B and Figure 3C The object speed during panning illustrated in FIG. 1 and the panning speed detected by the lens-side gyro sensor 111 are each converted into an angular velocity [degrees / second]. The bottom graph illustrates the shift amount of the correction lens 114. Figure 3A , Figure 3B and Figure 3C In the case where the camera is panned at a speed faster than the object speed for panning as shown in the figure, in order to eliminate blur of the object image during exposure, the camera MPU 102 notifies the lens MPU 110 of the object speed during the preparation period as the first period when the exposure preparation signal (Sw2-1) is turned on. Then, during the exposure period as the second period starting when the exposure instruction signal (Sw2-2) is turned on, the correction lens 114 (OIS) is adjusted at a speed for obtaining the angular velocity ωo at Figure 5 The bottom diagram illustrates the shift in the + direction.

[0047] Therefore, by shifting the correction lens 114 to reduce or eliminate the difference between the object speed and the pan speed and by tracking the object, good panning with reduced image blur during panning can be achieved.

[0048] However, although the photographer's desired composition is Fig. 6A The entire train as an object is contained in the captured image, but Figure 6B , the captured image actually obtained by panning (hereinafter referred to as the panned image) does not include a part of the train in the captured image. In this case, the panning is unsuccessful. Figure 5 In the object tracking shown in the figure, the position of the object image at the start of exposure is maintained, so if you use Figure 6B If you start panning from the composition shown in the figure, you will not be able to obtain Fig. 6A Therefore, in this embodiment, composition adjustment control (second control) is performed as a process for adjusting the position of the object image in the imaging frame image (ie, on the imaging surface of the image sensor 104) immediately before exposure - that is, the composition.

[0049] Fig. 7AThe diagram shows Figure 3A Same status. Figure 7B The diagram illustrates a composition adjustment in which the position of the object image in the imaging screen is adjusted by shifting the image sensor 104 before object tracking, that is, the composition. Figure 8 The flowchart in shows a panning assist process (control method) including composition adjustment control and object tracking control, which is executed by the camera MPU 102 as a control device for panning according to a program. The camera MPU 102 functions as an acquisition unit and a control unit.

[0050] In step S801 , when the photographer instructs to start exposure and Sw2 - 1 is turned on (starting a preparation period before imaging), in step S802 , the camera MPU 102 causes the object recognition processing unit 217 to calculate the object position (in this embodiment, the center of gravity of the object image).

[0051] Next, in step S803, the camera MPU 102 causes the camera-side cooperative control unit 207 to calculate the adjustment shift amount for IBIS based on the object position calculated in step S802, so that the position of the object image is moved to the center position (predetermined position) of the imaging screen in the panning direction. Then, the IBIS is shifted by the adjustment shift amount. Thus, the composition adjustment is performed.

[0052] Thereafter, in step S804, Sw2-2 is turned on and exposure is started (an exposure period is started during imaging), and in step S805, the camera MPU 102 causes the correction lens drive control unit 216 to control OIS for object tracking. At this time, the camera MPU 102 may perform control of IBIS to correct image blur caused by hand-held shake or the like, or may perform coordinated control of OIS and IBIS. After the panning exposure is thus completed, the flow ends.

[0053] Fig. 9 Added shift of image sensor 104 (IBIS) in composition adjustment to Figure 5 First, before exposure, that is, during the preparation period from when Sw2-1 is turned on to when exposure starts, the position of the object image within the imaging screen is adjusted by shifting the IBIS (one of the lens and the image sensor), and then the position of the object image is maintained (that is, the IBIS is stopped at the position after the composition adjustment).

[0054] Next, during the exposure period starting when Sw2-2 is turned on, the OIS (the other of the lens and the image sensor) is shifted for object tracking. Therefore, the position of the object image within the imaging screen can be adjusted while ensuring the OIS drive amount for object tracking. As a result, a signal having a certain degree of rotation such as 100% is obtained. Fig. 6AA panning image of a composition such as the one illustrated in .

[0055] The adjustment shift amount (including the shift direction) of the IBIS before exposure may be calculated by a method other than the above-described calculation method. For example, the type of object obtained from the object recognition processing unit 217 may be identified, and the adjustment shift amount may be calculated to approach the desired composition according to the recognition result. The photographer may be prompted to input (indicate) a target position of the object image within the imaging screen, and the adjustment shift amount may be calculated so that the position of the object image moves to the target position. Captured images in which the object falls within the imaging angle of view may be registered in advance, and the adjustment shift amount may be calculated by identifying the object during panning so that the entire object falls within the imaging angle of view.

[0056] IBIS does not need to shift the image sensor by the calculated adjustment shift amount (to the target position). IBIS can shift the image sensor so that the position of the object image moves by a predetermined amount in a direction (predetermined direction) close to a desired position in terms of composition (such as the center position of the imaging screen), in a direction indicated by the photographer, or in a direction according to the object recognition result. Second embodiment

[0057] Now, a description will be given of the second embodiment. The first embodiment performs composition adjustment by IBIS before exposure, then maintains the position of the object image whose composition has been adjusted during exposure, and performs object tracking using OIS. On the other hand, the second embodiment performs object tracking using cooperative control of OIS and IBIS during exposure.

[0058] Fig.10 Added IBIS shift during composition adjustment and coordinated control of OIS and IBIS to Figure 5 .

[0059] When controlling OIS and IBIS at the same time, by appropriately setting the ratio (OIS correction ratio and IBIS correction ratio) each responsible for object tracking based on the respective drivable amounts of OIS and IBIS, good object tracking can be performed. The OIS+ correction ratio and the OIS- correction ratio, which are the OIS correction ratios according to the OIS shift direction (positive side and negative side), and the IBIS+ correction ratio and the IBIS- correction ratio, which are the IBIS correction ratios according to the IBIS shift direction (positive side and negative side), are calculated as follows: θOIS+ and θOIS- are the shift drivable amounts of the OIS + side and - side, respectively. θIBIS+ and θIBIS- are the shift drivable amounts of the IBIS + side and - side, respectively.

[0060] OIS+ correction ratio: θOIS+ / {(θIBIS+)+(θOIS+)}

[0061] OIS-correction ratio: θOIS- / {(θIBIS-)+(θOIS-)}

[0062] IBIS+ correction ratio: θIBIS+ / {(θIBIS+)+(θOIS+)}

[0063] IBIS-correction ratio: θIBIS- / {(θIBIS-)+(θOIS-)}

[0064] IBIS is executed for composition adjustment during the preparation period starting when Sw2-1 is turned on, and then cooperative control of OIS and IBIS is performed for object tracking using the above correction ratio during the exposure period starting from Sw2-2. At this time, IBIS is controlled from the shift position after the composition adjustment.

[0065] Thus, the drivable amounts of OIS and IBIS can be effectively utilized to perform object tracking that supports a large error between the object speed and the pan speed.

[0066] As described above, each embodiment can perform panning an image with a composition more desired by the user by adjusting the composition before exposure for object tracking.

[0067] Although each embodiment uses IBIS to perform composition adjustment, OIS, or both IBIS and OIS may be used to perform composition adjustment. In other words, composition adjustment may be performed using at least one of IBIS and OIS. The lens MPU in the lens device (optical device) may be used as the above control device.

[0068] The embodiments have described the lens interchangeable type image pickup device having the IBIS function to which the interchangeable lens equipped with OIS is attached, but are applicable to a lens integrated type image pickup device having both the OIS function and the IBIS function. Other embodiments

[0069] The embodiment(s) of the present invention may also be implemented by a computer of a system or device that reads out and executes computer executable instructions (e.g., one or more programs) recorded on a storage medium (which may also be more completely referred to as a "non-transitory computer-readable storage medium") to perform the functions of one or more of the above-mentioned embodiment(s) and / or includes one or more circuits (e.g., application-specific integrated circuits (ASICs)) for performing the functions of one or more of the above-mentioned embodiment(s), and by a method executed by a computer of a system or device, for example, by reading out and executing computer executable instructions from a storage medium to perform the functions of one or more of the above-mentioned embodiment(s) and / or controlling one or more circuits to perform the functions of one or more of the above-mentioned embodiment(s). The computer may include one or more processors (e.g., a central processing unit (CPU), a microprocessing unit (MPU)), and may include a network of separate computers or separate processors to read out and execute computer executable commands. The computer executable commands may be provided to the computer, for example, from a network or a storage medium. The storage medium may include, for example, one or more of a hard disk, a random access memory (RAM), a read-only memory (ROM), a storage device of a distributed computing system, an optical disk (such as a compact disk (CD), a digital versatile disk (DVD), or a Blu-ray Disc (BD)™), a flash memory device, a memory card, etc. Other embodiments

[0070] The embodiments of the present invention may also be implemented by providing software (program) for executing the functions of the above embodiments to a system or device via a network or various storage media, and a computer or a central processing unit (CPU) or a microprocessing unit (MPU) of the system or device reads and executes the program.

[0071] Although the present disclosure has described example embodiments, it is to be understood that some embodiments are not limited to the disclosed embodiments.The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.

[0072] Each embodiment may perform composition adjustments during panning and then perform object tracking.

Claims

1. A control device, comprising: an acquisition unit configured to acquire an object position and an object speed from an image generated using an output of an image sensor configured to photoelectrically convert an object image formed by an optical system including an optical element; as well as A control unit, the control unit being configured to: performing a first control of moving at least one of the optical element and the image sensor based on the object speed and a detection result of a movement of the optical device caused by panning the optical device, the optical device including at least one of the optical system and the image sensor, and A second control is performed that moves the at least one of the optical element and the image sensor based on the object position before imaging so that the object image moves to a predetermined position or direction on the image sensor.

2. The control device according to claim 1, characterized in that: The control unit is configured to: moving one of the optical element and the image sensor in the second control, and The one of the optical element and the image sensor is stopped in the first control at the position moved in the second control, and the other of the optical element and the image sensor is moved.

3. The control device according to claim 1, characterized in that: The control unit is configured to move the optical element and the image sensor according to a ratio calculated based on respective drivable amounts of the optical element and the image sensor in the first control.

4. The control device according to claim 1, characterized in that: The control unit is configured to: The second control is performed in a first period from when the start of imaging is instructed to when imaging starts, and The first control is performed in a second period after the start of imaging.

5. The control device according to claim 1, characterized in that: The predetermined position or direction is a center position of the image sensor or a direction close to the center position.

6. The control device according to claim 1, characterized in that: The predetermined position or direction is a position or direction indicated by a user.

7. The control device according to claim 1, characterized in that: The predetermined position or direction is a position or direction according to a recognition result of the moving object from the image.

8. An optical device comprising: A control device according to any one of claims 1 to 7; as well as At least one of an optical system and an image sensor.

9. An image pickup device, comprising: A control device according to any one of claims 1 to 7; Image sensor; as well as A driving unit is configured to move the image sensor.

10. A lens device detachably connected to the image pickup device according to claim 9, the lens device comprising: Optical system; as well as A drive unit is configured to move the optical element.

11. A control method, comprising: acquiring an object position and an object speed from an image generated using an output of an image sensor configured to photoelectrically convert an object image formed by an optical system including an optical element, performing a first control of moving at least one of the optical element and the image sensor based on the object speed and a detection result of a movement of the optical device caused by panning the optical device, the optical device including at least one of the optical system and the image sensor, and A second control is performed that moves the at least one of the optical element and the image sensor based on the object position before imaging so that the object image moves to a predetermined position or direction on the image sensor. 12 . A non-transitory computer-readable storage medium storing a program for causing a computer to execute the control method according to claim 11 .

Images