Control device, optical instrument, and control method

Abstract

To adjust a composition before tracking a subject in a panning shot.SOLUTION: A control device 102 acquires a subject position and a subject speed from an image created by using output from an image pick-up device 104 that photoelectrically converts a subject image formed by an optical system including an optical element 114, and on the basis of the subject speed and a result of detection of the movement of an optical instrument having at least one of the optical system and the image pick-up device, performs subject tracking control of moving at least one of the optical element and the image pick-up device during panning imaging. A panning shot control device performs, before the imaging, composition adjustment control of moving at least one of the optical system and the image pick-up device so that the subject image moves to a predetermined position or direction on the image pick-up device on the basis of the subject position.SELECTED DRAWING: Figure 9

Description

[Technical field]

[0001] The present invention relates to a control for assisting panning in an optical device such as a digital camera. [Background technology]

[0002] When a camera is used to perform panning of a moving subject, it is common to slow down the shutter speed in order to bring out the sense of dynamism in the moving subject. However, slowing down the shutter speed is likely to cause image blur. Patent Document 1 discloses a subject tracking method in which the subject speed is calculated from the difference between the moving speed of the subject on the imaging element and the panning speed at which the user moves the imaging device, and the optical system is moved off-center during panning to correct the error between the subject speed and the panning speed. This makes it possible to perform panning while maintaining the position of the subject within the imaging screen.

[0003] In addition, there are two types of vibration reduction methods for optically reducing image blur: a lens shift method (OIS) that moves the lens along the optical axis, and an in-camera sensor shift method (IBIS) that moves the image sensor along the optical axis. Patent Document 2 discloses a camera system that improves vibration reduction performance by moving both the OIS and the IBIS at a ratio that effectively uses their movable ranges. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2007-139952 A [Patent Document 2] Patent No. 6410431 Summary of the Invention [Problem to be solved by the invention]

[0005] In the subject tracking method disclosed in Patent Document 1, the error between the subject speed calculated before imaging and the panning speed is corrected, so if imaging is started while the subject is in an undesirable position in the imaging screen, the subject position is maintained. In other words, the subject position cannot be adjusted. This problem cannot be solved even if both OIS and IBIS are used as in the camera system disclosed in Patent Document 2.

[0006] The present invention provides a control device that is capable of tracking a subject after adjusting the position of the subject, that is, the composition, during panning. [Means for solving the problem]

[0007] A control device according to one aspect of the present invention includes an acquisition means for acquiring a subject position and a subject speed from an image generated using an output of an image sensor that photoelectrically converts a subject image formed by an optical system including an optical element, and a control means for performing subject tracking control for moving at least one of the optical element and the image sensor during panning based on the subject speed and the detection result of the movement of an optical device having at least one of the optical system and the image sensor. The control means is characterized in that, before the above-mentioned image capture, it performs composition adjustment control for moving at least one of the elements so that the subject image moves to a predetermined position or direction on the image sensor based on the subject position. Note that an optical device such as an image capture device equipped with the above-mentioned control device also constitutes another aspect of the present invention.

[0008] A control method according to another aspect of the present invention includes a step of acquiring a subject position and a subject speed from an image generated using an output of an image sensor that photoelectrically converts a subject image formed by an optical system including an optical element, and a step of performing subject tracking control to move at least one of the optical element and the image sensor during panning based on the subject speed and the detection result of the movement of an optical device having at least one of the optical system and the image sensor.Furthermore, the control method includes a step of performing composition adjustment control to move at least one of the elements so that the subject image moves to a predetermined position or direction on the image sensor based on the subject position before the above-mentioned image capture.Note that a program for causing a computer to execute a process according to the above-mentioned control method also constitutes another aspect of the present invention. Effect of the Invention

[0009] According to the present invention, it is possible to adjust the composition when taking a panning shot and then track the subject. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram showing a configuration of an imaging system according to a first embodiment. [Diagram 2] FIG. 2 is a block diagram showing the configuration of an image blur correction system in the first embodiment. [Diagram 3] FIG. [Figure 4] FIG. 4 is a diagram showing calculation of a subject vector in the first embodiment. [Diagram 5] 5A to 5C are diagrams showing a subject angular velocity, a panning angular velocity, and the movement of an OIS during panning in the first embodiment. [Figure 6] 4A to 4C are diagrams showing successful and unsuccessful images taken with panning in the first embodiment. [Figure 7] 5A to 5C are diagrams illustrating a composition adjustment control before a panning shot in the first embodiment. [Figure 8] 5 is a flowchart showing a panning assist process in the first embodiment. [Figure 9]11 is a diagram showing the subject angular velocity, panning angular velocity, and the movements of the OIS and IBIS during composition adjustment control and panning. [Figure 10] 13A to 13C are diagrams showing a subject angular velocity, a panning angular velocity, and movements of the OIS and the IBIS during composition adjustment control and panning in the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, embodiments of the present invention will be described with reference to the drawings. EXAMPLES

[0012] 1 shows the configuration of an imaging system (camera system) according to Example 1. The camera system includes an interchangeable lens 101 as a lens device, and an imaging device (hereinafter referred to as a camera body) 100 as an optical device to which the interchangeable lens 101 is detachably and communicably connected.

[0013] The camera body 100 has a camera MPU 102 as a computer, an operation unit 103, an image sensor 104, a camera side contact terminal 105, a camera side gyro sensor 106, an acceleration sensor 109, and a rear display 116.

[0014] The camera MPU 102 is a controller that controls the entire camera system consisting of the camera body 100 and the interchangeable lens 101, and controls various operations such as AE, AF, and imaging in response to inputs from an operation unit 103 (described later). The camera MPU 102 communicates various commands and information with the lens MPU 110 via a camera side contact terminal 105 and a lens side contact terminal 112 provided on the interchangeable lens 101. The camera side contact terminal 105 and the lens side contact terminal 112 also include a power supply terminal for supplying power from the camera body 100 to the interchangeable lens 101.

[0015] The operation unit 103 includes a mode selection dial that is operated by the user (photographer) to select various imaging modes, a release button that is operated by the user to instruct imaging preparation operations and imaging operations, and the like. By operating the mode selection dial, imaging modes such as still image imaging, video imaging, and the flowing shot assist mode described later can be selected. When the release button is pressed halfway, the first switch (Sw1) is turned on, and when it is pressed fully, the second switch (Sw2) is turned on. In response to the turning on of Sw1, AE and AF as imaging preparation operations are performed. Also, in response to the turning on of Sw2, the AE setting is confirmed and AF is stopped, and the start of imaging (exposure) is instructed. Further, when the start of exposure is instructed, the camera MPU 102 first turns on Sw2-1 as an exposure preparation signal, and turns on Sw2-2 as an exposure instruction signal that instructs the start of actual exposure after a predetermined time from the start instruction of the imaging operation. Sw2-1 and Sw2-2 are turned off at the timing when the set exposure time has elapsed and imaging is completed. The on / off states of Sw1, Sw2-1, and Sw2-2 are notified from the camera MPU 102 to the lens MPU 110 by communication.

[0016] The imaging element 104 is composed of a photoelectric conversion element such as a CCD sensor or a CMOS sensor, and photoelectrically converts (images) the subject image formed by the imaging optical system described later to generate an imaging signal. The camera MPU 102 generates an imaging image (image data) using the imaging signal from the imaging element 104.

[0017] The camera-side gyro sensor 106 is a shake sensor that detects angular shake (camera shake) applied to the camera body 100 due to hand shake or the like, and outputs a camera shake detection signal as an angular velocity signal. The camera MPU 102 controls the driving of the image sensor actuator (driving means) 107 based on the camera shake detection signal or the like, and moves (shifts) the image sensor 104 in a direction perpendicular to the optical axis of the imaging optical system. At this time, the camera MPU 102 performs feedback control of the image sensor actuator 107 so that the position of the image sensor 104 detected by the image sensor position sensor 108 (the amount of movement from the position on the optical axis that is the shift center) approaches the target position. This allows image blur correction by a sensor shift method using the shift of the image sensor 104 (in-body sensor vibration isolation: hereinafter referred to as IBIS).

[0018] The camera MPU 102 also performs calculations to recognize the subject, separate the subject from the background, and obtain the subject's moving direction, moving speed, etc., based on image data from the image sensor 104 and lens information received from the lens MPU 110.

[0019] The acceleration sensor 109 is used to detect the attitude of the camera body 100 and to detect shift shake that is difficult to detect with the camera-side gyro sensor 106 described above.

[0020] The rear display 116 as a display means displays image data obtained by the camera MPU 102 through the image sensor 104 as an image. In the following description, imaging (exposure) means imaging for recording to obtain a still image or video for recording. Before imaging (before exposure), the user can observe the image displayed on the rear display 116 as a finder image (live view image). After imaging (after exposure), the image data can be displayed on the rear display 116 as a still image or video for recording.

[0021] The interchangeable lens 101 has an imaging optical system (not shown), a lens MPU 110 as a computer, a lens side contact terminal 112, and a lens side gyro sensor 111. The lens side gyro sensor 111 is a shake sensor that detects angular shake (lens shake) of the interchangeable lens 101 and outputs a lens shake detection signal as an angular velocity signal.

[0022] The lens MPU 110 controls the driving of the lens actuator (driving means) 113 based on the lens shake detection signal and an OIS correction ratio described later, thereby moving (shifting) a correction lens 114, which is an optical element that is part of the imaging optical system, in a direction perpendicular to the optical axis of the imaging optical system. At this time, the lens MPU 110 performs feedback control of the lens actuator 113 so that the position of the correction lens 114 detected by the lens position sensor 115 (the amount of movement from a position on the optical axis that is the shift center) approaches a target position. In this way, image shake correction (optical element anti-vibration: hereinafter referred to as OIS) is performed by shifting the correction lens 114.

[0023] 2 shows the configuration of an image shake correction system in the camera system of this embodiment. The image shake correction system includes a camera image shake correction unit 201 provided on the camera body 100 side and a lens image shake correction unit 209 provided on the interchangeable lens 101 side. The camera image shake correction unit 201 is part of the camera MPU 102, and the lens image shake correction unit 209 is part of the lens MPU 110.

[0024] The camera gyro offset removal unit 202 removes an offset component from a camera shake detection signal (angular velocity signal) output from the camera side gyro sensor 106 mounted on the camera body 100. The camera side angle conversion unit 203 converts the angular velocity signal output from the camera gyro offset removal unit 202 into an angle signal. The camera information storage unit 204 stores camera information such as the drivable amount of the IBIS and the size of the image sensor 104. The camera information is used for driving control of the IBIS and is transmitted from the lens communication transmission unit 205 to the lens MPU 110. The lens communication transmission unit 205 also transmits to the lens MPU 110 subject information (subject position, subject speed, etc.) obtained from a subject recognition processing unit 217 provided in the camera MPU 102 separately from the image shake correction system.

[0025] The lens communication receiving unit 206 receives OIS information related to image shake correction (information such as an OIS correction ratio and OIS sensitivity indicating the relationship between the shift amount of the correction lens 114 and the image shake correction amount) transmitted from the camera communication transmitting unit 213 of the interchangeable lens 101. The camera side cooperative control unit 207 determines the amount of image shake to be corrected by the IBIS based on the camera information read from the camera information storage unit 204 and the OIS information received via the lens communication receiving unit 206. The image sensor drive control unit 208 generates a drive control signal for shifting and driving the image sensor 104 in the IBIS based on the angle signal output from the camera side angle conversion unit 203 and the amount of image shake determined by the camera side cooperative control unit 207.

[0026] Furthermore, when the panning assist mode is set, the camera-side cooperation control unit 207 determines the adjustment shift amount of the image sensor 104 based on the subject position indicated by the subject information from the subject recognition processing unit 217. The image sensor drive control unit 208 generates a drive control signal for shifting the image sensor 104 by the adjustment shift amount.

[0027] The lens gyro offset removal unit 210 removes an offset component from a lens shake detection signal (angular velocity signal) output from the lens side gyro sensor 111 mounted on the interchangeable lens 101. The lens side angle conversion unit 211 converts the angular velocity signal output from the lens gyro offset removal unit 210 into an angle signal. The camera communication receiving unit 214 receives subject information and information on the drive amount of the correction lens 114 transmitted from the lens communication transmitting unit 205 of the camera body 100. The lens information storage unit 212 stores information on the OIS driveable amount and OIS sensitivity. The lens information storage unit 212 also stores IBIS sensitivity information indicating the relationship between the shift amount of the image sensor 104 and the image shake correction amount.

[0028] The lens side cooperative control unit 215 performs cooperative control of the OIS and IBIS based on information read from the lens information storage unit 212 and information received via the camera communication receiving unit 214. At this time, it calculates a correction ratio, which is the ratio of the amount of image blur corrected by the OIS and the IBIS (ratio related to the control of the OIS and the IBIS). The correction lens drive control unit 216 generates a drive control signal for shifting and driving the correction lens 114 in the OIS based on the angle signal from the lens side angle conversion unit 211. Furthermore, when the panning assist mode is set in the camera body 100, the correction lens drive control unit 216 generates a drive control signal for shifting and driving the correction lens 114 to perform subject tracking control based on subject information received via the camera communication receiving unit 214.

[0029] Next, panning will be explained. Figure 3 shows the movements of the subject and the camera (system) in a time series of (a), (b), and (c) when taking a panning shot of a subject (train) passing in front of the photographer. In panning, the camera is moved (panned) to match the moving speed of the subject even during the exposure period, thereby obtaining a captured image in which the subject movement stops and the background flows. However, as shown in Figure 3, even if the photographer intends to pan the camera to match the movement of the subject, there may actually be a difference between the speed at which the camera is panned (panning speed) and the moving speed of the subject (subject speed). Here, there is a correlation between the fluctuation in the subject speed and the fluctuation in the output of the vibration sensor provided in the camera.

[0030] As shown in FIG. 3(b), if the angular displacement of the camera movement is θ [deg], the subject distance is L, the imaging magnification is β, and the shake displacement of the subject image is D, then the relationship shown in the following formula (1) holds.

[0031] D=βLπθ / 180 (1) For this reason, if the subject speed is Va and the panning angular velocity of the camera detected by the shake sensor (the motion detection result) is ωa, the relationship shown in the following formula (2) holds.

[0032] Va=βLπωa / 180 (2) 4(a) and (b) show a subject image formed on the image sensor 104. By processing image data obtained by photoelectrically converting the subject image with the image sensor 104 in the subject recognition processing unit 217, it is possible to separate the subject and the background in the image data and obtain subject information indicating the size, type (car, train, bird, person, etc.), position, moving speed, etc. of the subject.

[0033] 4(a) and (b) respectively show subject images formed on the image sensor 104 at a first timing during panning and a second timing that is slower than the first timing by a predetermined sampling rate. FIG. 4(c) shows motion vector data obtained as a result of performing a comparison process on the subject images shown in (a) and (b) for each area divided into a grid pattern. Since panning is performed, the position of the subject, the train, in the captured image does not change much in (c), and a small value is output as the motion vector data. On the other hand, the background (buildings, etc.) moves at a speed equivalent to the subject speed, and therefore a large value of motion vector data is obtained. The subject speed Va is calculated from the amount of deviation between corresponding pixels between the predetermined sampling rates. By subtracting the angular velocity ωa described above from the output ω of the shake sensor during exposure (image capture) in panning, the angular velocity ωo for performing good panning, that is, for accurately tracking a moving subject, can be calculated as shown in the following equation (3).

[0034] ω 0 =ω-ωa =ω-180Va / (βLπ) (3) The upper diagram in FIG. 5 shows the subject speed when the panning shot shown in FIG. 3 is performed and the panning shot speed detected through the lens side gyro sensor 111, each converted into an angular velocity [deg / sec]. The lower diagram shows the shift amount of the compensation lens 114. When the camera is panned at a speed faster than the subject speed to perform panning shots as in FIG. 3, the subject speed is notified to the lens MPU 110 from the camera MPU 102 during the preparation period as the first period when the exposure preparation signal (Sw2-1) is turned on, in order to cancel blurring of the subject image during exposure. Then, during the exposure period as the second period that starts when the exposure instruction signal (Sw2-2) is turned on, the compensation lens 114 (OIS) is rotated in the + direction shown in the lower diagram in FIG. 5 at the above-mentioned angular velocity ω 0 The shift is made at a speed that gives

[0035] In this way, by shifting the compensation lens 114 to track the subject so as to reduce the difference between the subject speed and the panning speed (preferably to make it zero), it is possible to perform good panning with reduced image blurring during panning.

[0036] However, even if the photographer's desired composition is one in which the entire train, which is the subject, fits within the imaging screen as shown in FIG. 6(a), the image actually obtained by panning (hereinafter referred to as panning image) may not include part of the train within the imaging screen as shown in FIG. 6(b). In this case, panning is unsuccessful. In the subject tracking shown in FIG. 5, the position of the subject image at the start of exposure is maintained, so if panning is started with the composition shown in FIG. 6(b), a panning image with the composition shown in FIG. 6(a) cannot be obtained. For this reason, in this embodiment, composition adjustment control is performed as a process of adjusting the position of the subject image within the imaging screen (i.e., on the imaging surface of the image sensor 104) immediately before exposure, that is, the composition.

[0037] Fig. 7(a) shows the same state as Fig. 3(a). Fig. 7(b) shows a composition adjustment in which the position of the subject image in the imaging screen, that is, the composition, is adjusted by shifting the image sensor 104 before subject tracking. The flowchart in Fig. 8 shows a panning assist process (control method) including composition adjustment control and subject tracking control, which is executed according to a program by the camera MPU 102 as a control device for panning. The camera MPU 102 functions as an acquisition means and a control means.

[0038] In step S801, when the photographer instructs the start of exposure and Sw2-1 is turned on (entering the preparation period before imaging), the camera MPU 102 causes the object recognition processing unit 217 to calculate the object position (in this embodiment, the center of gravity position of the object image) in step S802.

[0039] Next, in step S803, the camera MPU 102 causes the camera-side cooperation control unit 207 to calculate an adjustment shift amount of the IBIS based on the subject position calculated in step S802 so that the position of the subject image moves to the center position (predetermined position) of the imaging screen in the panning direction. Then, the camera MPU 102 shifts the IBIS by the adjustment shift amount. This allows the composition adjustment to be performed.

[0040] After that, in step S804, Sw2-2 is turned on and exposure starts (entering the exposure period during image capture), and in step S805, camera MPU 102 causes correction lens drive control unit 216 to control the OIS for subject tracking. At this time, camera MPU 102 may also control the IBIS for correcting image blur caused by camera shake or the like, or may control the OIS and IBIS in cooperation. When exposure for panning ends in this manner, camera MPU 102 ends this process.

[0041] Figure 9 adds shifting of the image sensor 104 (IBIS) during composition adjustment to Figure 5. First, before exposure, that is, during the preparation period from when Sw2-1 is turned on to when exposure starts, the position of the subject image within the imaging screen is adjusted by shifting the IBIS (one of the elements), and thereafter the position of the subject image is maintained (in other words, the IBIS is stopped at the position after the composition adjustment).

[0042] Next, during the exposure period corresponding to the ON of Sw2-2, the OIS (the other element) is shifted to track the subject. In this way, the position of the subject image within the imaging screen can be adjusted while ensuring the drive amount of the OIS for tracking the subject. As a result, a panning image with the composition shown in Figure 6(a) is obtained.

[0043] The adjustment shift amount of the IBIS before exposure (including the shift direction) may be calculated by a method other than the calculation method described above. For example, the type of the subject obtained from the subject recognition processing unit 217 may be identified, and the adjustment shift amount may be calculated so as to approach a preferable composition according to the identification result. Further, the photographer may be allowed to input (indicate) a target position of the subject image within the imaging screen, and the adjustment shift amount may be calculated so that the position of the subject image moves to the target position. Furthermore, an imaging image in which the subject is within the imaging angle of view may be registered in advance, and the adjustment shift amount may be calculated so that the entire subject is within the imaging angle of view by identifying the same subject during continuous shooting.

[0044] Also, the IBIS does not necessarily have to be shifted by the calculated adjustment shift amount (to the target position). The IBIS may be shifted by a predetermined amount in a direction in which the position of the subject image approaches a compositionally preferable position such as the center position of the imaging screen (a predetermined direction), a direction instructed by the photographer, or a direction according to the identification result of the subject.

Example

[0045] Next, Example 2 will be described. In Example 1, the case where after performing composition adjustment by the IBIS before exposure, subject tracking using the OIS is performed while maintaining the position of the composition-adjusted subject image during exposure was described. In contrast, in Example 2, subject tracking using cooperative control of the OIS and the IBIS is performed during exposure.

[0046] FIG. 10 is obtained by adding the shift of the IBIS in composition adjustment and the cooperative control of the OIS and the IBIS to FIG. 5.

[0047] When the OIS and IBIS are controlled simultaneously, the ratio at which each is responsible for tracking the subject (OIS correction ratio and IBIS correction ratio) can be appropriately set based on the respective driveable amounts of the OIS and IBIS, enabling good subject tracking. The OIS+ correction ratio and OIS- correction ratio, which are the OIS correction ratios according to the OIS shift direction (+ side and - side), and the IBIS+ correction ratio and IBIS- correction ratio, which are the IBIS correction ratios according to the IBIS shift direction (+ side and - side), are calculated as follows: θOIS+ and θOIS- are the possible shift drive amounts on the + side and - side of the OIS, respectively. θIBIS+ and θIBIS- are the possible shift drive amounts on the + side and - side of the IBIS, respectively.

[0048] OIS+ correction ratio: θOIS+ / {(θIBIS+)+(θOIS+)} OIS-correction ratio: θOIS- / {(θIBIS-)+(θOIS-)} IBIS+ correction ratio: θIBIS+ / {(θIBIS+)+(θOIS+)} IBIS-correction ratio: θIBIS- / {(θIBIS-)+(θOIS-)} The IBIS is shifted to adjust the composition during the preparation period when Sw2-1 is turned on, and then during the exposure period that begins when Sw2-2 is turned on, subject tracking is performed through cooperative control of the OIS and IBIS. At this time, the IBIS is controlled from the shift position after the composition adjustment.

[0049] This makes it possible to effectively utilize the respective drive capacity of the OIS and IBIS to perform subject tracking that can accommodate a larger error between the subject speed and the panning speed.

[0050] As described above, according to each embodiment, by adjusting the composition before exposure for subject tracking, it is possible to obtain a panning image with a composition more desired by the user.

[0051] In the above embodiments, the composition adjustment is performed using the IBIS, but the composition adjustment may be performed using the OIS, or both the IBIS and the OIS. In other words, the composition adjustment may be performed using at least one of the IBIS and the OIS. In addition, the lens MPU in the lens device (optical device) may have the function of the control device.

[0052] Furthermore, in each of the above embodiments, a lens-interchangeable image pickup device equipped with an IBIS to which an interchangeable lens equipped with an OIS is attached has been described, but an embodiment also includes a lens-integrated image pickup device equipped with an OIS and an IBIS.

[0053] The above embodiment includes the following configurations as flowcharts.

[0054] (Configuration 1) A control device that assists in panning a moving subject, an acquisition means for acquiring a subject position and a subject velocity from an image generated using an output of an image sensor that photoelectrically converts a subject image formed by an optical system including an optical element; a control unit that performs subject tracking control to move at least one of the optical element and the image sensor during imaging of the panning shot based on a detection result of the subject speed and a movement of an optical device having at least one of the optical system and the image sensor, The control device is characterized in that, before the image is captured, the control means performs composition adjustment control to move at least one of the elements so that the subject image moves to a predetermined position or direction on the imaging element based on the subject position. (Configuration 2) The control means In the composition adjustment control, one of the optical element and the image pickup element is moved; The control device according to configuration 1, characterized in that in the subject tracking control, the one of the elements is stopped at a position to which it was moved in the composition adjustment control, and the other of the optical element and the imaging element is moved. (Configuration 3) The control device according to configuration 1, characterized in that, in the subject tracking control, the control means moves the optical element and the image sensor in accordance with a ratio calculated based on respective drivable amounts of the optical element and the image sensor. (Configuration 4) The control device according to any one of configurations 1 to 3, wherein the control means performs the composition adjustment control in a first period from when an instruction to start the imaging is given until the imaging starts, and performs the subject tracking control in a second period after the imaging starts. (Configuration 5) 5. The control device according to any one of configurations 1 to 4, wherein the predetermined position or direction is a center position of the imaging element or a direction approaching the center position. (Configuration 6) 5. The control device according to any one of configurations 1 to 4, wherein the predetermined position or direction is a position or direction designated by a user. (Configuration 7) 5. The control device according to any one of configurations 1 to 4, wherein the predetermined position or direction is a position or direction according to a result of identifying the subject from the image. (Configuration 8) A control device according to any one of configurations 1 to 7; An optical device comprising at least one of the optical system and the image sensor. (Configuration 9) A control device according to any one of configurations 1 to 7; The imaging element; and a driving means for moving the imaging element. (Configuration 10) A device according to claim 9, which is detachably connected to the imaging device, The optical system; and a driving means for moving the optical element. (Other Examples) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

[0055] The embodiments described above are merely representative examples, and various modifications and alterations are possible for each embodiment when implementing the present invention. [Explanation of symbols]

[0056] 100 Camera body 101 Interchangeable Lenses 102 Camera MPU 104 Image sensor 106 Camera side gyro sensor 111 Lens side gyro sensor 114 Corrective Lenses

Claims

1. A control device that assists in panning a moving subject, an acquisition means for acquiring a subject position and a subject velocity from an image generated using an output of an image sensor that photoelectrically converts a subject image formed by an optical system including an optical element; a control unit that performs subject tracking control to move at least one of the optical element and the image sensor during imaging of the panning shot based on a detection result of the subject speed and a movement of an optical device having at least one of the optical system and the image sensor, The control device is characterized in that, before the image is captured, the control means performs composition adjustment control to move at least one of the elements so that the subject image moves to a predetermined position or direction on the imaging element based on the subject position.

2. The control means In the composition adjustment control, one of the optical element and the image pickup element is moved; 2. The control device according to claim 1, wherein in the subject tracking control, the one of the elements is stopped at a position to which it was moved in the composition adjustment control, and the other of the optical element and the image pickup element is moved.

3. The control device according to claim 1 , wherein the control means, in the subject tracking control, moves the optical element and the image sensor in accordance with a ratio calculated based on respective drivable amounts of the optical element and the image sensor.

4. 2. The control device according to claim 1, wherein the control means performs the composition adjustment control during a first period from when an instruction to start the imaging is given until the imaging starts, and performs the subject tracking control during a second period after the imaging starts.

5. 2. The control device according to claim 1, wherein the predetermined position or direction is a center position of the imaging element or a direction approaching the center position.

6. The control device according to claim 1 , wherein the predetermined position or direction is a position or direction designated by a user.

7. 2. The control device according to claim 1, wherein the predetermined position or direction is a position or direction according to a result of identifying the subject from the image.

8. A control device according to any one of claims 1 to 7; An optical device comprising at least one of the optical system and the image sensor.

9. A control device according to any one of claims 1 to 7; The imaging element; and a driving means for moving the imaging element.

10. A device removably connected to the imaging device according to claim 9, The optical system; and a driving means for moving the optical element.

11. A control method for assisting in panning a moving subject, comprising: acquiring a subject position and a subject velocity from an image generated using an output from an image sensor that photoelectrically converts a subject image formed by an optical system including an optical element; and performing a subject tracking control for moving at least one of the optical element and the image sensor during imaging of the panning shot based on a detection result of the subject speed and a movement of an optical device having at least one of the optical system and the image sensor, The control method according to claim 1, further comprising a step of performing a composition adjustment control for moving at least one of the elements so that the subject image moves to a predetermined position or direction on the imaging element based on the subject position before the imaging.

12. A program causing a computer to execute a process according to the control method according to claim 11.

Images