Control device and method, image blur correction device, imaging device, program and storage medium

Abstract

To expand the tracking range while ensuring quality of use in subject tracking that utilizes both optical and electronic image stabilization functions. [Solution] The system includes: a first acquisition means for acquiring a shake amount representing the magnitude of the shake of the detected object; a second acquisition means for acquiring a tracking amount for holding a predetermined subject included in the image at a predetermined position in a partial image cut out by an electronic correction means; a third acquisition means for acquiring a first ratio for allocating the movable range of the optical correction means to shake correction and a second ratio for allocating it to tracking control, a third ratio for allocating the movable range of the partial image to shake correction and a fourth ratio for allocating it to tracking control, according to predetermined conditions; and a fourth acquisition means for determining the control amount of the optical correction means and the control amount of the electronic correction means for shake correction and tracking control based on the shake amount, the tracking amount, and the first to fourth ratios.

Description

Technical Field

[0001] The present invention relates to a control device and method, an image blur correction device, an imaging device, a program, and a storage medium, and particularly to a technique for image blur correction and subject tracking using an image blur correction function.

Background Art

[0002] Conventionally, there is an imaging device that has an image blur correction function and stabilizes camera shake during shooting of a moving image. In such an imaging device, for example, camera shake is detected using an angular velocity sensor or the like, and a correction lens or an imaging element is driven to cancel the detected camera shake, or an image obtained by cutting out a small area from the imaging area by image processing is geometrically deformed. Generally, the former is called optical image blur correction, and the latter is called electronic image blur correction.

[0003] Also, a subject tracking technique is known in which a subject is detected from an image obtained by shooting, the detected subject is tracked using an image blur correction function, and the subject is held at a predetermined position within the angle of view.

[0004] In Patent Document 1, a technique is proposed for determining which of an anti-shake function and a subject tracking function should be preferentially executed so as to achieve both anti-shake performance and tracking performance using a correction lens.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] However, when using only one of either optical or electronic image stabilization to track a subject, there are limitations to improving tracking performance. For example, when using only electronic image stabilization to track a subject, it is known that improving tracking performance requires increasing the size of the image sensor, and that improving tracking performance becomes difficult as the correction angle decreases with increasing focal length. On the other hand, when using only optical image stabilization to track a subject, it is not possible to improve the correction angle unless the drive range of the components used for optical image stabilization (correction lenses and image sensors) is widened. Furthermore, depending on how these components are moved, vibrations and operating noises may be generated, potentially compromising the quality of use.

[0007] This invention was made in view of the above-mentioned problems, and aims to expand the tracking range while ensuring quality of use in subject tracking that utilizes both optical image stabilization and electronic image stabilization functions. [Means for solving the problem]

[0008] To achieve the above objective, the control device of the present invention controls a first correction means for optically correcting shake and a second correction means for correcting shake by changing the cropping position of a partial image cropped from an image captured by an imaging means, and includes a first acquisition means for acquiring a shake amount representing the magnitude of shake of a detected object, a second acquisition means for acquiring a tracking amount for holding a predetermined subject included in the image at a predetermined position in the partial image, a third acquisition means for acquiring a first ratio for allocating the movable range of the first correction means to shake correction and a second ratio for allocating it to tracking control, a third ratio for allocating the movable range of the partial image in the second correction means to shake correction and a fourth ratio for allocating it to tracking control, according to predetermined conditions, and a fourth acquisition means for determining the control amounts of the first correction means and the second correction means for shake correction and tracking control based on the shake amount, the tracking amount, and the first to fourth ratios. [Effects of the Invention]

[0009] According to the present invention, when subject tracking using both optical image stabilization and electronic image stabilization functions, the tracking range can be expanded while ensuring quality of use. [Brief explanation of the drawing]

[0010] [Figure 1] A block diagram showing the configuration of the imaging system in an embodiment of the present invention. [Figure 2] A block diagram showing the configuration for performing image blur correction control and subject tracking control in the embodiment. [Figure 3] A figure showing the ratio table in the first embodiment. [Figure 4] A flowchart illustrating image blur correction control and subject tracking control in the first embodiment. [Figure 5] A figure showing another ratio table in the first embodiment. [Figure 6] A figure showing yet another ratio table in the first embodiment. [Figure 7] A figure showing yet another ratio table in the first embodiment. [Figure 8A] A graph showing an example of the transition of the tracking control quantity in the second embodiment. [Figure 8B] A graph showing another example of the transition of the tracking control quantity in the second embodiment. [Figure 9] A flowchart illustrating the control in the second embodiment. [Figure 10] A diagram illustrating the field of view to illustrate the problems in the third embodiment. [Figure 11] An explanatory diagram of the field of view when control is performed according to the third embodiment. [Figure 12] A graph showing an example of the transition of the tracking control amount in the third embodiment. [Figure 13] A flowchart illustrating subject tracking control in the third embodiment. [Modes for carrying out the invention]

[0011] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiments, not all of these plurality of features are essential for the invention, and the plurality of features may be arbitrarily combined. Further, in the accompanying drawings, the same or similar configurations are given the same reference numerals, and redundant descriptions are omitted.

[0012] <First Embodiment> FIG. 1 is a block diagram showing the configuration of an imaging system according to an embodiment of the present invention. The imaging system in the present embodiment mainly includes a camera body 1 and a lens unit 2 detachable from the camera body 1. When the lens unit 2 is attached to the camera body 1, information is transmitted via a camera-side communication unit 140 and a lens-side communication unit 128.

[0013] First, the configuration of the lens unit 2 will be described. The imaging optical system 200 includes a zoom lens 101, an image blur correction lens 102, a focus lens 103, and an aperture 104. The zoom lens driving unit 124 drives the zoom lens 101 in the optical axis direction to optically change the focal length of the imaging optical system 200, thereby changing the shooting angle of view. Further, the zoom lens control unit 127 controls the zoom lens driving unit 124 according to a zoom instruction performed by the photographer operating the operation unit 114, thereby controlling the position of the zoom lens 101. The focus lens driving unit 121 adjusts the focusing position by driving the focus lens 103 in the optical axis direction. The aperture driving unit 120 adjusts the incident light amount by controlling the aperture diameter of the aperture 104.

[0014] The image blur correction lens position detection unit 123 detects the position of the image blur correction lens 102. The lens-side shake detection unit 125 detects the vibration of shake or sway applied to the lens unit 2 (detection target) and outputs a shake detection signal. As the lens-side shake detection unit 125, a device that detects angular velocity such as a gyro sensor or a device that detects acceleration such as an acceleration sensor can be used. The image blur correction lens anti-vibration control unit 126 calculates an image blur correction amount for suppressing shake based on the shake detection signal output from the lens-side shake detection unit 125, or the camera-side shake detection unit 134, or both, and the current position of the image blur correction lens 102 detected by the image blur correction lens position detection unit 123, and notifies the image blur correction lens drive unit 122. The image blur correction lens drive unit 122 optically corrects the image blur caused by the shake of the imaging system by driving the image blur correction lens 102 in a direction perpendicular to the optical axis.

[0015] Next, the configuration of the camera body 1 will be described. The light that has passed through the imaging optical system 200 is received by the imaging element 106 using a CCD (charge-coupled device) or a CMOS sensor (complementary metal-oxide semiconductor) etc. via the shutter 105, and is photoelectrically converted from the optical signal to an electrical signal. The shutter 105 is driven by the shutter drive unit 135.

[0016] The AD converter 107 performs noise removal processing, gain adjustment processing, and AD conversion processing on the analog image signal output from the imaging element 106, and outputs a digital image signal.

[0017] The timing generator 108 controls the drive timing of the imaging element 106 and the processing timing of the AD converter 107 according to the command of the camera control unit 115. <00001​​The image processing circuit 109 performs pixel interpolation, color conversion, and other processing on the digital image signal output from the AD converter 107, and then sends the processed image data to the internal memory 110. The image processing circuit 109 includes a positioning circuit for multiple images captured in succession, a geometric transformation circuit that performs cylindrical coordinate transformation and distortion correction of lens groups, and a synthesis circuit that performs cropping and compositing processing.

[0019] Furthermore, the image processing circuit 109 extracts a partial region from the imaging area through image processing and controls the extraction position according to the direction and magnitude of image blur, thereby realizing electronic image blur correction and subject tracking. In this embodiment, the extraction position is controlled based on a control amount for the image extraction position calculated by the control amount calculation unit 1333, which will be described later. In this embodiment, electronic image blur correction is performed using a projection transformation circuit provided in the image processing circuit 109.

[0020] The display unit 111 displays image data stored in the internal memory 110, along with shooting information and other data. The compression / decompression processing unit 112 performs compression or decompression processing on the image data stored in the internal memory 110 according to the image format. The memory 113 stores various data such as parameters. The control unit 114 is a user interface for the user to perform various commands to the imaging system, such as various menu operations and mode switching operations.

[0021] The camera control unit 115 is composed of a processing unit such as a CPU (Central Processing Unit) and executes various control programs stored in the internal memory 110 in response to user operations by the operation unit 114. These control programs include, for example, programs for zoom control, image blur correction control, automatic exposure control, automatic focus adjustment control, and processes for detecting the subject's face and subject tracking control.

[0022] The luminance signal detection unit 137 detects the luminance of the subject and scene from the image signal read from the image sensor 106 and output from the AD converter 107. The exposure control unit 136 calculates the exposure value (aperture value and shutter speed) based on the luminance information obtained by the luminance signal detection unit 137, and notifies the aperture drive unit 120 of the calculation result via the shutter drive unit 135, the camera-side communication unit 140, and the lens-side communication unit 128. The exposure control unit 136 also simultaneously performs gain control to amplify the signal read from the image sensor 106. This enables automatic exposure control (AE control).

[0023] The evaluation value calculation unit 138 extracts specific frequency components from the luminance information obtained by the luminance signal detection unit 137 and calculates a contrast evaluation value based on the extraction results.

[0024] The focus lens control unit 139 issues a command to drive the focus lens 103 by a predetermined amount over a predetermined range. At the same time, it obtains a contrast evaluation value, which is the result of a calculation by the evaluation value calculation unit 138 on the brightness information of the image signal obtained at each focus lens position. The amount of defocus is calculated from the focus lens position where the curve of change of the contrast evaluation value obtained in this way is at its peak, and this is notified to the focus lens drive unit 121 via the camera-side communication unit 140 and the lens-side communication unit 128. The focus lens drive unit 121 then drives the focus lens 103 according to the amount of defocus, thereby performing contrast-based automatic focusing control (AF control) in which the light beam is in focus on the surface of the image sensor 106.

[0025] Although this explanation describes focus control using the contrast method, phase-difference method focus control is also acceptable. Since phase-difference method focus control is publicly known, its explanation will be omitted.

[0026] The camera shake detection unit 134 detects shake and vibration applied to the camera body 1 (the object to be detected). The camera shake detection unit 134 can be a device that detects angular velocity, such as a gyro sensor, or a device that detects acceleration, such as an accelerometer, but in this embodiment, it detects angular velocity and outputs an angular velocity signal.

[0027] The image blur correction control unit 133 can communicate with the image blur correction lens vibration damping control unit 126 via the camera-side communication unit 140 and the lens-side communication unit 128. Based on the vibration detection signals detected by the camera-side vibration detection unit 134, the lens-side vibration detection unit 125, or both, the image blur correction control unit 133 calculates the amount of vibration correction to suppress vibration using the image sensor 106. Then, based on the calculated amount of vibration correction and the current position of the image sensor 106 detected by the image sensor position detection unit 132, it transmits a drive signal for the image sensor 106 to the image sensor drive unit 130 to control vibration damping by the image sensor 106. Based on the drive signal received from the image blur correction control unit 133, the image sensor drive unit 130 drives the image sensor 106 in a direction perpendicular to the optical axis.

[0028] The motion vector detection unit 131 calculates the correlation value between the image of the current frame and the image of the previous frame in block units, obtained by dividing the frame using the block matching method. It then searches for the block in the previous frame with the smallest correlation value and detects the displacement of the other blocks relative to that block as a motion vector.

[0029] The subject identification unit 143 sets one of the subjects in the captured image as the subject to be tracked. The photographer can set any subject by touching or pressing buttons via the operation unit 114. If the photographer does not set a subject, the subject to be tracked may be determined by an automatic subject setting program.

[0030] The subject detection unit 141 detects the area of ​​the subject to be tracked, as set by the subject identification unit 143, and generates subject detection information. The subject detection information includes information such as the type of subject (e.g., person, animal, vehicle), part (e.g., eyes, face, body), position, and size. The subject tracking calculation unit 142 calculates the tracking amount for tracking the target subject based on the subject detection information. Details will be described later using Figure 2.

[0031] Figure 2 is a block diagram showing the configuration for image blur correction control and subject tracking control in the first embodiment, and in particular shows the detailed configuration of the image blur correction control unit 133 and the subject tracking calculation unit 142. In this embodiment, image blur correction can be performed by optical image blur correction and electronic image blur correction, and optical image blur correction includes a method of driving the image blur correction lens 102 and a method of driving the image sensor 106. In the following description, optical image blur correction will be described as being performed by driving the image sensor 106, but it may also be performed by driving the image blur correction lens 102 alone, or by driving the image blur correction lens 102 and the image sensor 106 together.

[0032] The integration unit 1331 converts the angular velocity signal output from the camera shake detection unit 134 into a shake angle by performing integration processing. For example, an integral low-pass filter (LPF) can be used for the integration processing. If the camera shake detection unit 134 is an acceleration sensor, the integration unit 1331 converts the output acceleration signal into a shake amount corresponding to the shake angle by performing a second integral.

[0033] The shake correction amount calculation unit 1332 calculates the shake correction amount to cancel out the shake angle, taking into consideration the frequency band of the shake angle obtained by the integration unit 1331 and the range (movable area) in which the image sensor 106 can be driven. Specifically, the shake correction amount is calculated by integrating the gain related to the zoom magnification and subject distance with respect to the shake angle.

[0034] The shooting status acquisition unit 1334 acquires the shooting status, including the vibration state, such as whether the camera is in a static state, mounted on a tripod or the like, or whether the vibration due to hand shake is large or small, based on the swing angle obtained by the integration unit 1331.

[0035] The electronic image stabilization setting unit 1337 receives user settings related to electronic image stabilization from the operation unit 114. Here, the user can select whether or not to perform electronic image stabilization, the strength of the stabilization effect from multiple stabilization modes, and whether or not to perform subject tracking, and the angle of view cropped from the imaging area is determined according to the settings.

[0036] The subject target position setting unit 1424 calculates and sets the subject target position, which is the on-screen position where the subject to be tracked is kept when subject tracking control is performed. The subject target position is changeable, and possible setting values ​​include, for example, the center of the field of view, the on-screen position touched by the photographer, or a pre-stored coordinate position.

[0037] The tracking control amount calculation unit 1423 calculates the tracking control amount according to the subject target position set by the subject target position setting unit 1424 and the position of the subject included in the subject detection information from the subject detection unit 141.

[0038] The control amount calculation unit 1333 determines the control amount for the optical image blur correction member (in this case, the image sensor 106) and the control amount for the cropping position of the partial region in electronic image blur correction, based on the shake correction amount calculated by the shake correction amount calculation unit 1332, the tracking control amount calculated by the tracking control amount calculation unit 1423, the shooting conditions acquired by the shooting conditions acquisition unit 1334, and the settings related to electronic image blur correction set by the electronic image blur correction setting unit 1337. In this embodiment, the control amount calculation unit 1333 determines the control amount using a ratio table described later. The determined control amount for the optical image blur correction member is output to the position control unit 1335, and the control amount for the cropping position of the partial region is output to the image processing circuit 109.

[0039] The position control unit 1335 performs PID control (ratio control, integral control, and fine control) on the deviation between the target position of the image sensor 106, which is determined by the control amount of the optical image blur correction member calculated by the control amount calculation unit 1333, and the current position detected by the image sensor position detection unit 132. This is then converted into a drive signal for the image sensor 106 and input to the image sensor drive unit 130, which controls the position of the image sensor 106 to become the target position, thereby achieving optical image blur correction and subject tracking. Note that a detailed explanation of PID control is omitted as it is a common technique.

[0040] Next, the method for calculating the controlled variable in the controlled variable calculation unit 1333 will be explained using Figure 3. Figure 3 is a ratio table showing the ratio of the movable area of ​​the partial region extracted in electronic image stabilization and the movable area of ​​the optical image stabilization member to be allocated to image stabilization and subject tracking. The left column of Figure 3 shows the vibration state (condition) acquired by the shooting condition acquisition unit 1334, and in this embodiment, it is classified into four stages in order of increasing vibration: "tripod / gimbal", "low vibration", "medium vibration", and "high vibration". The numerical values ​​in the table in Figure 3 represent the vibration stabilization / tracking ratio of the movable area allocated to image stabilization (vibration stabilization) and subject tracking, when the movable area of ​​the partial region extracted in electronic image stabilization and the movable area of ​​the optical image stabilization member are set to 100%. In this embodiment, the movable area is set according to the magnitude of the vibration.

[0041] Note that the values ​​in the table in Figure 3 indicate the extent to which the control correction capability of each image stabilization is utilized, and do not mean, for example, that 30% of the camera shake component is corrected in the case of 30% image stabilization. In addition, with electronic image stabilization, the higher the correction capability of the image stabilization, the smaller the size of the cropped area becomes, so a limit is placed on the size of the cropped area to prevent it from becoming too small. For example, 100% in electronic image stabilization represents the correction capability when the cropped area is made as small as possible within the limited range.

[0042] For example, if the camera's shooting status is determined to be "tripod / gimbal," as shown in the table in Figure 3, both electronic and optical image stabilization are allocated 0% to image stabilization and 100% to subject tracking. This indicates that the camera's movement is extremely small and image stabilization control is unnecessary, so image stabilization is not performed, and the entire range of motion is allocated to subject tracking, maximizing the tracking performance.

[0043] Furthermore, when the camera's shooting conditions are determined to be "low vibration," electronic image stabilization allocates 0% to image stabilization and 100% to subject tracking, while optical image stabilization allocates 30% to image stabilization and 70% to subject tracking. This indicates that the camera is detected to be vibrating slightly, and therefore, optical image stabilization allocates 30% of its movable range to image stabilization.

[0044] Similarly, in this embodiment, depending on whether the camera's vibration is determined to be "moderate" or "high," the allocation ratio of optical image stabilization to vibration damping is increased, and in the case of "high vibration," the entire movable range (100%) of the optical image stabilization is allocated to vibration damping.

[0045] In this way, by increasing the proportion of the movable area used for subject tracking when vibrations are minimal, and increasing the proportion of the movable area used for image blur correction as vibrations increase, it is possible to balance subject tracking control and image blur correction control according to the vibration state.

[0046] Next, the image blur correction control and subject tracking control in this embodiment will be explained using the flowchart shown in Figure 4. In S101, the camera control unit 115 acquires the shooting status (vibration state) from the shooting status acquisition unit 1334.

[0047] In S102, if the vibration state acquired in S101 is "Tripod / Gimbal", proceed to S103, where, according to the ratio table shown in Figure 3, 0% of the movable range for electronic image stabilization is allocated to image stabilization and 100% is allocated to subject tracking. Additionally, 0% of the movable range for optical image stabilization is allocated to image stabilization and 100% is allocated to subject tracking, and proceed to S109.

[0048] Furthermore, in S104, if the vibration state is "low vibration", the system proceeds to S105, where, according to the ratio table shown in Figure 3, 0% of the movable range for electronic image stabilization is allocated to image stabilization and 100% is allocated to subject tracking. Additionally, 30% of the movable range for optical image stabilization is allocated to image stabilization and 70% is allocated to subject tracking, and the system proceeds to S109.

[0049] Furthermore, in S106, if the vibration state is "vibrating," the system proceeds to S107, where, according to the ratio table shown in Figure 3, 0% of the movable range for electronic image stabilization is allocated to image stabilization and 100% is allocated to subject tracking. Additionally, 50% of the movable range for optical image stabilization is allocated to image stabilization and 50% is allocated to subject tracking, and the system proceeds to S109.

[0050] Furthermore, in S106, if the vibration state is not "vibrating" (i.e., "high vibration"), the system proceeds to S108, where, according to the ratio table shown in Figure 3, 0% of the movable range for electronic image stabilization is allocated to image stabilization and 100% is allocated to subject tracking. Additionally, 100% of the movable range for optical image stabilization is allocated to image stabilization and 0% is allocated to subject tracking, and the system proceeds to S109.

[0051] In S109, image blur correction and subject tracking processing are performed using the image blur correction and subject tracking control amounts calculated by the control amount calculation unit 1333 within the movable range based on the ratio set in S103, S105, S107, or S108. Specifically, image blur correction and subject tracking are performed by controlling the drive of the image sensor 106 and the cropping position of the partial region based on the control amount calculated by the control amount calculation unit 1333. If the image blur correction lens 102 is also used, the ratio of the movable range of the image blur correction lens 102 should be added to the ratio table shown in Figure 3, and the control amount of the optical image blur correction member determined by the control amount calculation unit 1333 should be assigned to the image sensor 106 and the image blur correction lens 102. If only the image blur correction lens 102 is used for optical image blur correction, the image blur correction lens 102 should be driven and controlled instead of the image sensor 106 based on the control amount of the optical image blur correction member determined by the control amount calculation unit 1333.

[0052] Note that the ratio table shown in Figure 3 above is just one example and may be modified as needed. Other examples of ratio tables are described below.

[0053] (Ratio table based on user intent) Although the ratio table shown in Figure 3 was explained as determining the range of motion according to the vibration state determined by the shooting status acquisition unit 1334 based on the swing angle, the range of motion may also be determined according to the mode selected by the user from the operation unit 114.

[0054] Figure 5 shows an example of a ratio table when the user can select "Tracking Priority" or "Image Stabilization Priority" (condition) via menu settings. As shown in Figure 5, when "Tracking Priority" is selected, both electronic and optical image stabilization are allocated 0% to image stabilization and 100% to subject tracking. On the other hand, when "Image Stabilization Priority" is selected, electronic image stabilization is allocated 100% to subject tracking, and optical image stabilization is allocated 100% to image stabilization and 0% to subject tracking. In this way, the ratio table may be set based on the user's settings regardless of the shooting situation.

[0055] (Ratio table due to limitations in the optical image stabilization range) In the proportional table shown in Figure 3, the ratio of the movable area due to optical image stabilization was set to 100% when combined with image stabilization and subject tracking. However, with lenses that have large distortion or lenses with a small angle of incidence of light rays to the periphery, there are concerns about reduced image stabilization performance at the peripheral image height, reduced image quality due to reduced peripheral illumination, and reduced autofocus performance. For example, if optical image stabilization is used to continuously track the periphery for a long period of time, the reduction in image quality due to the above concerns may become significant. Therefore, under the circumstances (conditions) in which the above concerns exist, the movable area due to optical image stabilization may be limited.

[0056] Figure 6 shows an example of a ratio table when image height is limited. Compared to the ratio table shown in Figure 3, the range of motion for optical image stabilization can be limited by setting the sum of the ratios for camera shake stabilization and subject tracking to 75% instead of 100%, thereby preventing the aforementioned degradation of performance quality. (Ratio table prioritizing image quality) In the ratio table explained in Figure 3, the ratio of the movable area in electronic image stabilization was set to 100% as the sum of image stabilization and subject tracking. However, in electronic image stabilization, which extracts an image corresponding to a partial area (hereinafter referred to as a partial image) from the captured image, the smaller the size of the extracted partial image, the smaller the field of view becomes, and image quality deteriorates when the partial image is enlarged in subsequent processing. Therefore, an "image quality priority mode" (condition) may be set up, and if the user selects this mode, the movable range of electronic image stabilization may be limited, and the size of the extracted partial image may be restricted to improve image quality.

[0057] Figure 7 shows an example of a ratio table in "Image Quality Priority Mode." This ratio table is used instead of the ratio table shown in Figure 3 when "Image Quality Priority Mode" is set. Compared to the ratio table shown in Figure 3, in "Image Quality Priority Mode," the combined ratio of vibration stabilization and subject tracking in the movable range of electronic image stabilization is limited to a smaller value depending on the vibration state, rather than being 100%. By reducing the movable range of electronic image stabilization in this way, the size of partial images can be increased.

[0058] For example, in the "Tripod / Gimbal" setting, the range of motion for electronic image stabilization is limited to 30% in total for both image stabilization and subject tracking. This limitation helps to suppress the reduction in the size of partial images, i.e., the field of view. On the other hand, in the "High Vibration" setting, the range of motion for electronic image stabilization is limited to 75% in total. Compared to the "Tripod / Gimbal" setting, this limitation is closer to the amount of cropping when not in "Image Quality Priority Mode," balancing vibration stabilization and tracking performance. In this way, by limiting the range of motion for electronic image stabilization according to the need for image stabilization and subject tracking, it is possible to reduce the image quality degradation associated with image cropping.

[0059] As described above, according to the first embodiment, optical image stabilization and electronic image stabilization can be effectively combined depending on the vibration state of the imaging system and the user settings, thereby achieving a good balance between image stabilization and subject tracking.

[0060] <Variation> The field of view may change significantly when the shooting conditions change. For example, if you remove a camera mounted on a tripod and lift it up, the control in the ratio table in Figure 3 switches from "Tripod / Gimbal" to "Low Vibration." In this case, the allocation ratio of the movable area in the optical image stabilization instantly switches from 100% subject tracking to 70% subject tracking, which may cause a sudden change in the field of view.

[0061] When the shooting conditions change in this way, the display unit 111 may display a warning indicating that a change in the field of view may occur, thereby notifying the user of the change in the field of view. Alternatively, as in the case of "tracking priority" shown in Figure 5, when subject tracking is prioritized, the field of view may be maintained without changing the movable range of the optical image stabilization assigned to tracking, by maintaining the movable range of the optical image stabilization before the shooting conditions changed even after the shooting conditions changed.

[0062] <Second Embodiment> Next, a second embodiment of the present invention will be described. One challenge when using optical image stabilization in addition to electronic image stabilization for subject tracking is the noise and vibration generated when the optical image stabilization member is moved rapidly. In this embodiment, as an example of a case where it is necessary to move the optical image stabilization member rapidly, we will describe the centering operation of the control amount when transitioning from a state in which subject tracking is performed using the image stabilization function (hereinafter referred to as "subject tracking mode") to normal image stabilization without subject tracking (hereinafter referred to as "normal image stabilization mode"). Note that the configuration of the imaging system in the second embodiment can be the same as that described in Figures 1 and 2, so the explanation will be omitted here.

[0063] Figure 8A(a) is a schematic graph showing the time evolution of the tracking control amount for electronic and optical image stabilization, as well as the total tracking control amount, when transitioning from subject tracking mode to normal image stabilization mode. Here, the vertical axis represents the number of pixels (pix) as the control amount, and the horizontal axis represents the frame number of the image as the change over time. Frames 0 to 10 in Figure 8A(a) are in subject tracking mode, showing how subject tracking is performed so that the total tracking control amount is 150 (pix).

[0064] In the 10th frame of Figure 8A(a), when the setting is changed from subject tracking mode to normal image stabilization mode, control is performed in the 11th frame to return the tracking control amount to 0 (pix). In other words, by returning the position of the optical image stabilization member and the position of the partial image cropping in the electronic image stabilization correction to the central position, the maximum range of motion is ensured in image stabilization from the 11th frame onward.

[0065] In this case, if the mode change involves a change in the cropping range of the partial image due to electronic image stabilization, the screen switches instantaneously before and after the change. In this case, by also instantly controlling the tracking correction amount to return it to 0 (pix), a seamless transition in the appearance of the screen can be achieved. However, as shown in Figure 8A(a), the optical image stabilization component (i.e., the image stabilization lens 102 and the image sensor 106) will operate in a step-response manner, which may generate operating noise and vibration, potentially impairing the user's tactile experience.

[0066] Therefore, in the second embodiment, as shown in Figure 8A(b), the partial image extraction position in the optical image stabilization member and the electronic image stabilization are gradually transitioned in opposite directions over several frames. That is, when the system changes from subject tracking mode to normal image stabilization mode in the 10th frame, the tracking control amount of the optical image stabilization member is gradually returned to 0 (pix) over several frames, while the partial image extraction position in the electronic image stabilization transitions in the opposite direction to that of the optical image stabilization member. Here, although the optical image stabilization member can be driven regardless of the frame period, if the optical image stabilization member is not driven in synchronization with the transition of the partial image extraction position in the electronic image stabilization, unwanted angle of view fluctuations may occur. Therefore, until the transition is complete, the optical image stabilization member is controlled to be driven in synchronization with the same frame update period as the transition of the extraction position by the electronic image stabilization.

[0067] As described above, according to the second embodiment, by canceling out the transition of the optical image stabilization member with electronic image stabilization, the total tracking correction amount does not change, and mode transitions can be performed without changing the field of view. Furthermore, by avoiding the abrupt movement of the optical image stabilization member, the generation of sound and vibration caused by the optical image stabilization member can be suppressed.

[0068] Furthermore, since electronic image stabilization controls the extraction of a portion of the image from the captured image, there is a limit to the amount of control that can be applied. Therefore, it is possible to reach the correction limit during the control shown in Figure 8A(b). Figure 8B(a) is a graph illustrating such an example. The dashed line represents the limit of electronic image stabilization, and the amount of electronic image stabilization during the transition reaches its limit in the 16th frame. On the other hand, optical image stabilization continues to transition down to 0, so the total tracking amount changes from the 16th to the 20th frame, which could appear as a change in the field of view during the mode transition and potentially cause discomfort to the user.

[0069] As described above, if the field of view change becomes unacceptably large when the electronic image stabilization reaches its correction limit, the correction may be performed as follows. That is, in the example of Figure 8B(a), the display control may be performed to suppress the display of the image during the field of view change by blacking out the displayed image in the section from the 16th frame to the 20th frame. Alternatively, as shown in Figure 8B(b), when the electronic image stabilization reaches its limit, the process of transitioning the optical image stabilization member to 0 (pix) may be stopped. This makes it possible to reduce user discomfort caused by field of view changes during state transitions, even when the electronic image stabilization reaches its limit.

[0070] Figure 9 is a flowchart showing the control in the second embodiment. In S201, the electronic image blur correction setting unit 1337 detects whether the subject tracking mode has been changed from the ON state to the OFF state by an operation on the operation unit 114. If no change is detected, the control amount calculation unit 1333 continues the tracking control in S202 and repeats the determination in S201.

[0071] If a change from the ON state to the OFF state of the subject tracking mode is detected in S201, the electronic image blur correction setting unit 1337 determines in S203 whether or not there is a change in the crop size of the partial image due to the mode change. If it is determined that there is no change in the crop size of the field of view, there is no need to instantly cancel the tracking, so in S205 the control amount calculation unit 1333 controls the tracking amount to be gradually returned to 0 over several frames and ends the mode transition.

[0072] If it is determined in S203 that there is a change in the field of view cropping size, then in S204, the control of the variation in the cropping position of the partial image by the optical image blur correction member and the electronic image blur correction is started based on the ratio shown in Figure 8A(b) above. That is, the control amount calculation unit 1333 starts a variation process that gradually transitions the tracking amount by the optical image blur correction to 0 and the tracking amount by the electronic image blur correction in the opposite direction.

[0073] Next, in S206, the control amount calculation unit 1333 determines, based on the current position of the optical image blur correction member and the current cropping position of the partial image in the electronic image blur correction, whether the cropping position will reach the limit of the movable range before the tracking amount of the optical image blur correction member becomes 0. If it is determined that the limit will not be reached, the variation processing continues at the ratio shown in Figure 8A(b), and the variation processing ends in S208.

[0074] On the other hand, if it is determined in S206 that the limit of the movable range has been reached, control is performed to mitigate the field of view fluctuations caused by reaching the limit. The control to mitigate field of view fluctuations may be, for example, as explained in Figure 8B(b), a control amount calculation unit 1333 that stops the transition of the optical image correction member at the same time that the partial image extraction position in the electronic image correction reaches the limit of the movable range. Alternatively, the display screen may be blacked out (display stopped) for the time from when the partial image extraction position in the electronic image correction reaches the limit of the movable range until the transition of the optical image correction member is completed, or the fluctuation processing described in Figure 8A(b) may be used as is if the amount of field of view fluctuation is within the acceptable range.

[0075] As described above, according to the second embodiment, even when transitioning from subject tracking mode, it is possible to reduce the operating noise and vibration that occur when the optical image stabilization member is moved rapidly, while also avoiding unnatural fluctuations in the displayed field of view.

[0076] <Third Embodiment> Next, a third embodiment of the present invention will be described. In the third embodiment, we will describe the challenges and control examples that arise when switching from subject tracking mode to still image shooting mode. Note that the configuration of the imaging system in the third embodiment can be the same as that described in Figures 1 and 2, so we will omit further explanation here.

[0077] Figure 10(a) shows the field of view 1001 of the image sensor 106 tracking the subject 1003 in subject tracking mode, the partial image 1002 extracted by electronic image stabilization, and the positional relationship of the subject 1003 being tracked. As a result of tracking the subject 1003 using both optical and electronic image stabilization, the captured image during tracking, i.e., the partial image 1002 shown by the dashed line, is moved, and the subject 1003 being tracked is kept in the center of the partial image 1002.

[0078] Figure 10(b) shows the field of view when switching from tracking mode, as shown in Figure 10(a), to still image shooting mode. In still image shooting, the entire field of view captured by the image sensor is usually displayed and recorded, so the field of view cropping and enlargement processing by electronic image stabilization, as in video mode, is not reflected. As a result, as shown in Figure 10(b), the subject may be off-center in the captured image, which may result in a composition that goes against the user's intention to keep the subject 1003 in the center.

[0079] Therefore, in this embodiment, when transitioning from subject tracking mode to still image shooting mode, the tracking control amount t is changed as shown in Figure 11. Figure 11 shows how the field of view is corrected when switching from the subject tracking mode shown in Figure 10(a) to the still image shooting mode. Specifically, the tracking control amount in the electronic image stabilization is gradually transitioned to 0 (pix) (center position of the image sensor), and the optical image stabilization member is transitioned in the opposite direction to the tracking control amount in the electronic image stabilization to control the field of view so that it does not change. As a result, even in the field of view when shooting still images where electronic image stabilization is not performed, the position of the subject 1003 can be maintained at the same position as in the subject tracking mode (the center of the cropping range 1102 of the partial image in the example of Figure 11). When still image shooting is instructed in this state, an image of the entire field of view 1001 of the image sensor 106 is recorded.

[0080] Figure 12 shows an example of the transition of the tracking control amount when switching to still image shooting mode. Similar to Figures 8A and 8B, Figure 12 is a schematic graph showing the time change of the tracking control amount for electronic image stabilization and optical image stabilization, as well as the total tracking control amount. From frame 0 to frame 10, the camera is in subject tracking mode, and the combined tracking correction amount of 150 (pix) is performed using both optical and electronic image stabilization.

[0081] In Figure 12, when the mode is changed from subject tracking mode to still image shooting mode in the 10th frame, the tracking control amount in the electronic image stabilization is controlled to gradually transition to 0 (pix), and the optical image stabilization member is controlled to transition in the opposite direction to the tracking control amount in the electronic image stabilization. As shown in Figure 12, the total tracking correction amount does not change before and after the mode change, and the tracking control amount in the electronic image stabilization transitions to 0 (pix), so the subject position can be kept at the same position as during tracking control. Furthermore, by gradually transitioning the position of the optical image stabilization member over several frames, abrupt fluctuations of the optical image stabilization member can be prevented, and the generation of operating noise and vibration can be reduced.

[0082] Here, the control cycle of the optical image stabilization should preferably be the same as the frame update cycle and synchronized with the electronic image stabilization until the transition to still image shooting mode is completed, as described above. However, after the transition to still image shooting mode is completed, it may be changed to a faster cycle, such as the detection cycle of the camera shake detection unit 134, in order to maximize the effect of image stabilization.

[0083] Figure 13 is a flowchart showing the control in the third embodiment. In S301, the control amount calculation unit 1333 obtains from the operation unit 114 whether the mode has been changed from subject tracking mode to still image shooting mode. If no change is detected, the control amount calculation unit 1333 generates a tracking control amount in S302 to continue tracking control, and repeats the determination in S301.

[0084] If a change to still image shooting mode is detected in S301, the control amount calculation unit 1333 performs the tracking control amount transition processing described in Figure 12 in S303. That is, the tracking control amount of the electronic image stabilization is gradually transitioned to 0 (pix), and the optical image stabilization member is moved in the opposite direction to the transition direction of the cropping position due to the tracking control amount of the electronic image stabilization, thereby transitioning to still image shooting mode.

[0085] As described above, according to the third embodiment, when transitioning from subject tracking mode to still image shooting mode, it is possible to reduce the operating noise and vibration generated when moving the optical image stabilization member, while still image shooting can be performed while tracking the subject at the intended angle of view.

[0086] <Other Embodiments> Furthermore, the present invention may be applied to a system consisting of multiple devices or to a device consisting of a single device.

[0087] Furthermore, the present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0088] <Summary> This embodiment includes the following configuration.

[0089] (Item 1) A control device for controlling a first correction means for optically correcting shake, and a second correction means for correcting shake by changing the cropping position of a partial image obtained by imaging with an imaging means, A first acquisition means for acquiring the amount of deflection, which represents the magnitude of the deflection of the object to be detected, A second acquisition means for acquiring a tracking amount to hold a predetermined subject included in the aforementioned image at a predetermined position in the partial image, A third acquisition means that, according to predetermined conditions, acquires a first ratio for allocating the movable range of the first correction means to shake correction and a second ratio for allocating it to tracking control, and a third ratio for allocating the movable range of the partial image in the second correction means to shake correction and a fourth ratio for allocating it to tracking control. A fourth acquisition means for determining the control amount of the first correction means and the control amount of the second correction means for the shake correction and tracking control, based on the shake amount, the tracking amount, and the ratios of the first to fourth means, A control device characterized by having the following features. (Item 2) The system further includes a determination means for determining the state of the vibration based on the amount of vibration, The control device according to item 1, characterized in that the predetermined condition is the state of the runout, and the greater the state of the runout, the higher the first ratio and the lower the second ratio. (Item 3) The predetermined conditions are the image quality at the peripheral image height of the imaging means, The control device according to item 2, characterized in that when the image quality at the peripheral image height decreases, the movable range of the first correction means is narrower than when it does not decrease. (Item 4) The system further includes a fifth acquisition means for acquiring information about the optical system that incident light onto the imaging means, The control device according to item 3, characterized in that the image quality at the peripheral image height decreases when the optical system includes at least one of a lens with large distortion and a lens with a small angle of incidence of light rays to the periphery. (Item 5) The control unit further includes a detection means for detecting whether the image quality priority mode, which prioritizes image quality, has been selected. The control device according to item 2, characterized in that when the image quality priority mode is selected, the movable range of the first correction means is narrower than when the image quality priority mode is not selected. (Item 6) The control unit further includes detection means for detecting whether a first priority mode that prioritizes correction of the amount of shake or a second priority mode that prioritizes tracking the subject has been selected. The first ratio is higher when the first priority mode is selected than when the second priority mode is selected. The second ratio is higher when the second priority mode is selected than when the first priority mode is selected. The control device according to item 1, characterized in that it is a control device. (Item 7) A determination means for determining the state of the runout based on the amount of runout, The system further includes a display control means for displaying the partial image extracted by the second correction means on a display means, The control device according to item 6, characterized in that the display control means controls the display means to display a warning when the state of the oscillation determined by the determination means changes. (Item 8) The system further includes a determination means for determining the state of the vibration based on the amount of vibration, The control device according to item 6, characterized in that, when the second priority mode is selected, the first to fourth ratios acquired by the third acquisition means are maintained even if the state of the swing determined by the determination means changes. (Item 9) It further includes a detection means for detecting whether the tracking mode, which tracks a subject, is ON or OFF as set by the control unit, The control device according to any one of items 1 to 8, characterized in that, when the tracking mode is switched from ON to OFF, the control amount for the second correction means determines a control amount for gradually driving the first correction means in the direction of the position where the tracking amount becomes 0, and a control amount for the second correction means for cutting out the partial image in such a way as to cancel out the change in the field of view caused by driving the first correction means. (Item 10) The control device according to item 9, characterized in that when the cropping position of the partial image in the second correction means reaches the limit of the movable range, the fourth acquisition means determines a control amount for the second correction means to stop the movement of the cropping position, and a control amount for the first correction means to stop driving in the direction of the position where the tracking amount becomes 0. (Item 11) The system further includes a display control means for displaying the partial image extracted by the second correction means on a display means, The control device according to item 9, characterized in that when the cropping position of the partial image in the second correction means reaches the limit of the movable range, the display control means stops displaying the partial image until the first correction means is driven to a position where the tracking amount becomes 0. (Item 12) It further includes a detection means for detecting the mode set by the operating unit, The control device according to any one of items 1 to 8, characterized in that when the system switches from a tracking mode for tracking the subject to a still image shooting mode for taking a still image, the fourth acquisition means determines a control amount for the second correction means which gradually drives the cropping position of the partial image in a direction where the tracking amount becomes 0, and a control amount for driving the first correction means to cancel out the change in the field of view caused by the second correction means. (Item 13) The first correction means is at least one of the following: a correction means that corrects image blur by driving a correction lens included in the imaging optical system in a direction perpendicular to the optical axis; and a correction means that corrects image blur by driving an image sensor that converts light incident through the imaging optical system into an image signal in a direction perpendicular to the optical axis. The fourth acquisition means acquires, as a control amount for the first correction means, a control amount that drives at least one of the correction lens and the image sensor in a direction perpendicular to the optical axis of the imaging optical system. A control device according to any one of items 1 to 12, characterized in that it is a control device. (Item 14) The control device according to any one of items 1 to 13, characterized in that the first acquisition means acquires at least one of the amount of shake detected by the shake detection means and the motion vector detected from the image output from the imaging means. (Item 15) A control method for a first correction means for optically correcting shake and a second correction means for correcting shake by changing the cropping position of a partial image extracted from an image captured by an imaging means, A first acquisition step involves obtaining the amount of deflection, which represents the magnitude of the deflection of the object to be detected. A second acquisition step involves acquiring a tracking amount to hold a predetermined subject included in the aforementioned image at a predetermined position in the aforementioned partial image, A third acquisition step is to acquire, according to predetermined conditions, a first ratio for allocating the movable range of the first correction means to shake correction and a second ratio for allocating it to tracking control, and a third ratio for allocating the movable range of the partial image extraction position by the second correction means to shake correction and a fourth ratio for allocating it to tracking control. A fourth acquisition step in which, based on the amount of vibration, the tracking amount, and the ratios of the first to fourth, the control amount of the first correction means and the control amount of the second correction means for vibration correction and tracking control are determined, A control method characterized by having the following features. (Item 16) A control device described in any one of items 1 to 14, The first correction means and, The second correction means and Based on the control amount of the first correction means acquired by the fourth acquisition means, a first driving means drives the first correction means, Based on the control amount of the second correction means acquired by the fourth acquisition means, the second driving means drives the second correction means and An image blur correction device characterized by having the following features. (Item 17) The image blur correction device described in item 16, The imaging means and An imaging device characterized by having the following features. (Item 18) A program for causing a computer to function as one of the control devices described in any one of items 1 through 14. (Item 19) A computer-readable storage medium containing the program described in item 18.

[0090] The invention is not limited to the embodiments described above, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, claims are attached to disclose the scope of the invention. [Explanation of Symbols]

[0091] 102...Image stabilization lens, 106...Image sensor, 109...Image processing circuit, 110...Internal memory, 111...Display unit, 113...Storage memory, 114...Operation unit, 115...Camera control unit, 200...Shooting optical system, 122...Image stabilization lens drive unit, 123...Image stabilization lens position detection unit, 125...Lens side shake detection unit, 126...Image stabilization lens vibration damping control unit, 130...Image sensor drive unit, 131...Motion Vector detection unit, 133... Image blur correction control unit, 134... Camera side shake detection unit, 141... Subject detection unit, 142... Subject tracking calculation unit, 143... Subject identification unit, 1331... Integration unit, 1332... Shake correction amount calculation unit, 1333... Control amount calculation unit, 1334... Shooting status acquisition unit, 1335... Position control unit, 1337... Electronic image blur correction setting unit, 1423... Tracking control amount calculation unit, 1424... Subject target position setting unit

Claims

1. A control device for controlling a first correction means for optically correcting shake, and a second correction means for correcting shake by changing the position of the partial image extracted from an image captured by an imaging means, A first acquisition means for acquiring the amount of deflection, which represents the magnitude of the deflection of the object to be detected, A second acquisition means for acquiring a tracking amount to hold a predetermined subject included in the aforementioned image at a predetermined position in the partial image, A third acquisition means that, according to predetermined conditions, acquires a first ratio for allocating the movable range of the first correction means to shake correction and a second ratio for allocating it to tracking control, and a third ratio for allocating the movable range of the partial image in the second correction means to shake correction and a fourth ratio for allocating it to tracking control. A fourth acquisition means for determining the control amount of the first correction means and the control amount of the second correction means for the shake correction and tracking control, based on the shake amount, the tracking amount, and the first to fourth ratios, A control device characterized by having the following features.

2. The system further includes a determination means for determining the state of the vibration based on the amount of vibration, The control device according to claim 1, characterized in that the predetermined condition is the state of the runout, and the greater the state of the runout, the higher the first ratio and the lower the second ratio.

3. The predetermined conditions are the image quality at the peripheral image height of the imaging means, The control device according to claim 2, characterized in that when the image quality at the peripheral image height decreases, the movable range of the first correction means is narrower than when it does not decrease.

4. The system further includes a fifth acquisition means for acquiring information about the optical system that incident light onto the imaging means, The control device according to claim 3, characterized in that the image quality at the peripheral image height decreases when the optical system includes at least one of a lens with large distortion and a lens with a small angle of incidence of light rays to the periphery.

5. The control unit further includes a detection means for detecting whether the image quality priority mode, which prioritizes image quality, has been selected. The control device according to claim 2, characterized in that when the image quality priority mode is selected, the movable range of the first correction means is narrower than when the image quality priority mode is not selected.

6. The control unit further includes detection means for detecting whether a first priority mode that prioritizes correction of the amount of shake or a second priority mode that prioritizes tracking the subject has been selected. The first ratio is higher when the first priority mode is selected than when the second priority mode is selected. The second ratio is higher when the second priority mode is selected than when the first priority mode is selected. The control device according to feature 1.

7. A determination means for determining the state of the runout based on the amount of runout, The system further includes a display control means for displaying the partial image extracted by the second correction means on a display means, The control device according to claim 6, characterized in that the display control means controls the display means to display a warning when the state of the oscillation determined by the determination means changes.

8. The system further includes a determination means for determining the state of the vibration based on the amount of vibration, The control device according to claim 6, characterized in that, when the second priority mode is selected, the first to fourth ratios acquired by the third acquisition means are maintained even if the state of the swing determined by the determination means changes.

9. It further includes a detection means for detecting whether the tracking mode, which tracks a subject, is ON or OFF as set by the control unit. The control device according to claim 1, characterized in that the fourth acquisition means determines, when the tracking mode is switched from ON to OFF, a control amount for gradually driving the first correction means in the direction of the position where the tracking amount becomes 0, and a control amount for the second correction means for cutting out the partial image in such a way as to cancel out the change in the field of view due to the driving of the first correction means.

10. The control device according to claim 9, characterized in that when the cropping position of the partial image in the second correction means reaches the limit of the movable range, the fourth acquisition means determines a control amount for the second correction means to stop the movement of the cropping position, and a control amount for the first correction means to stop driving in the direction of the position where the tracking amount becomes zero.

11. The system further includes a display control means for displaying the partial image extracted by the second correction means on a display means, The control device according to claim 9, characterized in that when the cropping position of the partial image in the second correction means reaches the limit of the movable range, the display control means stops displaying the partial image until the first correction means is driven to a position where the tracking amount becomes 0.

12. It further includes a detection means for detecting the mode set by the operating unit, The control device according to claim 1, characterized in that when the system switches from a tracking mode for tracking the subject to a still image shooting mode for taking a still image, the fourth acquisition means determines a control amount for the second correction means which gradually drives the cropping position of the partial image in a direction where the tracking amount becomes zero, and a control amount for driving the first correction means to cancel out the change in the field of view caused by the second correction means.

13. The first correction means is at least one of the following: a correction means that corrects image blur by driving a correction lens included in the imaging optical system in a direction perpendicular to the optical axis; and a correction means that corrects image blur by driving an image sensor that converts light incident through the imaging optical system into an image signal in a direction perpendicular to the optical axis. The fourth acquisition means acquires a control amount for driving at least one of the correction lens and the image sensor in a direction perpendicular to the optical axis of the imaging optical system, as a control amount for the first correction means. The control device according to feature 1.

14. The control device according to claim 1, characterized in that the first acquisition means acquires at least one of the amount of shake detected by the shake detection means and the motion vector detected from the image output from the imaging means.

15. A control method for a first correction means for optically correcting shake and a second correction means for correcting shake by changing the cropping position of a partial image obtained by imaging with an imaging means, A first acquisition step involves obtaining a deflection amount that represents the magnitude of the deflection of the object to be detected, A second acquisition step involves acquiring a tracking amount to hold a predetermined subject included in the aforementioned image at a predetermined position in the partial image, A third acquisition step is to acquire, according to predetermined conditions, a first ratio for allocating the movable range of the first correction means to shake correction and a second ratio for allocating it to tracking control, and a third ratio for allocating the movable range of the partial image extraction position by the second correction means to shake correction and a fourth ratio for allocating it to tracking control. A fourth acquisition step in which, based on the amount of vibration, the tracking amount, and the ratios of the first to fourth, the control amount of the first correction means and the control amount of the second correction means for vibration correction and tracking control are determined, A control method characterized by having the following features.

16. A control device according to any one of claims 1 to 14, The first correction means and, The second correction means and Based on the control amount of the first correction means acquired by the fourth acquisition means, a first driving means drives the first correction means, Based on the control amount of the second correction means acquired by the fourth acquisition means, the second driving means drives the second correction means. An image blur correction device characterized by having the following features.

17. The image blur correction device according to claim 16, The imaging means and An imaging device characterized by having the following features.

18. A program for causing a computer to function as one of the means of the control device described in any one of claims 1 to 14.

19. A computer-readable storage medium storing the program described in claim 18.

Images