Control device and method, image stabilizing device, image pickup device, computer program product, and storage medium

Abstract

The present application relates to a control device and method, an image stabilizing device, an image pickup device, a computer program product, and a storage medium. A control device for controlling an optical image stabilizing section and an electronic image stabilizing section includes: a first acquisition section configured to acquire a shake amount representing a size of a shake; a second acquisition section configured to acquire a tracking amount; a third acquisition section configured to acquire a first ratio for allocating a movable region of the optical image stabilizing section to image stabilization and a second ratio for allocating the same to tracking control, and a third ratio for allocating a movable region of a partial image of the electronic image stabilizing section to image stabilization and a fourth ratio for allocating the same to tracking control; and a fourth acquisition section configured to acquire a control amount of the first and second image stabilizing sections for image stabilization and tracking control, based on the shake amount, the tracking amount, and the first to fourth ratios.

Description

Technical Field

[0001] This disclosure relates to control devices and methods, image stabilization devices, camera devices, programs and storage media, and more particularly to techniques used for image stabilization and subject tracking using image stabilization functions. Background Technology

[0002] Traditionally, there exist camera devices with image stabilization capabilities to stabilize camera shake when capturing moving images. Such devices use, for example, angular velocity sensors to detect camera shake, then actuate corrective lenses or image sensors to counteract the detected camera shake, or perform image processing to cut out small areas relative to the captured area and geometrically distort the resulting image. Typically, the former is called optical image stabilization, and the latter is called electronic image stabilization.

[0003] Subject tracking technology is also known, which detects a subject from a captured image, uses image stabilization to track the detected subject, and keeps the subject at a predetermined position within the field of view.

[0004] Japanese Patent Application Publication No. 2017-215350 discloses a technique for determining whether to prioritize image stabilization or subject tracking when performing image stabilization and subject tracking functions, so as to achieve both image stabilization performance and subject tracking performance using a corrective lens.

[0005] However, there are limitations to improving subject tracking performance when using only optical or electronic image stabilization. For example, if only electronic image stabilization is used, problems such as the need to increase the image sensor size to improve subject tracking performance, and the difficulty in improving subject tracking performance due to the decreasing correction angle as the focal length approaches the telephoto side, are known. On the other hand, if only optical image stabilization is used, the correction angle cannot be improved unless the movable area of ​​the components used for optical image stabilization (correction lens and / or image sensor) is expanded. Furthermore, depending on how these components move, vibration and operational noise may occur, which could impair the quality when using optical image stabilization. Summary of the Invention

[0006] This disclosure has been made with the above considerations in mind, and in subject tracking using both optical and electronic image stabilization, the tracking range is expanded while maintaining the quality when using image stabilization.

[0007] According to this disclosure, a control device is provided for controlling a first image stabilization component and a second image stabilization component. The first image stabilization component is configured to perform optical image stabilization, and the second image stabilization component is configured to perform image stabilization by changing the cut-out position of a portion of an image captured from an image sensor. The control device includes: a first acquisition component configured to acquire a jitter amount representing the magnitude of jitter in a detected object; a second acquisition component configured to acquire a tracking amount for holding a predetermined subject included in the image at a predetermined position in the portion of the image; and a third acquisition component configured to acquire, according to predetermined conditions... A first ratio for allocating the movable region of the first image stabilization component to image stabilization, a second ratio for allocating the movable region of the first image stabilization component to tracking control, a third ratio for allocating the movable region of the portion of the image of the second image stabilization component to the image stabilization, and a fourth ratio for allocating the movable region of the portion of the image to the tracking control; and a fourth acquisition component configured to acquire, based on the jitter amount, the tracking amount, and the first to the fourth ratios, control amounts of the first image stabilization component and the second image stabilization component for the image stabilization and the tracking control.

[0008] Furthermore, according to this disclosure, a control device is provided for controlling a first image stabilization component and a second image stabilization component, the first image stabilization component being configured to perform optical image stabilization, and the second image stabilization component being configured to perform image stabilization by changing the cut-out position of a portion of an image captured from an image sensor. The control device includes: a first acquisition component configured to acquire a jitter amount representing the magnitude of jitter in a detected object; a second acquisition component configured to acquire a tracking amount for holding a predetermined subject included in the image at a predetermined position in the portion of the image; a third acquisition component configured to acquire a first value indicating the degree to which the first image stabilization component is assigned to image stabilization, a second value indicating the degree to which the first image stabilization component is assigned to tracking control, a third value indicating the degree to which the second image stabilization component is assigned to the image stabilization, and a fourth value indicating the degree to which the second image stabilization component is assigned to the tracking control; and a fourth acquisition component configured to acquire control amounts of the first image stabilization component and the second image stabilization component for image stabilization and tracking control based on the jitter amount, the tracking amount, and the first to fourth values.

[0009] Furthermore, according to this disclosure, an image stabilization device is provided, comprising: a control device as described above; a first image stabilization component; a second image stabilization component; a first actuation component configured to actuate the first image stabilization component based on a control quantity of the first image stabilization component acquired by the fourth acquisition component; and a second actuation component configured to actuate the second image stabilization component based on a control quantity of the second image stabilization component acquired by the fourth acquisition component.

[0010] Furthermore, according to this disclosure, a camera device is provided, comprising: an image stabilization device as described above; and the image sensor.

[0011] Furthermore, according to this disclosure, a control method is provided for controlling a first image stabilization component and a second image stabilization component, the first image stabilization component being configured to perform optical image stabilization, and the second image stabilization component being configured to perform image stabilization by changing the cut-out position of a portion of an image captured from an image sensor. The control method includes: a first acquisition step for acquiring a jitter amount representing the magnitude of jitter in a detected object; a second acquisition step for acquiring a tracking amount for holding a predetermined subject included in the image at a predetermined position in the portion of the image; a third acquisition step for acquiring, according to predetermined conditions, a first ratio for allocating a movable region of the first image stabilization component to image stabilization, a second ratio for allocating a movable region of the first image stabilization component to tracking control, a third ratio for allocating a movable region of the portion of the image of the second image stabilization component to the image stabilization, and a fourth ratio for allocating a movable region of the portion of the image to the tracking control; and a fourth acquisition step for acquiring control amounts of the first image stabilization component and the second image stabilization component for image stabilization and tracking control based on the jitter amount, the tracking amount, and the first to fourth ratios.

[0012] Furthermore, according to this disclosure, a computer program product is provided, which includes a program for causing a computer to perform the control method described above.

[0013] Furthermore, according to this disclosure, a computer-readable storage medium is provided that stores a program for causing a computer to perform the control method described above.

[0014] The features of this disclosure will become apparent from the following description of embodiments with reference to the accompanying drawings. The following description of the embodiments is given by way of example. Attached Figure Description

[0015] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the present disclosure and, together with the specification, serve to explain the principles of the embodiments.

[0016] Figure 1 This is a block diagram illustrating the configuration of a camera system according to an embodiment of the present disclosure.

[0017] Figure 2 This is a block diagram illustrating a configuration for image stabilization control and subject tracking control according to an embodiment.

[0018] Figure 3 This is a diagram illustrating a ratio table according to the first embodiment.

[0019] Figure 4 This is a flowchart illustrating image stabilization control and subject tracking control according to the first embodiment.

[0020] Figure 5 This is a diagram illustrating another ratio table according to the first embodiment.

[0021] Figure 6 This is a diagram illustrating yet another ratio table according to the first embodiment.

[0022] Figure 7 This is a diagram illustrating yet another ratio table according to the first embodiment.

[0023] Figure 8AA and Figure 8AB This is a graph illustrating an example of the change in the tracking control quantity according to the second embodiment.

[0024] Figure 8BA and Figure 8BB This is a graph illustrating other examples of changes in the tracking control quantity according to the second embodiment.

[0025] Figure 9 This is a flowchart illustrating the control according to the second embodiment.

[0026] Figure 10A and Figure 10B This is an explanatory diagram based on the third embodiment, used to illustrate the problem to be solved from a certain perspective.

[0027] Figure 11 This is an explanatory diagram showing the perspective under control according to the third embodiment.

[0028] Figure 12 This is a graph illustrating an example of the change in the tracking control quantity according to the third embodiment.

[0029] Figure 13 This is a flowchart illustrating subject tracking control according to the third embodiment. Detailed Implementation

[0030] In the following, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments are not intended to limit the scope of the claims. Several features are described in the embodiments, but not all of them are required, and several such features can be appropriately combined. Furthermore, in the drawings, the same reference numerals are given the same or similar configuration, and redundant descriptions are omitted.

[0031] <First Embodiment> Figure 1 This is a block diagram illustrating the configuration of a camera system according to an embodiment of the present disclosure. The camera system in this embodiment mainly includes a camera body 1 and a lens unit 2 detachable from the camera body 1. When the lens unit 2 is attached to the camera body 1, information is transmitted via a camera-side communication unit 140 and a lens-side communication unit 128.

[0032] First, the configuration of lens unit 2 will be explained.

[0033] The camera optical system 200 includes a zoom lens 101, an image stabilizing lens 102, a focusing lens 103, and an aperture 104. A zoom lens actuator 124 actuates the zoom lens 101 along the optical axis to optically change the focal length of the camera optical system 200, thereby changing the angle of view. Additionally, a zoom lens controller 127 controls the position of the zoom lens 101 by controlling the zoom lens actuator 124 according to zoom commands input by a photographer operating the operation unit 114. A focusing lens actuator 121 adjusts the focus position by moving the focusing lens 103 along the optical axis. An aperture actuator 120 adjusts the amount of incident light by controlling the aperture diameter of the aperture 104.

[0034] Image stabilizing lens position detection unit 123 detects the position of image stabilizing lens 102. Lens-side shake detection unit 125 detects shake or vibration applied to lens unit 2 (the detection object) and outputs a shake detection signal. Lens-side shake detection unit 125 may be a device for detecting angular velocity, such as a gyroscope sensor, or a device for detecting acceleration, such as an accelerometer sensor. Image stabilizing lens control unit 126 calculates an image stabilization amount to suppress shake based on the shake detection signal output from lens-side shake detection unit 125, camera-side shake detection unit 134, or both, and the current position of image stabilizing lens 102 detected by image stabilizing lens position detection unit 123, and notifies image stabilizing lens actuator 122 of the calculated image stabilization amount. Image stabilizing lens actuator 122 optically corrects image blur caused by camera system shake by moving image stabilizing lens 102 in a direction perpendicular to the optical axis.

[0035] Next, the configuration of the camera body 1 will be explained.

[0036] Light passing through the camera optical system 200 is received by the image sensor 106 (which may be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) sensor, etc.) via the shutter 105, and undergoes photoelectric conversion from light signal to electrical signal. The shutter 105 is actuated by the shutter actuator 135.

[0037] The AD converter 107 performs noise reduction, gain adjustment, and AD conversion on the analog image signal output from the image sensor 106, and outputs a digital image signal.

[0038] The timing generator 108 controls the actuation timing of the image sensor 106 and the processing timing of the AD converter 107 according to the instructions from the camera control unit 115.

[0039] The image processing circuit 109 performs pixel interpolation, color conversion, and other processing on the digital image signal output from the AD converter 107, and then sends the processed image data to the internal memory 110. The image processing circuit 109 includes circuits for aligning multiple images captured in succession, geometric transformation circuits for performing cylindrical coordinate transformation and distortion correction corresponding to the lens group, and synthesis circuits for performing cropping and compositing processing.

[0040] The image processing circuit 109 also performs electronic image stabilization and subject tracking by performing image processing to cut out a portion of the captured image area, and controlling the cut-out position based on the direction and size of the image blur when cutting out the portion of the image. In this embodiment, the cut-out position is controlled based on a control quantity for the cut-out position in the image calculated by the control quantity calculation unit 1333 (described later). Note that the projective transformation circuit provided in the image processing circuit 109 is used for electronic image stabilization in this embodiment.

[0041] The display unit 111 displays the shooting information and other data together with the image data stored in the internal memory 110.

[0042] The compression / decompression processing unit 112 compresses or decompresses the image data stored in the internal memory 110 according to the image format.

[0043] Storage memory 113 stores various data such as parameters.

[0044] The operation unit 114 is a user interface that enables users to issue various commands to the camera system (such as various menu operations and mode switching operations).

[0045] The camera control unit 115 consists of an arithmetic unit such as a central processing unit (CPU), and executes various control programs stored in the internal memory 110 in response to user operation via the operation unit 114. These control programs are used for functions such as zoom control, image stabilization control, automatic exposure control, automatic focus adjustment control, face detection of the subject, and subject tracking control.

[0046] The luminance signal detection unit 137 detects the brightness of the subject and the scene based on the image signal read from the image sensor 106 and output from the AD converter 107. The exposure control unit 136 calculates the exposure value (aperture value and shutter speed) based on the brightness information obtained by the luminance signal detection unit 137, and notifies the calculation result to the shutter actuator 135 and the aperture actuator 120 via the camera-side communication unit 140 and the lens-side communication unit 128. The exposure control unit 136 also simultaneously performs gain control in the image sensor 106 to amplify the signal to be read, thereby performing automatic exposure control (AE control).

[0047] The evaluation value calculation unit 138 extracts specific frequency components from the brightness information obtained by the brightness signal detection unit 137, and calculates the contrast evaluation value based on the extraction result.

[0048] The focusing lens control unit 139 issues a command to actuate the focusing lens 103 by a predetermined actuation amount within a predetermined range. Simultaneously, a contrast evaluation value is acquired, which is the result of a calculation by the evaluation value calculation unit 138 based on the brightness information of the image signal obtained at each focusing lens position. The amount of defocus is calculated based on the focusing lens position where the curve of the obtained contrast evaluation value reaches its peak, and this amount of defocus is communicated to the focusing lens actuator 121 via the camera-side communication unit 140 and the lens-side communication unit 128. The focusing lens actuator 121 then actuates the focusing lens 103 according to the amount of defocus, thereby performing contrast-based autofocus control (AF control) so that the light beam converges onto the surface of the image sensor 106.

[0049] Although a contrast-based focus adjustment control method has been described here, a phase-difference-based focus adjustment control method can also be used. Since phase-difference-based focus adjustment control is well-known, its description will be omitted.

[0050] The camera-side shake detection unit 134 detects vibrations and shakes applied to the camera body 1 (the object being detected). As the camera-side shake detection unit 134, a device for detecting angular velocity, such as a gyroscope sensor, or a device for detecting acceleration, such as an accelerometer sensor, can be used. However, in this embodiment, it is assumed that the camera-side shake detection unit 134 detects angular velocity and outputs an angular velocity signal.

[0051] The image stabilization control unit 133 can communicate with the image stabilization lens control unit 126 via the camera-side communication unit 140 and the lens-side communication unit 128. The image stabilization control unit 133 calculates an image stabilization amount for suppressing image blur using the image sensor 106 based on shake detection signals detected by the camera-side shake detection unit 134, the lens-side shake detection unit 125, or both. Then, based on the calculated image stabilization amount and the current position of the image sensor 106 detected by the image sensor position detection unit 132, the image stabilization control unit 133 transmits an actuation signal to the image sensor actuation unit 130 to actuate the image sensor 106, thereby controlling image stabilization using the image sensor 106. Based on the actuation signal received from the image stabilization control unit 133, the image sensor actuation unit 130 actuates the image sensor 106 in a direction perpendicular to the optical axis.

[0052] The motion vector detection unit 131 uses a block matching method to calculate the correlation value between the current frame image and the previous frame image for each block obtained by segmenting each frame, then searches for the block of the previous frame with the minimum correlation value, and detects the deviation of other blocks relative to that block as motion vectors.

[0053] The subject designation unit 143 sets one of the subjects in the captured image as the tracking subject. The photographer can set any subject by touching or operating the buttons via the operation unit 114. Furthermore, if the photographer does not set a subject, the tracking subject can be determined through an automatic subject setting procedure.

[0054] The subject detection unit 141 detects the area of ​​the tracked subject set by the subject designation unit 143 and generates subject detection information. The subject detection information includes information such as the type of subject (e.g., person, animal, vehicle), body parts (e.g., pupil, face, body), position, and size.

[0055] The subject tracking calculation unit 142 calculates the tracking amount for tracking the subject based on the subject detection information. The following will use... Figure 2 Let me explain the details.

[0056] Figure 2This is a block diagram illustrating the configuration for image stabilization control and subject tracking control in the first embodiment, and specifically shows the detailed configuration of the image stabilization control unit 133 and the subject tracking calculation unit 142. In this embodiment, image stabilization can be performed by optical image stabilization and electronic image stabilization. Furthermore, optical image stabilization includes a method for actuating the image stabilizing lens 102 and a method for actuating the image sensor 106. In the following description, optical image stabilization is described as being performed by actuating the image sensor 106, but optical image stabilization can also be performed by actuating the image stabilizing lens 102 alone or by actuating both the image stabilizing lens 102 and the image sensor 106.

[0057] Integrator 1331 integrates the angular velocity signal output from camera-side shake detection unit 134 and converts it into a shake angle. For example, an integrating low-pass filter (LPF) can be used for integration processing. Note that if camera-side shake detection unit 134 is an accelerometer, integrator 1331 performs second-order integration to convert the output acceleration signal into a shake amount corresponding to the shake angle.

[0058] The image stabilization calculation unit 1332 calculates the image stabilization amount used to counteract the shake angle by considering the frequency band of the shake angle obtained by the integrator 1331 and the range (movable area) that can actuate the image sensor 106. Specifically, the image stabilization amount is calculated by multiplying the shake angle by a gain related to the zoom ratio and the distance to the subject.

[0059] The shooting status acquisition unit 1334 acquires shooting status information, including vibration states such as whether the camera is in a stationary state mounted on a tripod or the like, and whether the vibration caused by hand shaking is large or small, based on the shaking angle obtained by the integrator 1331.

[0060] The electronic image stabilization setting unit 1337 receives user settings related to electronic image stabilization from the operation unit 114. Here, users can select whether to perform electronic image stabilization, select the intensity of the correction effect from multiple correction modes, select whether to perform subject tracking, and determine the angle cut out from the camera area based on these settings.

[0061] The subject target position setting unit 1424 calculates and sets the subject target position, which is the position on the screen where the subject to be tracked will be captured during subject tracking control. The subject target position is changeable, and possible setting values ​​include the center of the viewpoint, the position on the screen touched by the photographer, and the position corresponding to pre-stored coordinates.

[0062] The tracking control quantity calculation unit 1423 calculates the tracking control quantity based on the subject target position set by the subject target position setting unit 1424 and the position of the subject included in the subject detection information from the subject detection unit 141.

[0063] The control quantity calculation unit 1333 calculates the control quantity of the optical image stabilizing component (here, the image sensor 106) and the control quantity of the cutout position of the partial area used for electronic image stabilization based on the image stabilization quantity calculated by the image stabilization quantity calculation unit 1332, the tracking control quantity calculated by the tracking control quantity calculation unit 1423, the shooting status acquired by the shooting status acquisition unit 1334, and the electronic image stabilization related settings set by the electronic image stabilization setting unit 1337. In this embodiment, the control quantity calculation unit 1333 uses a ratio table, which will be described later, to calculate the control quantity. The calculated control quantity of the optical image stabilizing component is output to the position control unit 1335, and the control quantity of the cutout position of the partial area is output to the image processing circuit 109.

[0064] The position control unit 1335 performs PID control (ratio-integral-derivative control) on the deviation between the target position of the image sensor 106 based on the control quantity of the optical image stabilization component calculated by the control quantity calculation unit 1333 and the current position detected by the image sensor position detection unit 132. Then, the position control unit 1335 converts this deviation into an actuation signal for the image sensor 106 and inputs this actuation signal to the image sensor actuation unit 130, thereby controlling the position of the image sensor 106 to be at the target position, thus achieving optical image stabilization and subject tracking. Note that PID control is a commonly used technique, therefore its detailed description will be omitted.

[0065] Next, refer to Figure 3 Explain the calculation method of the control quantity in the control quantity calculation unit 1333.

[0066] Figure 3 An example ratio table is shown, which represents the ratio of the movable area of ​​the portion to be cut out in electronic image stabilization and the movable area of ​​the optical image stabilization component allocated to image stabilization and subject tracking. Figure 3 The left column shows the vibration state (condition) acquired by the shooting status acquisition unit 1334. In this embodiment, the detected shaking is classified into four levels: "tripper / universal joint", "small vibration", "medium vibration" and "large vibration". Figure 3The values ​​in the table also represent the image stabilization / tracking ratio allocated to the movable areas of image stabilization and subject tracking, respectively, when the movable areas of the portion to be cut out in electronic image stabilization and the movable areas of the optical image stabilization component are both set to 100%. In this embodiment, the movable area is set according to the magnitude of the jitter.

[0067] Notice, Figure 3 The values ​​in the table indicate the degree of image stabilization capability controlled by each image stabilization measure, and do not imply, for example, that 30% image stabilization corrects for 30% of camera shake. Furthermore, in electronic image stabilization, the higher the stabilization capability, the smaller the size of the cut-out region. Therefore, the size of the cut-out region is limited to prevent it from becoming too small. For example, 100% electronic image stabilization indicates the image stabilization capability when a minimum region within a limited size is cut out.

[0068] For example, if the camera's shooting condition is determined to be "tripod / gimbal", then... Figure 3 As shown in the table, 0% of both electronic and optical image stabilization are allocated to image stabilization, and 100% is allocated to subject tracking. This indicates that camera shake will be very small and image stabilization control will not be necessary, so no image stabilization is performed and the entire movable area is allocated to subject tracking, and this control state maximizes subject tracking performance.

[0069] Furthermore, if the camera's shooting condition is judged as "minor vibration," then 0% of electronic image stabilization is assigned to image stabilization and 100% to subject tracking, while 30% of optical image stabilization is assigned to image stabilization and 70% to subject tracking. This indicates that the camera is judged to be under slight vibration, therefore 30% of the movable area of ​​optical image stabilization is assigned to image stabilization.

[0070] Similarly, in this embodiment, when the camera vibration condition is judged as "moderate vibration" and "large vibration", the allocation ratio of optical image stabilization to image stabilization increases, and under the "large vibration" condition, the entire movable area (100%) of optical image stabilization is allocated to image stabilization.

[0071] In this way, by allocating a larger proportion of the movable area to subject tracking when vibration is low, and to image stabilization as vibration increases, a good balance can be achieved between subject tracking control and image stabilization control based on the vibration state.

[0072] Next, we will use Figure 4 The flowchart shown illustrates the image stabilization control and subject tracking control in this embodiment.

[0073] In step S101, the camera control unit 115 acquires the shooting status (vibration status) from the shooting status acquisition unit 1334.

[0074] In step S102, if the vibration state indication obtained in step S101 is "tripod / universal joint", then the process proceeds to step S103, where according to Figure 3 The ratio table shown assigns 0% of the movable region for electronic image stabilization to image stabilization and 100% to subject tracking. Similarly, 0% of the movable region for optical image stabilization is assigned to image stabilization and 100% to subject tracking, and the process proceeds to step S109.

[0075] If the vibration state indicates "small vibration" in step S104, the process proceeds to step S105, where according to... Figure 3 The ratio table shown indicates that 0% of the movable region for electronic image stabilization is allocated to image stabilization and 100% is allocated to subject tracking. Furthermore, 30% of the movable region for optical image stabilization is allocated to image stabilization and 70% is allocated to subject tracking, and the process proceeds to step S109.

[0076] If the vibration state indicates "moderate vibration" in step S106, the process proceeds to step S107, where according to... Figure 3 The ratio table shown indicates that 0% of the movable area for electronic image stabilization is allocated to image stabilization and 100% is allocated to subject tracking. Furthermore, 50% of the movable area for optical image stabilization is allocated to image stabilization and 50% is allocated to subject tracking, and the process proceeds to step S109.

[0077] If the vibration state in step S106 does not indicate "moderate vibration" (i.e., "large vibration"), the process proceeds to step S108, where according to Figure 3 The ratio table shown assigns 0% of the movable region for electronic image stabilization to image stabilization and 100% to subject tracking. Similarly, 100% of the movable region for optical image stabilization is assigned to image stabilization and 0% to subject tracking, and the process proceeds to step S109.

[0078] In step S109, within a movable area based on the ratio set in any of steps S103, S105, S107, and S108, image stabilization and subject tracking processing are performed using control quantities calculated by the control quantity calculation unit 1333. Specifically, image stabilization and subject tracking are performed by controlling the actuation of the image sensor 106 and the cutout position of a portion of the area based on the control quantities calculated by the control quantity calculation unit 1333. Note that when the image stabilizing lens 102 is also used, [the image stabilization lens is used to...]. Figure 3 The ratio table shown adds the ratio of the movable area of ​​the image stabilizing lens 102, and the control quantity of the optical image stabilizing member calculated by the control quantity calculation unit 1333 is assigned to the image sensor 106 and the image stabilizing lens 102. Furthermore, if only the image stabilizing lens 102 is used for optical image stabilization, the image stabilizing lens 102 can be controlled and actuated instead of the image sensor 106 based on the control quantity of the optical image stabilizing member calculated by the control quantity calculation unit 1333.

[0079] It should be noted that, as mentioned above Figure 3 The ratio table shown is an example and can be modified as appropriate. Other examples of ratio tables will be described below.

[0080] (Ratio table based on user intent) Figure 3 The ratio table shown is described as determining the movable area based on the vibration state determined by the shooting status acquisition unit 1334 based on the shake angle. However, the movable area can also be determined based on the mode selected by the user via the operation unit 114.

[0081] Figure 5 This example shows a ratio table with the user able to select "Priority Tracking" or "Priority Image Stabilization" (condition) via menu settings. Figure 5 As shown, when "Priority Tracking" is selected, 0% of both electronic and optical image stabilization are allocated to image stabilization, and 100% of both are allocated to subject tracking. Conversely, when "Priority Image Stabilization" is selected, 100% of electronic image stabilization is allocated to subject tracking, 0% is allocated to image stabilization, 100% is allocated to image stabilization, and 0% is allocated to subject tracking. This allows the ratio table to be set based on the user's settings regardless of the shooting conditions.

[0082] (Ratio table of limited stable regions in optical images) exist Figure 3In the ratio table shown, the movable area of ​​optical image stabilization is allocated to image stabilization and subject tracking, resulting in a total percentage of 100%. However, for lenses with significant distortion or lenses where light incident at the periphery occurs at a small angle, there are concerns related to deterioration in image stabilization performance at the periphery, image quality degradation due to decreased edge illumination, and deterioration in autofocus performance. For example, if optical image stabilization is used to continuously track a subject located at the periphery over an extended period, the image quality degradation caused by the aforementioned concerns may become significant. Therefore, in situations (conditions) where the above concerns exist, the movable area of ​​optical image stabilization may be limited.

[0083] Figure 6 An example of a ratio table is shown in cases where high limits are imposed. (Compared to...) Figure 3 Compared to the ratio table shown, the total percentage to be allocated to image stabilization and subject tracking is set to 75% instead of 100% to limit the movable area of ​​optical image stabilization, thereby preventing the degradation of each performance quality as described above.

[0084] (Ratio table of priority image quality) exist Figure 3 In the ratio table described, the movable region of electronic image stabilization is allocated to image stabilization and subject tracking, resulting in a total allocation percentage of 100%. However, in electronic image stabilization where an image corresponding to a portion of the captured image (hereinafter referred to as a partial image) is cut out, the smaller the size of the cut partial image, the narrower the viewing angle, leading to image quality degradation due to subsequent magnification processing of the partial image. Therefore, an "image quality priority mode" (condition) can be used, and when the user selects this mode, the movable region of electronic image stabilization can be limited, thereby imposing a limit on the size of the cut partial image and thus improving image quality.

[0085] Figure 7 This shows an example of a ratio table used in "Image Quality Priority Mode". This ratio table is used instead of the one used when "Image Quality Priority Mode" is set. Figure 3 The ratio table shown. (Compared to...) Figure 3 Compared to the ratio table shown, in "Image Quality Priority Mode," the sum of the percentages allocated to the movable area of ​​electronic image stabilization for image stabilization and subject tracking is limited to a smaller value, rather than 100%, based on the vibration state. This allows for an increase in the size of the majority of the image by reducing the movable area of ​​electronic image stabilization.

[0086] For example, in the "tripod / gimbal" case, the percentage of the movable area for electronic image stabilization allocated to image stabilization and subject tracking is limited to a total of 30%. By limiting the total percentage in this way, the size (i.e., angle of view) of parts of the image can be prevented from becoming too small. On the other hand, in the "large vibration" case, the percentage of the movable area for electronic image stabilization is limited to a total of 75%. Compared to the "tripod / gimbal" case, this limitation is closer to the cut-out amount when "image quality priority mode" is not selected, thus balancing image stabilization and subject tracking performance. In this way, limiting the movable area of ​​electronic image stabilization according to the needs of image stabilization and subject tracking can reduce the image quality degradation associated with image cut-out.

[0087] As described above, according to the first embodiment, optical image stabilization and electronic image stabilization can be effectively combined based on the vibration state of the camera system and the user's settings, thereby allowing a good balance between image stabilization and subject tracking.

[0088] <Variation Example> There are situations where the field of view fluctuates significantly when shooting conditions change. For example, if the camera mounted on a tripod is removed from the tripod and lifted, the control from... Figure 3 The ratio table shown indicates that "Tripod / Gimbal" is switched to "Small Vibration". In this case, the ratio of the movable area allocated to subject tracking in optical image stabilization suddenly changes from 100% to 70%, resulting in a sharp change in the field of view.

[0089] When the shooting conditions change as described above, the user can be notified of the change in viewing angle by displaying a warning on the display unit 111 indicating a possible change in viewing angle. Alternatively, in situations such as Figure 5 In the case of "priority tracking" as shown, subject tracking is prioritized, and the viewpoint can be maintained without changing the movable area allocated to subject tracking in optical image stabilization. This is achieved by maintaining the movable area allocated to subject tracking in optical image stabilization as it was before the change in shooting conditions, even after the shooting conditions have changed.

[0090] <Second Embodiment> Next, a second embodiment of this disclosure will be described.

[0091] One problem that arises when using optical image stabilization for subject tracking in addition to electronic image stabilization is the noise and vibration that may occur when the optical image stabilization component moves suddenly. In this embodiment, as an example of a situation requiring sudden movement of the optical image stabilization component, the centering operation of the control quantity when transitioning from a state using image stabilization for subject tracking (hereinafter referred to as "subject tracking mode") to normal image stabilization without subject tracking (hereinafter referred to as "normal image stabilization mode") will be described. Note the configuration and reference of the camera system in the second embodiment. Figure 1 and Figure 2 The camera systems described are configured the same, so a detailed description will be omitted here.

[0092] Figure 8AA This is a schematic graph illustrating the tracking control amount of electronic image stabilization, the tracking control amount of optical image stabilization, and the total tracking control amount over time when transitioning from subject tracking mode to normal image stabilization mode. Here, the vertical axis represents the number of pixels as the control amount, and the horizontal axis represents the change of the image frame number over time. Figure 8AA Frames 0 through 10 are in subject tracking mode, and it shows how subject tracking is performed so that the total tracking control amount is 150 (pixels).

[0093] exist Figure 8AA In this mode, when the setting changes from subject tracking mode to normal image stabilization mode, control is applied to return the tracking control value to 0 (pixel) in the 11th frame. In other words, by shifting the position of the optical image stabilization component and the cut-out position of a portion of the image in electronic image stabilization to the center position, the maximum movable area is ensured for image stabilization from the 11th frame onwards.

[0094] If the mode change involves a change in the cut-out region of a portion of the image during electronic image stabilization, the image displayed on the screen will suddenly appear different after the mode change. In this case, the tracking control variable can be immediately returned to 0 (pixels), resulting in a natural change in the appearance of the image. However, as... Figure 8AA As shown, the optical image stabilization component (i.e., the image stabilization lens 102 or the image sensor 106) will move in a sudden, step-like manner, thereby generating operational noise and vibration, which may impair the user's operating experience.

[0095] Therefore, in the second embodiment, as Figure 8ABAs shown, the cutout positions of the optical image stabilization component and the portion of the image in electronic image stabilization gradually shift in opposite directions over several frames. That is, when the subject tracking mode switches to normal image stabilization mode in the 10th frame, the tracking control amount of the optical image stabilization component gradually shifts to 0 (pixels) over several frames, while the cutout position of the portion of the image in electronic image stabilization shifts in the opposite direction to that of the optical image stabilization component. Here, the optical image stabilization component can be moved regardless of the frame period, but if it does not move synchronously with the change in the cutout position of the portion of the image in electronic image stabilization, unnecessary fluctuations in the viewpoint may occur. Therefore, before the transition is complete, the optical image stabilization component is controlled to move synchronously with the timing when the frame is updated, which is the same timing when the cutout position in electronic image stabilization is shifted.

[0096] As described above, according to the second embodiment, by using electronic image stabilization to counteract the movement of the optical image stabilization member, the mode can be changed without altering the total tracking control amount and the viewing angle. Furthermore, by avoiding sudden movement of the optical image stabilization member, noise and vibration caused by its movement can be suppressed.

[0097] Note that electronic image stabilization controls the portion of the image cropped from the captured image, therefore there are limits to the control parameters. Thus, there exists... Figure 8AB The indicated control period may reach the correction limit. Figure 8BA This is a graph illustrating such an example. The dashed line represents the limit value of the control amount of electronic image stabilization, and the control amount of electronic image stabilization reaches its limit in frame 16. On the other hand, as the control amount of the optical image stabilization component continues to change to 0, the total tracking amount changes from frame 16 to frame 20. This can manifest as a change in viewing angle due to the change in mode, which can cause discomfort to the user.

[0098] If, after the electronic image stabilizes to its correction limit as described above, the viewing angle fluctuation becomes unacceptably large, the following correction can be performed. That is, in Figure 8BA In the example, display control can be implemented to show a black image from frame 16 to frame 20, so that the captured image is not displayed during periods of viewpoint fluctuation. Alternatively, as... Figure 8BB As shown, when electronic image stabilization reaches its limit, the process of moving the optical image stabilization component to the 0 (pixel) position can be stopped. Even when electronic image stabilization reaches its limit, this can reduce the discomfort experienced by the user due to viewing angle fluctuations during state transitions.

[0099] Figure 9 This is a flowchart illustrating the control process in the second embodiment.

[0100] In step S201, the electronic image stabilization setting unit 1337 detects whether the subject tracking mode has changed from ON to OFF through the operation of the operation unit 114. If the change is not detected, the control quantity calculation unit 1333 continues the subject tracking control in step S202 and repeats the determination in step S201.

[0101] If a change from on to off in the subject tracking mode is detected in step S201, the electronic image stabilization setting unit 1337 determines in step S203 whether the cropping size of a portion of the image will change in association with the mode change. If it is determined that the cropping size of the portion of the image will not change, tracking does not need to be canceled immediately. Therefore, in step S205, the control quantity calculation unit 1333 performs control to gradually change the tracking quantity to 0 over several frames, and the mode change ends.

[0102] If it is determined in step S203 that the cropping size of a portion of the image will change, then in step S204, the control quantity calculation unit 1333 begins the following control, which is used to change the cropping size based on... Figure 8AB The ratios shown represent the positions of the optical image stabilizing components and the cut-out positions of the portion of the image in electronic image stabilization. In other words, the control quantity calculation unit 1333 initiates a change process that gradually changes the tracking quantity through optical image stabilization to 0 and changes the tracking quantity through electronic image stabilization in the opposite direction.

[0103] Next, in step S206, the control quantity calculation unit 1333 determines, based on the current position of the optical image stabilizing component and the current cut-out position of a portion of the image in electronic image stabilization, whether the cut-out position will reach the limit of the movable area before the tracking quantity of the optical image stabilizing component becomes 0. If it is determined that the limit will not be reached, the processing is changed to... Figure 8AB The ratio shown continues and ends in step S208.

[0104] On the other hand, if it is determined in step S206 that the limit of the movable area will be reached, then in step S207, control is implemented to mitigate the viewpoint fluctuation that occurs when the limit is reached. Note that the control used to mitigate viewpoint fluctuation is, for example, as... Figure 8BB The control could be as follows: in this control, the control quantity calculation unit 1333 stops moving the optical image stabilizing member when the cut-out position of a portion of the image in electronic image stabilization reaches the limit of the movable region. Alternatively, control could be implemented to display a black image (stop display) on the display screen for the period from when the cut-out position of the portion of the image in electronic image stabilization reaches the limit of the movable region until the displacement of the optical image stabilizing member ends, or if the amount of viewing angle fluctuation is within an allowable range. Figure 8ABThe changes described herein can be handled in a way that preserves the original state.

[0105] As described above, according to the second embodiment, operational noise and vibration that occur when the optical image stabilizing member moves suddenly during a mode change from the subject tracking mode can be reduced, while also avoiding unnatural fluctuations in the display viewing angle.

[0106] <Third Embodiment> Next, a third embodiment of this disclosure will be described.

[0107] In the third embodiment, an example of a problem that arises when switching from subject tracking mode to still image capture mode, and an example of control measures in that situation, are given. Note that reference... Figure 1 and Figure 2 The configuration of the camera system described can be used in the third embodiment, so its description will be omitted here.

[0108] Figure 10A The image sensor 106's viewpoint 1001, the partial image 1002 cut out during electronic image stabilization, and the positional relationship between the subject 1003 being tracked in subject tracking mode are shown. As a result of using optical and electronic image stabilization to track the subject 1003, the captured image during tracking (i.e., the partial image 1002 indicated by the dashed line) moves, and the subject 1003 being tracked remains centered in the partial image 1002.

[0109] Figure 10B Shown in Figure 10A The image shown is the viewpoint when the tracking state is switched to still image capture mode. In still image capture, the full-angle image captured by the image sensor is typically displayed and recorded, therefore, partial image cropping and magnification, as is done in video mode, are not performed. Therefore, as... Figure 10B As shown, the subject may be off-center in the captured image, resulting in a composition contrary to the user's intention to keep the subject 1003 in the center.

[0110] Therefore, in this embodiment, when changing from subject tracking mode to still image capture mode, such as Figure 11 As shown, change the tracking control variable.

[0111] Figure 11 Shown in Figure 10AThis illustrates how the viewing angle is corrected when switching from subject tracking mode to still image capture mode. Specifically, the tracking control value in electronic image stabilization is gradually changed to 0 (pixel) (the center of the image sensor), and the optical image stabilization component is rotated in the opposite direction to the tracking control value in electronic image stabilization to prevent changes in viewing angle. This allows the position of the subject 1003 to be maintained in the same position as during subject tracking mode, even when still image capture is performed without electronic image stabilization. Figure 11 In the example, this is the center of the cropped area 1002 of a portion of the image. Then, when a still image is captured in this state, an image of the entire field of view 1001 of the image sensor 106 is recorded.

[0112] Figure 12 An example illustrating the change in tracking control values ​​when switching to still image capture mode. Similar to... Figure 8AA , Figure 8AB , Figure 8BA and Figure 8BB , Figure 12 This is a schematic graph illustrating the changes in tracking control amounts for electronic image stabilization and optical image stabilization, as well as the total tracking control amount, over time. Frames 0 to 10 are in subject tracking mode, and the graph shows the tracking correction state using a combination of optical and electronic image stabilization with a tracking control amount of 150 (pixels).

[0113] exist Figure 12 In the 10th frame, when the mode changes from subject tracking mode to still image shooting mode, the tracking control amount for electronic image stabilization is controlled to gradually change to 0 (pixels), and the optical image stabilization component is controlled to move in the opposite direction to the tracking control amount for electronic image stabilization. Figure 12 As shown, the total tracking control quantity remains unchanged before and after the mode change, and the tracking control quantity for electronic image stabilization changes to 0 (pixels). Therefore, the subject position can be maintained at the same position as during tracking control. Furthermore, by gradually moving the position of the optical image stabilization component over several frames, sudden movement of the optical image stabilization component can be prevented, thereby reducing operational noise and vibration.

[0114] Here, as described above, it is desirable that the control period for optical image stabilization be the same as the frame period and synchronized with the control period for electronic image stabilization until the transition to still image shooting mode is complete. However, after the transition to still image shooting mode is complete, the control period for optical image stabilization can be changed to a faster period (such as the detection period of the camera-side shake detection unit 134) to maximize the effectiveness of image stabilization.

[0115] Figure 13This is a flowchart illustrating the control in the third embodiment.

[0116] In step S301, the control quantity calculation unit 1333 obtains from the operation unit 114 whether the subject tracking mode has changed to the still image shooting mode. If no change is detected, in step S302, the control quantity calculation unit 1333 generates a tracking control quantity to continue subject tracking control and repeats the judgment in step S301.

[0117] If a change to still image capture mode is detected in step S301, then in step S303, the control quantity calculation unit 1333 performs a reference calculation. Figure 12 The described tracking control variable transition process. That is, the tracking control variable of electronic image stabilization is gradually changed to 0 (pixel), and the optical image stabilization component moves in the opposite direction to the movement direction of the cut-out position according to the tracking control variable of electronic image stabilization, thereby transitioning to still image capture mode.

[0118] As described above, according to the third embodiment, when the subject tracking mode is changed to a still image shooting mode, still images can be captured while tracking the subject from the intended perspective, and operational noise and vibration that occur when the optical image stabilizing member moves are reduced.

[0119] <Other Embodiments> This disclosure can be applied to a system consisting of multiple devices, or to a device consisting of a single device.

[0120] The embodiments of the present invention can also be implemented by the following method: providing software (including computer program products of computer programs) that performs the functions of the above embodiments to a system or device via a network or various storage media, and the computer (central processing unit (CPU) or microprocessor unit (MPU) of the system or device) reads and executes the computer program.

[0121] While this disclosure has been described with reference to exemplary embodiments, it should be understood that this disclosure is not limited to the disclosed exemplary embodiments. The scope of the appended claims should be given the broadest interpretation to cover all such modifications and equivalent structures and functions.

Claims

1. A control device for controlling a first image stabilizing component and a second image stabilizing component, the first image stabilizing component being configured to perform optical image stabilization, and the second image stabilizing component being configured to perform image stabilization by changing the cropping position of a portion of an image cropped from an image captured by an image sensor, the control device comprising: The first acquisition component is configured to acquire a jitter amount representing the magnitude of the jitter of the detected object; The second acquisition component is configured to acquire a tracking amount for holding a predetermined subject included in the image at a predetermined position in the partial image; The third acquisition component is configured to acquire, according to predetermined conditions, a first ratio for allocating the movable region of the first image stabilization component to image stabilization, a second ratio for allocating the movable region of the first image stabilization component to tracking control, a third ratio for allocating the movable region of the portion of the image of the second image stabilization component to the image stabilization, and a fourth ratio for allocating the movable region of the portion of the image to the tracking control. as well as A fourth acquisition component is configured to acquire, based on the jitter amount, the tracking amount, and the first to the fourth ratios, the control amount of the first image stabilization component and the control amount of the second image stabilization component for the image stabilization and the tracking control.

2. The control device according to claim 1, further comprising: A judgment component is configured to determine the state of jitter based on the amount of jitter. The predetermined condition is the state of the jitter, and the greater the jitter, the higher the first ratio and the lower the second ratio.

3. The control device according to claim 2, wherein, The predetermined condition is the quality of the image at the peripheral image height of the image sensor, and When the image quality at the peripheral image height deteriorates, the movable area of ​​the first image stabilizing component becomes smaller compared to when the image quality is not deteriorated.

4. The control device of claim 3, further comprising a fifth acquisition component configured to acquire information relating to the optical system for incident light onto the image sensor. in, When the optical system includes at least one of a lens with large distortion and a lens in which light is incident at a small angle at its periphery, the image quality at the higher part of the peripheral image deteriorates.

5. The control device according to claim 2 further includes a detection component, the detection component being configured to detect whether an image quality priority mode with priority image quality has been selected using the operation unit. in, When the image quality priority mode is selected, the movable area of ​​the first image stabilizing component is narrower compared to when the image quality priority mode is not selected.

6. The control device according to claim 1, further comprising a detection component configured to detect whether the operation unit has selected a first priority mode that prioritizes reducing the amount of jitter or a second priority mode that prioritizes tracking the subject. in, When the first priority mode is selected, the first ratio is higher than when the second priority mode is selected. When the second priority mode is selected, the second ratio is higher than when the first priority mode is selected.

7. The control device according to claim 6, further comprising: A judgment component is configured to determine the state of jitter based on the amount of jitter. as well as The display control unit is configured to display the portion of the image cropped out by the second image stabilization unit on the display unit. When the jitter state determined by the judgment component changes, the display control component controls the display component to display a warning.

8. The control device according to claim 6 further includes a judging component, the judging component being configured to judge the state of jitter based on the jitter amount. in, When the second priority mode is selected, even if the jitter state determined by the judgment component changes, the first to fourth ratios obtained by the third acquisition component are maintained.

9. The control device according to claim 1 further includes a detection component, the detection component being configured to detect the activation / deactivation of a tracking mode for tracking a subject set by the operation unit. in, When the tracking mode is switched from on to off, the fourth acquisition unit calculates a control amount for gradually moving the first image stabilizing unit in the direction toward the position where the tracking amount becomes 0, and a control amount for cutting out the portion of the image to counteract the change in viewing angle caused by moving the first image stabilizing unit.

10. The control device according to claim 9, wherein, When the cut-out position of the second image stabilization component reaches the limit of the movable area, the fourth acquisition component determines the control amount of the second image stabilization component to stop moving the cut-out position, and the control amount of the first image stabilization component to stop moving in the direction toward the position where the tracking amount becomes 0.

11. The control device according to claim 9, further comprising a display control component, the display control component being configured to display a portion of the image cropped by the second image stabilization component on a display component. in, When the cut-out position of the second image stabilization component reaches the limit of the movable area, the display control component stops displaying the partial image until the first image stabilization component moves to the position where the tracking amount becomes 0.

12. The control device according to claim 1, further comprising a detection component, the detection component being configured to detect a mode set by the operation unit. in, When the mode switches from a tracking mode for tracking the subject to a still image shooting mode for capturing still images, the fourth acquisition unit calculates a control amount for the second image stabilization unit to gradually move the cut-out position of the partial image in the direction toward the position where the tracking amount becomes 0, and a control amount for moving the first image stabilization unit to counteract the change in viewing angle caused by the second image stabilization unit.

13. The control device according to claim 1, in, The first image stabilization component is at least one of the following correction components: a correction component configured to reduce image blur by moving a correction lens included in the imaging optical system in a direction perpendicular to the optical axis; and a correction component configured to reduce image blur by moving an image sensor in a direction perpendicular to the optical axis, the image sensor being used to perform photoelectric conversion on light incident through the imaging optical system and output an image signal. The fourth acquisition component acquires a control amount for moving at least one of the correction lens and the image sensor in a direction perpendicular to the optical axis of the camera optical system as the control amount of the first image stabilization component.

14. The control device according to claim 1, wherein, The first acquisition component acquires at least one of the jitter amount detected by the jitter detection component and the motion vector detected in the image output from the image sensor.

15. A control device for controlling a first image stabilizing component and a second image stabilizing component, the first image stabilizing component being configured to perform optical image stabilization, and the second image stabilizing component being configured to perform image stabilization by changing the cut-out position of a portion of an image cropped from an image captured by an image sensor, the control device comprising: The first acquisition component is configured to acquire a jitter amount representing the magnitude of the jitter of the detected object; The second acquisition component is configured to acquire a tracking amount for holding a predetermined subject included in the image at a predetermined position in the partial image; The third acquisition unit is configured to acquire a first value indicating the degree to which the first image stabilization unit is assigned to image stabilization, a second value indicating the degree to which the first image stabilization unit is assigned to tracking control, a third value indicating the degree to which the second image stabilization unit is assigned to the image stabilization, and a fourth value indicating the degree to which the second image stabilization unit is assigned to tracking control. as well as A fourth acquisition component is configured to acquire, based on the jitter amount, the tracking amount, and the first to the fourth values, the control amount of the first image stabilization component and the control amount of the second image stabilization component for the image stabilization and the tracking control.

16. An image stabilization device, comprising: The control device according to any one of claims 1 to 15; The first image stabilizing component; The second image stabilization component; The first actuating component is configured to actuate the first image stabilizing component based on the control quantity of the first image stabilizing component acquired by the fourth acquiring component; as well as The second actuation component is configured to actuate the second image stabilization component based on the control quantity of the second image stabilization component acquired by the fourth acquisition component.

17. A camera device, comprising: The image stabilization device according to claim 16; and The image sensor.

18. A control method for controlling a first image stabilization component and a second image stabilization component, the first image stabilization component being configured to perform optical image stabilization, and the second image stabilization component being configured to perform image stabilization by changing the cut-out position of a portion of an image cropped from an image captured by an image sensor, the control method comprising: The first acquisition step is used to acquire the jitter amount, which represents the magnitude of the jitter of the detected object; The second acquisition step is used to acquire the tracking amount used to keep a predetermined subject included in the image at a predetermined position in the partial image; The third acquisition step is used to acquire, according to predetermined conditions, a first ratio for allocating the movable region of the first image stabilizing component to image stabilization, a second ratio for allocating the movable region of the first image stabilizing component to tracking control, a third ratio for allocating the movable region of the portion of the image of the second image stabilizing component to the image stabilization, and a fourth ratio for allocating the movable region of the portion of the image to the tracking control. as well as The fourth acquisition step is used to acquire, based on the jitter amount, the tracking amount, and the first to the fourth ratios, the control amount of the first image stabilization component and the control amount of the second image stabilization component for the image stabilization and the tracking control.

19. A computer program product comprising a program for causing a computer to perform the control method according to claim 18.

20. A computer-readable storage medium storing a program for causing a computer to perform the control method according to claim 18.

Images