Control device for image blur correction and control method

Abstract

To provide a control device capable of properly reducing the influence of blur, even when a blur amount generated according to an image height is different.SOLUTION: The control device controls image blur correction by first image blur correction means for moving an optical element included in a photographic optical system, second image blur correction means for moving an imaging element for imaging a subject image formed by light transmitted through the photographic optical system in a direction crossing an optical axis, and third image blur correction means for correcting the image blur of a tilt component caused by the shake of an imaging device by image processing in an image. The control device controls the first image blur correction means and the second image blur correction means by a plurality of systems including at least a first control system and a second control system, and controls the third image blur correction means so that the level of the image blur correction of the tilt component by the third image blur correction means in the control by the second control system is higher than that in the control by the first control system.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a control device for image stabilization used in an imaging device, an imaging device including the control device, and a control method. [Background technology]

[0002] In recent years, many imaging devices such as digital cameras and video cameras have been equipped with an image stabilization function that corrects shake and other motions that occur in the imaging device. This image stabilization function has made it possible to capture images with higher image quality.

[0003] The image stabilization mechanisms in the imaging devices described above include a type that reduces image shake by shifting the image stabilization lens relative to the optical axis of the imaging optical system (hereafter referred to as OIS), and a type that reduces image shake by shifting the image sensor relative to the optical axis of the imaging optical system (hereafter referred to as IBIS).

[0004] Patent Document 1 discloses that the angle of blur that can be corrected (hereinafter referred to as the image blur correction angle) can be widened by simultaneously driving these two image blur correction mechanisms. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent No. 6410431 Summary of the Invention [Problem to be solved by the invention]

[0006] However, the amount of relative movement between the subject image and the imaging element caused by movement of the imaging device (the amount of blur of the subject image on the image) may differ depending on the image height. Therefore, the optimal amount of image blur correction differs depending on the image height. For example, image blur in the central part where the image height is low may be corrected, but image blur in the peripheral part where the image height is high may be noticeable.

[0007] The method disclosed in Patent Document 1 does not take into consideration any measures to address the above-mentioned problems.

[0008] The present invention has been made in consideration of the above-mentioned problems, and has as its object to provide a control device that can appropriately reduce the effects of blur even when the amount of blur that occurs varies depending on the image height. [Means for solving the problem]

[0009] According to one aspect of the present invention, there is provided a control device that controls image blur correction by a first image blur correction unit that changes the direction of the optical axis of the photographing optical system by moving an optical element included in the photographing optical system, a second image blur correction unit that moves an image sensor that captures an object image formed by light transmitted through the photographing optical system in a direction intersecting the optical axis, and a third image blur correction unit that corrects, by image processing, image blur caused by a tilt component resulting from a shake of the photographing device for an image generated by processing a signal output from the image sensor, the control device further comprising: a determination unit that determines the intensity of image blur correction for the tilt component in the third image blur correction unit; and a control unit configured to control the first and second image blur correction units using a plurality of control methods including at least a first control method and a second control method, wherein the determination unit determines a first correction intensity as the image blur correction intensity when the control unit controls the first and second image blur correction units using the first control method, and determines a second correction intensity greater than the first correction intensity as the image blur correction intensity when the control unit controls the first and second image blur correction units using the second control method. Other aspects of the present invention will become apparent from the following description of the embodiments. [Effects of the Invention]

[0010] According to the present invention, it is possible to provide a control device that can appropriately reduce the influence of blur even when the amount of blur that occurs varies depending on the image height. [Brief explanation of the drawings]

[0011] [Figure 1] FIG. 1 is a block diagram showing a schematic configuration of an imaging system according to an embodiment. [Figure 2] FIG. 1 is a block diagram showing a schematic configuration of an image stabilization control system for a camera and a lens according to an embodiment. [Figure 3] FIG. 1 is a diagram illustrating a first cooperation method. [Figure 4] FIG. 10 is a diagram illustrating a second cooperation method. [Figure 5] 5A and 5B are diagrams illustrating the relationship between the image blur correction amount and the tilt correction upper limit value in the embodiment. [Figure 6] 5A and 5B are diagrams illustrating the relationship between the image blur correction amount and the tilt correction gain in the embodiment. [Figure 7] 4 is a flowchart showing processing of a camera in an embodiment. [Figure 8] 5 is a flowchart showing processing of an interchangeable lens in an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0012] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention claimed. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0013] 1 is a block diagram showing a schematic configuration of an imaging system according to this embodiment. The imaging system includes a camera body 100 and a lens device (hereinafter referred to as an "interchangeable lens") 200 that is detachable from the camera body 100. The camera body 100 may be a still camera or a video camera. This embodiment describes image stabilization in an imaging system in which an interchangeable lens is detachably attached to the camera body (a so-called interchangeable lens camera).

[0014] First, each component of the camera body 100 will be described. The image sensor 101 is an image sensor such as a CMOS (Complementary MOS) image sensor, and captures a subject image by photoelectrically converting a subject image formed by light transmitted through the imaging optical system 210 of the interchangeable lens 200. The image sensor 101 is configured to be movable in a direction intersecting an optical axis OP of the imaging optical system 210 by a shift mechanism 101a, and the image sensor 101 and the shift mechanism 101a function as an image stabilization unit (second image stabilization unit). For example, the image sensor 101 can shift within a plane perpendicular to the optical axis OP, or rotate around the optical axis OP within a plane perpendicular to the optical axis OP. The following description will focus on shifting the image sensor 101. The shift mechanism 101a has an actuator and can shift the image sensor under control of an image stabilization control unit 103 on the camera body side. An output signal from the image sensor 101 (hereinafter referred to as an image signal) is input to an image processing unit .

[0015] The image processing unit 108 performs various image processes on the input imaging signal to generate image data. The image data generated by the image processing unit 108 is output to a motion vector detection unit 109 and an image correction unit 110.

[0016] The motion vector detection unit 109 detects, as a motion vector, motion information of feature points in the image data, using a plurality of image data (image data of a plurality of frames) captured at different timings successively by the image sensor 101 and generated by the image processing unit 108. Any known method such as a correlation method or a block matching method may be used to detect the motion vector, and a description of the detection method will be omitted.

[0017] The image correction unit 110 performs image processing based on information about the amount of image blur correction output from an image blur correction control unit 103 (described later), thereby performing electronic image blur correction (hereinafter, sometimes referred to as EIS). The image blur correction method will be described in detail later. The image blur corrected image data generated by the image correction unit 110 is displayed on a monitor (not shown) or recorded on a recording medium (not shown).

[0018] Camera shake detection unit 105 is composed of inertial sensors such as an angular velocity sensor and an acceleration sensor, and detects shake of camera body 100 caused by hand shake by the user or the like (hereinafter referred to as "camera shake"). Then, a camera shake detection signal, which is the detection result, is output to image stabilization control unit 103 via camera microcomputer 102. In this embodiment, camera shake detection unit 105 is an angular velocity sensor, and a form in which the angular velocity signal is output as the camera shake detection signal will be described.

[0019] The camera microcomputer 102 controls the overall processing of the camera body 100. It is also capable of communicating with the lens microcomputer 226 via the camera communication unit 106 and a lens communication unit 229 in the interchangeable lens 200.

[0020] The camera communication unit 106 has electrical contacts, and is connected to the electrical contacts of the lens communication unit 229 of the attached interchangeable lens 200 to perform communication between the interchangeable lens 200 and the camera body 100 .

[0021] The lens information management unit 129 holds and manages various types of information about the interchangeable lens 200 acquired through communication with the interchangeable lens 200 via the camera communication unit 106. The various types of information include optical property information about the image stabilization lens of the interchangeable lens 200, correction position information about the image stabilization lens, and information about the movable range (upper limit of the drive amount).

[0022] The image stabilization control unit 103 on the camera body side functions as a control unit that controls the IBIS by controlling the movement of the image sensor 101. It also functions as a control unit that controls the EIS by controlling electronic image stabilization by the image correction unit 110. Based on a camera shake detection signal detected by the camera shake detection unit 105, the image stabilization control unit 103 calculates the shift amount of the image sensor 101 (target value for the image stabilization amount) for reducing image shake caused by camera shake and the amount of electronic image stabilization by the image correction unit 110. In this embodiment, the image stabilization control unit 103 on the camera body side also functions as a control unit that controls the OIS by outputting the OIS correction ratio to the interchangeable lens 200 side. The OIS correction ratio will be described later.

[0023] The image stabilization control unit 103 controls the actuator of the shift mechanism 101a based on the calculated shift amount of the image sensor 101, thereby shifting the image sensor 101 by the shift amount. This allows the subject image to move on the image plane (sensor surface) of the image sensor 101, thereby enabling image stabilization (IBIS) by shifting the image sensor 101.

[0024] The image stabilization control unit 103 controls the EIS by outputting the calculated image stabilization amount of the image corrector 110 to the image corrector 110. The image corrector 110 performs geometric deformation by projective transformation based on the image stabilization amount output from the image stabilization control unit 103, thereby performing electronic image stabilization (EIS) by image stabilization. By setting an EIS cropping range smaller than the readout range of the image sensor, the difference between the readout range of the image sensor and the cropping range becomes surplus pixels, which becomes the range that can be corrected by EIS (EIS correctable range). Note that the EIS in this embodiment can perform translational component shake correction, rotational component shake correction, and tilt component shake correction. Note that although the present embodiment will be described using projective transformation, other transformation methods such as affine transformation may also be used.

[0025] The image sensor position detector 132 is a position detection sensor such as a Hall sensor, and detects the position of the image sensor and outputs the result to the image blur correction controller 103 .

[0026] The following describes each component of the interchangeable lens 200. The imaging optical system 210 has a variable magnification lens 201, an aperture 202, a focus lens 203, and an image blur correction lens 204, which is an optical element that can change the position at which a subject image is formed.

[0027] The zoom control unit 221 can detect the position of the variable magnification lens 201 (hereinafter referred to as the "zoom position") and varies the magnification by driving the variable magnification lens 201 in response to a zoom drive command from the camera microcomputer 102. Information about the zoom position is transmitted to the camera body 100 via the lens microcomputer 226 and the lens communication unit 229. The transmitted zoom position may be information about the position of the variable magnification lens 201, or may be information about the focal length corresponding to that zoom position.

[0028] The aperture control unit 222 can detect the aperture diameter of the aperture 202 (hereinafter referred to as "aperture position") and adjusts the amount of light by driving the aperture 202 in response to an aperture drive command from the camera microcomputer 102. The aperture control unit 222 may detect and control the aperture position continuously, or may detect and control the aperture position discontinuously, such as fully open, two stops (intermediate), and one stop (minimum). The aperture position may also be detected using the drive amount of the drive mechanism that drives the aperture 202. Information on the aperture position is transmitted to the camera body 100 via the lens microcomputer 226 and lens communication unit 229.

[0029] The focus control unit 223 can detect the position of the focus lens 203 (hereinafter referred to as the "focus position") and performs focus adjustment by driving the focus lens 203 in response to a focus drive command from the camera microcomputer 102. Information on the focus position is transmitted to the camera body 100 side via the lens microcomputer 226 and the lens communication unit 229.

[0030] The image blur correction lens 204 is configured to be shiftable by a shift mechanism 204a in directions that include a directional component perpendicular to the optical axis, and the image blur correction lens 204 and the shift mechanism 204a function together as an image blur correction unit (first image blur correction unit). That is, the image blur correction lens 204 is configured to be able to shift within a plane perpendicular to the optical axis and to rotate around a point on the optical axis as the rotation center. The following explanation will focus on the case where the image blur correction lens 204 is shifted. Shifting the image blur correction lens 204 changes the direction of the optical axis of the photographing optical system, and moves the position of the subject image formed on the image plane of the image sensor 101, thereby enabling image blur correction.

[0031] The shift mechanism 204a has an actuator and is capable of shifting the image stabilization lens 204 under control of an image stabilization control unit 224 on the interchangeable lens side.

[0032] The lens shake detection unit 228 is composed of inertial sensors such as an angular velocity sensor and an acceleration sensor. The lens shake detection unit 228 detects shake of the interchangeable lens 200 caused by a user's hand shake or the like (hereinafter referred to as "lens shake"), and outputs a lens shake detection signal representing the lens shake to the image stabilization control unit 224 via the lens microcomputer 226. When the interchangeable lens 200 is attached to the camera body 100, lens shake and camera shake are nearly identical, and therefore shake detected by the lens shake detection unit 228 is also referred to as camera shake. In this embodiment, the lens shake detection unit 228 is an angular velocity sensor, and a form in which the angular velocity is output as a lens shake detection signal will be described.

[0033] The image stabilization control unit 224 on the interchangeable lens side functions as a control unit that controls the OIS by controlling the movement of the correction lens 204. The image stabilization control unit 224 calculates the shift amount of the image stabilization lens 204 to reduce (correct) image shake caused by lens shake, based on the lens shake detection signal detected by the lens shake detection unit 228. Then, by controlling the actuator of the shift mechanism 204a based on the shift amount, the image stabilization lens 204 is shifted by the calculated shift amount. In this way, image stabilization can be performed by shifting the image stabilization lens 204.

[0034] The image stabilization (IBIS) by shifting the image sensor 101 and the lens image stabilization (OIS) by shifting the image stabilization lens 204 are generally called optical image stabilization. The image stabilization (EIS) by electronic image correction in the image correction unit 110 is generally called electronic image stabilization.

[0035] In this embodiment, whether or not to perform optical image stabilization (OIS, IBIS) can be set independently for image stabilization by shifting the image sensor 101 and lens image stabilization. The camera microcomputer 102 may set whether or not to perform optical image stabilization based on a user instruction, or may automatically set the setting based on various information such as the mode of the camera body 100. In this embodiment, whether or not to perform electronic image stabilization can be set when optical image stabilization is set to be performed, but electronic image stabilization may also be set to be performed when optical image stabilization is set to be not performed. The camera microcomputer 102 may set whether or not to perform electronic image stabilization based on a user instruction, or may automatically set the setting based on various information such as the mode of the camera body 100.

[0036] The lens position detector 234 is a position detection sensor such as a Hall sensor, which detects the position of the image stabilization lens 204 and outputs the position to the image stabilization controller 103 .

[0037] The lens microcomputer 226 controls the overall processing of the interchangeable lens 200. It is also capable of communicating with the camera microcomputer 102 via a lens communication unit 229 and the camera communication unit 106 in the camera body 100. The lens microcomputer 226 also functions as a transmission control unit that reads out information such as image circle information (described later) stored in the data storage unit 227, and transmits the image circle information, etc. to the camera body 100.

[0038] The lens communication unit 229 has electrical contacts, and by connecting to the electrical contacts of the camera communication unit 106 of the attached camera body 100 , communication is performed between the interchangeable lens 200 and the camera body 100 .

[0039] The camera information management unit 237 holds and manages various types of information about the camera body 100 obtained through communication with the camera body 100 via the lens communication unit 229. The various types of information include camera setting information, image sensor position information, and movable range information.

[0040] The data storage unit 227 is a non-volatile storage unit that stores optical information such as the zoom range (variable range of focal length) of the imaging optical system 210, the focus range (focusable distance range), and the variable range of aperture value.

[0041] 2 is a block diagram showing the detailed configuration of the image stabilization control unit 103 on the camera body side and the image stabilization control unit 224 on the interchangeable lens side. First, the configuration of the image stabilization control unit 103 on the camera body side will be described.

[0042] The image stabilization control unit 103 on the camera body side receives the detection result of the shake angular velocity as a shake detection signal from the camera shake detection unit 105. The camera integrator 161 of the image stabilization control unit 103 on the camera body side converts the shake angular velocity into a shake angle by integrating the input shake angular velocity. In this embodiment, a pseudo-integral low-pass filter (hereinafter referred to as an integral LPF) is used for the camera integrator 161.

[0043] Camera shake correction amount calculation unit 162 calculates the amount of correction required to cancel the shake angle, taking into account the frequency band of the converted shake angle and the drivable range of the camera. A specific example of this processing is to perform band-pass filtering on the input shake angle signal, extracting only a specific frequency band that is the target of shake correction.

[0044] The camera control method determination unit 166 determines whether the cooperative control in which the camera body (IBIS, EIS) and the interchangeable lens (OIS) share the responsibility of image stabilization should be performed using the first cooperative method or the second cooperative method. In this embodiment, if at least one of the camera body 100 and the interchangeable lens 200 does not support the second cooperative method, the first cooperative method is selected, and if both support the second cooperative method, the second cooperative method is selected.

[0045] The first and second coordination methods will now be described with reference to Figures 3 and 4. Figure 3 is a diagram illustrating the first coordination method, and Figure 4 is a diagram illustrating the second coordination method. In the graphs of Figures 3 and 4, the horizontal axis represents the amount of shake detected by camera shake detection unit 105, and the vertical axis represents the amount of shake correction performed by the OIS, IBIS, and EIS as a whole (sometimes referred to as the first correction amount). When performing shake correction, it is basically preferable that the amount of shake and the amount of shake correction be equal, because residual shake can be eliminated by correcting the same amount as the detected amount of shake.

[0046] The first cooperative method will be described using FIG. 3. In the first cooperative method, the ratio of the OIS, IBIS, and EIS is constant regardless of the amount of shake. Therefore, the shake correction amounts provided by the OIS, IBIS, and EIS each increase as the amount of shake increases until they reach the end (upper limit) of the correctable range of the OIS, IIS, and EIS. In this embodiment, this ratio is determined by the size of the correctable range of the OIS, IBIS, and EIS. Note that the correctable range here refers to the distance that can be caused by driving the correction lens 204 or the image sensor 101 to cause relative movement between the subject image and the image plane of the image sensor. Furthermore, for the EIS, it refers to the range that can be corrected by geometrically deforming the image within the surplus pixels, which are the difference between the readout range and the cropping range of the image sensor. FIG. 3 shows, as an example, a case where the ratio of the correctable range of the OIS, IBIS, and EIS is OIS:IBIS:EIS=35%:35%:30%. In this case, the correction ratio of OIS, IBIS, and EIS is always OIS:IBIS:EIS=35%:35%:30%, regardless of the amount of blur.

[0047] At this time, the direction of relative movement between the subject image generated by the OIS and the image plane of the image sensor, the direction of relative movement between the subject image generated by the IBIS and the image plane of the image sensor, and the direction of relative movement between the subject image generated by the EIS and the image plane (image crop position) are the same. This allows the OIS, IBIS, and EIS to share and correct image blur caused by camera shake. For example, if a correction lens is used that moves the subject image upward when moved upward, the OIS will move downward when camera shake occurs in the upward direction, moving the subject image downward relative to the image plane of the image sensor. At this time, the IBIS moves upward, and the EIS changes the crop position on the image upward, moving the subject image downward relative to the image plane of the image sensor (i.e., moving the image plane upward relative to the subject image). This method of moving the OIS, IBIS, and EIS so that relative movement occurs at the same ratio and in the same direction regardless of the amount of blur is called the first cooperative method. At this time, the amount of shake correction performed by the OIS, the amount of shake correction performed by the IBIS, and the amount of shake correction performed by the EIS do not exceed the amount of shake that is actually detected. Here, when correction is shared among the OIS, IBIS, and EIS, the amount of shake correction performed by the EIS is the amount of shake correction for the translational component. Note that while the OIS cannot perform shake correction for the rotational component, the IBIS and EIS can, so the IBIS and EIS can share the role of correcting the rotational component of shake.

[0048] Although FIG. 3 illustrates an example in which the correction ratio of the EIS is determined based on the movable range in the same way as the OIS and IBIS, the first cooperative method is not limited to this. Other methods may be used as long as the relative movement direction between the subject image generated by the OIS and the image plane of the image sensor coincides with the relative movement direction between the subject image generated by the IBIS and the image plane of the image sensor. For example, a method may be used in which the correction ratio between the OIS and IBIS is determined based on the ratio of the correctable ranges, and the portion that cannot be corrected by the OIS and IBIS is corrected by the EIS. When translational blur is corrected by the EIS using projective transformation, unlike the OIS and IBIS, image blur occurring during the capture of a single image cannot be corrected. Instead, image blur is corrected by adjusting the crop position between frames. Therefore, by using the OIS and IBIS in preference to the EIS, image blur occurring during the capture of a single image can be corrected more effectively than the method shown in FIG. 3.

[0049] The second cooperative method will be described using FIG. 4. In the second cooperative method, during section AB, where the amount of movement of the image stabilization lens 204 required to correct the amount of shake is equal to or less than the length of the movable range of the image stabilization lens 204, the OIS is driven with an excessive amount of shake compensation that exceeds the amount necessary to correct the amount of shake. Here, the movable range of the image stabilization lens 204 refers to the length from the reference position of the OIS to the movable end. In other words, section AB is a section where the amount of shake occurring can be corrected by the OIS alone. This control to drive with an amount of shake compensation that exceeds the amount necessary to correct the amount of shake occurring is referred to as overcorrection control in this invention and this specification. During this time, the IBIS performs inverse correction control to cancel the amount of shake compensation by the OIS that exceeds the amount required to correct the amount of shake occurring, i.e., the excessive correction amount. At this time, the relative movement directions between the subject image generated by the OIS and the image plane of the image sensor are opposite to each other. In this specification, the method in which the OIS performs overcorrection and the IBIS cancels the overcorrection is called the first control method.

[0050] On the other hand, when the amount of shake correction required to correct the amount of shake exceeds the length of the OIS's movable range and the OIS alone is no longer able to correct the amount of shake (section C), the IBIS and EIS share the responsibility of correcting the amount of shake that the OIS cannot correct. At this time, the direction of movement of the IBIS and the direction of image cropping by the EIS are directions that reduce the movement between the subject image and the image sensor that accompanies the shake, and the direction of relative movement between the subject image generated by the OIS, IBIS, and EIS and the image plane of the image sensor are the same.

[0051] In this specification, the method for controlling the image pickup device so that the direction of relative movement between the subject image generated by the OIS, IBIS, and EIS and the image plane of the image sensor coincides when a magnitude of blur that cannot be fully corrected occurs using the first control method is referred to as the second control method. If, during control using the first control method, an amount of blur exceeding the amount corresponding to the boundary between sections B and C is detected (i.e., the amount of blur increases from less than a predetermined value to more than the predetermined value), the first control method, which performs inverse correction, is switched to the second control method, which does not perform inverse correction. On the other hand, if, during control using the second control method, an amount of blur less than the amount corresponding to the boundary between sections B and C is detected, the second control method, in which the OIS and IBIS perform correction in the same direction, is switched to the first control method, which performs inverse correction.

[0052] The control in each section will be explained in detail. Section A is a section in which the OIS is overcorrected and the IBIS is inversely corrected, and in particular, is a section in which the OIS is operated at its maximum ratio. In the example shown in FIG. 4, the maximum ratio is 200%, and the correction amount is twice the amount of correction required to correct image blur caused by camera shake. Because overcorrection by the OIS would cause image blur if left unchecked, the IBIS correction ratio is set to -100%, and the sum of the OIS and IBIS correction ratios is set to 100%. Note that a correction ratio of -100% refers to driving with a shake correction amount that causes the same amount of blur in the same direction as the image blur caused by camera shake. In other words, it refers to driving with a shake correction amount that doubles the image blur caused by camera shake.

[0053] Section B extends from the end of Section A until the amount of image stabilization used to correct the amount of blur exceeds the length of the OIS's movable range. Section B is a section in which the OIS correction ratio and the IBIS inverse correction amount gradually decrease. It can also be considered a section in which the absolute values ​​of the OIS correction ratio and the IBIS correction ratio gradually decrease. In the example of Figure 4, at the boundary between Sections A and B, the OIS correction ratio is 200%, and the IBIS correction ratio is -100%. From the boundary between Sections A and B to the boundary between Sections B and C, the absolute values ​​of the correction ratios monotonically decrease, with the OIS correction ratio transitioning from 200% to 100% and the IBIS correction ratio transitioning from -100% to 0%. By controlling the sum of the OIS and IBIS correction ratios to be 100%, it is possible to achieve just the right amount of blur correction. Sections A and B are sections in which the first control method is implemented.

[0054] In section C, the OIS cannot correct any more shake, so the IBIS and EIS share the shortfall in shake correction. During this period, the amount of shake correction by the OIS remains constant, and only the amounts of shake correction by the IBIS and EIS increase. As a result, the correction ratios between the OIS, IBIS, and EIS do not remain constant in section C; when the amount of shake is large, the IBIS correction ratio (β) and EIS correction ratio (γ) increase, while the OIS correction ratio (α) decreases. Even in this section, by controlling the total correction ratios of the OIS, IBIS, and EIS to be 100%, it is possible to correct shake neither excessively nor insufficiently.

[0055] FIG. 4 shows an example in which the correction ratio between the IBIS and the EIS in section C is determined based on the ratio of the maximum movable ranges, as in the first cooperative method. However, as in the first cooperative method, the correction ratio between the IBIS and the EIS in this section does not have to be based on the ratio of the maximum movable ranges. As described above, when translational blur correction is performed using the EIS with geometric deformation, it is not possible to correct image blur while capturing a single image. Therefore, for example, a method may be used in which the IBIS first compensates for the deficiency in the OIS in section C, and then the EIS corrects any deficiencies in either the OIS or the IBIS.

[0056] Compared to the first coordination method, the second coordination method shown in FIG. 4 can reduce image blur in the peripheral areas of an image, which are areas with high image heights. When shooting video while carrying the camera body 100, significant blur occurs in the camera body 100 due to the impact of the photographer landing, etc. In this case, image blur occurring in the camera body 100 is corrected by the OIS, IBIS, and EIS. However, since the optimal amount of image blur correction may differ between the center of the image, where the image height is low, and the peripheral areas, where the image height is high, performing optimal correction for image blur in the center of the image using the first coordination method may result in noticeable image blur in the peripheral areas of the image. This is particularly noticeable when shooting wide-angle images or video. By performing the first control in the second coordination method, it is possible to reduce image blur in the peripheral areas of the image.

[0057] The camera control method determination unit 166 determines whether to use the first cooperation method or the second cooperation method described with reference to Figures 3 and 4 to control the OIS, IBIS, and EIS. There may be three or more cooperation methods, and the camera control method determination unit 166 may select a cooperation method from among the three or more cooperation methods.

[0058] Returning to the explanation of camera-side image stabilization control unit 103 using FIG. 2, camera ratio calculation unit 163 obtains the camera correction ratio provided by the camera, assuming that the sum of the camera-side (IBIS+EIS) and lens-side image stabilization amounts is 100%. Then, the camera ratio is multiplied by the first correction amount calculated by camera-side image stabilization amount calculation unit 162 to calculate the second correction amount. The camera correction ratio is obtained based on the cooperative control method (first cooperative method or second cooperative method). When the second cooperative method is used, the camera correction ratio is obtained based on the detected amount of shake and data indicating the relationship between the amount of shake and the ratio, as shown in the graph in FIG. 4. Furthermore, since the first correction amount corresponds to the total correction amount provided by the camera body side (IBIS and EIS) and the interchangeable lens side (OIS), the second correction amount provided by the camera body-side image stabilization is calculated by multiplying the first correction amount by the camera correction ratio provided by the camera.

[0059] Furthermore, the camera ratio calculation unit 163 obtains the IBIS correction ratio performed by the IBIS side when the sum of the shake correction amounts of the IBIS and EIS is 100%, and multiplies the IBIS correction ratio by the second correction amount to calculate the third correction amount, which is the IBIS correction amount. The IBIS correction ratio is obtained based on the cooperative control method (first cooperative method or second cooperative method). When the second cooperative method is used, the IBIS correction ratio is further obtained based on data showing the relationship between the shake amount and the ratio, as shown in the graph in FIG. 4, and the detected shake amount. Furthermore, since the second correction amount corresponds to the total correction amount corrected by the camera body side (IBIS and EIS), the third correction amount corrected by the IBIS-side shake correction is calculated by multiplying the IBIS correction ratio performed by the IBIS side by the second correction amount. The EIS correction ratio performed by the EIS side is calculated by subtracting the IBIS correction ratio from 100%. The fourth correction amount, which is the EIS correction amount, is calculated by multiplying the second correction amount by the EIS correction ratio. Since the second correction amount corresponds to the total correction amount made by the camera body (IBIS and EIS), the fourth correction amount made by image stabilization on the EIS side is calculated by multiplying it by the EIS correction ratio made by the EIS side.

[0060] Here, the second correction amount, which is the camera-side correction amount, is calculated and then distributed between the IBIS correction amount and the EIS correction amount, but this is not limited to this. For example, the IBIS correction amount may be calculated by multiplying the first correction amount by the IBIS correction ratio, and the EIS correction amount may be calculated by multiplying the first correction amount by the EIS correction ratio. The IBIS correction ratio and the EIS correction ratio can be obtained based on the cooperative control method and information indicating the relationship between the shake amount and the correction ratio (information showing the graphs in FIGS. 3 and 4).

[0061] If the target position of the image sensor 101, which corresponds to the third correction amount, exceeds the drive limit, the camera drive range limiter 164 performs limit processing and adjusts the correction amount so that the drive limit is not exceeded. The output of the camera drive correction limiter 164 becomes the final target correction amount for IBIS.

[0062] The camera feedback control unit 165 performs feedback control on the shift mechanism 101a using the current position acquired by the image sensor position detection unit 132 so that the image sensor 101 follows the target position corresponding to the third correction amount. As a result, the shift mechanism 101a drives the image sensor 101 to perform image stabilization control. In this embodiment, the camera feedback control unit 165 performs PID control based on the current position and the target correction amount. However, the feedback control method is not limited to PID control, and P control, PI control, or PD control may also be used.

[0063] When camera control method determination unit 166 selects the second cooperative method, control interval determination unit 170 determines in which interval of the control intervals shown in Fig. 4 correction control is being performed based on the current amount of shake (output of camera integration unit 161). Note that if camera ratio accumulation unit 163 has acquired in which interval correction control is being performed in order to acquire the correction ratio, the interval determination result may be acquired from camera ratio accumulation unit 163. Conversely, the determination result of control interval determination unit 170 may be output to camera ratio accumulation unit 163, and camera ratio accumulation unit 163 may acquire the ratio using this determination result.

[0064] When performing image stabilization using the second cooperative method, the correction intensity determination unit 171 determines the intensity of image stabilization for the tilt component of EIS (hereinafter sometimes referred to as tilt correction) based on the determination result of the control section determination unit 170 and the first correction amount. A strong image stabilization intensity indicates a strong effect of correcting the detected shake, and a weak image stabilization intensity indicates a weak effect of correcting the detected shake. The correction intensity determination unit 171 reduces the intensity of tilt correction in the case of the first control method, in which the optical image stabilization is relatively effective in reducing image blur in the peripheral areas of the image, compared to the case of the second control method, in which the optical image stabilization is less effective than the first control method in reducing image blur in the peripheral areas of the image. In this embodiment, the correction intensity determination unit 171 determines the correction intensity so that the upper limit value ω of the correction amount for the tilt component of EIS is larger in the second control method than in the first control method. Details of the method for determining the correction strength will be explained later, along with the image stabilization method using projective transformation by the image correction unit 110. However, when the first control method is performed with the second cooperative method, image blur in the peripheral areas of the image can be reduced by optical image stabilization more effectively than when the second control method or the first cooperative method is performed. Therefore, if the correction amount for the EIS tilt component is calculated in the second cooperative method as in the first cooperative method, image blur in the peripheral areas of the image will be reduced by both optical image stabilization and EIS. As a result, image blur in the peripheral areas of the image may be over-corrected, potentially preventing appropriate reduction of image blur in the peripheral areas of the image. Therefore, as described above, by determining the strength of EIS tilt correction by the correction strength determination unit 171, it is possible to prevent such excessive correction. This makes it possible to prevent image blur in the peripheral areas of the image from being over-corrected in the first control method, and to prevent excessive limitation of image blur correction in the peripheral areas of the image in the second control method. In this way, by changing the tilt correction strength based on the control method, image blur in the peripheral areas of the image can be reduced using both optical image stabilization and EIS correction methods, which makes it possible to further reduce image blur in the peripheral areas of the image compared to when optical image stabilization and EIS are used separately to correct image blur in the peripheral areas of the image.

[0065] The EIS correction amount calculation unit 180 calculates a fifth correction amount, which is the EIS correction amount reflecting the intensity of the tilt correction determined by the correction intensity determination unit, based on a fourth correction amount, which is the EIS correction amount, a first correction amount, which is the overall correction amount, and the correction intensity determination unit 171. When the translational component and rotational component are corrected by the EIS, the fifth correction amount is calculated based on the fourth correction amount. On the other hand, for the tilt component, since it is not the same as the shake correction component corrected by the OIS and IBIS, the correction amount of the tilt correction is calculated based on the first correction amount. Then, if the correction amount of the tilt correction calculated based on the first correction amount is equal to or less than the upper limit value ω of the tilt correction determined by the correction intensity determination unit 171, the calculated correction amount of the tilt correction is set as the correction amount of the tilt correction in the fifth correction amount. On the other hand, if the correction amount of the tilt component calculated based on the first correction amount exceeds the upper limit value ω, a limit process is performed on the calculated correction amount of the tilt correction. The amount of tilt correction subjected to limit processing is then set as the amount of tilt correction in the fifth correction amount. There is no particular limit processing method. For example, the calculated amount of tilt correction may be replaced with the upper limit value ω. Furthermore, when the calculated amount of tilt correction approaches ω, it may be multiplied by a coefficient of 1 or less to prevent it from exceeding ω. In this way, by having the EIS correction amount calculation unit 180 acquire an EIS tilt correction amount within a range that does not exceed the upper limit value ω of the tilt correction amount, EIS can be performed with a tilt correction strength appropriate to the control method. A method for calculating the fifth correction amount from the first and fourth correction amounts will be described in detail in the processing by the image correction unit 110, which will be described later.

[0066] An EIS cut-out range limiting unit 181 performs limiting processing when the fifth correction amount, which is the EIS correction amount calculated by the EIS correction amount calculating unit 180, exceeds the cut-out range of EIS, and adjusts the correction amount so that it does not exceed the cut-out range. The output of the EIS cut-out range limiting unit 181 becomes the final EIS correction amount.

[0067] The image correction unit 110 performs image processing based on the amount of EIS correction output from the EIS cutout range limiter 181, thereby executing EIS.

[0068] Here, we will explain in detail the electronic image stabilization (EIS) method performed by the image correction unit 110. The image correction unit 110 corrects the image by geometric deformation using projective transformation. For example, if the coordinates of each pixel in the image before correction are (X0, Y0) and the coordinates of each pixel in the image after correction are (X1, Y1), then the following (Equation 1) is obtained.

[0069]

number

[0070] Here, the 3x3 matrix in (Equation 1) is generally called a projective transformation matrix. The values ​​of each element of this matrix, h1, h2, h4, and h5, indicate the amount of shake correction for the rotational component, h3 and h6 indicate the amount of shake correction for the translational component, and h7 and h8 indicate the amount of shake correction for the tilt component. Transforming (Equation 1) yields (Equation 2) and (Equation 3), which allow the coordinates (X1, Y1) of each pixel in the corrected image to be calculated.

[0071]

number

[0072]

number

[0073] Here, the amount of angular deviation in the pitch direction is θ p , the amount of angular vibration in the Yaw direction is θ y , the amount of angular deviation in the roll direction is θ r , the horizontal translation correction amount is t x、 The vertical translation correction amount is t y , the horizontal tilt correction amount is v x , the vertical tilt correction amount is v y , and the focal length is f. As a result, the elements h1 to h8 of the 3×3 matrix in (Equation 1) can be expressed as (Equation 4) to (Equation 11). h1=cosθ r(Formula 4) h2=-sinθ r (Formula 5) h3=t x =ftanθ y (Formula 6) h4=sinθ r (Formula 7) h5=cosθ r (Formula 8) h6=t y =ftanθ p (Formula 9)

[0074]

number

[0075]

number

[0076] Pitch direction angle deviation θ p In order to correct the blur due to the vertical translation component, the blur correction amount t y , the amount of blur correction for the vertical tilt component v y Calculate the angular deviation θ in the yaw direction. y In order to correct the blur due to the horizontal translation component, the blur correction amount t x , the amount of shake correction for the horizontal tilt component v x Calculate the angular deviation in the roll direction θ r In order to correct the shake caused by the rotation, the amount of shake correction in the rotation direction is calculated from (Equation 4), (Equation 5), (Equation 7), and (Equation 8).

[0077] In this embodiment, the calculation of each of the parameters h1 to h8 using the above-mentioned equations is performed by the EIS correction amount calculation unit 180 and the EIS cutout range limiter 181, and projective transformation using each parameter is performed by the image correction unit 110. In this way, each blur component can be corrected, and thus image stabilization (EIS) can be performed by electronic image correction.

[0078] The EIS correction parameters h1, h2, h4, and h5 for the rotational component are calculated in the same way in the first and second coordination methods. The EIS correction amount calculation unit 180 calculates the roll correction angle in the fourth correction amount by multiplying the angular shake amount θ of the EIS rotational component by r and calculate the EIS correction parameters h1, h2, h4, and h5 for the rotational component from (Equation 4), (Equation 5), (Equation 7), and (Equation 8). These are set as the correction amounts for the rotational component in the fifth correction amount.

[0079] The EIS correction parameters h6 and h9 for the translational components are calculated in the same way in the first and second coordination methods. The EIS correction amount calculation unit 180 calculates the pitch correction angle in the fourth correction amount by multiplying the pitch angle shake amount θ p The Yaw correction angle in the fourth correction amount is the Yaw angle shake amount θ y The EIS correction parameters h6 and h9 for the translational component are calculated from (Equation 6) and (Equation 9) by substituting the above equations. These are used as the translational component correction amounts in the fifth correction amount.

[0080] As described above, the EIS correction parameters h7 and h8 for the tilt component are limited by the EIS correction amount calculation unit 180. However, since limit processing is not necessary in the first coordination method, the calculation method differs between the first coordination method and the second coordination method. In the first coordination method, the EIS correction amount calculation unit 180 calculates the pitch correction angle of the first correction amount by multiplying the pitch angle shake amount θ p Then, the Yaw correction angle of the first correction amount is calculated by substituting the Yaw angle shake amount θ yis substituted into EIS correction parameter h7 to calculate the EIS correction parameter h7 for the tilt component. In the second coordination method, the above-mentioned limiting process is also performed in addition to this. First, as in the first coordination method, the tilt correction parameters h7 and h8 are calculated using the first correction amount and (Equation 10) and (Equation 11). Next, the calculated EIS tilt correction parameters h7 and h8 are compared with the upper limit value ω of EIS tilt correction determined by the correction intensity determination unit 171. Then, if the EIS tilt correction parameters h7 and h8 exceed the upper limit value ω of correction, limiting process is performed to adjust the EIS tilt correction parameters h7 and h8 so that they do not exceed the upper limit value ω of correction.

[0081] The upper limit value ω of the tilt correction amount (tilt correction parameter) determined by the correction intensity determination unit 171 will be described using FIGS. 5(a) to 5(c). In FIG. 5, the horizontal axis represents the image shake correction amount (first correction amount), and the vertical axis represents the upper limit values ​​of the EIS tilt correction parameters h7 and h8, which are the EIS tilt correction amount. As described above, the first control method is more effective at reducing image blur in the peripheral areas of an image using optical image stabilization than the second control method, and therefore the upper limit value of the tilt correction amount in the first control method is set smaller than the upper limit value in the second control method.

[0082] 5(a) shows an example of the upper limit value of the tilt correction amount determined by the correction intensity determination unit 171. For the correction amount controlled by the first control method, the upper limit value is set to ω1, and for the correction amount controlled by the second control method, the upper limit value is set to ω2, with the magnitude relationship between the two being ω1<ω2. As a result, the upper limit values ​​of the EIS tilt correction parameters h7 and h8 are limited to ω1 during control by the first control method, and the upper limit values ​​of the EIS tilt correction parameters h7 and h8 are limited to ω2 during control by the second control method. Therefore, the intensity of tilt correction during the second control can be stronger than during the first control.

[0083] FIG. 5(b) shows another example of the upper limit of the amount of tilt compensation. As shown in FIG. 5(b), in the section controlled by the first control method, the upper limit ω1 of the amount of tilt compensation may be varied depending on the amount of shake. While FIG. 5(b) shows an example in which the upper limit of compensation increases from ω1 to ω2 in proportion to the increase in the amount of compensation in the section controlled by the first control method, the upper limit may also be increased in multiple stages from ω1 to ω2. However, since the effect of optical image stabilization on peripheral areas tends to decrease as the amount of image shake increases, it is desirable for the upper limit to increase monotonically as the amount of shake increases. Furthermore, as shown in FIG. 5(c), in the section controlled by the first control method, the upper limit of compensation may be varied between the section in which the OIS is operated at the maximum ratio (section A in FIG. 4) and the section in which the IBIS reduces the amount of inverse compensation as the amount of image shake compensation increases (section B in FIG. 4), as described in FIG. 4. FIG. 5(c) shows an example in which the upper limit of correction in section A is set to ω1 (a constant value), and in section B the upper limit of correction increases from ω1 to ω2 in proportion to an increase in the amount of image shake correction. Because the OIS cannot be operated at its maximum ratio in section B, the effect of optical image stabilization in reducing image blur in the peripheral areas of the image is smaller than in section A. Therefore, in section B the upper limit of EIS tilt correction amount is gradually increased in proportion to the amount of image shake, and the effect of optical image stabilization in reducing image blur in the peripheral areas of the image is compensated for by EIS tilt correction, thereby further reducing image blur in the peripheral areas of the image.

[0084] The method of changing the sway correction strength may be other methods. For example, it may be performed by changing the gain multiplied by the correction amount. In this case, the correction strength determination unit 171 determines the correction gain g of the EIS sway correction as the strength of the correction. Then, the EIS correction amount calculation unit 180 calculates the EIS sway correction parameter used for projective transformation by multiplying the EIS sway correction parameters h7 and h8 calculated in the same manner as in the first cooperation method by the correction gain g. An example of the correction gain g will be described using FIGS. 6((a) to (c)). The horizontal axis is the first correction amount, and the vertical axis is the determined gain g of the sway correction. FIG. 6(a) is an example of the correction gain g determined by the correction strength determination unit 171. Let the gain of the sway correction for the correction amount controlled by the first control method be g1, and the gain of the sway correction for the correction amount controlled by the second control method be g2, and the magnitude relationship between the two be g1 < g2. That is, during the control by the first control method, the sway correction parameters h7 and h8 are multiplied by the correction gain g1, and during the control by the second control method, the sway correction parameters h7 and h8 are multiplied by the correction gain g2 to calculate the actual sway correction parameters h7 and h8 used for projective transformation. Thereby, the strength of the sway correction during the second control can be made stronger than during the first control.

[0085] FIG. 6(b) shows another example of the correction gain for tilt correction. As shown in FIG. 6(b), in the section controlled by the first control method, the correction gain may be varied according to the correction amount. FIG. 6(b) shows an example in which the correction gain increases from g1 to g2 in proportion to the increase in the correction amount in the section controlled by the first control method. However, as with the upper limit value, the correction gain may be increased in multiple stages. However, it is preferable that the correction gain increases monotonically as the amount of image blur increases. Also, as shown in FIG. 6(c), in the section controlled by the first control method, the correction gain may be varied between a section in which the OIS is operated at the maximum ratio (section A in FIG. 4) as described in FIG. 4, and a section in which the IBIS corrects by reducing the amount of inverse correction as the amount of image blur correction increases (section B in FIG. 4). FIG. 6(c) shows an example in which the correction gain is g1 (a constant value) in section A, and the correction gain increases from g1 to g2 in proportion to the increase in the amount of image blur correction in section B. As described above, the effect of optical image stabilization in reducing image blur in the peripheral areas of the image is smaller in Section B than in Section A. For this reason, in Section B, the gain of EIS tilt correction is gradually increased in proportion to the amount of blur, and the effect of EIS tilt correction in reducing image blur in the peripheral areas of the image is compensated for by the optical image stabilization, thereby making it possible to further reduce image blur in the peripheral areas of the image.

[0086] The configuration of the interchangeable lens-side image stabilization control unit 224 will now be described. The interchangeable lens-side image stabilization control unit 224 receives an input of the detection result of the blur angular velocity as a shake detection signal from the lens-side shake detection unit 228. A lens integration unit 251 of the image stabilization control unit 224 converts the blur angular velocity into a blur angle by performing integration processing on the input blur angular velocity. In this embodiment, an integral LPF is also used for the lens integration unit 251.

[0087] Lens shake correction amount calculation unit 252 calculates the correction amount to cancel the shake angle, taking into account the frequency band of the converted shake angle and the driveable range of the lens. Specific processing examples include performing band-pass filtering on the input shake angle signal to extract only a specific frequency band that is the target of shake correction.

[0088] Like the camera control method determination unit 166, the lens control method determination unit 256 determines whether the cooperative control of the IBIS, OIS, and EIS will be performed using the first cooperative method or the second cooperative method. As with the camera control method determination unit 166, the determination method is to select the second cooperative method if both the camera body 100 and the interchangeable lens 200 support the second cooperative method. Note that instead of the lens control method determination unit 256 making the determination, the determination result may be acquired from the camera control method determination unit 166. Conversely, the lens control method determination unit 256 may determine the cooperative method and transmit the determination result to the camera side, thereby replacing the camera control method determination unit 166.

[0089] Lens ratio calculation unit 253 obtains the correction ratio provided by the lens side when the sum of the shake correction amounts on the camera side and the lens side is 100%, and multiplies the obtained ratio by the sixth correction amount calculated by lens shake correction amount calculation unit 252 to calculate a seventh correction amount. As with camera ratio calculation unit 163, the correction ratio is obtained based on a cooperative method, and when the cooperative method is the second cooperative method, the correction ratio is obtained based on data indicating the relationship between the amount of shake and the ratio and the detected amount of shake. Note that the correction ratio provided by the lens side may be obtained by camera ratio calculation unit 163 also obtaining the lens-side correction ratio and transmitting it to lens ratio calculation unit 253, which then receives it. Alternatively, camera ratio calculation unit 163 may transmit the camera-side correction ratio to lens ratio calculation unit 253, and lens ratio calculation unit 253 may obtain the lens-side correction ratio based on the camera-side correction ratio received. In addition, the roles of the lens ratio calculation unit 253 and the camera ratio calculation unit 163 may be reversed, so that the lens ratio calculation unit 253 acquires the correction ratio of the camera and the lens and transmits the camera-side correction ratio to the camera side, or transmits the lens correction ratio to the camera side.

[0090] If the target position of the image blur correction lens 204, which corresponds to the fourth image blur correction amount, exceeds the drive limit, the lens drive range limiter 254 performs limit processing and adjusts the correction amount so that the drive limit is not exceeded. The output of the lens drive correction limiter 254 becomes the final target value of the OIS correction amount.

[0091] The lens feedback control unit 255 performs feedback control on the shift mechanism 204a using the current position acquired by the lens vibration reduction position detection 234 so that the image blur correction lens 204 follows the target position corresponding to the seventh correction amount. As a result, the shift mechanism 204a drives the OIS to perform vibration reduction control.

[0092] The image stabilization processing performed in this embodiment will be described using the flowcharts in Figures 7 and 8. Figure 7 is a flowchart of the image stabilization processing performed on the camera body 100 side in this embodiment, and Figure 8 is a flowchart of the image stabilization processing performed on the interchangeable lens 200 side in this embodiment.

[0093] The image stabilization process by camera body 100 will be described with reference to Figure 7. Unless otherwise specified, this process is performed by image stabilization control unit 103. First, in step S101, image stabilization control unit 103 receives an instruction from camera microcomputer 102 and starts image stabilization control.

[0094] In step S102, camera control method determination unit 166 of image stabilization control unit 103 determines whether the attached interchangeable lens 200 is an interchangeable lens that supports the second cooperative method, based on information indicating the model number of the interchangeable lens, etc. If the attached interchangeable lens 200 supports the second cooperative method, the process proceeds to step S103, where it is determined that the following image stabilization control will be performed using the second cooperative method, and the second cooperative method is set as the cooperative method to be used for image stabilization control. If the attached interchangeable lens does not support the second cooperative method, the process proceeds to step S104, where it is determined that the following image stabilization control will be performed using the first cooperative method, and the first cooperative method is set as the cooperative method to be used for image stabilization control.

[0095] Once the cooperative method to be used for image stabilization control is set in step S103 or S104, the process proceeds to step S105, where the camera microcomputer 102 acquires lens information from the interchangeable lens 200 via the camera communication unit 106 and stores the lens information in the lens information management unit 129. The lens information to be stored has been described above, so a detailed description thereof will be omitted.

[0096] Next, in step S106, the camera microcomputer 102 transmits the camera information via the camera communication unit to the interchangeable lens 200. The camera information to be transmitted is as described above, and therefore a description thereof will be omitted.

[0097] In step S107, the image stabilization control unit 103 acquires the detection result from the camera shake detection unit 105. The acquired shake detection result is input to the camera integration unit 161.

[0098] In step S108, camera integration unit 161 performs LPF processing on the detection result input from camera shake detection unit 105 and performs pseudo-integration.

[0099] In step S109, camera shake correction amount calculation unit 162 calculates a first correction amount based on the shake amount input from camera integration unit 161. Details of the calculation of the first correction amount are as described above and will be omitted here.

[0100] 4, which are sections in which the OIS and IBIS perform inverse correction, and outputs the determination result to the camera ratio accumulator 163 and the correction intensity calculator 171. As described above, this determination is made based on the magnitude of the shake amount, but if the shake amount and the first correction amount are approximately the same, it may also be made based on the first correction amount.

[0101] In step S111, camera ratio accumulator 163 acquires the camera correction ratio to be used by camera body 100 based on the determination result by camera control compensation method determiner 166. Furthermore, camera ratio accumulator 163 multiplies the acquired camera correction ratio by the first correction amount input from camera shake compensation amount calculator 162 to calculate a second correction amount, which is the shake compensation amount on the camera side. Furthermore, an IBIS correction ratio to be used by the IBIS side is acquired, and a third correction amount, which is the shake compensation amount to be used by the IBIS, is calculated by multiplying the camera shake compensation amount by the IBIS correction ratio. Furthermore, an EIS correction ratio to be used by the EIS side is acquired, and a fourth correction amount, which is the shake compensation amount to be used by the EIS, is calculated by multiplying the camera shake compensation amount by the EIS correction ratio. Since the method for acquiring each correction ratio is as described above, details are omitted here. In the case of the second cooperative method, a correction ratio is acquired according to which control section (sections A to C) the system corresponds to.

[0102] In step S112, if the target position of the image sensor 101 corresponding to the calculated camera-side shake correction amount exceeds the limit of the image sensor's drive range, the camera drive range limit unit 164 performs limit processing and calculates the final target correction amount on the IBIS side.

[0103] In step S113, the camera feedback control unit 165 controls the shift mechanism 101a based on the position of the image sensor 101 detected by the image sensor position detection unit 132 and the target correction amount on the camera side input from the camera driving range limit unit 164. In this way, the camera feedback control unit 165 controls the position of the image sensor 101 and performs driving processing of the IBIS.

[0104] In step S114, correction strength determination unit 171 determines the strength of image blur correction for the tilt component of EIS, based on the determination result of control section determination unit 170 and the first correction amount calculated by camera shake correction amount calculation unit 162. The determination method has already been explained, so a detailed description will be omitted here. However, in cases where the strength can be determined based only on the control method, as in Figures 5(a) and 6(a), the strength can be determined based on the determination result of control section determination unit 170.

[0105] In step S115, the EIS correction amount calculation unit 180 calculates the EIS correction amount based on the EIS shake correction amount (fourth correction amount) output from the camera ratio integration unit 163, the first correction amount, and the EIS tilt correction intensity determined by the correction intensity calculation unit 171. The calculation method has already been explained and will not be described here, but even if the rotational component and translational component are not electronically corrected, the EIS correction parameters h7 and h8 used in the projective transformation in this step are calculated.

[0106] In step S116, if the EIS correction amount calculated by the EIS correction amount calculation unit 180 exceeds the EIS cut-out range, the EIS cut-out range limiter 181 performs limit processing to adjust the correction amount so that it does not exceed the cut-out range.

[0107] In step S117, the image correction unit 110 performs geometric transformation by projective transformation based on the correction amount output from the EIS cropping range limit unit 181, thereby performing image blur correction by electronic image correction, and generates image blur-corrected image data.

[0108] In step S118, the camera-side image stabilization control unit 103 determines whether or not to continue image stabilization control on the camera side, and if so, returns to step S102. On the assumption that the attached interchangeable lens remains the same, the process may return to step S105. In this case, if it is detected that the attached interchangeable lens has been changed, the process may start again from step S102. If it is determined not to continue image stabilization, for example, if the user has turned off the camera-side image stabilization function or the camera body 100 has transitioned to playback mode, this process ends.

[0109] The image stabilization process performed by the interchangeable lens 200 will be described with reference to Figure 8. Unless otherwise specified, this process is performed by the image stabilization control unit 224. First, in step S201, the image stabilization control unit 224 receives an instruction from the lens microcomputer 226 and starts image stabilization control.

[0110] In step S202, lens control method determination unit 256 of image stabilization control unit 224 determines whether attached camera body 100 is a camera body that supports the second cooperation method, based on information indicating the model number of the attached camera body, etc. If attached camera body 100 supports the second cooperation method, the process proceeds to step S203, where it is determined that the following image stabilization control will be performed using the second cooperation method, and the second cooperation method is set as the cooperation method to be used for image stabilization control. If attached camera body 100 does not support the second cooperation method, the process proceeds to step S204, where it is determined that the following image stabilization control will be performed using the first cooperation method, and the first cooperation method is set as the cooperation method to be used for image stabilization control. Regardless of whether the process proceeds to step S203 or S204, lens control method determination unit 256 outputs the determination result to lens ratio accumulator 253 and lens driving range limiter 254. Instead of using the camera model number to determine whether the camera supports the second cooperative method, the camera body may obtain a determination result as to whether the image stabilization control will be performed using the first cooperative method or the second cooperative method, and the cooperative method may be set according to the obtained determination result.

[0111] Once the cooperative method to be used for image stabilization control is set in step S203 or S204, the process proceeds to step S205, where the lens microcomputer 226 transmits lens information to the camera body 100 via the lens communication unit 229. This step corresponds to step S105 in FIG.

[0112] Next, in step S206, the lens microcomputer 226 acquires camera information from the camera body 100 via the lens communication unit 229, and stores the camera information in the camera information management unit 237. This is a step corresponding to step S106 in FIG.

[0113] In step S207, the image stabilization control unit 224 acquires the detection result from the lens shake detection unit 228. The acquired shake detection result is input to the lens integration unit 251.

[0114] In step S208, the lens integration unit 251 performs LPF processing on the detection result of the lens shake detection unit 228 and performs pseudo-integration.

[0115] In step S209, lens shake correction amount calculation unit 252 calculates the shake correction amount based on the shake amount input from lens integration unit 251. The details of the shake correction amount calculation are as described above and will not be repeated here.

[0116] In step S210, lens ratio integrator 253 acquires the correction ratio to be handled by interchangeable lens 200 based on the determination result by lens control compensation method determiner 256. Then, the acquired correction ratio is multiplied by the shake compensation amount input from lens shake compensation amount calculator 252 to calculate the shake compensation amount on the lens side. The method for acquiring the correction ratio is as described above, so details will be omitted, but in the case of the second control method, a correction ratio is acquired according to which control section (one of sections A to C) the control method corresponds to. For this reason, the determination result by control section determiner 170 acquired in step S206 is referenced.

[0117] In step S211, the lens driving range limiting unit 254 performs limiting processing when the target position of the image blur correction lens 204 corresponding to the calculated shake correction amount on the lens side exceeds the limit of the driving range of the image blur correction lens 204.

[0118] In step S212, the lens feedback control unit 255 controls the shift mechanism 204a based on the position of the image stabilization lens 204 detected by the lens position detection unit 234 and the target correction amount on the lens side input from the lens drive range limit unit 254. In this way, the lens feedback control unit 255 controls the position of the image stabilization lens 204 and performs drive processing for the OIS.

[0119] In step S213, the lens-side image stabilization control unit 224 determines whether to continue lens-side image stabilization control, and if so, returns to step S202. On the assumption that the attached camera body remains unchanged, the process may return to step S205. In this case, if it is detected that the attached interchangeable lens has been changed, the process may start again from step S202. If it is determined not to continue image stabilization, for example, if the user has turned off the lens-side image stabilization function, this process ends.

[0120] This embodiment makes it possible to perform image blur correction of tilt components using electronic image stabilization (EIS) in accordance with the magnitude of the effect of image blur correction in the peripheral areas of an image using optical image stabilization (OIS, IBIS). Therefore, it is possible to provide an image stabilization device that can further reduce image blur in the peripheral areas of an image compared to when image blur correction in the peripheral areas of an image is performed using only optical image stabilization or electronic image stabilization, or when the strength of image blur correction of tilt components using electronic image stabilization is uniform regardless of the control method. This is particularly effective in reducing image blur that occurs in the peripheral areas of an image, such as when shooting video while carrying an imaging device around.

[0121] [Variations] In the above-described embodiment, the target correction amount for the IBIS is calculated on the camera body side, and the target correction amount for the OIS is calculated on the interchangeable lens side. However, it is also possible for the target correction amounts for both the IBIS and OIS to be calculated by either the camera body or the interchangeable lens, and the calculated target correction amounts to be transmitted to the other.

[0122] In the above embodiment, the target correction amount is calculated based on the amount of shake calculated by pseudo-integrating the angular velocity detected by the angular velocity sensor, but the method for calculating the target correction amount is not limited to this. For example, the target correction amount may be calculated based on the acceleration detected by an acceleration sensor or a motion vector calculated based on an image, or the amount of shake may be calculated using a combination of these pieces of information.

[0123] In the above-described embodiment, a camera system (image capture system) including a camera body 100 and an interchangeable lens 200 detachable from the camera body has been described. However, this embodiment can also be applied to a lens-integrated camera as long as it has the functions of OIS, IBIS (which performs image stabilization by moving the image sensor), and EIS (which performs tilt correction). Furthermore, this embodiment can be applied to various electronic devices, such as smartphones, tablets, wearable devices, and drones, as long as it has a camera unit with the functions of OIS, IBIS, and EIS (tilt correction). Furthermore, the above-described control does not need to be performed entirely on the camera body or the interchangeable lens. For example, some or all of the processing performed by the image stabilization control units 103 and 224 on the camera and interchangeable lens sides in this embodiment may be performed by an external device or the cloud. In this case, the control devices that control the OIS, IBIS, and EIS are configured not only within the camera or interchangeable lens, but also by the external device or the cloud.

[0124] In the above-described embodiment, the image stabilization control unit 103 on the camera body side also functions as a control unit for controlling the OIS by outputting the OIS correction ratio to the interchangeable lens 200 side. However, the division of roles between the image stabilization control unit 103 on the camera body side and the image stabilization control unit 224 on the interchangeable lens side can be changed as appropriate. For example, the image stabilization control unit 224 on the interchangeable lens side may determine the IBIS correction ratio or the EIS correction strength and output it to the camera body 100. Alternatively, the image stabilization control unit 103 on the camera body side may output information that serves as a basis for determining the OIS correction ratio to the interchangeable lens side, rather than the OIS correction ratio itself. For example, in the case of the first cooperative method shown in FIG. 3 , the information that serves as the basis is the IBIS and EIS correction ratio. By outputting the IBIS and EIS correction ratio to the interchangeable lens 200 side, the OIS correction ratio can be calculated inversely. Furthermore, in the first control method in which the OIS performs overcorrection control and the IBIS cancels the overcorrection, when the OIS is operated at the maximum ratio as in section A shown in Fig. 4, information indicating section A can be used as material information. Information indicating the relationship between the amount of shake of the imaging device and the OIS correction ratio, as shown in Figs. 3 and 4, can also be used as material information. The image stabilization control unit 103 outputs information indicating the relationship between the amount of shake of the imaging device and the OIS correction ratio to the interchangeable lens 200, so that the image stabilization control unit 224 on the interchangeable lens 200 can determine the OIS correction ratio based on the current amount of shake.

[0125] Conversely, the image stabilization control unit 224 on the interchangeable lens 200 side may output information for determining the correction ratios of the IBIS and EIS to the camera body 100 side. The information used as the basis is the same as the information output from the camera body side to the interchangeable lens side, and in the case of the first cooperative method, for example, information on the correction ratio of the OIS. It is also possible to use information indicating the relationship between the amount of shake of the imaging device and the correction ratio of the IBIS, or information indicating the relationship between the amount of shake and the correction ratio of the EIS.

[0126] In this way, information that serves as the basis for the OIS correction ratio or the IBIS correction ratio is sometimes referred to as "information regarding the OIS to IBIS correction ratio" in this invention and this specification. Furthermore, when EIS is also involved in the correction as shown in Figures 3 and 4, the information regarding the OIS to IBIS correction ratio uses information regarding the correction ratios of not only OIS and IBIS but also OIS, IBIS, and EIS.

[0127] Alternatively, instead of information about the correction ratio, the OIS and IBIS can be controlled by outputting the OIS correction amount or the IBIS correction amount. For example, the image stabilization control unit 103 on the camera body can control the OIS by outputting the OIS correction amount to the interchangeable lens, or the camera correction amount (IBIS correction amount + EIS correction amount) can be output to the interchangeable lens. In the latter case, the image stabilization control unit 224 on the interchangeable lens can calculate the insufficient correction amount based on the detection result by the lens shake detection unit 228 and the camera correction amount, and correct this insufficiency using the OIS. This control allows the image stabilization control unit 224 on the interchangeable lens to calculate the OIS correction amount regardless of the cooperation method or interval. The same applies when the image stabilization control unit 224 on the interchangeable lens controls the IBIS and EIS. The image stabilization control unit 224 on the interchangeable lens can output the IBIS correction amount and the EIS correction amount, or the OIS correction amount, to the camera body 100.

[0128] Furthermore, tilt correction may not be performed in the first control method. In this case, the tilt correction strength is 0, and if an upper limit is set, the upper limit in the first control method is set to 0. Furthermore, if a gain is set, the gain in the first control method is set to 0. This allows electronic tilt correction not to be performed in the first control method, and electronic tilt correction to be performed in the second control method. Furthermore, the first control method may also be configured not to perform electronic tilt correction only when certain conditions are met. For example, it may be configured such that electronic tilt correction is not performed in section A of FIG. 4, and electronic tilt correction is performed in section B even during the first control.

[0129] [Other embodiments] The present invention can also be realized by providing a program that implements one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions. The invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Therefore, the following claims are appended to clarify the scope of the invention. [Explanation of symbols]

[0130] 100 camera body 101 Image sensor 101a Shift mechanism 102 Camera microcomputer 103 Image stabilization control unit 105 Camera shake detection unit 106 Camera communication unit 108 Image Processing Unit 109 Motion vector detection unit 110 Image correction unit 106 Camera communication unit 129 Lens Information Management Unit 132 Image sensor position detection unit 161 Camera integration unit 162 Camera shake correction amount calculation unit 163 Camera ratio calculation unit 164 Camera driving range limit section 165 Camera feedback drive unit 166 Camera control method determination unit 170 Control section determination unit 171 Corrected intensity calculation unit 180 EIS correction amount calculation section 181 EIS cutout range limit 200 interchangeable lenses 204 Image stabilization lens 204a Shift mechanism 222 Lens vibration prevention drive unit 223 Lens position detection unit 224 Image stabilization control unit 226 Lens Microcomputer 227 Data Storage Unit 228 Lens shake detection unit 229 Lens Communication Department 237 Camera Information Management Department 251 Lens integration section 252 Lens shake correction amount calculation unit 253 Lens ratio calculation unit 256 Lens control method determination section

Claims

1. a first image blur correction unit that changes the direction of an optical axis of the photographing optical system by moving an optical element included in the photographing optical system; a control device for controlling image blur correction by a second image blur correction means which moves an image sensor which captures an object image formed by light transmitted through the photographing optical system in a direction intersecting the optical axis, and a third image blur correction means which corrects, by image processing, an image blur caused by a tilt component caused by a shake of an image capturing device, for an image generated by processing a signal output from the image sensor, a determination unit that determines a strength of the image blur correction for the tilt component in the third image blur correction unit; a calculation means for calculating a correction amount of the third image blur correction means based on the strength of the image blur correction determined by the determination means and an amount of shake of the imaging device; a control unit that controls the first image blur correction unit and the second image blur correction unit using a plurality of control methods including at least a first control method and a second control method, The determining means is when the control means controls the first image blur correction means and the second image blur correction means by the first control method, a first correction intensity is determined as an intensity of the image blur correction; the control device determines, when the control device controls the first image blur correction device and the second image blur correction device by the second control method, a second correction intensity that is greater than the first correction intensity, as an intensity of the image blur correction.

2. 2. The control device according to claim 1, wherein the first control method is more effective in reducing image blur in a region of the image with a high image height than the second control method.

3. the first control method is a control method for moving the image sensor in the same direction as a moving direction of the subject image and the image sensor caused by a shake of the image capturing apparatus, 2. The control device according to claim 1, wherein the second control method is a control method for moving the image sensor in a direction that reduces movement between the subject image and the image sensor caused by shaking of the image pickup device.

4. the first control method is a control method for moving the optical element so that a position of the subject image moves by a larger amount than an amount of movement between the subject image and the image sensor caused by a shake of the imaging device, The control device according to claim 1 , wherein the second control method is a control method for moving the optical element so that the position of the subject image moves by an amount smaller than the amount of movement between the subject image and the imaging element caused by shaking of the imaging device.

5. the first control method is a method of moving the optical element excessively in response to a shake of the imaging device, and moving the imaging element so as to cancel out the excessive movement of the optical element; 2. The control device according to claim 1, wherein the second control method is a method of moving the optical element and the imaging element such that an amount of movement between the imaging element and the subject image due to movement of the optical element and an amount of movement between the imaging element and the subject image due to movement of the imaging element are each smaller than an amount of movement between the imaging element and the subject image caused by shake of the imaging device.

6. the control means controls the first image blur correction means and the second image blur correction means in the first control method when the shake amount is a first shake amount; 2. The control device according to claim 1, wherein when the shake amount is a second shake amount that is larger than the first shake amount, the control device controls the first image blur correction means and the second image blur correction means by the second control method.

7. an upper limit value of the amount of blur correction by the third image blur correction means is changed to change the strength of the correction; 2. The control device according to claim 1, wherein an upper limit value of an amount of blur correction by the third image blur correction means when the first correction intensity is set is smaller than an upper limit value of an amount of blur correction by the third image blur correction means when the second correction intensity is set.

8. 2. The control device according to claim 1, wherein the third image blur correction means corrects blur by geometrically transforming the image using projective transformation.

9. 9. The control device according to claim 8, wherein the third image blur correction means performs blur correction of a translational component, blur correction of a rotational component, and blur correction of a tilt component.

10. the magnitude of the first correction is 0; 2. The control device according to claim 1, wherein when the first image blur correction means and the second image blur correction means are controlled by the first control method, the third image blur correction means does not correct the tilt component of the image blur.

11. the determining means determines a gain of the image blur correction in the third image blur correcting means as the correction strength, 2. The control device according to claim 1, wherein the calculation means calculates the correction amount using the gain.

12. The control means 2. The control device according to claim 1, wherein the control device outputs, to the first image blur correction means and the second image blur correction means, information on a correction ratio between the first image blur correction means and the second image blur correction means and at least one of a correction amount of the first image blur correction means and a correction amount of the second image blur correction means, thereby controlling the first image blur correction means and the second image blur correction means in the plurality of methods.

13. The information regarding the correction ratio is 13. The control device according to claim 12, further comprising at least one of information indicating a correction ratio of the first image blur correction means determined on the basis of a detected amount of shake of the imaging device and information indicating a correction ratio of the second image blur correction means determined on the basis of the amount of shake.

14. The information regarding the correction ratio is 13. The control device according to claim 12, further comprising at least one of information indicating a relationship between an amount of shake of an imaging device and a correction ratio of the first image shake correction means and information indicating a relationship between the amount of shake and a correction ratio of the second image shake correction means.

15. 13. The control device according to claim 12, wherein the information on the correction ratio is information on a correction ratio between the first image blur correction means, the second image blur correction means, and the third image blur correction means.

16. A control device according to any one of claims 1 to 15; The imaging element; the second image blur correction means, the determination means determines whether or not the attached lens device supports a cooperative control method in which the first control method and the second control method are switched depending on the amount of blur, and determines the strength of the correction based on the result of the determination.

17. The determining means is 17. The imaging device according to claim 16, wherein, when the attached lens device is compatible with the cooperative method, the strength of the correction is determined based on an amount of shake of the imaging device.

18. The determining means is 17. The imaging device according to claim 16, wherein when the attached lens device does not support the cooperative method, the strength of the correction is set to a constant value regardless of an amount of shake of the imaging device.

19. a first image blur correction unit that changes the direction of an optical axis of the photographing optical system by moving an optical element included in the photographing optical system; a second image blur correction unit that moves an image sensor that captures a subject image formed by light transmitted through the photographing optical system in a direction intersecting with the optical axis; a third image blur correction unit that corrects, by image processing, an image blur caused by a tilt component caused by a shake of an imaging device, for an image generated by processing a signal output from the imaging element; A method for controlling blur correction using a camera, comprising the steps of: a determination step of determining a strength of the image blur correction for the tilt component in the third image blur correction means; a calculation step of calculating a correction amount of the third image blur correction unit based on the strength of the image blur correction determined in the determination step and an amount of shake of the imaging device; a control step of controlling the first image blur correction means, the second image blur correction means and the third image blur correction means by a plurality of control methods including at least a first control method and a second control method, the determining step determines a first correction intensity as the intensity of the image blur compensation when the first image blur compensation means and the second image blur compensation means are controlled by the first control method in the control step, and determines a second correction intensity larger than the first correction intensity as the intensity of the image blur compensation when the first image blur compensation means and the second image blur compensation means are controlled by the second control method in the control step.

Images