Control device and method, imaging device, imaging system, program and storage medium

Abstract

To improve the accuracy of motion vector detection when image blur correction is performed. [Solution] A control device for controlling image blur correction using a first correction means that corrects image blur using a correction lens included in the imaging optical system, and a second correction means that corrects image blur using an image sensor, comprising: a first acquisition means for acquiring the amount of shake detected by a shake detection means; a determination means for determining a method for calculating a correction amount for controlling the first and second correction means based on the amount of shake and the control method of the first and second correction means; a calculation means for calculating the correction amounts of the first and second correction means using the calculation method based on the amount of shake; a second acquisition means for acquiring a representative motion vector based on a motion vector detected from an image output from the image sensor; and a setting means for setting a method for acquiring a representative motion vector by the second acquisition means based on the calculation method.

Description

Technical Field

[0001] The present invention relates to a control device and method, an imaging device, an imaging system, a program, and a storage medium, and particularly relates to control for detecting a motion vector in an apparatus that performs image blur correction.

Background Art

[0002] In recent years, many imaging devices such as digital cameras and video cameras have been equipped with an image blur correction function for correcting camera shake or the like applied to the imaging device. And, with this image blur correction function, it has become possible to capture images with better image quality.

[0003] There are roughly two types of image blur correction mechanisms in such imaging devices. One is a method of reducing image blur by shifting an image blur correction lens with respect to the optical axis of the imaging optical system (Optical Image Stabilization, hereinafter referred to as "OIS"). The other is a method of reducing image blur by shifting an image pickup device with respect to the optical axis of the imaging optical system (In Body Image Stabilization, hereinafter referred to as "IBIS").

[0004] Patent Document 1 discloses that by simultaneously driving these two types of image blur correction mechanisms, the range of correctable blur angles can be widened.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] However, the relative movement of the subject image and the image sensor (the amount of blur in the subject image on the image) caused by the movement of the imaging device can vary depending on the image height, and this effect is particularly noticeable when using a wide-angle lens. Therefore, the optimal amount of image blur correction differs depending on the image height. For example, image blur may be corrected for the subject image in the central part where the image height is low, while image blur may be noticeable for the subject image in the peripheral part where the image height is high. In this case, when motion vectors are detected using the subject image output from the imaging device, it may not be possible to detect motion vectors with the same accuracy in the peripheral part where the image height is high compared to the central part where the image height is low. As a result, for example, when image blur correction is performed using the motion vector detection results, the desired image blur correction effect may not be obtained.

[0007] However, the method disclosed in Patent Document 1 does not consider any measures to address the problem that motion vectors cannot be detected accurately in the peripheral area where the image height is higher compared to the central area where the image height is lower.

[0008] This invention was made in view of the above-mentioned problems, and aims to improve the accuracy of motion vector detection when image blur correction is performed. [Means for solving the problem]

[0009] To achieve the above objective, the control device of the present invention controls image blur correction by a first correction means that corrects image blur by driving a corrective lens included in the imaging optical system in a direction perpendicular to the optical axis, and a second correction means that corrects image blur by driving an image sensor that converts light incident through the imaging optical system into an image signal in a direction perpendicular to the optical axis, the control device of the present invention includes a first acquisition means for acquiring the amount of shake detected by a shake detection means, a determination means for determining a method for calculating a correction amount for controlling the first correction means and the second correction means based on the amount of shake and the control method of the first correction means and the second correction means, a calculation means for calculating the correction amounts of the first correction means and the second correction means using the calculation method based on the amount of shake, a second acquisition means for acquiring a representative motion vector based on a motion vector detected from an image output from the image sensor, and a setting means for setting a method for acquiring a representative motion vector by the detection means based on the calculation method. [Effects of the Invention]

[0010] According to the present invention, the accuracy of motion vector detection can be improved when image blur correction is performed. [Brief explanation of the drawing]

[0011] [Figure 1] A block diagram showing the schematic configuration of the imaging system in an embodiment of the present invention. [Figure 2] A block diagram showing the schematic configuration of the image blur correction control unit in the camera body and interchangeable lens device in the embodiment. [Figure 3] A diagram illustrating the first cooperation method in the embodiment. [Figure 4] A diagram illustrating the second coordination method in the embodiment. [Figure 5] A figure showing an example of the division of the motion vector detection region in the embodiment. [Figure 6] A diagram illustrating an example of setting the motion vector detection region in an embodiment. [Figure 7]A diagram illustrating the weighting of the motion vector detection region in the embodiment. [Figure 8] A diagram illustrating the correction gain of the motion vector detection amount in the embodiment. [Figure 9] A flowchart illustrating the image blur correction process in the camera body of the embodiment. [Figure 10] A flowchart illustrating the image blur correction process in the interchangeable lens device of the embodiment. [Modes for carrying out the invention]

[0012] The embodiments will be described in detail below with reference to the attached drawings. Note that the following embodiments do not limit the invention as defined in the claims. While the embodiments describe multiple features, not all of these features are essential to the invention, and the features may be combined in any way. Furthermore, in the attached drawings, identical or similar configurations are given the same reference numerals, and redundant descriptions are omitted.

[0013] Figure 1 is a block diagram showing the schematic configuration of the imaging system in this embodiment. The imaging system includes a camera body 100 and a detachable interchangeable lens device (hereinafter referred to as "interchangeable lens") 200 attached to the camera body 100. The camera body 100 may be a still camera or a video camera. In this embodiment, image blur correction in an imaging system (a so-called interchangeable lens camera) in which the interchangeable lens 200 is detachably attached to the camera body 100 will be described.

[0014] First, let's explain the components of the camera body 100. The imaging element 101 is, for example, an image sensor such as a CMOS (Complementary MOS) image sensor, and images the subject image formed by the light incident through the imaging optical system 210 of the interchangeable lens 200 by photoelectrically converting it, and outputs an image signal. The imaging element 101 is configured to be movable in a direction intersecting the optical axis OP of the imaging optical system 210 by the shift mechanism 101a, and the imaging element 101 and the shift mechanism 101a function as an image blur correction means. The imaging element 101 can, for example, shift within a plane orthogonal to the optical axis OP or rotate within a plane orthogonal to the optical axis OP with the optical axis OP as the center of rotation. In the following description, the case of shifting the imaging element 101 will be mainly described. The shift mechanism 101a has an actuator and can shift the imaging element 101 based on the control from the image blur correction control unit 103. The image signal output from the imaging element 101 is input to the image processing unit 108.

[0015] The image processing unit 108 performs various image processes on the input image signal to generate image data. The generated image data is displayed on a monitor (not shown) or recorded on a recording medium (not shown). Further, the image data generated by the image processing unit 108 is output to the motion vector detection unit 109.

[0016] The motion vector detection unit 109 uses multiple image data (multiple frames of image data) captured at different timings, which are continuously captured by the image sensor 101 and generated by the image processing unit 108, to detect motion information of feature points within the image data as motion vectors. Any known algorithm, such as the correlation method or the block matching method, may be used as the detection algorithm for detecting motion vectors from multiple image data. The motion vector detection unit 109 generates a histogram of multiple motion vector detection quantities obtained from the motion vector detection region, calculates the average value of the interval where the distribution is most concentrated, and obtains one representative motion vector quantity. The representative motion vector quantity obtained here is converted to units of angular velocity and used for image blur correction control, which will be described later. Compared to image blur detection using an angular velocity sensor, which will be described later, image blur detection using motion vectors is suitable for detecting camera shake components in the relatively low frequency range.

[0017] In this embodiment, the motion vector detected by the motion vector detection unit 109 is used for image blur correction, but it may be used for purposes other than image blur correction control. For example, the amount of movement of the subject to be tracked may be detected as a motion vector, and subject tracking control may be performed using the detected motion vector. Alternatively, the detected value of the angular velocity sensor (described later) may be compared with the detected value of the motion vector, and the offset component may be removed from the detected value of the angular velocity sensor. Furthermore, the motion vector detection unit 109 may change the method of detecting the motion vector depending on the control state of the camera. For example, the detection area of ​​the motion vector, the weighting of each area within the detection area of ​​the motion vector, or the correction gain multiplied by the detected amount of the motion vector may be changed. Details of the processing of the motion vector detection unit 109 will be described later.

[0018] The camera shake detection unit 105 is composed of inertial sensors such as an angular velocity sensor and an acceleration sensor, and detects the shake of the camera body 100 (hereinafter referred to as "camera shake") caused by the user's hand shake or the like. Then, a camera shake detection signal representing the detected camera shake is output to the image blur correction control unit 103 via the camera microcomputer 102. In the present embodiment, the camera shake detection unit 105 is assumed to be an angular velocity sensor, and a form in which the obtained angular velocity signal is output as a camera shake detection signal will be described.

[0019] The camera microcomputer 102 controls the processing of the entire camera body 100. It can also communicate with the lens microcomputer 226 via the camera communication unit 106 and the lens communication unit 229 in the interchangeable lens 200. The camera communication unit 106 has electrical contacts and communicates between the interchangeable lens 200 and the camera body 100 by being connected to the electrical contacts of the lens communication unit 229 of the mounted interchangeable lens 200.

[0020] The lens information management unit 129 holds and manages various information of the interchangeable lens 200 obtained by communicating with the interchangeable lens 200 via the camera communication unit 106. The various information includes optical characteristic information of the image blur correction lens 204 of the interchangeable lens 200, correction position information of the image blur correction lens, and information on the movable range (upper limit value of the driving amount).

[0021] The image blur correction control unit 103 of the camera body 100 has a function as a control unit that controls IBIS by controlling the movement of the imaging device 101. The image blur correction control unit 103 calculates the shift amount (target correction amount) of the imaging device 101 for reducing (correcting) the image blur caused by camera shake based on the camera shake detection signal and the motion vector detected by the camera shake detection unit 105 and the motion vector detection unit 109. Then, by controlling the actuator of the shift mechanism 101a based on the shift amount, the imaging device 101 is shift-driven by the calculated shift amount. As a result, the subject image can be moved on the image plane (sensor plane) of the imaging device 101, so that image blur correction (IBIS) by the shift of the imaging device 101 can be performed.

[0022] The image sensor position detection unit 132 is a position detection sensor such as a Hall sensor, which detects the position of the image sensor 101 and outputs it to the image blur correction control unit 103.

[0023] Next, we will explain the components of the interchangeable lens 200. The imaging optical system 210 includes a variable magnification lens 201, an aperture 202, a focus lens 203, and an image blur correction lens 204, which is an optical element capable of changing the position in which the subject image is formed.

[0024] The zoom control unit 221 can detect the position of the variable magnification lens 201 (hereinafter referred to as the "zoom position") and performs magnification by driving the variable magnification lens 201 in response to a zoom drive command from the camera microcontroller 102. Information on the zoom position is transmitted to the camera body 100 via the lens microcontroller 226 and the lens communication unit 229. The transmitted zoom position may be information on the position of the variable magnification lens 201, or it may be information on the focal length corresponding to that zoom position.

[0025] The aperture control unit 222 can detect the aperture diameter of the aperture 202 (hereinafter referred to as "aperture position") and adjusts the amount of light by driving the aperture 202 in response to an aperture drive command from the camera microcontroller 102. The aperture control unit 222 may continuously detect and control the aperture position, or it may discontinuously detect and control the aperture position, such as wide open, 2 stops (intermediate), and 1 stop (minimum). In addition, the aperture position may be detected using the drive amount of the drive mechanism that drives the aperture 202. Information on the aperture position is transmitted to the camera body 100 via the lens microcontroller 226 and the lens communication unit 229.

[0026] The focus control unit 223 can detect the position of the focus lens 203 (hereinafter referred to as "focus position") and adjusts the focus by driving the focus lens 203 in response to a focus drive command from the camera microcontroller 102. The focus position information is transmitted to the camera body 100 via the lens microcontroller 226 and the lens communication unit 229.

[0027] The image stabilization lens 204 is configured to be shiftable in a direction including a directional component perpendicular to the optical axis by a shift mechanism 204a, and the image stabilization lens 204 and the shift mechanism 204a function as an image stabilization means. That is, the image stabilization lens 204 is configured to be able to shift in a plane perpendicular to the optical axis or rotate around a point on the optical axis as the pivot point. The following description will focus on the case where the image stabilization lens 204 is shifted. By shifting the image stabilization lens 204, the direction of the optical axis of the imaging optical system is changed, and the position of the subject image formed on the image plane of the image sensor 101 moves, thereby enabling image stabilization. The shift mechanism 204a has an actuator and can shift the image stabilization lens 204 based on control from the image stabilization control unit 224 of the interchangeable lens 200.

[0028] The lens shake detection unit 228 is composed of inertial sensors such as an angular velocity sensor and an acceleration sensor, and detects shake of the interchangeable lens 200 caused by the user's hand shake, etc. (hereinafter referred to as "lens shake"). It then outputs a lens shake detection signal representing the detected lens shake to the image shake correction control unit 224 via the lens microcontroller 226. When the interchangeable lens 200 is attached to the camera body 100, the lens shake and camera shake are almost the same, so the shake detected by the lens shake detection unit 228 is also called "camera shake". In this embodiment, the lens shake detection unit 228 is assumed to be an angular velocity sensor, and a configuration in which the obtained angular velocity signal is output as the lens shake detection signal will be described.

[0029] The image blur correction control unit 224 of the interchangeable lens 200 functions as a control unit that controls OIS by controlling the movement of the image blur correction lens 204. Based on the lens shake detection signal detected by the lens shake detection unit 228, the image blur correction control unit 224 calculates the amount of shift of the image blur correction lens 204 to reduce (correct) image blur caused by lens shake. Then, by controlling the actuator of the shift mechanism 204a based on that shift amount, the image blur correction lens 204 is driven to shift by the calculated shift amount. This enables image blur correction (OIS) by shifting the image blur correction lens 204.

[0030] The image stabilization by shifting the image sensor 101 (IBIS) and the image stabilization by shifting the image stabilization lens 204 (OIS) are generally referred to as optical image stabilization. In this embodiment, the presence or absence of optical image stabilization can be set independently for IBIS and OIS, respectively. The presence or absence of optical image stabilization may be set by the camera microcontroller 102 based on user instructions, or it may be set automatically based on various information such as the mode of the camera body 100.

[0031] The lens vibration damping position detection unit 258 is a position detection sensor such as a Hall sensor, which detects the position of the image blur correction lens 204 and outputs the result to the image blur correction control unit 224.

[0032] The lens microcontroller 226 controls the overall processing of the interchangeable lens 200. It can also communicate with the camera microcontroller 102 via the lens communication unit 229 and the camera communication unit 106 of the camera body 100. The lens microcontroller 226 also functions as a transmission control unit, reading information such as image circle information (described later) stored in the data storage unit 227 and transmitting the image circle information to the camera body 100.

[0033] The lens communication unit 229 has electrical contacts and connects to the electrical contacts of the camera communication unit 106 of the attached camera body 100, thereby enabling communication between the interchangeable lens 200 and the camera body 100.

[0034] The camera information management unit 237 stores and manages various information of the camera body 100, which is acquired through communication with the camera body 100 via the lens communication unit 229. This information includes camera setting information, image sensor position information, and movable range information.

[0035] The data storage unit 227 is a non-volatile memory unit that stores optical information such as the zoom range (variable range of focal length), focus range (range of distances at which focus is possible), and variable range of aperture value of the imaging optical system 210.

[0036] Figure 2 is a block diagram showing the detailed configuration of the image stabilization control unit 103 of the camera body 100 and the image stabilization control unit 224 of the interchangeable lens 200. First, the configuration of the image stabilization control unit 103 of the camera body 100 will be described.

[0037] The image shake correction control unit 103 of the camera body 100 generates a shake detection signal by adding the camera shake detection signal from the camera shake detection unit 105 and the representative motion vector amount from the motion vector detection unit 109. Of the camera shake, shakes in the relatively high frequency range are suitable for detection by the angular velocity sensor, while shakes in the relatively low frequency range are suitable for detection as motion vectors. Therefore, by adding the camera shake detection signal, which is the detection signal from the camera shake detection unit 105, and the representative motion vector amount, which is the detection signal from the motion vector detection unit 109, camera shakes in a wide frequency range can be detected. The camera integration unit 161 of the image shake correction control unit 103 of the camera body 100 converts the input angular velocity signal into an angle signal by integrating it. In this embodiment, a pseudo-integral low-pass filter is used in the camera integration unit 161 (hereinafter referred to as "integral LPF").

[0038] The camera shake correction amount calculation unit 162 calculates a correction amount to cancel the shake angle, taking into account the frequency band of the converted shake angle and the range in which the image sensor 101 can be driven. A specific example of this process is to perform a bandpass filter process on the input angle signal to extract only the specific frequency band that is the target of shake correction.

[0039] The camera control method determination unit 166 determines whether to perform the coordinated control, in which IBIS and OIS share the responsibility for correcting image blur, using the first coordinated method or the second coordinated method. In this embodiment, if at least one of the camera body 100 or the interchangeable lens 200 does not support the second coordinated method, the first coordinated method is selected; if both support the second coordinated method, the second coordinated method is selected.

[0040] Here, the first and second coordination methods will be explained using Figures 3 and 4. Figure 3 is a diagram illustrating the first coordination method, and Figure 4 is a diagram illustrating the second coordination method. In the graphs in Figures 3 and 4, the horizontal axis represents the camera shake amount, which is the angle signal obtained by integrating the angular velocity signal of the camera shake detected by the camera shake detection unit 105 with the camera integration unit 161, and the vertical axis represents the shake correction amount. When performing camera shake correction, it is preferable that the camera shake amount and the shake correction amount are equal, as this eliminates residual camera shake.

[0041] First, the first coordination method will be explained using Figure 3. In the first coordination method, the ratio of OIS to IBIS is constant regardless of the amount of camera shake, and the amount of shake correction performed by OIS and the amount of shake correction performed by IBIS increase as the amount of camera shake increases until they reach the limits (upper limits) of the movable ranges of OIS and IBIS. This ratio is determined by the size (length) of the movable ranges of OIS and IBIS. However, the movable range here refers not to the distance that the image stabilization lens 204 or image sensor 101 can actually drive, but to the distance over which relative movement between the subject image and the image plane of the image sensor 101 can be caused by driving the image stabilization lens 204 or image sensor 101. As an example, Figure 3 shows the case where the movable ranges of OIS and IBIS are the same, and the correction ratio of OIS and IBIS is always 50% regardless of the amount of camera shake. In this case, the relative movement direction between the subject image and the image plane of the image sensor 101 caused by OIS coincides with the relative movement direction between the subject image and the image plane of the image sensor 101 caused by IBIS, and the image blur caused by camera shake is corrected by both OIS and IBIS.

[0042] For example, when using an image stabilization lens 204 that moves upward when moved upward, the OIS moves the image stabilization lens 204 downward when upward camera shake occurs, causing the subject image to move downward relative to the image plane of the image sensor 101. At the same time, the IBIS moves the image sensor 101 upward, causing the subject image to move downward relative to the image plane of the image sensor 101. In other words, the image plane is moved upward relative to the subject image. This method of moving the OIS and IBIS so that relative movement occurs in the same direction at the same ratio regardless of the amount of camera shake is called the first coordinated method. In this case, the amount of image stabilization performed by the OIS and the amount of image stabilization performed by the IBIS do not exceed the amount of shake actually detected.

[0043] Next, the second coordination method will be explained using Figure 4. In the second coordination method, in the section (section AB in Figure 4) where the amount of movement of the image stabilization lens 204 corresponding to the amount of shake correction required to correct the amount of camera shake occurring is less than or equal to the length of the movable range of the image stabilization lens 204 (length from the reference position to the movable end of the image stabilization lens 204), the OIS drives the image stabilization lens 204 with a shake correction amount that exceeds the amount required to correct the amount of camera shake occurring (an excessive correction amount). In other words, section AB is the section in which the amount of shake occurring can be corrected by OIS alone. This control, which drives the image stabilization lens 204 with a shake correction amount that exceeds the amount required to correct the amount of camera shake occurring, is called overcorrection control. During this time, the IBIS performs inverse correction control so that the amount of shake correction by OIS exceeds the amount required to correct the amount of camera shake occurring, i.e., the excessive correction amount. At this time, the relative movement direction between the subject image and the image plane of the image sensor generated by the OIS and the IBIS are in opposite directions. Thus, in the second cooperative control method, the method in which OIS performs overcorrection and IBIS cancels out the overcorrection is called the first control method (cooperative control method).

[0044] On the other hand, when the amount of camera shake correction required to compensate for the amount of camera shake occurring exceeds the range of motion of the image shake correction lens 204, and the shake occurring cannot be corrected by OIS alone (section C), the amount of camera shake that cannot be corrected by OIS is corrected by IBIS. At this time, the direction of movement of the image sensor 101 is in a direction that reduces the movement between the subject image and the image sensor 101 due to the amount of camera shake occurring, and the relative direction of movement between the subject image and the image plane of the image sensor 101, which are generated by OIS and IBIS, coincides. Thus, in the second cooperative method, when a shake of a magnitude that cannot be corrected by the first control method occurs, a method is used to control the system so that the relative direction of movement between the subject image and the image plane of the image sensor, which are generated by OIS and IBIS, coincides. This method is called the second control method (cooperative control method).

[0045] If, while using the first control method, a camera shake amount exceeding the amount corresponding to the boundary between sections B and C is detected (i.e., the camera shake amount changes from below a predetermined value to above a predetermined value), the system switches from the first control method, which performs inverse correction, to the second control method, which does not perform inverse correction. On the other hand, if, while using the second control method, a camera shake amount less than the amount corresponding to the boundary between sections B and C is detected, the system switches from the second control method, in which OIS and IBIS perform correction in the same direction, to the first control method, which performs inverse correction.

[0046] The following provides a detailed explanation of the control systems used in each section. Section A is the section in which OIS is overcorrected and IBIS is inversely corrected, and in particular, the section in which OIS is operated at the maximum ratio. In the example shown in Figure 4, the maximum ratio is 200%, and the correction amount is twice the amount of correction required to correct the image blur caused by the resulting camera shake. At this time, due to the overcorrection of OIS, image blur will occur if left as is, so the correction ratio of IBIS is set to -100%, and the sum of the correction ratios of OIS and IBIS is set to 100%. Note that a correction ratio of -100% means driving with a shake correction amount that produces the same amount of shake in the same direction as the image blur caused by the resulting camera shake, or in other words, driving with a shake correction amount that amplifies the image blur caused by camera shake by a factor of two.

[0047] Section B is the section from the end of Section A until the amount of shake correction used to compensate for the resulting camera shake exceeds the length of the movable range of the image stabilization lens 204. Section B is the section in which the OIS correction ratio and the inverse IBIS correction amount gradually decrease, and can also be described as the section in which the absolute values ​​of the OIS correction ratio and the IBIS correction ratio gradually decrease. In the example in Figure 4, at the boundary point between Section A and Section B, the OIS correction ratio is 200%, and the IBIS correction ratio is -100%. From the boundary point between Section A and Section B to the boundary point between Section B and Section C, the absolute value of the correction ratio decreases monotonically, and the OIS correction ratio changes from 200% to 100%, and the IBIS correction ratio changes from -100% to 0%. At this time, by controlling the sum of the OIS and IBIS correction ratios to be 100%, it is possible to correct the shake without excess or deficiency.

[0048] In section C, OIS cannot correct the shake any further, so IBIS corrects the remaining amount. During this period, the amount of shake correction by OIS remains constant, while only the amount of shake correction by IBIS increases. Therefore, in section C, the correction ratio of OIS and IBIS is not constant; when the amount of camera shake is large, the correction ratio of IBIS (β) increases and the correction ratio of OIS (α) decreases. Even in this section, by controlling the sum of the correction ratios of OIS and IBIS to be 100%, it is possible to correct the shake without excess or deficiency.

[0049] The second coordination method shown in Figure 4 can reduce image blur at the edges of the image compared to the first coordination method.

[0050] For example, when shooting video while carrying camera body 100, the camera body 100 experiences significant shaking due to the impact of the photographer landing, etc. In this case, the shaking that occurs in camera body 100 is corrected by OIS and IBIS. However, in the first cooperative method, the optimal amount of image blur correction differs between the low-profile center of the image and the high-profile periphery of the image. Therefore, if the optimal correction is applied to the image blur in the center of the image, the image blur in the periphery of the image may become noticeable. This is particularly noticeable when shooting with a wide-angle lens or when shooting video. The second cooperative method can reduce this image blur in the periphery of the image.

[0051] The camera control method determination unit 166 determines whether to use the first or second coordination method, as described with reference to Figures 3 and 4, to control the OIS and IBIS. There may be three or more coordination methods, and the camera control method determination unit 166 may select a coordination method from three or more coordination methods.

[0052] Returning to the explanation of the image blur correction control unit 103 of the camera body 100 using Figure 2, the camera ratio calculation unit 163 acquires the correction ratio handled by IBIS when the sum of the blur correction amounts from OIS and IBIS is set to 100%, and calculates the second correction amount by multiplying this ratio by the first correction amount calculated by the camera blur correction amount calculation unit 162. The correction ratio is acquired based on a cooperative method (first cooperative method or second cooperative method). When using the second cooperative method, the correction ratio is further acquired based on data showing the relationship between camera blur amount and ratio, as shown in the graph in Figure 4, and the detected camera blur amount. In addition, since the first correction amount corresponds to the total correction amount corrected by IBIS and OIS, the second correction amount corrected by IBIS blur correction is calculated by multiplying it by the correction ratio handled by IBIS.

[0053] The camera drive range limit unit 164 performs limit processing if the target position of the image sensor 101, which corresponds to the second correction amount, exceeds the drive limit, and adjusts the correction amount so that it does not exceed the drive limit. The output of the camera drive range limit unit 164 becomes the final target correction amount for IBIS.

[0054] The camera feedback control unit 165 performs feedback control using the current position acquired by the image sensor position detection unit 132 so that the image sensor 101 tracks the target position corresponding to the target correction amount, and drives the image sensor 101 with the shift mechanism 101a to perform vibration isolation control. In this embodiment, the camera feedback control unit 165 performs PID control based on the current position and the target correction amount. However, the feedback control method is not limited to PID control, and P control, PI control, or PD control may also be used.

[0055] The control section determination unit 170 determines, based on the current camera shake amount (output of the camera integration unit 161), which section of the control section shown in Figure 4 is being corrected when the camera control method determination unit 166 selects the second cooperative method. The camera ratio calculation unit 163 also determines which section is being corrected based on the camera shake amount and obtains the correction ratio, so the result of that calculation unit may also be obtained. Conversely, the determination result of the control section determination unit 170 may be output to the camera ratio calculation unit 163, and the camera ratio calculation unit 163 may use this determination result to obtain the ratio.

[0056] Next, the configuration of the image blur correction control unit 224 of the interchangeable lens 200 will be described. The image blur correction control unit 224 of the interchangeable lens 200 receives an angular velocity signal as a shake detection signal from the lens shake detection unit 228. The lens integration unit 251 of the image blur correction control unit 224 converts the input angular velocity signal into an angle signal by integrating it. In this embodiment, an integral LPF is also used in the lens integration unit 251.

[0057] The lens shake correction amount calculation unit 252 calculates a correction amount to cancel the shake angle, taking into account the frequency band of the converted shake angle and the driveable range of the image shake correction lens 204. A specific example of this process is to perform a bandpass filter on the input angle signal to extract only the specific frequency band to be corrected for shake.

[0058] The lens control method determination unit 256, like the camera control method determination unit 166, determines whether to perform the coordinated control of IBIS and OIS using the first coordinated method or the second coordinated method. The determination method is the same as that of the camera control method determination unit 166, and the second coordinated method is selected when both the camera body 100 and the interchangeable lens 200 support the second coordinated method. Alternatively, instead of the lens control method determination unit 256 making the determination, the determination result may be obtained from the camera control method determination unit 166. Conversely, the lens control method determination unit 256 may also act as a substitute for the camera control method determination unit 166 by determining the coordinated method and transmitting the determination result to the camera body 100.

[0059] The lens ratio calculation unit 253 obtains the correction ratio performed by OIS when the sum of the shake correction amounts from IBIS and OIS is set to 100%, and multiplies this ratio by the third correction amount calculated by the lens shake correction amount calculation unit 252 to calculate the fourth correction amount. Similar to the camera ratio calculation unit 163, the correction ratio is obtained based on a cooperative method, and if the cooperative method is the second cooperative method, the correction ratio is obtained based on data showing the relationship between the camera shake amount and the ratio, and the detected camera shake amount. Alternatively, the acquisition of the correction ratio performed by OIS may be performed by the camera ratio calculation unit 163 also acquiring the OIS correction ratio and transmitting it to the lens ratio calculation unit 253, which then receives it. Alternatively, the camera ratio calculation unit 163 may transmit the IBIS correction ratio to the lens ratio calculation unit 253, and the lens ratio calculation unit 253 may acquire the OIS correction ratio based on the IBIS correction ratio received by the lens ratio calculation unit 253. Alternatively, the roles of the lens ratio calculation unit 253 and the camera ratio calculation unit 163 may be reversed, so that the lens ratio calculation unit 253 acquires the correction ratios for IBIS and OIS and transmits the IBIS correction ratio to the camera body 100, or transmits the OIS correction ratio to the camera body 100.

[0060] The lens drive range limit unit 254 performs limit processing if the target position of the image blur correction lens 204, which corresponds to the fourth correction amount, exceeds the drive limit, and adjusts the correction amount so that it does not exceed the drive limit. The output of the lens drive range limit unit 254 becomes the final target correction amount for OIS.

[0061] The lens feedback control unit 255 performs feedback control using the current position acquired by the lens vibration isolation position detection unit 258 so that the image vibration correction lens 204 follows a target position corresponding to the target correction amount, and drives the image vibration correction lens 204 with the shift mechanism 204a to perform vibration isolation control.

[0062] Next, the processing by the motion vector detection unit 109 will be described. The motion vector detection unit 109 determines the motion vector detection method based on the determination result of the control interval determination unit 170. As described above, the second control method of the second cooperative method cannot optically reduce image blur at the edges of the image as well as the first control method. Therefore, when the motion vector detection unit 109 detects a motion vector, if the edges of the image are included in the motion vector detection area, the image blur component at the edges of the image will be detected as a motion vector, and the accuracy of motion vector detection will decrease.

[0063] On the other hand, the first control method can optically reduce image blur in the peripheral areas of the image compared to the second control method, making it possible to detect motion vectors using an image with optically reduced image blur in the peripheral areas. Therefore, in the first control method, the entire image, including the peripheral areas, can be set as the motion vector detection area in the motion vector detection unit 109. In this way, since image blur in the peripheral areas of the image is sufficiently reduced in the first control method, the motion vector detection unit 109 can accurately detect motion vectors by setting the entire image, including the peripheral areas, as the motion vector detection area. In other words, while the motion vector detection unit 109 cannot detect motion vectors by setting the entire image, including the peripheral areas, as the motion vector detection area when controlled by the second control method, it can detect motion vectors by setting the entire image, including the peripheral areas, as the motion vector detection area when controlled by the first control method.

[0064] For the reasons stated above, in this embodiment, the motion vector detection method in the second cooperative method is determined based on the correction control interval (interval A to interval C) shown in Figure 4.

[0065] Next, we will explain how to determine the motion vector detection method based on the correction control interval. Here, we will explain four types of methods using the division of the motion vector detection region shown in Figure 5 as an example. Note that the following motion vector detection methods will be used to obtain representative motion vectors, so these motion vector detection methods can be rephrased as methods for obtaining representative motion vectors.

[0066] <Method 1 for detecting motion vectors> The motion vector detection method 1 will be explained using Figures 5 and 6. The motion vector detection unit 109 acquires the determination result of the control interval determination unit 170. If the motion vector detection region is as shown in Figure 5(a), and corresponds to intervals A and B in Figure 4, and the determination result is obtained indicating that OIS and IBIS are being controlled by the first control method that performs inverse correction, the motion vector detection unit 109 sets regions 501 and 502, i.e., the entire image, as the motion vector detection region, as shown in Figure 6(a), and detects the motion vector.

[0067] On the other hand, if the control section determination unit 170 obtains a determination result indicating that the control is in operation using the second control method, corresponding to section C in Figure 4, the motion vector detection unit 109 sets only region 502 as the motion vector detection region (region 501 is not included in the motion vector detection region) as shown in Figure 6(a), and detects the motion vector.

[0068] Furthermore, for example, if the motion vector detection region is as shown in Figure 5(b), and the control section determination unit 170 obtains a determination result indicating that it corresponds to section A in Figure 4 and that control is being performed using the first control method, the motion vector detection unit 109 sets regions 505, 506, and 507, i.e., the entire image, as the motion vector detection region, as shown in Figure 6(b). If a determination result indicating that it corresponds to section B in Figure 4 is obtained, regions 506 and 507 are set as the motion vector detection region. If a determination result indicating that it corresponds to section C in Figure 4 is obtained, only region 507 is set as the motion vector detection region. The motion vector detection unit 109 then detects motion vectors within the set motion vector detection region.

[0069] <Method 2 for detecting motion vectors> Next, the motion vector detection method 2 will be explained using Figures 5 and 7. In motion vector detection method 2, the motion vector detection areas are weighted according to each detection area. At that time, a weight value for each detection area is calculated based on the judgment result described above. Then, when the motion vector detection unit 109 generates a histogram of multiple motion vector detection amounts detected from each detection area, it multiplies each detection area by its respective weight value and aggregates the frequency of the interval. For example, in the case of the motion vector detection areas shown in Figure 5(a), the weight values ​​for the motion vector detection areas shown in Figure 7(a) are set based on the judgment result of the control interval judgment unit 170.

[0070] In other words, if the motion vector detection region is as shown in Figure 5(a), and the control interval determination unit 170 obtains a determination result indicating that the control is being performed using the first control method which performs inverse correction and corresponds to intervals A and B in Figure 4, the motion vector detection unit 109 sets 1.0 to relatively increase the weighting of the motion vector detection region at the edge of the image (region 501 in Figure 5(a)), as shown in Figure 7(a).

[0071] On the other hand, if the control interval determination unit 170 obtains a determination result indicating that control is being performed using the second control method, corresponding to interval C in Figure 4, the motion vector detection unit 109 sets the weighting of the motion vector detection area at the edge of the image (area 501 in Figure 5(a)) to 0.1, as shown in Figure 7(a). Note that the motion vector detection area at the center of the image (area 502 in Figure 5(a)) is an area with a low image height, so the effect of image blur is small, and therefore the weighting value is set to 1.0 regardless of the control interval.

[0072] Furthermore, for example, if the motion vector detection region is as shown in Figure 5(b), and the control interval determination unit 170 obtains a determination result indicating that it corresponds to interval A in Figure 4 and that control is being performed using the first control method, the motion vector detection unit 109 sets 1.0 to relatively increase the weighting of the motion vector detection region at the edge of the image (regions 505 and 506 in Figure 5(b)), as shown in Figure 7(b). If a determination result corresponding to interval B in Figure 4 is obtained, 0.5 is set for region 505 and 0.8 for region 506 to relatively decrease the weighting of the motion vector detection region at the edge of the image (regions 505 and 506 in Figure 5(b)). Region 506 has a lower image height and is less affected by image blur, so the weighting of region 506 is set to a larger value compared to region 505.

[0073] Furthermore, if the control interval determination unit 170 obtains a determination result indicating that control is being performed using the second control method, corresponding to interval C in Figure 4, the motion vector detection unit 109 sets the weighting of the motion vector detection area in the peripheral part of the controlled image (areas 505 and 506 in Figure 5(b)) to 0.1 for area 505 and 0.5 for area 506, as shown in Figure 7(b). The motion vector detection area in the central part of the image (area 507 in Figure 5(b)) is a low-image-height area where the effect of image blur is small, so the weighting is set to 1.0 regardless of the control interval.

[0074] Thus, when the motion vector detection unit 109 obtains a determination result from the control interval determination unit 170 indicating that control is being performed using the second control method, corresponding to interval C in Figure 4, it relatively reduces the weighting of the vector detection region in the peripheral part of the image (for example, region 501 in Figure 5(a)). As a result, when OIS and IBIS are being controlled in a control interval where image blur correction is weak in the peripheral part of the image, corresponding to interval C in Figure 4, the motion vector detection unit 109 can reduce the proportion of motion vector detection in the peripheral part of the image compared to the proportion of motion vector detection in the central part of the image when generating the histogram, thereby improving the accuracy of motion vector detection.

[0075] <Method 3 for detecting motion vectors> Next, the motion vector detection method 3 will be explained using Figures 5 and 8. In motion vector detection method 3, the gain multiplied by the motion vector detection amount in the motion vector detection region at the edge of the image is changed. In this case, the correction gain multiplied by the motion vector detection amount in the motion vector detection region at the edge of the image is called g.

[0076] For example, as shown in Figure 8(a), the correction gain is set to g1 when controlled by the first control method and to g2 when controlled by the second control method, with the relationship between the two being g1 > g2. When controlled by the first control method, the motion vector detection amount in the motion vector detection region at the edge of the image is multiplied by the correction gain g1, and when controlled by the second control method, the motion vector detection amount in the motion vector detection region at the edge of the image is multiplied by the correction gain g2.

[0077] Furthermore, as shown in Figure 8(b), the correction gain g may be varied according to the amount of image blur correction in the section controlled by the first control method. Figure 8(b) shows an example in which, in the section controlled by the first control method, the correction gain is decreased from g1 to g2 in proportion to the increase in the amount of image blur correction.

[0078] Furthermore, as shown in Figure 8(c), in the section controlled by the first control method, the correction gain may be changed between the section in which the OIS is moved at the maximum ratio (section A in Figure 4) and the section in which the IBIS corrects by reducing the inverse correction amount as the amount of image blur correction increases (section B in Figure 4). In Figure 8(c), an example is shown where the correction gain is g1 (a constant value) in section A, and in section B, the correction gain is reduced from g1 to g2 in proportion to the increase in the amount of image blur correction. In section B, the OIS cannot be moved at the maximum ratio, so the effect of optical image stabilization (OIS, IBIS) in reducing image blur in the peripheral parts of the image is smaller compared to section A. Therefore, in section B, the correction gain g of the motion vector detection amount is gradually reduced in proportion to the amount of image blur correction. As a result, in the case of section C in Figure 4, where the image blur correction in the peripheral parts of the image is weak, the detection accuracy of the motion vector calculated by the motion vector detection unit 109 can be improved by reducing the correction gain in the peripheral parts of the image.

[0079] <Method 4 for detecting motion vectors> In motion vector detection methods 1 to 3, the motion vector detection method of the motion vector detection unit 109 was determined based on the image blur correction state of the peripheral areas of the image where the image height is high. In contrast, in motion vector detection method 4, an arbitrary subject is selected within the screen, and the motion vector detection method is determined based on the image blur correction state of the selected arbitrary subject.

[0080] For example, suppose the user selects a "car" as the subject within area 513 shown in Figure 5(c) displayed on the camera screen. The subject can be selected by touching the screen with a finger, by displaying a selection frame on the screen and using buttons such as the camera's directional pad, or by any other method. Then, image blur correction is performed so that any blur in the image of the subject selected by the user is corrected.

[0081] When a user selects a subject located at a high image height, such as the selected subject (for example, the "car" in area 513 in Figure 5(c)), image blur in subjects in areas other than the selected subject (for example, the "human face" in area 512) may increase. However, priority is given to correcting the image blur of the subject selected by the user. That is, the correction amount for image blur correction is calculated using the user-selected arbitrary area as the reference for image blur correction, and the image blur in the arbitrary area is reduced compared to other areas. The correction amount at this time can be determined by adding a preset offset value according to the image height of the arbitrary area to the correction amount for image blur correction calculated using the center of the image as the reference for image blur correction. Based on the image blur correction state of the selected subject, the motion vector detection unit 109 determines how to detect the motion vector.

[0082] If the image blur of the selected subject (the "car" in region 513 in Figure 5(c)) is small, the region 513 surrounding the selected subject is set as the motion vector detection region. Alternatively, the weighting of the motion vector detection region (region 513) around the subject may be increased, or the motion vector detection region around the subject may be set. If the user selects a subject that is at a high position in the image height, such as the selected subject (for example, the "car" in region 513 in Figure 5(c)), the image blur of subjects in regions other than the selected subject (for example, the "human face" in region 512) may increase, but priority is given to correcting the image blur of the subject selected by the user. That is, the correction amount for image blur correction is calculated using an arbitrary region selected by the user as the reference for image blur correction, and the image blur of the arbitrary region is reduced compared to other regions. The correction amount at this time should be calculated by adding a preset offset value according to the image height of the arbitrary region to the correction amount for image blur correction calculated using the center of the image as the reference for image blur correction.

[0083] Based on the image blur correction status of the selected subject, the motion vector detection unit 109 determines how to detect the motion vector. If the image blur of the selected subject (the "car" in region 513 in Figure 5(c)) is small, the region 513 surrounding the selected subject is set as the motion vector detection region. Alternatively, the weighting of the motion vector detection region (region 513) surrounding the subject may be increased compared to other regions, or the correction gain of the motion vector detection region (region 513) surrounding the subject may be increased compared to other regions. In other words, motion vector detection methods 1 to 3 set the surrounding region based on the distance (image height) from the center of the image, but motion vector detection method 4 sets the surrounding region based on the distance from the position of the selected subject. Therefore, even if the position of the selected subject is near the corners of the image, it may be set as a surrounding region even if it is near the center of the image. This improves the accuracy of motion vector detection for any subject selected by the user.

[0084] The motion vectors detected by the method described above may be used for image blur correction of the selected subject, or for subject tracking control of the selected subject. Furthermore, motion vector detection method 4 modifies the settings related to the motion vectors in each region (detection region settings, weighting, gain, etc.) based on the position of the selected subject. Therefore, motion vector detection method 4 may be applied when the second cooperative method is not being used (for example, when the first cooperative method is being used, or when only one of OIS or IBIS is being performed).

[0085] Next, the image blur correction process performed in this embodiment will be explained using the flowcharts in Figures 9 and 10. Figure 9 is a flowchart of the image blur correction process performed in the camera body 100 in this embodiment, and Figure 10 is a flowchart of the image blur correction process performed in the interchangeable lens 200 in this embodiment.

[0086] First, the image blur correction process in the camera body 100 will be explained using Figure 9. Unless otherwise specified, this process is performed by the image blur correction control unit 103. First, in S101, the image blur correction control unit 103 receives instructions from the camera microcontroller 102 and starts image blur correction control.

[0087] In S102, the camera control method determination unit 166 of the image blur correction control unit 103 determines whether the attached interchangeable lens 200 is an interchangeable lens that supports the second cooperative method, based on information such as the model number of the interchangeable lens. If it supports the second cooperative method, the process proceeds to S103, where it is determined that the following image blur correction control will be performed using the second cooperative method, and the second cooperative method is set as the cooperative method to be used for image blur correction control. If the attached interchangeable lens does not support the second cooperative method, the process proceeds to S104, where it is determined that the following image blur correction control will be performed using the first cooperative method, and the first cooperative method is set as the cooperative method to be used for image blur correction control. Regardless of whether the process proceeds to S103 or S104, the camera control method determination unit 166 outputs the determination result to the camera ratio calculation unit 163, the camera drive range limit unit 164, and the control section determination unit 170.

[0088] Once the cooperative method used for image blur correction control is set, the process proceeds to S105. The camera microcontroller 102 acquires lens information from the interchangeable lens 200 via the camera communication unit 106 and stores the lens information in the lens information management unit 129. The details of the lens information to be stored are as described above, so no further explanation is provided. Next, in S106, the camera microcontroller 102 transmits camera information to the interchangeable lens 200 via the camera communication unit 106. The camera information to be transmitted is as described above, so no further explanation is provided.

[0089] In S107, the control section determination unit 170 determines whether the OIS and IBIS were controlled by the first control method of the second cooperative method in the previous image blur correction processing cycle (whether the control section corresponds to section A or section B in Figure 4). In other words, this determination determines whether it corresponds to the second cooperative method and whether the current amount of camera shake is in a section where inverse correction is performed by OIS and IBIS to move the relative position between the subject image and the image sensor in opposite directions. This determination is made by referring to the result of S113 in the previous image blur correction processing cycle. If it is determined to be the first control method of the second cooperative method, the process proceeds to S108. If it is determined not to be the first control method of the second cooperative method, that is, in this embodiment, if it is determined to be the first cooperative method or the second control method of the second cooperative method, the process proceeds to S109.

[0090] In S108, the motion vector detection unit 109 sets the motion vector detection area to the entire image, including the peripheral areas. On the other hand, in S109, the motion vector detection unit 109 sets the motion vector detection area to only the central part of the image. Once the motion vector detection area is set in S108 and S109, the process proceeds to S110.

[0091] In this description, we have set the motion vector detection area using motion vector detection method 1 in the detection area shown in Figure 5(a). However, weighting values ​​and gains may also be set using motion vector detection methods 2 or 3 as described above. Furthermore, in the case of the detection area shown in Figure 5(b), the first control method is further divided into area A and area B for setting. In addition, when the motion vector detection area is set using motion vector detection method 4, the vector detection area becomes a predetermined range including the selected subject (for example, an area where the distance from the selected subject is less than a predetermined value).

[0092] In S110, the motion vector detection unit 109 detects motion vectors in the motion vector detection region set in S108 or S109. Details of motion vector detection are as described above and will be omitted here. In step S111, the image shake correction control unit 103 acquires the detection result from the camera shake detection unit 105. The acquired shake detection result is input to the camera integration unit 161. In S112, the camera integration unit 161 performs pseudo-integration by applying an LPF (low-pass filter) to the signal obtained by adding the detection result from the input camera shake detection unit 105 and the detection result from the motion vector detection unit 109.

[0093] In S113, the control section determination unit 170 determines whether the control section (corresponding to section A or section B in Figure 4) is a control section in which the OIS and IBIS are controlled by the first control method of the second cooperative method, and outputs the determination result to the camera ratio calculation unit 163. This determination is made based on the settings in S103 or S104 and the magnitude of the camera shake, as described above.

[0094] In S114, the camera shake correction amount calculation unit 162 calculates the shake correction amount based on the shake amount input from the camera integration unit 161. In S115, the camera ratio calculation unit 163 acquires the correction ratio to be handled by the camera body 100 based on the determination result by the camera control method determination unit 166. Furthermore, the camera ratio calculation unit 163 calculates the amount of shake correction by IBIS by multiplying the acquired correction ratio by the shake correction amount input from the camera shake correction amount calculation unit 162. The method for acquiring the correction ratio is as described above, so the details are omitted, but in the case of the second control method, the correction ratio is acquired according to which control section (any of sections A to C) it corresponds to.

[0095] In S116, the camera drive range limit unit 164 performs limit processing if the target position of the image sensor 101 corresponding to the calculated IBIS shake correction amount exceeds the limit of the drive range of the image sensor 101, and calculates the final target correction amount for IBIS.

[0096] In S117, the camera feedback control unit 165 controls the shift mechanism 101a of the image sensor 101 based on the position of the image sensor 101 detected by the image sensor position detection unit 132 and the target correction amount of the IBIS input from the camera drive range limit unit 164. As a result, the camera feedback control unit 165 controls the position of the image sensor 101 and performs vibration damping drive processing using IBIS.

[0097] In S118, the image stabilization control unit 103 of the camera body 100 determines whether to continue image stabilization control by IBIS, and if so, returns to S102. Alternatively, assuming that the attached interchangeable lens 200 remains unchanged, the process may return to S105. In this case, the process may be restarted from S102 when a change in the attached interchangeable lens 200 is detected. If the user turns off the image stabilization function by IBIS, or if the camera body 100 switches to playback mode, or if it is determined that image stabilization will not be continued, this process is terminated.

[0098] Next, Figure 10 will be used to explain the image blur correction process in the interchangeable lens 200. Unless otherwise specified, this process is performed by the image blur correction control unit 224. First, in S201, the image blur correction control unit 224 receives instructions from the lens microcontroller 226 and starts image blur correction control.

[0099] In S202, the lens control method determination unit 256 of the image blur correction control unit 224 determines whether the attached camera body 100 supports the second coordination method based on information such as the model number of the attached camera body 100. If the attached camera body 100 supports the second coordination method, the process proceeds to S203, where it is determined that the following image blur correction control will be performed using the second coordination method, and the second coordination method is set as the coordination method to be used for image blur correction control. If the attached camera body 100 does not support the second coordination method, the process proceeds to S204, where it is determined that the following image blur correction control will be performed using the first coordination method, and the first coordination method is set as the coordination method to be used for image blur correction control. In addition, regardless of whether the process proceeds to S203 or S204, the lens control method determination unit 256 outputs the determination result to the lens ratio calculation unit 253 and the lens drive range limit unit 254. Alternatively, instead of using the camera model number to determine whether the camera supports the second coordination method, the camera body 100 may be used to obtain a determination result indicating whether to perform image blur correction control using the first coordination method or the second coordination method, and the coordination method may be set according to the obtained determination result.

[0100] Once the cooperative method used for image blur correction control is set, the process proceeds to S205, where the lens microcontroller 226 transmits lens information to the camera body 100 via the lens communication unit 229. This corresponds to the process shown in S105 of Figure 9. Next, in S206, the lens microcontroller 226 acquires camera information from the camera body 100 via the lens communication unit 229 and stores the acquired camera information in the camera information management unit 237. This corresponds to the process in S106 of Figure 9.

[0101] In S207, the image blur correction control unit 224 acquires detection results from the lens shake detection unit 228. The acquired shake detection results are input to the lens integration unit 251. In S208, the lens integration unit 251 performs pseudo-integration by applying LPF processing to the detection result from the input lens shake detection unit 228. In S209, the lens shake correction amount calculation unit 252 calculates the shake correction amount based on the shake amount input from the lens integration unit 251. The details of the shake correction amount calculation are as described above and are therefore omitted here.

[0102] In S210, the lens ratio calculation unit 253 obtains the correction ratio to be handled by the interchangeable lens 200 based on the determination result by the lens control method determination unit 256. Then, the OIS shake correction amount is calculated by multiplying the acquired correction ratio by the shake correction amount input from the lens shake correction amount calculation unit 252. The method for obtaining the correction ratio is as described above, so the details are omitted, but in the case of the second control method, the correction ratio is obtained according to which control section (any of sections A to C) it corresponds to. For this reason, the determination result by the control section determination unit 170 obtained in S206 is referred to.

[0103] In S211, the lens drive range limit unit 254 performs limit processing if the target position of the image stabilization lens 204 corresponding to the calculated OIS shake correction amount exceeds the limit of the drive range of the image stabilization lens 204.

[0104] In S212, the lens feedback control unit 255 controls the shift mechanism 204a of the image stabilization lens 204 based on the position of the image stabilization lens 204 detected by the lens vibration isolation position detection unit 258 and the target correction amount of OIS input from the lens drive range limit unit 254. As a result, the lens feedback control unit 255 controls the position of the image stabilization lens 204 and performs the OIS drive process.

[0105] In S213, the image stabilization control unit 224 of the interchangeable lens 200 determines whether to continue image stabilization control by OIS, and if so, returns to S202. It may also return to S205, assuming that the attached camera body 100 does not change. In this case, it may be configured to restart from S202 when it detects that the attached camera body 100 has changed. If it is determined that image stabilization will not be continued, for example, when the user has turned off the OIS image stabilization function, this process is terminated.

[0106] In this embodiment, the target correction amount for IBIS is calculated by the camera body 100 and the target correction amount for OIS is calculated by the interchangeable lens 200. However, it is also possible for either the camera body 100 or the interchangeable lens 200 to calculate the target correction amounts for both IBIS and OIS.

[0107] Furthermore, in this embodiment, the target correction amount was calculated based on the amount of shake calculated by pseudo-integrating a signal obtained by adding the motion vector detection amount and the angular velocity sensor detection amount, but the method for calculating the target correction amount is not limited to this. For example, the target correction amount may be calculated based on the acceleration detected by the acceleration sensor, or the amount of shake may be calculated using multiple pieces of information such as motion vector detection, angular velocity sensor, and acceleration sensor.

[0108] Furthermore, in this embodiment, a camera system (imaging system) consisting of a camera body 100 and a detachable interchangeable lens 200 attached to the camera body has been described. However, this embodiment can also be applied to a camera with an integrated lens, as long as it has OIS and IBIS functions. In addition, if the camera unit has OIS and IBIS functions, it can be applied to various electronic devices such as smartphones, tablets, wearable devices, and drones. Moreover, for example, some or all of the processing performed by the image blur correction control unit 103 of the camera body 100 and the image blur correction control unit 224 of the interchangeable lens 200 in this embodiment may be performed by an external device or the cloud.

[0109] According to this embodiment, even when the amount of image blur varies depending on the image height, motion vectors can be accurately detected using the subject image output from the imaging device. In particular, even when the amount of image blur varies depending on the image height, which is a particularly noticeable effect when shooting with a wide-angle lens, motion vectors can be accurately detected.

[0110] <Other Embodiments> Furthermore, the present invention may be applied to a system consisting of multiple devices or to a device consisting of a single device.

[0111] Furthermore, the present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0112] <Summary> This embodiment includes the following configuration.

[0113] (Item 1) A control device for controlling image blur correction comprising: a first correction means that corrects image blur by driving a corrective lens included in the imaging optical system in a direction perpendicular to the optical axis; and a second correction means that corrects image blur by driving an image sensor that converts light incident through the imaging optical system into an image signal in a direction perpendicular to the optical axis, wherein the first correction means corrects image blur by driving the image sensor in a direction perpendicular to the optical axis. A first acquisition means for acquiring the amount of vibration detected by the vibration detection means, A determination means for determining a method for calculating a correction amount for controlling the first correction means and the second correction means, based on the amount of the deflection and the control method of the first correction means and the second correction means, A calculation means that calculates the correction amounts of the first correction means and the second correction means based on the amount of the aforementioned fluctuation using the calculation method, A second acquisition means for acquiring a representative motion vector based on the motion vector detected from the image output from the image sensor, A setting means for setting a method for acquiring a representative motion vector by the second acquisition means based on the calculation method described above, A control device characterized by having the following features. (Item 2) The control device according to item 1, characterized in that the determination means determines whether the first correction means and the second correction means are compatible with a cooperative control method that includes overcorrection, which corrects the first correction means by an excess amount exceeding the correction amount within a range in which the first correction means can be driven, and inverse correction, which cancels out the excess correction by the second correction means, and if it is compatible with the cooperative control method, it further determines the calculation method according to the amount of vibration. (Item 3) When the first correction means and the second correction means correspond to the cooperative control method, the determination means determines the amount of the swing. When the amount of shake is such that image blur correction can be performed by performing the aforementioned overcorrection and inverse correction, a first calculation method that performs the aforementioned overcorrection and inverse correction is determined as the calculation method. If the correction amount exceeds the range within which the first correction means can be driven, a second calculation method is determined in which the correction amount of the first correction means is driven by the maximum drive amount that can be driven, and the deficit is allocated as the correction amount of the second correction means. The control device according to item 2, characterized in that (Item 4) The setting means is, In the case of the first calculation method, the method for acquiring the representative motion vector is set based on the motion vector detected from a region including a first region including the center of the image and a second region including the peripheral part of the image, In the case of the second calculation method, the method for obtaining the representative motion vector is set based on the motion vector detected from the region that includes the first region but does not include the second region. The control device according to item 3, characterized in that (Item 5) The setting means is, In the first calculation method described above, the method for acquiring the representative motion vector is set by assigning the same weight to the motion vectors detected in the first region including the center of the image and the second region of the peripheral part of the image. In the case of the second calculation method described above, the method for obtaining the representative motion vector is set by assigning a smaller weight to the motion vector obtained from the second region than to the motion vector detected from the first region. The control device according to item 4, characterized in that it is a control device. (Item 6) The control device according to item 4, characterized in that, in the case of the first calculation method, the method for acquiring the representative motion vector is set by applying a higher gain to the motion vector than in the case of the second calculation method. (Item 7) When the first correction means and the second correction means correspond to the cooperative control method, the determination means determines the amount of the swing. In the case where the amount of shake is such that image blur correction can be performed by driving the first correction means within the range in which it can be driven in the overcorrection and performing the inverse correction, the calculation method is determined to be the first calculation method which performs the overcorrection and the inverse correction. In the case where the amount of shake is such that image blur correction can be achieved by driving the first correction means with the maximum drive amount that can be driven in the overcorrection and performing the inverse correction, a second calculation method is determined as the calculation method, which performs the overcorrection and the inverse correction. If the correction amount exceeds the range in which the first correction means can be driven, a third calculation method is determined in which the correction amount of the first correction means is driven by the maximum drive amount that can be driven, and the deficit is allocated as the correction amount of the second correction means. The control device according to item 2, characterized in that (Item 8) The setting means is, In the first calculation method described above, the method for acquiring the representative motion vector is set based on motion vectors detected from a region including a first region including the center of the image, a second region that is peripheral to the image beyond the first region, and a third region that is even further peripheral to the image beyond the second region. In the case of the second calculation method described above, the method for acquiring the representative motion vector is set based on the motion vector detected from the region including the first region and the second region, In the case of the third calculation method, the method for acquiring the representative motion vector is set based on the motion vector detected from the first region. The control device according to item 7, characterized in that it is a control device. (Item 9) The setting means is, In the case of the first calculation method described above, the method for obtaining the representative motion vector is set by assigning the same weight to the motion vectors detected from a region including a first region containing the center of the image, a second region that is peripheral to the image beyond the first region, and a third region that is even further peripheral to the image beyond the second region. In the case of the second calculation method described above, the method for obtaining the representative motion vector is set by assigning a smaller weight to the motion vector obtained from the second region than to the motion vector detected from the first region, and by assigning a smaller weight to the motion vector obtained from the third region than to the motion vector detected from the second region. In the third calculation method described above, the motion vector obtained from the second region is given a smaller weight than the motion vector detected from the first region, the motion vector obtained from the third region is given a smaller weight than the motion vector detected from the second region, and the motion vectors detected from the second and third regions are given smaller weights than in the second calculation method, thereby setting the method for obtaining the representative motion vector. The control device according to item 7, characterized in that it is a control device. (Item 10) The setting means is, In the case of the second calculation method, the method for obtaining the representative motion vector is set by applying a lower gain to the motion vector than in the case of the first calculation method. In the third calculation method, a lower gain than in the second calculation method is applied to the motion vector to set the method for acquiring the representative motion vector. The control device according to item 7, characterized in that it is a control device. (Item 11) The system further includes means for the user to select any region in the image, The control device according to any one of items 1 to 10, characterized in that when the area is selected, the setting means sets the method for acquiring the representative motion vector based on the motion vector detected from the selected area, regardless of the calculation method. (Item 12) The control device according to any one of items 1 to 11, characterized in that the calculation means further calculates the correction amounts of the first correction means and the second correction means using the representative motion vector. (Item 13) The control device according to any one of items 1 to 11, further comprising a second calculation means for calculating a drive amount for tracking a subject using the aforementioned representative motion vector. (Item 14) The control device according to any one of items 2 to 10, characterized in that, if at least one of the first correction means and the second correction means does not correspond to the cooperative control method, the setting means sets a predetermined method for acquiring a representative motion vector regardless of the calculation method. (Item 15) A control device for controlling image blur correction, which corrects image blur by driving at least one of a corrective lens included in an imaging optical system and an image sensor that converts light incident through the imaging optical system into an image signal in a direction perpendicular to the optical axis, An operating means for the user to select an arbitrary region in the image output from the image sensor to be used as the reference for image blur correction, An acquisition means for acquiring a representative motion vector based on motion vectors detected from multiple regions of an image output from the image sensor after the image blur correction has been performed, A setting means for changing the settings relating to the motion vectors of the plurality of regions according to the position of the selected arbitrary region, A control device characterized by having the following features. (Item 16) A control device described in any one of items 1 to 15, An imaging device characterized by having the aforementioned image sensor. (Item 17) The imaging device described in item 16, An imaging system characterized by having the aforementioned imaging optical system. (Item 18) A control method for correcting image blur by a first correction means that corrects image blur by driving a corrective lens included in the imaging optical system in a direction perpendicular to the optical axis, and a second correction means that corrects image blur by driving an image sensor that converts light incident through the imaging optical system into an image signal in a direction perpendicular to the optical axis, A first acquisition step involves acquiring the amount of vibration detected by the vibration detection means, A determination step of determining a method for calculating a correction amount for controlling the first correction means and the second correction means based on the amount of the runout and the control method of the first correction means and the second correction means, A calculation step of calculating the correction amounts of the first correction means and the second correction means based on the amount of the aforementioned fluctuation using the calculation method, A second acquisition step involves acquiring a representative motion vector based on the motion vector detected from the image output from the image sensor, A setting step to set a method for acquiring the representative motion vector to be used in the second acquisition step, based on the calculation method described above. A control method characterized by having the following features. (Item 19) A control method for controlling image blur correction, which corrects image blur by driving at least one of a corrective lens included in an imaging optical system and an image sensor that converts light incident through the imaging optical system into an image signal in a direction perpendicular to the optical axis, An acquisition step in which a representative motion vector is acquired based on motion vectors detected from multiple regions of the image output from the image sensor after the image blur correction has been performed, A setting step of changing the settings related to the motion vectors of the plurality of regions according to the position of an arbitrary region in the image output from the image sensor selected by the user as the reference for image blur correction, A control method characterized by having the following features. (Item 20) A program for causing a computer to function as one of the control devices described in any one of items 1 through 15. (Item 21) A computer-readable storage medium containing the program described in item 20.

[0114] The invention is not limited to the embodiments described above, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, claims are attached to disclose the scope of the invention. [Explanation of symbols]

[0115] 100...Camera body, 101...Image sensor, 101a...Shift mechanism, 102...Camera microcontroller, 103...Image blur correction control unit, 105...Camera shake detection unit, 106...Camera communication unit, 108...Image processing unit, 109...Motion vector detection unit, 129...Lens information management unit, 132...Image sensor position detection unit, 161...Camera integration unit, 162...Camera shake correction amount calculation unit, 163...Camera ratio calculation unit, 164...Camera drive range limit unit, 165...Camera feedback control unit, 166...Camera control method determination unit, 170...Control section determination unit, 2 00...Interchangeable lens, 204...Image shake correction lens, 204a...Shift mechanism, 222...Aperture control unit, 223...Focus control unit, 224...Image shake correction control unit, 226...Lens microcontroller, 227...Data storage unit, 228...Lens shake detection unit, 229...Lens communication unit, 237...Camera information management unit, 251...Lens integration unit, 252...Lens shake correction amount calculation unit, 253...Lens ratio calculation unit, 254...Lens drive range limit unit, 255...Lens feedback control unit, 256...Lens control method determination unit, 258...Lens vibration isolation position detection unit

Claims

1. A control device for controlling image blur correction comprising: a first correction means that corrects image blur by driving a corrective lens included in the imaging optical system in a direction perpendicular to the optical axis; and a second correction means that corrects image blur by driving an image sensor that converts light incident through the imaging optical system into an image signal in a direction perpendicular to the optical axis, wherein the first correction means corrects image blur by driving the image sensor in a direction perpendicular to the optical axis. A first acquisition means for acquiring the amount of vibration detected by the vibration detection means, A determination means for determining a method for calculating a correction amount for controlling the first correction means and the second correction means, based on the amount of the deflection and the control method of the first correction means and the second correction means, A calculation means that calculates the correction amounts of the first correction means and the second correction means based on the amount of the aforementioned fluctuation using the calculation method, A second acquisition means for acquiring a representative motion vector based on the motion vector detected from the image output from the image sensor, A setting means for setting a method for acquiring a representative motion vector by the second acquisition means based on the calculation method described above, A control device characterized by having the following features.

2. The control device according to claim 1, wherein the determination means determines whether the first correction means and the second correction means are compatible with a cooperative control method that includes overcorrection, which corrects the first correction means by an excess amount exceeding the correction amount within a range in which the first correction means can be driven, and inverse correction, which cancels out the excess correction by the second correction means, and if it is compatible with the cooperative control method, it further determines the calculation method according to the amount of vibration.

3. When the first correction means and the second correction means correspond to the cooperative control method, the determination means determines the amount of the swing. When the amount of shake is such that image blur correction can be performed by performing the aforementioned overcorrection and inverse correction, a first calculation method is determined as the calculation method, which performs the aforementioned overcorrection and inverse correction. If the correction amount exceeds the range in which the first correction means can be driven, a second calculation method is determined in which the correction amount of the first correction means is driven by the maximum drive amount that can be driven, and the deficit is allocated as the correction amount of the second correction means. The control device according to claim 2.

4. The setting means is, In the first calculation method described above, the method for acquiring the representative motion vector is set based on the motion vector detected from a region including a first region including the center of the image and a second region including the peripheral part of the image. In the case of the second calculation method, the method for acquiring the representative motion vector is set based on the motion vector detected from the region that includes the first region but does not include the second region. The control device according to claim 3.

5. The setting means is, In the first calculation method described above, the method for acquiring the representative motion vector is set by assigning the same weight to the motion vectors detected in the first region including the center of the image and the second region of the peripheral part of the image. In the case of the second calculation method described above, the method for obtaining the representative motion vector is set by assigning a smaller weight to the motion vector obtained from the second region than to the motion vector detected from the first region. The control device according to feature 4.

6. The control device according to claim 4, characterized in that, in the case of the first calculation method, the method for acquiring the representative motion vector is set by applying a higher gain to the motion vector than in the case of the second calculation method.

7. When the first correction means and the second correction means correspond to the cooperative control method, the determination means determines the amount of the swing. In the case where the amount of shake is such that image blur correction can be achieved by driving the first correction means within the range in which it can be driven in the overcorrection and performing the inverse correction, the calculation method is determined to be the first calculation method which performs the overcorrection and the inverse correction. In the case where the amount of shake is such that image blur correction can be achieved by driving the first correction means with the maximum drive amount that can be driven in the overcorrection and performing the inverse correction, a second calculation method is determined as the calculation method, which performs the overcorrection and the inverse correction. If the correction amount exceeds the range in which the first correction means can be driven, a third calculation method is determined in which the correction amount of the first correction means is driven by the maximum drive amount that can be driven, and the deficit is allocated as the correction amount of the second correction means. The control device according to claim 2.

8. The setting means is, In the first calculation method described above, the method for acquiring the representative motion vector is set based on motion vectors detected from a region including a first region including the center of the image, a second region that is peripheral to the image beyond the first region, and a third region that is even further peripheral to the image beyond the second region. In the case of the second calculation method described above, the method for acquiring the representative motion vector is set based on the motion vector detected from the region including the first region and the second region, In the case of the third calculation method, the method for acquiring the representative motion vector is set based on the motion vector detected from the first region. The control device according to feature 7.

9. The setting means is, In the first calculation method described above, the method for acquiring the representative motion vector is set by assigning the same weight to the motion vectors detected from a region including a first region containing the center of the image, a second region further to the periphery of the image than the first region, and a third region further to the periphery of the image than the second region. In the case of the second calculation method described above, the motion vector obtained from the second region is given a smaller weight than the motion vector detected from the first region, and the motion vector obtained from the third region is given a smaller weight than the motion vector detected from the second region, thereby setting the method for obtaining the representative motion vector. In the third calculation method described above, the motion vector obtained from the second region is given a smaller weight than the motion vector detected from the first region, the motion vector obtained from the third region is given a smaller weight than the motion vector detected from the second region, and the motion vectors detected from the second and third regions are given a smaller weight than in the second calculation method, thereby setting the method for obtaining the representative motion vector. The control device according to feature 7.

10. The setting means is, In the case of the second calculation method, the method for obtaining the representative motion vector is set by applying a lower gain to the motion vector than in the case of the first calculation method. In the third calculation method, a lower gain than in the second calculation method is applied to the motion vector to set the method for acquiring the representative motion vector. The control device according to feature 7.

11. The system further includes means for the user to select any region in the image, The control device according to claim 1, wherein the setting means, when the region is selected, sets the method for acquiring the representative motion vector based on the motion vector detected from the selected region, regardless of the calculation method.

12. The control device according to claim 1, wherein the calculation means further calculates the correction amounts of the first correction means and the second correction means using the representative motion vector.

13. The control device according to claim 1, further comprising a second calculation means for calculating a drive amount for tracking a subject using the aforementioned representative motion vector.

14. The control device according to claim 2, wherein if at least one of the first correction means and the second correction means does not correspond to the cooperative control method, the setting means sets a predetermined method for acquiring a representative motion vector regardless of the calculation method.

15. A control device for controlling image blur correction, which corrects image blur by driving at least one of a corrective lens included in an imaging optical system and an image sensor that converts light incident through the imaging optical system into an image signal in a direction perpendicular to the optical axis, An operating means for the user to select an arbitrary region in the image output from the image sensor to be used as the reference for image blur correction, An acquisition means for acquiring a representative motion vector based on motion vectors detected from multiple regions of an image output from the image sensor after the image blur correction has been performed, A setting means for changing the settings relating to the motion vectors of the plurality of regions according to the position of the selected arbitrary region, A control device characterized by having the following features.

16. A control device according to any one of claims 1 to 15, An imaging device characterized by having the aforementioned image sensor.

17. The imaging device according to claim 16, The aforementioned yard optical system and An imaging system characterized by having the following features.

18. A control method for controlling image blur correction comprising: a first correction means that corrects image blur by driving a corrective lens included in the imaging optical system in a direction perpendicular to the optical axis; and a second correction means that corrects image blur by driving an image sensor that converts light incident through the imaging optical system into an image signal in a direction perpendicular to the optical axis, wherein A first acquisition step involves acquiring the amount of vibration detected by the vibration detection means, A determination step of determining a method for calculating a correction amount for controlling the first correction means and the second correction means based on the amount of the runout and the control method of the first correction means and the second correction means, A calculation step of calculating the correction amounts of the first correction means and the second correction means based on the amount of the aforementioned fluctuation using the calculation method, A second acquisition step involves acquiring a representative motion vector based on the motion vector detected from the image output from the image sensor, A setting step to set a method for acquiring a representative motion vector to be used in the second acquisition step, based on the calculation method described above. A control method characterized by having the following features.

19. A control method for controlling image blur correction, which corrects image blur by driving at least one of a corrective lens included in an imaging optical system and an image sensor that converts light incident through the imaging optical system into an image signal in a direction perpendicular to the optical axis, An acquisition step in which a representative motion vector is acquired based on motion vectors detected from multiple regions of the image output from the image sensor after the image blur correction has been performed, A setting step of changing the settings related to the motion vectors of the plurality of regions according to the position of an arbitrary region in the image output from the image sensor selected by the user as the reference for image blur correction, A control method characterized by having the following features.

20. A program for causing a computer to function as one of the means of the control device described in any one of claims 1 to 15.

21. A computer-readable storage medium storing the program described in claim 20.

Images